Operation timed out communicating with the workflow service

asked on June 1, 2022 • Show version history

We had a client getting the below message every time they tried to send an email, sending a test email from the workflow config also threw the error stating it was timing out trying to contact the workflow management service. In all other actions, Workflow was working, but the problem could not be resolved without restarting the workflow service to essentially re-construct it's state in memory.

What causes it to stop allowing SMTP messages like this?

 <InnerException>
    <ExceptionType>System.Net.Mail.SmtpException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
    <Message>The operation has timed out.</Message>
    <Source>System</Source>
    <HelpLink />
    <Property name="StatusCode">GeneralFailure</Property>
    <Property name="EmailSyncLevel">1</Property>
    <Property name="StartTime">6/1/2022 3:20:04 PM</Property>
    <Property name="EndTime">6/1/2022 3:21:44 PM</Property>
    <Property name="Duration">00:01:40.0127120</Property>
    <Property name="InstanceId">395fe6ee-b947-4b52-a0d4-8ad0212b0a5d</Property>
    <Property name="ActivityName">Email_EmailWithAttachments</Property>
    <Property name="Workflow">NotifyofFailedEmails</Property>
    <Property name="ContextGuid">395fe6ee-b947-4b52-a0d4-8ad0212b0a5d</Property>
    <Property name="TargetSite">Void Send(System.Net.Mail.MailMessage)</Property>
    <Property name="HResult">-2146233088</Property>
    <StackTrace>   at System.Net.Mail.SmtpClient.Send(MailMessage message)
   at Laserfiche.Workflow.Activities.Email.Runtime.EmailTaskExecutor.Execute(EmailTaskOptions options, TaskExecutorContext status)</StackTrace>
  </InnerException>

0 0

replied on June 1, 2022

That's not timing out communicating with the Workflow server. That's Workflow Server timing out when attempting to reach the mail server. It could be a network issue or something else on the mail server.

Please open a support case to investigate it.

0 0

View 2 previous replies

replied on June 1, 2022

I can not see how this could be on the SMTP provider side if I fixed it with a restart of my local service "Laserfiche Workflow"

It failed and failed and failed all day yesterday and this morning, immediatly after a failed test I restarted the workflow service and the next test went right through. Re-building the application code into Memory was the solution, which means the problem was in the app, not the external provider.

0 0

replied on June 1, 2022

Or it's a coincidence and the network issue cleared up around the same time. Or, you may have run into throttling on the mail server side (I've seen Microsoft 365 throw timeout errors instead of the actual error when you run into sending limits) and that expired by the time the server was restarted.

It's really not possible to guess from one snippet.

0 0

replied on June 1, 2022

There was a 10 second window and it was not the first time I have resolved this issue always by restarting the workflow service.

The message also specifically mentions it is timing out reaching the management service, which is what workflow calls itself.

I have no evidence I can take to the SMTP provider, I have plenty of evidence my local service is the problem. There is just no way the SMTP providers are fixing it immediatly as I restart my service, this is probably the 4th time I have seen this issue in the wild.

0 0

replied on June 1, 2022

I'm only saying all we have here is the error above which is about communication between Workflow and the mail server timing out on the mail server side and based on that message alone, I can't hazard a guess as to the cause.

Please open a support case and attach the contents of Workflow's Logs folder from the install directory and your Application, System and LFWorkflow event logs.

0 0

replied on June 1, 2022

Ok but I am not having a timout communicating with the SMTP server. I am having a corruption of the state of code running as a service on the server in local memory, proven by restarting the service to fix the problem with a 10 second window between full failure state and fully operational after a 24 hour downtime.

This is what happens if the communication is severed with the SMTP server

For a failure in DNS

    <InnerException>
      <ExceptionType>System.Net.WebException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
      <Message>The remote name could not be resolved: 'mail.smtp2go.com2'</Message>
      <Source>System</Source>
      <HelpLink />
      <Property name="Status">NameResolutionFailure</Property>
      <Property name="Response">&lt;undefined value&gt;</Property>
      <Property name="TargetSite">System.Net.Sockets.Socket GetConnection(System.Net.PooledStream, System.Object, Boolean, System.Net.IPAddress ByRef, System.Net.Sockets.Socket ByRef, System.Net.Sockets.Socket ByRef)</Property>
      <Property name="HResult">-2146233079</Property>
      <StackTrace>   at System.Net.ServicePoint.GetConnection(PooledStream PooledStream, Object owner, Boolean async, IPAddress&amp; address, Socket&amp; abortSocket, Socket&amp; abortSocket6)
   at System.Net.PooledStream.Activate(Object owningObject, Boolean async, GeneralAsyncDelegate asyncCallback)
   at System.Net.ConnectionPool.GetConnection(Object owningObject, GeneralAsyncDelegate asyncCallback, Int32 creationTimeout)
   at System.Net.Mail.SmtpConnection.GetConnection(ServicePoint servicePoint)
   at System.Net.Mail.SmtpClient.GetConnection()
   at System.Net.Mail.SmtpClient.Send(MailMessage message)</StackTrace>
    </InnerException>
  </InnerException>

For a disconnected server

    <InnerException>
        <ExceptionType>System.Net.Sockets.SocketException, System, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089</ExceptionType>
        <Message>A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond 100.0.0.1:25</Message>
        <Source>System</Source>
        <HelpLink />
        <Property name="ErrorCode">10060</Property>
        <Property name="SocketErrorCode">TimedOut</Property>
        <Property name="NativeErrorCode">10060</Property>
        <Property name="TargetSite">Void DoConnect(System.Net.EndPoint, System.Net.SocketAddress)</Property>
        <Property name="HResult">-2147467259</Property>
        <StackTrace>   at System.Net.Sockets.Socket.DoConnect(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.ServicePoint.ConnectSocketInternal(Boolean connectFailure, Socket s4, Socket s6, Socket&amp; socket, IPAddress&amp; address, ConnectSocketState state, IAsyncResult asyncResult, Exception&amp; exception)</StackTrace>
      </InnerException>

0 0

Question

Question

Operation timed out communicating with the workflow service

Replies

Sign in to reply to this post.