Does HTTP Web Request have a concurrency limit in self-hosted Workflow?

asked on September 29

I’m trying to understand how self-hosted Laserfiche Workflow limits concurrent HTTP Web Request activities across workflow instances and definitions.

The Task Threads documentation mentions activity-specific caps for searches, database/directory tasks, and email, but I haven’t found an explicit limit for HTTP Web Request.

Web service configuration also includes “Run activities that use this web service as a task.”

Could someone clarify:

With that option enabled, which thread pool handles HTTP Web Request activities, and what controls the maximum concurrent requests?
Is there an additional limit per web service?
Does a request occupy a task/thread slot for the entire time it waits for the remote server to respond?
If the option is disabled, what concurrency limits apply instead?
Does this behavior differ between Workflow 11 and 12?

For example, if 100 workflow instances reach an HTTP Web Request activity calling the same endpoint, how many requests can actually be in flight simultaneously, and where do the remaining requests queue?

I’m specifically asking about throttling within Workflow, separate from rate limits imposed by the destination API or Laserfiche Cloud.

0 0

replied on September 29 • Show version history

Hi Blake,

External Tasks handles HTTP Web Request activities. The maximum number is 4 × CPU cores.
No.
Yes.
There is no concurrency limit when the option is disabled.
No, there is no difference.

In this example, the concurrency limit is determined by taking the greater of 4 × the CPU count and the value configured in External Tasks. Web request tasks that exceed this limit are queued in the database.

Thanks,

Zhujie

1 0

View 4 previous replies

replied on September 30

Thank you for the reply. When a HTTP Web Request activity completes, does a pending one immediately start or how does that work?

0 0

replied on September 30

One thing to note is that Laserfiche recommends against clearing the option to run activities as a task. Moving these activities from the task system to Workflow itself increases the resource usage on the Workflow Server itself and the risk of crashes if data coming back from the web service is malformed in a way that the activity can't handle. On the web service side, you may be running a risk of being rate limited due to the higher amount of traffic.

You mentioned Laserfiche Cloud, so this is a reminder that self-hosted Workflow is not included in Laserfiche Cloud plans and not supported with Laserfiche Cloud.

0 0

replied on September 30

I was curious if you were going to chime in. Thank you for the extra words of caution about clearing that setting.

0 0

replied on September 30

Nah, Zhujie's got it. I'm just adding the Greek chorus commentary of our experience to prevent future support cases

Out of curiosity though, what is the use case for it?

0 0

replied on September 30

There is a workflow that kicks off thousands of instances within a short amount of time. The workflow makes an API call to get some data, but we are seeing some instances take over 30-60 minutes to complete and looking at them, it is the API call that is taking the longest amount of time. Workflow shows the activity started at a specific time, but the API server is not showing it was contacted until 30 minutes later. So, we are trying figure out where the disconnect is and what we can do to help prevent the delay in processing.

0 0

replied on October 1

Right, so Workflow did put the activities in the task system, but then no slots were available to run because of the flood of activities. How fast did the web service respond once it got the call?

If you move the activities out of the task system, then you'll likely see this load move to starting instances. Because they would now run in process, the instance would hold an active slot for the duration of the call, so Workflow wouldn't be able to start another because it would have no free slots. This is also limited to 4 per CPU per Workflow definition and not editable because it comes from Windows Workflow Foundation. The task system allows Workflow to do more work while waiting on calls that may take a while, like web service calls, repository searches, data queries and such. It's the equivalent of "call me back when you have info and I'll pick where we left off" vs. sitting there waiting for the info to be retrieved. Or having minions run your errands.

1 0

replied on October 1

The web service responded within seconds.

0 0

Question

Question

Does HTTP Web Request have a concurrency limit in self-hosted Workflow?

Replies

Sign in to reply to this post.