You are viewing limited content. For full access, please sign in.

Question

Question

Running workflows concurrently

Workflow
Updated April 26, 2015
asked on April 24, 2015

If we have 4x vCPUs provisioned (i.e. 16 Workflow threads), does this means that we can run 4 concurrent Workflows (to accommodate requirement for 1x Top Priority and 2x Medium Priority Workflows to be run concurrently) using a single Workflow server?

1 0

Answer

SELECTED ANSWER
replied on April 24, 2015

Ok, so we're actually talking about task priorities for custom scripts (which run as tasks so they are "sandboxed". Since script code is not our code, we have no way of knowing if the author handled all possible exceptions and so on. To prevent a poorly written script from crashing the entire server, they are run in separate threads, isolated from the rest of the server). In that case, yes, the tasks are throttled by the option in the Advanced Server Options and that pool of threads applies to all workflows definitions.

The same rules that govern the design of custom activities apply to scripts as well, so they shouldn't be long running. If they are, they do have the potential to slow down execution of other workflows.

Depending on what your scripts are doing, you could increase the number of threads, monitor performance and see if that is enough to solve your problem.

You could choose to not run scripts as tasks (in the Activity Performance tab in Adv. Server Options). That way instances in different workflow definitions would no longer impact each other. However, the Workflow Server may use more resources since there would be potentially more concurrent activities. And more importantly, you would be losing the protection of the "sandbox" feature, so an unhandled exception in your code in a script could propagate out and crash the Workflow Server.

You other option, as you mentioned, would be to run separate servers for the low and high priority workflows. That would be the safer option.

Currently Workflow does no support a priority system for instances. The concept of "system workflows" that would get lower priority and possibly track less information as they run is something we've thought about and it will make it into a future version.

1 0

Replies

replied on April 24, 2015
1 0
replied on April 24, 2015

No, it means you can have 4 concurrent actively running activities per CPU per Workflow definition. Additionally, activities are considered "actively running" only when they do work. So for ex, an Assign Fields activity is actively running while setting fields. A Deadline or Wait for Entry Change activity is only actively running when checking conditions, not when waiting for a timer to go off or for a user action.

1 0
replied on April 24, 2015

Thanks Miruna.

So, does that mean that there is no way of setting workflows to run concurrently (and only tasks can run concurrently using multiple threads)?

What would be your advice; if we have a low priority workflow triggering continuously on documents creations and then we get some priority documents in priority folder which triggers the priority workflow, so apparently priority workflow will keep going in queues.

Is there a way to prioritise the processing of priority workflow by temporarily queuing the low priority workflow threads?

1 0
View 1 previous reply
replied on April 24, 2015

That's not what I said at all. Different Workflow definitions do not have an impact on each other in terms of the number of instances. The low priority workflows do not hold up high priority ones, they're independent as far as the number of instances per CPU is concerned.

Are there custom scripts in these workflows? What exactly are you seeing "going in queues"?

1 0
replied on April 24, 2015

Yes there will be custom scripts in the workflow. I am still in the process of requirements gathering stage.

I do understand, as per say, that low priority wfs will not hold up high priority wfs, but what I meant was, if we have large number of low priority wfs keep triggering while we have the high priority wfs triggered as well, then WF server will be running the tasks for both types of wfs concurrently. But what we want is, if high priority wf triggers, we want to pause the low priority wfs processing and only process the high priority wfs. And once high priority wfs finished processing, start the low priority wfs again. Does this make sense? Is there a way of implementing this?

Thanks

1 0
replied on April 24, 2015

Or the only solution is, to have two WF servers? One for processing low priority WFs and second one to process high priority WFs?

1 0
SELECTED ANSWER
replied on April 24, 2015

Ok, so we're actually talking about task priorities for custom scripts (which run as tasks so they are "sandboxed". Since script code is not our code, we have no way of knowing if the author handled all possible exceptions and so on. To prevent a poorly written script from crashing the entire server, they are run in separate threads, isolated from the rest of the server). In that case, yes, the tasks are throttled by the option in the Advanced Server Options and that pool of threads applies to all workflows definitions.

The same rules that govern the design of custom activities apply to scripts as well, so they shouldn't be long running. If they are, they do have the potential to slow down execution of other workflows.

Depending on what your scripts are doing, you could increase the number of threads, monitor performance and see if that is enough to solve your problem.

You could choose to not run scripts as tasks (in the Activity Performance tab in Adv. Server Options). That way instances in different workflow definitions would no longer impact each other. However, the Workflow Server may use more resources since there would be potentially more concurrent activities. And more importantly, you would be losing the protection of the "sandbox" feature, so an unhandled exception in your code in a script could propagate out and crash the Workflow Server.

You other option, as you mentioned, would be to run separate servers for the low and high priority workflows. That would be the safer option.

Currently Workflow does no support a priority system for instances. The concept of "system workflows" that would get lower priority and possibly track less information as they run is something we've thought about and it will make it into a future version.

1 0
replied on April 26, 2015

Thanks a lot Miruna. That is very helpful.

1 0
You are not allowed to follow up in this post.

Sign in to reply to this post.

Details
Related Posts
Loading ...