You are viewing limited content. For full access, please sign in.

Question

Question

Cloud Workflow - Creating over 10,000 folders

asked on January 6, 2021

We have a cloud customer that would like to generate over 10k new client folders at the start of each year. However the workflow system reports back this error in the middle of creating them

This workflow instance has been terminated due to exceeding activity invocation limit of 10000 activities. 

I assume this means a new instance must be started where the current left off, but instead it just terminates. How can we handle this better and continue where we left off with a new instance?

0 0

Replies

replied on January 6, 2021

Are you saying that if you starts another instance, it terminates? Or are you saying that this workflow is trying to invoke itself to start a new instance? If the latter, then that is expected, workflows are themselves activities.

0 0
replied on January 6, 2021 Show version history

Right now the workflow just has a for each and create entry, but when the for each reaches 10,000 it terminates.

So I am not sure how to know where it left off and continue where it left off with a new workflow.

0 0
replied on January 6, 2021

Ok, so that would be about 4999 entries in (Find Entries + (4999 For Each + 4999 Create Entry) + 1 For Each = 10,000).

 

0 0
replied on January 6, 2021

So how do I continue where I left off if the workflow just terminates? We are looking to create folders in every client subfolder returned, which surpasses the 10k limit.

I am not even sure if I tracked iterations, how I would tell the next flow to find entries and begin at iteration.

0 0
replied on January 11, 2021

I have something similar that runs on a weekly basis.  I only want to impact the new items on the search.  I used a tag to keep track of what items were already processed.

The search looks for items that match the search criteria that do not have the tag.

Then it processes the required actions on all the of search results, and one of the actions is to add the tag to the item to say it was processed, then it doesn't come up next time.

You could do something similar, adding a tag to each item after it processes.  Then you could just keep running your Workflow until every record is processed.  If you want to avoid having to re-run the Workflow after each failed attempt (every 10,000 actions), the I would recommend setting up your Workflow to limit the number of search results returned, may to something like 500 records.  Then have a condition to determine if the number of results was 0 or more.  If it was zero, you'll just end the Workflow without further action.  But if it is between 1-500, add the subfolder, and then add the tag to the entry, and then start a new instance of the Workflow, which will start the search all over.  The reason it does nothing if there are no results, is so that it has a way to end, since each instance of the Workflow that finds records, will trigger the workflow to load again (you don't want that to continue triggering over and over until your server crashes).  You could even do a reverse workflow to clean up the tags afterwards, for a fresh start in the new year, it would do the search just for the records with the tag, and then remove the tag.

0 0
replied on January 12, 2021 Show version history

Hi Matthew

I don't think searching by tags is an option. Only Field, Path, Template, and Type

0 0
replied on January 12, 2021

You should be able to do it with Search Syntax:

This example returns all folders in the path "MAIN\Operations\Accounts" that do not have a flag named "Security Flag".

{LF:Name="*", Type=F}
- {LF:Tags="Security Flag"}
& {LF:Lookin="MAIN\Operations\Accounts", Subfolders=N}

 

0 0
replied on January 12, 2021

I wish I could use search syntax, this was a good way to perform a search using the built in search tool and then link that search to a workflow by simply copying the syntax. I have asked about this and was told search syntax is not supported.

0 0
replied on January 12, 2021

Oh, I didn't realize that limitation existed in the cloud version of Workflow.

You might be able to do the same kind of thing by temporarily renaming the folders.

Let's say they are all named for account numbers, example:

1000001

1000002

1000003

After searching for them, it renames the folders with an extra character (that your initial search excluded), example:

1000001xxx

1000002xxx

1000003xxx

I assume you can use wildcards in the name search, so this example should work like this: [0-9][0-9][0-9][0-9][0-9][0-9][0-9] which would return the 7 digit numbers, but not return the ones where the numbers had the xxx after them.  Then the reverse search would be [0-9][0-9][0-9][0-9][0-9][0-9][0-9]xxx,  At least I think that'll work, I haven't tested that specific example.

You would definitely need the reverse Workflow afterwards to return the folder names to their original values.

0 0
replied on January 12, 2021

That might be the only way, was trying to use metadata since that should be more reliable, but it seems search can not search by metadata fields. When I try to search for HiddenWFField != X it doesn't return any entries. Even though there are plenty  of entries with no HiddenWFField applied.

1 0
replied on May 10, 2021

" HiddenWFField != X " would translate to "HiddenWFField is applied to the document and is either blank or has a value other than X". If the field is not set on the document, it wouldn't match that criteria. We likely won't change this behavior because if implemented, it has the potential to return very large data sets.

0 0
You are not allowed to follow up in this post.

Sign in to reply to this post.