Question

Updating Large Meta Data Workflow

Workflow

Updated March 23

asked on March 11

Hi all,

First time doing a big metadata update and I have crashed my workflow server! Maybe 48 000 000 condition tests were not a good idea.

I have a legacy Folder which has a whole bunch of files very messy, each of these files have an employee number in their meta data. They are about 12000 legacy files

I have created a new folder with employee folders with matching employee numbers.

I used a workflow to search through the files and if the employee numbers were matching for it to update some of the meta data in the new files from the old files. In future I might also move some files across to the new folders.

However after running it it seemed to work for about 10 entries in the new folder but then crashed my whole workflow server.

What is a better way that I can achieve this task

Please see my workflow below

Any help appreciated!

0 0

Answer

SELECTED ANSWER

replied on March 12

12 thousand legacy documents times however many new employee folders is definitely too large a number to process in one instance. Did the server crash or is it just really busy crunching the millions of iterations and conditions?

The problem with this design is you're not actually searching, you are looking at every record and asking "is it this one? no, ok, next, is it this one?". The way For Each Entry works, even after you find a match, you are still going through the rest of the data set looking for matches. So for each new employee folder, you go through the entire stack of 12 thousand legacy folders trying to find the matching one.

You want to replace Find Entries Historic with a Search Repository and put it after Retrieve Field Values New (inside For Each Entry New). The search criteria should include whatever matching information you're checking in the Match branch. This way, Workflow will look at the new record and find only the matching old record.

1 0

Replies

replied on March 12

Thanks Miruna,

That workflow makes more sense I will give that a shot.

Just a side question, is there any way to exit a For loop once a condition has been met?

0 0

replied on March 23

Short of terminating the instance, there is not.

1 0

You are not allowed to follow up in this post.

Question

Question

Updating Large Meta Data Workflow

Answer

Replies

Sign in to reply to this post.