replied on August 1, 2024
What are your limitations? I haven't done LF Cloud migrations, but I've done on-prem. This might not be what you're asking, but I will let you know my thoughts. Since simultaneously running many workflows during production hours could negatively affect performance, here was my approach to importing millions of already-digitized pages from a different repository system. Once they were imported, instead of having a workflow run as soon as an entry is created (which can generate quite a few simultaneous workflows), I instead used an hourly workflow which searched for entries that weren't yet processed and returned the average amount that could be processed in an hour (in my case this was about 400). I then used a "for each entry" loop to perform the work (like generating pages, OCR, renaming, updating metadata, moving) within a "deadline" activity. This deadline would end the workflow if it ran for 59 minutes, which prevented it from running as the same time as the next hourly workflow.
This approach allows easy scalability by the hour, because I can schedule it to only run once every hour for any hour of the day by only changing the starting rule, not the workflow itself. Additionally, each iteration of the "for each entry" loop is only a single workflow running at a single time, so there was essentially zero chance that it could spawn runaway workflows and bog the server down.