It is not recommended to disable all activity tracking. If something goes wrong, with your current configuration you will have no way of troubleshooting. At least activity errors, warnings, conditions and tokens should be on.

When you open the instance details, the Designer still has to check if there are any rows in the table in SQL. Depending on when the instance ran, it may have data tracked already (as Advanced Server Options apply to new instances started after you made changes). Also, disabling activity tracking does not mean the WF Server will not log each activity it runs in the instance. So when you open the instance, it still goes through the data for 30000 times however many activities you run in For Each Row.

Ok, so basically check the Excel spreadsheet every day for new values and handle those.

I'd recommend letting SQL do most of the work of figuring out what the new values are. I get the appeal of the UI in Workflow (), but SQL is a better tool.

So here is what I would do:

1. Use SQL SSIS (or the Import Data wizard in SQL Management Studio) to import the Excel spreadsheet into a table, let's call it "Reference" and run your workflow once.

2. At this point, every row in the Reference table would have a corresponding folder in LF. Add a "Processed" column to your table and set it to 1.

Now, you'd have a reference point for data.

3. Create a SQL Scheduled Job (or save the Import Data package you created above) that runs daily that imports the Excel file into a TEMP table.

4. Create a scheduled job that runs a query that gets the values that are in the TEMP table but not the REFERENCE one and inserts them into the REFERENCE table. Something like

INSERT INTO reference([column1],[column2],[column3])
SELECT [column1],[column2],[column3] FROM temp
WHERE [column1] NOT IN 
(SELECT [column1] FROM reference WHERE status = 1)

In your WF

5. Modify the Query Data activity to look at the REFERENCE table instead of the spreadsheet and only get the values where the Status column is not 1.

6. Do whatever you did before to create the folders

7. Run an Update Data activity to set the Status column to 1 for everything in the table.

8. Run a Custom Query to drop the TEMP table (the SQL import job would re-create it next time it runs)

This would basically eliminate all searches. At the end of each run, your REFERENCE table would be updated to have all the values you've processed so far. And most days, the workflow instance would only run a Query Data activity since you're saying the spreadsheet is not updated daily.

If whatever application is generating the Excel spreadsheet could write it to SQL directly, things would be even easier.

Miruna,

You are actually correct. I guess I overlooked that matter. But you can use update/insert data into a spreadsheet. Or if you do not wish to do that, you can easily make the spreadsheet into a SQL table. 

To reduce the overhead you can have the workflow invoke itself after incrementing a counter variable and passing it through to the next invoked workflow, it will always try the list of exported documents for the next ID and if it's not there (say it's a folder perhaps that we dont care about) then you increment the counter and do a find entry for the next number when that workflow is called again. You would be making the find entry return one item at a time until you were done with the entire repository. Once done you would have a list of the files needing to be made. 

EDIT: I am having to do a similar amount of searching for one of our clients and I was planning on doing something similar to this so i can reduce overhead at any given time. I also have implemented a solution to get information in stages and limit the workflow to working during non-business hours