You are viewing limited content. For full access, please sign in.

Question

Question

laserfiche search using SDK -- what's the limitation of returning result list?

asked on March 3, 2020

Hello, I am going to create a program that using Laserfiche SDK and the first thing to do a search for all documents under certain folder -- however the targeting repository have millions records. So I am wondering, is there a limitation how many results will return by the search API?  Thanks.

0 0

Answer

SELECTED ANSWER
replied on March 4, 2020

There is no limit as long as your database has enough hardware resources to handle the large search.

If your goal is to find all documents under a folder, you could also crawl the folder tree. This bypasses the search engine and avoids a large temporary table with the search results. Here is some sample code:

List<DocumentInfo> FindAllDocumentsUnderFolder(Session session, string path)
{
    List<DocumentInfo> allDocuments = new List<DocumentInfo>();

    EntryListingSettings settings = new EntryListingSettings();
    settings.AddColumn(SystemColumn.Id);
    settings.AddColumn(SystemColumn.Name);
    settings.AddColumn(SystemColumn.EntryType);
    settings.SetSortColumn(SystemColumn.Name, SortDirection.Ascending);

    Stack<string> folders = new Stack<string>();
    folders.Push(path);

    while (folders.Count > 0)
    {
        string currentFolderPath = folders.Pop();

        Console.WriteLine($"Processing {currentFolderPath}");

        using (EntryInfo currentEntry = Entry.GetEntryInfo(currentFolderPath, session))
        using (FolderListing listing = ((FolderInfo)currentEntry).OpenFolderListing(settings))
        {
            foreach (EntryListingRow row in listing)
            {
                int entryId = (int)row[SystemColumn.Id];
                string entryName = (string)row[SystemColumn.Name];
                EntryType entryType = (EntryType)row[SystemColumn.EntryType];

                string entryPath = EntryPath.Make(currentFolderPath, entryName);

                if (entryType == EntryType.Folder || entryType == EntryType.RecordSeries)
                {
                    // Add folders to the stack so they can be processed later
                    folders.Push(entryPath);
                }
                else
                {
                    allDocuments.Add(Document.GetDocumentInfo(entryId, session));
                }
            }
        }
    }

    return allDocuments;
}

 

1 0
replied on March 4, 2020

Thank you Robert. Your code provide a good and safe way of traversing among all nodes of folder tree -- I might use this idea for some other situation that doesn't have time limitation of processing every single nodes (whether document or folder). It's better than "search" API because of uncertain returning size of the result list.

0 0

Replies

replied on March 4, 2020

Depending heavily on what kind of work I'm needing to do, sometimes I'll first query the database directly and cache the results.

You can make a temporary table that's as simple as a single column called EntryId, and run a query that populates it with the entries that you are going to work on. You can even still use the SDK to populate the values. However, I like to take it a step further, and include a bit field called Done. Then, in my script, I loop through that table, and use the SDK to act on each Entry Id that it finds. While it's doing that, it sets the Done flag. This is useful, especially for many thousands, or millions of documents, because it means that the process can be restarted if there is a problem.

1 0
replied on March 4, 2020

Hey Devin, after talk with some colleagues who had experience with Laserfiche Database I had the same idea of directly read from "toc" table to grab all exist "entry" with some of the key information and write into another custom table -- a typical "insert into [table] select from [toc] where [condition]. This should be the fastest way even though requires knowledge of LF database.

Cheers. 

1 0
replied on March 3, 2020

one of the sub-folder

2.jpg
2.jpg (28.81 KB)
0 0
You are not allowed to follow up in this post.

Sign in to reply to this post.