You are viewing limited content. For full access, please sign in.

Question

Question

Import Agent & XML files not merging folders correctly

asked on October 26, 2016

Hello all,

 

I'm encountering a strange issue with Import Agent (latest version 10.0.0.219) and using XMLs to import files. My understanding is XML's are the way Laserfiche is going and that LSTs will go away eventually, so I recently started using XMLs instead of LSTs. The problem doesn't exist with LSTs though. The problem I have is that if Import Agent is importing multiple files at the same time to a folder that doesn't exist yet in Laserfiche, it will create the first folder correctly, then on the second file, it will create a duplicate destination folder and stick the second file under there, then put the remaining files under the first folder.

 

Test scenario

Preparation:

  1. Have 3 or more images to import into the same destination directory.
  2. Have the "on_folder_conflict="merge">" in the XML set so that it will merge files into the destination directory if it exists already (instead of "unique", which will create a new folder if it exists).

 

The basic XML for import looks like this:

<?xml version="1.0" encoding="utf-8"?>
<LF:importengine xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://laserfiche.com/namespaces/importengine ImportEngine.xsd" xmlns:LF="http://laserfiche.com/namespaces/importengine" version="1.0">
  <LF:toc on_document_conflict="unique" on_folder_conflict="merge">
    <LF:folder name="Folder1">
      <LF:childentries>
        <LF:folder name="Folder2">
          <LF:childentries>
            <LF:document name="documentname" template="templatename">
              <LF:metadata>
                <LF:field_values>
                  <LF:field_value name="field1">field1data</LF:field_value>
                </LF:field_values>
              </LF:metadata>
              <LF:pages>
                <LF:page ocr="false">
                  <LF:imagepart>
                    <LF:fileref ref="pathtoimage\image1.jpg" />
                  </LF:imagepart>
                </LF:page>
              </LF:pages>
            </LF:document>
          </LF:childentries>
        </LF:folder>
      </LF:childentries>
    </LF:folder>
  </LF:toc>
</LF:importengine>

 

The bolded parts are things that you can change.

Folder1 - First folder

Folder2 - Subfolder/destination folder

documentname - The name of the document when it goes into Laserfiche

templatename - Whatever your template name is in Laserfiche (specified in the Import Agent config).

field1 - The field name of a field that's in the template chosen.

field1data - Whatever information you want stored in that field.

pathtoimage\image1.jpg - The path to the jpg and the file name of it. For me, I just had 5 jpg's, with the file names image1.jpg-image5.jpg, all stored in the same directory as the monitored folders as setup in Import Agent.

You will need 1 XML file per JPG to be imported. Just change the document name and "image1.jpg" part for each of the files.

 

In the import agent profile, setup the import filter to only import *.xml. You will also have to go under Profile>Options and then the File Types tab, and check off the option "Recognize files with XML and LSTX extensions as list files".

 

Test case:

Once you have all the JPG's in the monitored folder, place all the XML files at once into the monitored folder as well. What I encounter is the following:

  1. Import agent will start with the first file. Since the destination folder (folder2) doesn't exist in Laserfiche yet, it will create the folder, and place the first image into that folder.
  2. When it processes the second image, it's supposed to place it into the same folder as the first image since the "folder_conflict" is set to merge. What actually happens is that Import Agent will create a second "folder2" (but since Laserfiche cant have 2 folders with the same name, it appends a (1) after the folder name). It then places the second image into this folder.
  3. For the third and any further/future files, it will import the files into the first "folder2" folder that was created with the first file.

 

Again, if you use LSTs instead, it does not create a second "folder2" folder and instead places all the files in the same folder. Also, if you space out the files to be imported (ie place the first XML in the monitored folder, wait for it to be imported, then place the rest of the XMLs in the folder), it will place all the files in the same folder and not create the duplicate folder.

 

What I believe is happening is that Import Agent is processing the files too quickly so that it's not seeing that from the first file it had created the directory already. By the time it does the third file, it now sees the folder and is able to properly "merge" the files into the same folder. This is creating somewhat of an intermittent problem for me since it's only arising if 2 or more files are going to the same destination folder at the same time and the destination isn't created yet. This of course leaves me with the odd folder with "(1)" appended to it and only 1 file in it, while the everything else is in the original folder.

 

1 0

Answer

SELECTED ANSWER
replied on October 26, 2016

Thank you for the detailed information! You are right. The folder structure may be incorrect when importing XML file with multi-threading. To workaround it, navigate to HKLM\Software\Laserfiche\Import Agent\Settings and add a new string value "ImportThreadCount". set the value 1. Note, the setting will decrease the thread count and reduce the efficiency. We will investigate if there is other way to fix the issue. Thanks again for reporting this.

0 0
replied on October 27, 2016

Thanks for the quick reply. Ive implemented the registry fix for now then.

0 0

Replies

You are not allowed to reply in this post.
You are not allowed to follow up in this post.

Sign in to reply to this post.