Question

import agent email archive - what happens if LF repo is offline?

Import Agent

Updated November 19, 2020

asked on November 17, 2020

Hi All

We use Import Agent Email archive with SMTP to import emails into LF. When everything is working this works nicely.

Issue - when the LF repo/target that the Import Agent is supposed to post/create records in LF is offline, or down, what happens to the incoming emails landing in Import Agent. Import Agent is a email server, SMTP - so what does it do with incoming emails if it cannot store then into LF?

Do they spool/queue up and wait for LF Repo to come back online or are they "lost"?

Thanks

Jeff

0 0

Replies

replied on November 17, 2020

Looks like the emails may simply be lost - seeing error in Event Viewer, screenshot attached.

If this is the case, not very resilient architecture.

0 0

replied on November 17, 2020

Hi Jeff,

Looks like the screenshot attachment didn't come through.

Email Archive has an internal queuing mechanism. If the target repository is unavailable, EA should write emails to disk and periodically retry sending them. You shouldn't lose emails from the repository being offline. However, you can completely lose emails if Email Archive is offline or otherwise unavailable at that time.

For that and other reasons, we re-architected how Email Archive works in 10.4 to use IMAP instead of SMTP. Rather than trying to catch emails Exchange throws at it, Email Archive connects to a mailbox via IMAP and downloads unprocessed emails in that mailbox. This is a much more resilient and secure architecture that avoids known data loss scenarios possible with the old SMTP method, which we are depreciating.

I recommend upgrading to Import Agent/Email Archive 10.4 (download) and updating your email capture process to use the new IMAP method.

1 0

replied on November 17, 2020

Thanks - is there a folder on the server that shows the spooled email?

I attached what we are seeing in Event Viewer.

CleanShot 2020-11-17 at 09.29.02@2x.jpg (203.79 KB)

| Download

0 0

replied on November 17, 2020 • Show version history

They're stored in a local sqlite database rather than as files in a folder. The "File input/output" error you're seeing there seems to be failing on PDF page generation though. If you look at the stack trace, the last two visible lines in your screenshot are:

at Laserfiche.PdfServices.PdfExtractor.ImportPDFStream(<>)
at Laserfiche.EmailArchive.Workers.ImportDocumentWork.GeneratePdfPages(<>)

The Release Notes for Import Agent/Email Archive 10.3.1 list two Known Issues that might be at play:

Password protected PDF files cannot be imported. (132928)
For some PDFs, PDFServices will generate pages incorrectly. (144445)

The first is self-explanatory and the latter I unfortunately couldn't easily find more detailed information on. I would still recommend upgrading to Email Archive 10.4 which might have fixes that address your issue. If they still persist, try disabling PDF Page Generation in your Email Archive profile(s) as a test.

For your knowledge, I've also attached a diagram showing how Email Archive queues messages.

Laserfiche-EmailArchive10.3-SMTP-DataFlows.png (130.57 KB)

| Download

0 0

replied on November 17, 2020

Screen of event viewer

0 0

replied on November 18, 2020

We are on version 10.4, see attached.

Also - this looks like it should work, however we have many missing emails (100s) over a period of a few days when the LF Target Repo was offline (san disk offline - LF server was running but the repo disk was offline).

We do see errors in event viewer, but spot checking files that should have queued, it appears they are lost.

These are PDF files, that we are storing. The issue is a IO error, which makes sense as the target repo disk was offline.

CleanShot 2020-11-18 at 09.58.49@2x.jpg (189.91 KB)

| Download

0 0

replied on November 19, 2020

Hi Jeff, I see you've opened a support case so we'll handle further discussion of the issue through that channel.

I did want to note that an Import Agent/Email Archive 10.4. Update 1 patch with a long list of fixes was released on October 14th, 2020 (now incorporated into the main 10.4 installer). I recommend applying it when you have a chance.

0 0

replied on November 18, 2020

Here is a sample from the event log:

Log Name: Laserfiche-EmailArchive-Service/Admin
Source: Laserfiche-EmailArchive-Service
Date: 11/16/2020 6:39:27 AM
Event ID: 22
Task Category: ImportAttachmentError
Level: Error
Keywords: Session0,Session1,Session2,Session3,Import
User: SYSTEM
Computer: pgc-lfswork.pacificgroup.net
Description:
Accout archiv~1: Exception occurred when processing mail (2d568cf2-a724-4db0-9c4c-f627e82b675e) attachment.

Laserfiche.RepositoryAccess.LaserficheRepositoryException

Message: File input/output error. [9004]

StackTrace: at Laserfiche.RepositoryAccess.LaserficheWriteStream.FinishWrite()
at Laserfiche.RepositoryAccess.LaserficheWriteStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at Laserfiche.PdfServices.PdfExtractor.ImportPDFStream(Stream pdfStream, DocumentInfo doc, DateTime dtLastModified)
at Laserfiche.EmailArchive.Workers.ImportDocumentWork.GeneratePdfPages(DocumentInfo doc, IImportDocumentWorkSettings settings, IImportDocumentSource source)
at Laserfiche.EmailArchive.Workers.ImportDocumentWork.ImportDoc(DocumentInfo doc, IImportDocumentWorkSettings settings, IImportDocumentSource source)
at Laserfiche.EmailArchive.EmailExecuteJob.AttachmentDistWrapper.CreateAttachments(LfAttachmentCollection attachments, ICreateDocumentWorkParameters emailCreateDocParameter, IImportDocumentWorkSettings importSettings, AssignMetadataWork assignMetaWork)
Event Xml:
<Event xmlns="http://schemas.microsoft.com/win/2004/08/events/event">
<System>
<Provider Name="Laserfiche-EmailArchive-Service" Guid="{DA244C5A-571F-59A8-DD40-A50F8854D1AB}" />
<EventID>22</EventID>
<Version>0</Version>
<Level>2</Level>
<Task>65512</Task>
<Opcode>0</Opcode>
<Keywords>0x4000f00000000020</Keywords>
<TimeCreated SystemTime="2020-11-16T14:39:27.616795400Z" />
<EventRecordID>8501</EventRecordID>
<Correlation />
<Execution ProcessID="1384" ThreadID="4900" />
<Channel>Laserfiche-EmailArchive-Service/Admin</Channel>
<Computer>pgc-lfswork.pacificgroup.net</Computer>
<Security UserID="S-1-5-18" />
</System>
<EventData>
<Data Name="accountId">archiv~1</Data>
<Data Name="type">Laserfiche.RepositoryAccess.LaserficheRepositoryException</Data>
<Data Name="message">File input/output error. [9004]</Data>
<Data Name="stackTrace"> at Laserfiche.RepositoryAccess.LaserficheWriteStream.FinishWrite()
at Laserfiche.RepositoryAccess.LaserficheWriteStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at Laserfiche.PdfServices.PdfExtractor.ImportPDFStream(Stream pdfStream, DocumentInfo doc, DateTime dtLastModified)
at Laserfiche.EmailArchive.Workers.ImportDocumentWork.GeneratePdfPages(DocumentInfo doc, IImportDocumentWorkSettings settings, IImportDocumentSource source)
at Laserfiche.EmailArchive.Workers.ImportDocumentWork.ImportDoc(DocumentInfo doc, IImportDocumentWorkSettings settings, IImportDocumentSource source)
at Laserfiche.EmailArchive.EmailExecuteJob.AttachmentDistWrapper.CreateAttachments(LfAttachmentCollection attachments, ICreateDocumentWorkParameters emailCreateDocParameter, IImportDocumentWorkSettings importSettings, AssignMetadataWork assignMetaWork)</Data>
<Data Name="msgId">2d568cf2-a724-4db0-9c4c-f627e82b675e</Data>
</EventData>
</Event>

0 0

You are not allowed to follow up in this post.

Question

Question

import agent email archive - what happens if LF repo is offline?

Replies

Sign in to reply to this post.