Question

Increase number of processor cores utilised by DCC

Distributed Computing Cluster

Updated May 26, 2023

asked on May 23, 2023

Hi,

We have a client wishing to OCR a large volume of PDFs and are using DCC to do so. We have several workers configured, including a machine with 36 cores. However, the CPU usage rarely exceeds 30% since increasing the number of CPUs (from 16). We have the setting "Limit concurrent tasks based on the number of processor cores" set and have now tried manually "limiting" it to 32 cores (by specifying a custom limit) to try and override any internal limiter.

We have already increased the max number of tasks in the scheduler config file (to 20K) but wondered if there was another way to increase CPU utilisation?

Thanks,

Nigel.

0 0

Replies

replied on May 23, 2023

How are you sending documents in? OCRing one doc would only use a CPU, so you would need to have 32 tasks coming in at roughly the same time to see the CPU usage spike up.

1 0

replied on May 24, 2023

HI Miruna,

The documents are being sent by workflow in batches of 3000 according to a schedule (based on the average time it takes to process 3000, plus some breathing space). If I look at the number of assigned tasks to each machine it sits at between 90 and 100 until the batch completes but on the machine with 36 processors it never maxes the CPU like it does on the others so it seems like it can't utilise all the cores after a certain point.

Thanks,

Nigel.

0 0

replied on May 25, 2023

While the worker is processing, how many lfomniocr processes can you see from the task manager? Normally every ocr task would create a new lfomniorc19.exe process.

Please also monitor if there is other machine resources(memory/disk/network) that are becoming the bottleneck so that cpu is not fully taken advantage of.

0 0

replied on May 25, 2023 • Show version history

I run a test on my environment, I set the custom limit to 32 on a 4 cores cpu machine, there will be 32 lfomniorc19.exe processes running and cpu would max out after a while of preparation (some preparation work like retrieving documents to local)

0 0

replied on May 26, 2023

There are over 30 or so OCR processes running but the total CPU utilisation never really goes above 30%, most of the OCR processes sit at 0% CPU usage. There are no other processes consuming large amount of memory/CPU - neither exceed 50% of the total. The LfDccModuleHost process consume most of the CPU and memory (15% CPU, 2.5GB memory). The machine has 16GB memory so that isn't an issue.

0 0

replied on May 26, 2023

Hi Nigel, can you open a support case so we can investigate this further?

0 0

You are not allowed to follow up in this post.

Question

Question

Increase number of processor cores utilised by DCC

Replies

Sign in to reply to this post.