Our current permanent records repository is approaching 5 TB and will continue to grow.
We are growing about 1 TB annually for the last 3 years.
What is considered best practice when it comes Repository size?
Just chiming in on this. Does anyone have any views on where the 'sweet spot' is in terms of repository size. I.e. at what point does the repository size start to impact performance.
In our own testing, SQL seems to scale quite well in terms of performing a template search, the bottleneck seems to be LFFTS. At around 10TB the system seems to slow down regardless of how much resource you throw at LFFTS. E.g. in one scenario we have a dedicated LFFTS server with 128GB ram and 16 cores etc. but LFFTS doesn't seem to be able to use anywhere near that much in terms of resource, it caps out at the 30GB RAM mark and the CPU cores don't even break a sweat.
This means that even specific text searches take up to 2 minutes to return a result even with no columns docked. As I've said, SQL performs well.
As Laserfiche themselves offer zero information around performance benchmarks I'm just curious what others have found on this?
It really depends on your infrastructure and resources.
Or largest repository is over 100TB, so more than anything it just comes down to how you implement and manage your system, and what kind of resources you throw at Laserfiche.
General recommendations:
Of course, I'd still recommend looking at ways to keep file size (i.e., growth) to a minimum, like using monochrome instead of color whenever possible or only generating pages when needed, because you want to be efficient about the growth, but Laserfiche can handle a pretty large repository.
The number of entries/documents in the repository is a much more influential factor because that affects database performance, and even then, a well-maintained database with sufficient resources can handle a lot (we have over 50 million documents in the 100TB repository).
Basically, database performance is usually the biggest bottleneck for a repository.
Hi Jason,
Our Logical volume is set for 20GB.
Default Volume with Default0000, Default00001
Are you able to have a New Volume name that is connected to the same repository?
I'm not sure what you mean exactly, but you can have as many logical volumes as you want for a single repository.
The important thing is just to make sure you have a legitimate reason for creating separate logical volumes because it does add complexity to administration and maintenance.
Jason, 4TB on hard drive size...are you just creating additional drives in your Windows Laserfiche Server VM? Isn't that eventually unsustainable? We wondered about doing a network share from the SAN so they aren't drives but then wondered about backup/restore, etc.
We also want to take older drives and roll them off to slower storage, which should be transparent to Laserfiche.
@████████ No, we do not have them on the main Laserfiche application server, we use network shares and a high-speed SAN.
Although having multiple smaller drives does require more maintenance as the repository grows, having them on one massive drive is far more unsustainable because moving the files, backups, etc., all become problematic.
We've always used network shares and originally, we had some larger ones in the mix, but the size was causing problems, like hindering backup processes.
Hi Ivan,
I'm not sure if there is a limit, but ours is currently one drive with 9 TB of data and 2 TB on another Drive. Searches aren't the fastest, but certainly not terrible. Alot of this is more our SQL server which would benifit from more RAM.
Something unexpected we had occur, was how the disk was formatted.
The 9 TB drive used to be a 10TB NTFS iSCSI drive, but when it was converted to ReFS perfomance dropped so it got shrunk to 9TB and the expected drive preformance returned. The reason for the change was due to a change in backup software.
Stewart