Maximum Library size in fcpx 10.3

Hi folks,
Does anyone know what the "practical" maximum library size is in 10.3 ? We are planning to set up a "catalog" server where we would have libraries with possibly 500+ events, each with up to 100 clips (all the actual media would be very lowres proxy) ... the idea is to catalog a whole pile of camera card images (each one as an event), which we could search by keyword ... just wondering what sort of problems we might be heading into, and if they could be resolved by better hardware / SSDs etc ... thanks in advance for any advice ...

I would take into consideration the solutions of LUMAFORGE, lumaforge.com/

contact owner Sam Mestman, he's not only very profound on topics 'network storage/collaboration in FCPX' but a very friendly and helpful guy … I met him in person at the FCPX@tour in Amsterdam, unfortunately his products are way beyond my needs.


… … oops, told you already at ASC

anyhow, Welcome, Thom to fcp.,co!

Thanks Karsten, have arranged to have a chat with Sam, as you suggested ... looks like a Jellyfish could do exactly what we want ! ... cheers, Thom

thom.cookes wrote: Thanks Karsten, have arranged to have a chat with Sam, as you suggested ... looks like a Jellyfish could do exactly what we want ! ... cheers, Thom

… 4999 arranged deals left, and I get one for free, yippee! :silly:

… kiddin'! I don't participate in any means, just saw some demoes of it and was … impressed.

thom.cookes wrote: Hi folks,
Does anyone know what the "practical" maximum library size is in 10.3 ? We are planning to set up a "catalog" server where we would have libraries with possibly 500+ events, each with up to 100 clips (all the actual media would be very lowres proxy) ... the idea is to catalog a whole pile of camera card images (each one as an event), which we could search by keyword ... just wondering what sort of problems we might be heading into, and if they could be resolved by better hardware / SSDs etc ... thanks in advance for any advice ...

On 10.2.3 I have 7,100 clips of mostly H264 1080p and 4k media comprising about 160 hr of content, mostly in a single event of one library totaling about 5 terabytes. Most of the content is unmanaged, imported with "leave files in place", although a subset is proxy.

Search/lookup performance is a little sluggish but mostly OK. Hardware: 2015 top-spec iMac 27, 16TB OWC Thunderay 4, RAID-5 (SoftRAID 5.5.5), plus several other backup Thunderbolt arrays.

In my case it's a documentary project and will all be keyworded anyway, so I didn't see the advantage of spreading it across events. I don't know the performance pros/cons (if any) of doing that. As of 10.2, FCPX has library-wide searching so putting content into multiple events does not hinder searching that.

The biggest performance issue involves I/O for thumbnail generation if using the Event Browser in filmstrip mode. The thumbnails are rendered "just in time" as you scroll down. Once they are rendered they are persistent and performance is good until you make certain changes (difficult to define what those are), where they must be re-rendered. Overall it is usable on this hardware.

There is a related secondary issue if using bare .MTS files which have been extracted from the AVCHD file bundle. In that case it apparently greatly increases I/O when scrolling in the Event Browser. My solution was delete those and transcode to ProRes/MOV using EditReady, then re-import those.

For more details, see my post: www.fcp.co/forum/4-final-cut-pro-x-fcpx/...e-to-mts-files#80859

I only briefly tested 10.3 on this library. It seemed to work but the thumbnail I/O issue seemed worse. I don't know if that would have been persistent or would have eventually "settled out". I filed a report with Apple, then rolled back to 10.2.3.

As Karsten said, I/O performance is absolutely critical in cases like this. The main problem is not large sequential I/Os but small random I/Os done when scrolling in the Event Browser. So simple I/O tests like Black Magic are not revealing. I have not personally tested a Lumaforge device but they have studied the FCPX I/O behavior extensively and optimized their storage solutions for this.

The problem with SSD is it's often too expensive for the needed size -- at the upper end. It must also be backed up like any hard drive. However if it fits your storage space and price constraints, it solves potential issues with small random I/Os. OWC has a 4TB model for $1,700, and I think you could order one with 8TB (4 x 2TB SSDs) but that would probably be around $3,000: eshop.macsales.com/shop/Thunderbolt/Exte...WC/ThunderBay-4-mini

I have library ( also documentary 150h+, mostly 1080pProResHQ and 1080PAVCHD) with nearly 10000 files.
What I experienced in 10.2.3 was that separating the files in different events would increase reaction times a lot eg. the slight pause between hitting the space bar and the videos beginning to play.
From a non-scientific perspective the slugginess increased with the number of clips in the active browser.

In 10.3. it seems to be much better though I changed some things in my system which could also have led to this behaviour.

You could make a test library with 50k files by simply duplicationg some existing ones. (not the source media but the clips in FCPX) and try it ?!