FCPX struggling with large library

Hi guys,

I'm another television docs Editor who previously cut a lot with FCP7 and am now trying to find my feet with FCPX with a really large library and would appreciate some advice. By biggest problem is that when I click on an event, FCPX has a tendency to want to load up lots and lots of projects that will either take ages to load or crash FCPX, often forcing me to trash preferences in order to get the event open. In Avid in projects this size on lower spec machines I'm used to being able to double click a bin and go straight into a list of projects without the computer crashing. It feels like FCPX is trying to do too much when all I want to do is open an event and select one project to open rather than FCPX trying to load all the projects and overloading itself.

I don't think there should be an issue with my specs as this is my setup:

- I'm running a Raid with 115-140MB/sec read/write speeds
- Working with Red Epic R3D files up to 5K but using FCPX Proxy workflow
- Running the latest maxed out spec Macbook Pro Retina

After I trash preferences I don't seem to have a problem but very quickly FCPX seems to bottleneck again by trying to open too many projects and events that don't even seem relevant to the event I've clicked on. Any advice would be greatly appreciated.

Thanks,

Pete

Hi Pete,

What version of FCP X are you using?

- Ronny

Hi Ronny,

Thanks very much for the reply. I'm on 10.1.4 so should be current.

Thanks,

Pete

Okay. How many events do you have inside this Library, and how many projects?

Is your media external or managed?

- Ronny

About 70events and countless projects! A lot. Maybe several hundred projects.

The original media is kept externally and I don't have that hooked up. I've got the proxy files inside the library file which is what I'm working with.

Thanks

Pete

Are all these events and projects related to the same doc you are cutting?

- Ronny

Yes, unfortunately. Otherwise I'd get what I didn't need out of the library but it is all going to be accessed. It's three years of natural history filming!

Thanks

Pete

FCP X indeed loads everything related to the Library, and with such an amount of clips and projects you would need loads of RAM to cache everything.

When you delete your prefs you close your Library, which purges your RAM. That's why it works for awhile after deleting the prefs. But as soon as you start editing you fill up your RAM with Undo actions and project caches again.

I'm afraid for a project this size 16 GB RAM is not enough. I have a 22 TB Library with a similar amount of events and some 70 projects running on a new iMac with 32 GB Ram and it gets filled up quite easily. But I have no crashes or slowdowns. I think you are just working on the edge having such a large production on an MBP.

If you could find a way to spilt up your production into two or more Libraries I think this would solve your issue.

- Ronny

Thanks Ronny. That's much appreciated and great info. I hadn't anticipated that. Looks like I should have gone for the Mac Pro after all. I guess I'll hobble through for the moment and see if I can make it. It would be really interesting to see how the latest version of Avid or Premiere hold up with projects this size on the same spec. I just don't know if I could go through the pain of transcoding that many proxies again. Thanks so much for your help.

Pete

Avid and Premiere would probably be able to load this project because they don't use the same RAM caching. OTOH you would not be able to comfortably edit such a production at all on either of them using an MBP.

- Ronny

Thanks Ronny. I'll battle on in that case and see if I can start whittling the project down in size somehow.

Many thanks again

Pete

obardain wrote: ...
- I'm running a Raid with 115-140MB/sec read/write speeds...

If your RAID is really that speed, that's not very fast. That's no faster than some USB 3 bus-powered drives.

Maybe the bottleneck is RAM (as already postulated). But if it's I/O at least that can be helped with faster hardware. Are those performance numbers really correct? Can you verify that with BlackMagic or QuickBench? There is no shortage of fast Thunderbolt RAID systems, often not that expensive.

You say it's especially slow when clicking on an event. During that period is it CPU-bound or I/O bound? You can evaluate this using the built-in Activity Monitor or using iStat Menus: bjango.com/mac/istatmenus/

You could also use Final Cut Library Manager to clean up any extraneous space: www.arcticwhiteness.com/finalcutlibrarymanager/

Indeed. Every NLE can get overloaded. I helped out a production company that was cutting their first feature in FCPX (moving from 7). A similarly sized project. R3D with proxy workflow, and they were having all sorts of performance problems. After about an hour of talking through their workflow, and then some cleanup and reorganization, FCPX was singing again. I've worked with many large projects and have a few suggestions that can help.

But to help you better, can you tell me a bit more about the organization of your project:

What is the TRT?
Are you editing it all on one project timeline or did you break it up into "reels" to so speak?
What are in all the project timelines? Revisions? Alternative storylines?
Are all your timelines in one event?
Are you editing in compound clip timelines or project timelines?
How extensive is your use of Multicam or Compound clips?

- Raid is indeed on the slow side. FCPX doesn't just need a fast HD for media access, but also fast small random reads and writes for constant writing to the various databases (hence there is no "save" because FCPX saves with every keystone or edit operation). If you switched to a Thunderbolt raid (400+ MBS) you would see a significant performance jump.

Having done quite a few features with r3D's myself. You definitely need more RAM 24GB+ and a faster media drive to use FCPX in a reasonable matter with the amount of media you are dealing with.

Hi,

Thanks very much for the reply. Yes those performance numbers are correct. Verified with black magic. I'll have a go at upgrading to a faster RAID and see if that has any effect. I'm transferring the project over night though I could do with upgrading to a fast thunderbolt RAID.

I'll double check whether it's CPU bound or I/O bound tomorrow once this transfer completes.

Many thanks!!!

Pete

Brill! Thanks. The TRT is 3 x 60 mins.

I've got hundred of select sequences (as projects) spread across several events. I've just started editing and am editing each sequence (approx 3-4 mins each) in their own project. Once I've assembled each sequence then I'd normally start working in a 60mins (approx) timeline. Currently all my cuts are in one event, but as I say I have lots of selects across several events. I'm editing in project timelines.

Many thanks!

Pete

Thanks. I'll definitely have to struggle through for the moment having just parted with a wedge of money to Apple for my new Macbook Pro. Unfortunately this project is self-funded so there isn't the budget to upgrade to something meatier at this time.

Thanks,

Pete

Pete,

Shoot me an email at my FCP.co username and then my company which is at onecreative and then dot net. Perhaps we can screen share.

- T.

Hi guys,

I'm just trying out a slightly faster RAID at the moment so will let you know how I get on.

Thanks

Pete

I would divide up your project timelines to multiple events, perhaps reels and see if there is any difference.