How to manage huge events? (4TB of HD material --really sloooow )

Hi, I'm working on the making of for a feature film and we have 7292 HD clip, about 3TB.
To be able to organize all the material and easily go through it, I thought the best way was to keep everything under one single event and label each clip.

It works but sometimes the editing process gets really slow. Should I split it into different events. Have you ever worked with so many clips, any suggestion to make everything smoother?

Here's my configuration:
Working on a MacBook Pro 2.3 Ghz Intel Core i5 with 8 GB RAM
G-DRIVE 4TB connected through Firewire 800

My goodness, that's a lot of drive access going on with that much footage. I'm still pretty green myself with FCPX, but I would sure think you would need a faster drive than Firewire 800 for that much footage. Is your G-Drive setup as a RAID 0 or 5 or what? Im no expert on RAIDS either, (have never used one myself, not yet anyway) but I think a Thunderbolt RAID would be your best bet with a project that large. And, if you don't have Thunderbolt on that MacBook then possibly a faster RAID itself would help. More RAM certainly wouldn't hurt either.

The faster drive is not related to how much footage, but rather any single piece of HD footage. Definitely an esata drive or SSD would improve that workflow.
The bigger issue is probably trying to manage all those clips in one single event. I would make multiple events for that project and try to organize similar clips that way. Others on this forum have more knowledge about organizing your clips than I do, but I think you'll definitely want them NOT to be all in one event.

Another useful way to manage a large number of clips is to group them into compound clips. FCPX can look at 100 clips, for example, when grouped into compound clips of 10 clips each as just 10 clips.

While I haven't used FCPX in the context of a feature film yet, on shorts we've grouped clips into Events for each Scene with a lot of success. Do not just drop everything into one event. With this many clips, it will kill your resources no matter the setup probably.

With Event Manager X, you can limit which Events FCPX opens on startup, giving you a boost in performance overall.

Create an Event for Each Scene. Import the Clips for the Scene into the Event and use Keyword Collections to designate camera angles.

Edit your Scenes using Compound Clips, which are associated with Events. You can then nest those Compound Clips into an overall Project when necessary, but in the meanwhile it keeps the workflow modular and accessible to multiple editors.

Hope that helps. I'm really interested to hear how others are working collaboratively with FCPX.

I had a project with 7500+ clips in summer (on version 1.0.0!!). The first you need to do is turn off the waveforms in the event! And I also keep them turned off in the timeline until I need them. It is better than Legacy FCP, but waveforms still take time to render when coming in masses.

I also have the feeling when I have many projects (timelines) it helps to store them in folders when I don´t need access to them. I usually have a lot of versions of different projects and when they are in folders they don´t all load all the time.

you can also create Disk Images that you only mount when needed. Either one dmg per Event or a small group of related Events - a no-cost alternative to Event Manager.

The smaller the Event/Project libraries are, the more fluid I get FCP X to work (aside from fast drives on fast ports).

Thank you guys,

my G-DRIVE is this one www.g-technology.com/products/g-raid.cfm 4TB and it is connected to my macbook through Firewire 800.

A QUESTION ABOUT COMPOUND CLIPS:
I tried to use the compound clips in the event. I thought it would "replace" the clips as it does in the timeline, but now I see the compound clip AND the clips I used to generate it, is that normal? Because, even if it's a great feature, that increases the amount of clips in the event instead of reducing them.

What I'm doing now is moving grops of clips in different events. This process is EXTREMELY slow, spin wheel for 10-15 minutes, I wait with my fingers crossed hoping that I don't loose any clip.

Thank you again for all your advices..

Yes, when creating a Compound Clip from existing clips in the Browser, it creates a new Compound Clip, which contains the clips, but does not replace anything, this is normal.

I don't make multiple Events for several reasons. I separate my Scene clips out by metadata, which is the whole reason for FCP X running on top of the old Final Cut Server DB engine. I assign the Scene number as keyword. Then go through that Keyword Collection and assign other keywords to sub-group them. Then create a Folder labeled with the Scene number to group them in. That way I don't have to jump between multiple Events, keeps the Library much more clean.

An alternative is to use the Scene metadata field in the Inspector, and group them with a Smart Collection based on that.

I make Compound Clips for each Scene to edit them in more easily, and create a Smart Collection based on Clip Type to be Compound to keep them together.

I'd never separate out clips for a production over multiple Events, much too much clutter. I use Smart Collections, Keyword Collections, and Folders inside one single Event to much more easily and much more quickly organize them.

Thank you, right now FPX it's not responding, 30min went by while moving clips to a new event. I'm patiently waiting hoping I don't loose any data. A good sign is that once in a while the hard drive writes, that means it's actually doing something..

I tried to use the compound clips in the event. I thought it would "replace" the clips as it does in the timeline, but now I see the compound clip AND the clips I used to generate it, is that normal?

The compound clip should be used in your timeline to "group" your clips for a scene together. Then you can work with a single scene at a time by opening and editing the compound clip on the timeline.

I did a feature length project in the late summer fall, and this worked well.

Also I would recommend that you keep your timelines in a true Project rather that edit in a compound clip inside an event.

What I'm doing now is moving grops of clips in different events. This process is EXTREMELY slow, spin wheel for 10-15 minutes, I wait with my fingers crossed hoping that I don't loose any clip.

There is a difference between moving clips and copying clips between events. When you simply drag a clip to another event it should "move" that clip to another event. But if you hold down the option key you'll see the arrow with a green "plus" indicating that it will keep the clip on the first event and copy it to the second. I'm sure you are moving clips and not copying them, but just be aware that there is a difference.

So my last 2 cents.

Hard drive speed is huge in FCP X. Mainly because of the "current version" databases that hold the events and projects. Every keystroke or change you make on a timeline will be stored to the "current version" project. So the slower the drive, the most strain on performance.

I have a Raid that is FW800 and also ESATA compatible and there is no comparison, FCP X is much much happier with an eSATA connection.

So if you have any way of getting an eSATA PC Card or Thunderbolt to eSATA adapter, I think you will find that FCP X performs much better.

Editing a Compound Clip inside of a Project timeline defeats the whole purpose of making Compound Clips. I always just edit my Compound Clips in their own timeline. I've never had problems with it. If you edit Scenes in a Project, then skip making Compound Clips. You're going to do the Select All > Copy > Switch Timeilnes > Paste dance to export anyway.

Thanks, I also have several backup projects, after loosing a whole project I started doing hourly backup of my timelines, that multiplies the amount of data. But I think that what slows down more is the amount of clips in just one event..


// by the way, after one hour FCPX it's still not responding, drive reads once in a while..

Ben - I misspoke. What I mean was creating compound clips for each scene and then opening them in their own timeline.

It sounded as if he was making compound clips in the event while I was suggested he make compound clips in the project.

Are you on 10.0.3? In 10.0.3 FCP X automatically creates backups of projects every 15 minutes, in additional to constantly saving.

If this were my project and it has been siting for an hour doing nothing, I would just force quit it and perhaps move items in smaller chunks.

Thank you. I'm on 10.0.3 I didn't know about this new feature! It was necessary, so now I can gradually delete the old backups that I don't need and have less projects.

"It sounded as if he was making compound clips in the event while I was suggested he make compound clips in the project."

I'd make the Compounds in the Event, and edit in their own timeline. Assemble in a Project when done. Then you have smaller timelines to deal with. If they're all created in a Project, the whole project database has to be processed. CC's in an Event only process that CC's timeline, you have quicker access to each, and can duplicate the CC's to do alternative edits faster/easier. I've never had problems with it.

BenB wrote: "It sounded as if he was making compound clips in the event while I was suggested he make compound clips in the project."

I'd make the Compounds in the Event, and edit in their own timeline. Assemble in a Project when done.

Completely agree with this workflow. It's the way I do things too. Also make a smart keyword collection that selects clip types that are compounds. This way you won't have to go hunting through your master collection every time you create a new compound clip and you can just work from this collection. (Bin)

I'm gonna try the one event, keyword collections for each scene in the next project I'm working on and see how that goes.

Im still a bit confused about how you handle this workflow when sharing the Event between multiple editors? The reason I was organizing each Scene to an Event before was because it was extremely easy to duplicate an Event to a disk image, send it out to the editor who was working on it and then when I got it back I could easy drop it back into the workflow. So when organizing scenes by keyword, how do you organize the production effectively? Is there something about Smart Collections that I am missing? How do you merge the changes between multiple instances of the Event?

BenB wrote: I'd make the Compounds in the Event, and edit in their own timeline. Assemble in a Project when done. Then you have smaller timelines to deal with.

T. Payton reported a drastic processing slowdown when doing this back in August. His guess for the cause was constant updates to a very large event database. Has that changed in current versions of FCPX? And T. Payton, did I characterize your complaint correctly?

I've never had problems with it.