Library with too many events and too much media

This is a real head twister!

What I want to do in a nut shell is split a massive library into smaller libraries so that I can back them up individually and regularly because the library as it is at the moment is over 4TB and contains thousands of clips, which dramatically slows things down even with my ThunderboltB2 8TB GRaid Studio and 32GB Ram in my Mac Pro 6 Core.

I have a library containing 18 events (see attached grab). This is a managed library. What I would like to do is split it into multiple managed libraries relating to the specific companies but where would I keep the Projects so that they wouldn't have to copy any used clips into their own library? Surely there must be a way of having projects simply reference used media as apposed to copying it into its own library?

The problem is that the films that I am making reference media from most, if not all, of the events.

Ideally I would have 4-5 libraries and the fifth one could be exclusively for projects but I'm guessing that it would be impossible to do it that way without duplicating media, because as I say, the projects will reference most if not all of the Libraries and won't they just copy any media from used shots into their own library effectively duplicating it.

Any help, much appreciated!

You can now have a managed library that stores media outside the library. Select your library, open the inspector, and point your media/proxy directory to an external folder on your hard drive. At our company, every project is a new library which points to its own folder (where all of the project's assets including media are kept, and everything is organized into subfolders that make sense).

Then, you can select file->consolidate library files, and move ALL your media out of your library into your external folder that you selected in the previous step. You can also alternately select each event and do it by event a few at a time if you want.

When all of your media has been moved outside your library, you can make new libraries and copy over your events at will, and they will just reference the media in that external folder. The big advantage of this is that all your libraries will be very small, and they will all point to a centralized location for your media.

Thanks, I think I understand.

But if I want to go through an event in the browser and delete a bunch of old or duplicated clips, will they be deleted from the external media folder and if not then how would you go about doing that?

Also, can you just confirm the way to make sure that the Projects library doesn't duplicate media when I use clips from the other Libraries in this production. Is it by making the destination drive the same as the one for all the other libraries ie the same external folder?

Thanks,

FCPX only delete what it manages inside of its own Libraries. If your media is External Media, then it only delete the Alias file inside the Library, it won't touch the media file in the external folder on your hard drive.

When you use media from Library A for a timeline in Library B, it must copy the media file to Library B. Libraries do NOT talk to each other at all.
- If the media file is Managed, a duplicate is made of the original, physical media file, inside of Library B. You end up with two copies on the hard drive.
- If the media file is External, the Alias file (point back to the original) is duplicated, not the full, original, physical media file. You end up with one copy on the hard drive, and two Aliases.

Thus in your case, you want to keep everything as External Media, and only have one physical media file for each clip, and only populate your Libraries with Alias files (only a few k, very tiny).

What Ben said is correct, and after you have consolidated all your media outside your library, they will all be external, and your library will just contain aliases that point to the media. Thereafter, if you copy events or anything else between libraries, it will just copy the alias and not the media itself. If you want to actually delete the media, I would SHIFT-CMD-R to open it in the finder, delete it in FCPX, and then delete it in the finder. It's an extra step, but you have to do it.

As for your question about pointing the libraries to the same external folder, yes you can do that, but you said your goal was to make archiving easier. If you do that, FCPX will dump all your media into that one folder, and then you'll have to manually root around in there to archive what you need.

If you want to say, have one library per client, I would make a separate folder structure for each client, and point each library to their respective folders. Personally, I make a library per PROJECT (and by project, I don't mean FCPX projects, which are timelines, I mean a client's project), and each project (library) gets its own folder structure where all the assets live. This is because we will decide which projects get archived on a per-project basis. The assistant will just archive this one folder to LTO tape when it's done. It just depends on how you want to organize your projects.

Random Thoughts:

A student in my class a couple weeks ago discovered that if you move (consolidate) original files from Managed to External via Project Settings, FCPX does in fact COPY (not MOVE) the original media to the external folder, AND KEEPS the original copy inside of the Library. It ADDS Aliases to the Library.

I'm going to experiment with this some and figure out what it going on.

You can also create a folder manually on a hard drive, manually make sub-folders in it, set it up as you wish. Then physically place your media, as you gather it, into those folders.
- When "importing", get the media from those folders, and tell FCPX to "Keep In Place".
- When setting up a Project Timeline, point the Media folder to that one you manually created, if you optimize media.

But yeah, what's being said here is right on.

Ben, I've tried to replicate this by changing a Library from managed to an external folder and then consolidate a project to that external folder. In my case, the media gets moved, NOT copied. I don't see any way to control this via the project settings.

Interesting, I've used Larry Jordan's method for moving the media from a massive library containing 18 events by selecting the Library and consolidating the media to an external folder. Only took about 30 seconds to do. However, prior to that I tried moving an event to a library with an external media folder on the same drive and it did indeed start copying the media over as apposed to moving it. The way to avoid this is to make the library that you're moving your event to a managed library and by that I mean, keep the media 'In library' and once you're moved the event, change the media folder to the external media folder and then use consolidate to move the media. That method seems to work every time and avoids FCP copying the media instead of moving it.

As far as archiving goes, it's pretty straight forward and I'm just backing up the folders within the 'Original media' folder to external Passport drives and since they are all labelled by date it's easy to see which ones I have copied.

Emailed my student, got clarification. She was not clear in her original explanations.

Move a Project's media setting to an external folder, it gets moved. Change that setting back to "Library, it gets copied.
And, this is as it should be, FCPX won't destroy media outside of the physical Library itself. When deleting or copying; External media is never moved to the Trash, Managed Media (full files and/or Aliases) is moved to the Trash.

As stated, I'd go with making all your stuff External Media from the start.