Long-term archival with FCPX

Hi,


I’m looking for a long-term archival solution for my FCPX libraries. I’ve been storing my libraries on about 14TB of external hard drives (duplicated in the cloud). I produce about 1TB of raw media per month, though, and I can’t afford to keep buying hard drives.


Ideally, I’d be looking for some kind of lossy compression solution that would allow me to keep my libraries intact but with lower-bitrate media. Does anything like this exist?


Any advice would be very much appreciated.


Thanks,
Oliver

I could do a batch Handbrake compress of all my media files, delete the original media, and manually relink.

Would this reliably work?

HI Oliver
Welcome to the forums.
There are a few options.
Backing up to cloud services, slow and monthly costs.
Lots of folks like LTO tape archives.
Then there is M-Disc which claims 1,000 year life span. Uses special M-Disc Blu-ray Drives and the disk itself uses a rock recording layer instead of dye layer. Capacity up to 100GB/disc.
I see M-Disc is now also providing a archiving service. This is new.
I have used M-Disc for some of my archives through a ASUS BD burner. Haven't tested the 1,000 year claim yet, but working on it. Let you know :silly:

Hope this helps, Greg

I used to have the same problem when I was shooting 35mm transparencies. I had to buy two or three 140-slide Carousel trays a month to hold my precious photographs. And then I had to build more bookshelves on which to store them. When I grew up, I learned to cull my stuff with dispassionate objectivity.

You can buy 8T drives at Costco for $350. You can get 1T drives at Best Buy for $50. You can probably get a bulk discount if you buy a case of devices from an online seller someplace. Assuming you're in business, an excellent model is to include the cost of bulk storage on your invoice and hand your clients the drives containing their media.

You can compress your original media way down to something far cheaper to store and manage but lossy compression means you don't ever get it all back. How low can you go? If you can accept the quality hit and go quite low, maybe you're shooting too high. And inventorying multiple drives or stacks of optical disks is an intellectual nightmare.

I wish I had a simple answer.

One question is for how long time you should store your media? One year, three years, five years or forever. The other is for whom.

Standard contract for photographers in Sweden says that you have to store orginal footage for five years, unless other is stated. Forever is a long time. I still have some film footage from 1936 at home and glass plates from 1800ish.

My plan is to store everything done for others on harddisk for five years, with a copy in the cloud. After five years, erase harddisks and keep copy in cloud, unless somebody is paying me. For my own footage, store forever, one copy on disk and on in the cloud.

I currently use fast RAID0 local disks for editing, Network disks RAID1 until three years old, then migrated to harddisk archive.

Hourly backup to local NAS of local disks and network disks. cloud backup of network disks and harddisk archive.

Yes, I'm known at the local Apple store as the guy who buy the biggest drives.

My experience is that loss-less compressing (e.g. zip archives) doesn't work well with media files as they already are compressed, unless shooting in un-compressed raw.

Hi everyone, and sorry for the late response!

Most of the stuff I shoot doesn’t have much movement, and lends itself well to compression. Using reasonable h.264 settings in Handbrake, I can get the filesize down to about 1/8 of the original without losing very much visual quality. The quality loss is obvious, but it’s not too bad.

I work for a small news company in Canada. I’m under no obligation to store the footage after I deliver my projects, but I like to keep an archive so I can reuse footage if necessary.

Question is why make worse quality if you want to reuse footage?

How much of the footage is usable? is it 1%, 5%, 25% or 75%?
What kind of codec do you use for camera(s)?

If a very small portion is usable, I would store the clips used in the master.

If majority of clips usable, store everything.

Building a clip Library based on masterdata and store those with full quality, if you have the time.

I second Artmann's sentiments: Edit mercilessly, cull deeply, keep only the best, keep only the irreplaceable, keep only stuff that can make you money in the future.

Reduce the bulk of your storage by not storing what shouldn't be stored. You still need offsite backups.

Here's my long-term solution.

From your scratch drive, open the library icon of the project you want to archive (the purple 'four star' FCP library icon for the project... Control>Show Package Contents). Then go into the main project folder... usually titled by the date it was imported. Then, trash your 'Original Media' and 'Transcoded Media' folders.

Empty the trash.

Now your project is a shell of it's former self, size wise, and can be archived to an 'archive drive' (at least that's what I call them). Usually turns a 500GB library file into something down to 10-20GB's for me (if that).

If you ever need to bring that project back up for whatever reason, all you have to do is re-import the original files. I keep all of my compressed (AVCHD) master camera files on a separate 'Master File' drive. So it's easy to just re-import it all. Might have to manually make a few connections to other external media too, but this is a great and relatively quick way to archive large projects.

I've successfully resurrected a many of wedding projects this way with no problem.

To my knowledge of FCP/X, you are making it to complicated.

I use Final Cut Library Manager to clear all remporary files, after project is finished.

You can select to have a master catalogue of all clips, and then don't copy into Final Cut when importing, or import all clips to separate Library, and use references to those clips.

IMHO, never fiddle with clips in a Final Cut Library in the finder, unless something doesn't work in the NLE.