How do you recommend backing up Final Cut Projects with External Data Storage

Hi Everyone, 

Reaching out for advice on the best strategy for a) backing up
b) moving footage around without creating a relinking nightmare. 

Background problem 
- too much footage to fit on one SSD

I am working on a documentary which captures miracles a mission trip to 2 locations in Kenya. 

We have some 3TB of original data which we shot at two locations. At the first location we  got 2TB of footage and the second we had around 1.3 TB. When I tried consolidating all my footage on a large drive, I ended up with the spinning beach ball problem that people here kindly helped me with so I ended up editing the film in 2 parts - with one SSD for the first location, and the second for the second location plus all the FCPX Libraries / projects for both locations. 

As FCPX started generating optimised and render files, my second SSD filled up. 
Creating proxies filled up my second drive - so I am working with original media on the SSD (stored outside the libraries). 

FCPX Libraries 
I am working in 2 libraries - one for each location. 
The footage was not moved into the library - imported but left in place. 
I imported footage into the respective library only. 

I ended up using a slower HDD spinning drive as an overflow location when I generate a ProRes version of the first half of the film, which I bring into the edit of part II. 


Challenge
It is getting tedious to have to render first 25 mins of film and import to second film each time I do an update. 
I would prefer to have all in one library, but I am a little concerned after past spinning beachball problems that having too much footage/too many keywords etc. in one library could get messy, slow or difficult. 

Questions

1) about consolidate projects in one library

Is it wise to consolidate the projects in one library – leaving Data in original folder structures. 

Or is my concern that it might be too much justified?

Will I be fine if I move the project timeline from Location 1 (first half of the film) into the Library/project for Location 2. 

Are you guys working on long films with many TBs in one library with no problem? 

If I put all the projects timelines into one library, will I end up with problems relinking because the original footage is spread across 2 SSDs and one HDD? Or will I be okay provided I have no proxies to relink and all my files have unique names. 

2) when it comes to backups 

How should I back up all this material to avoid a relinking nightmare if I have to restore from that backup. 

1) Should I be using something like Carbon Copy Cloner to backup volumes to own partitions so that the volume names don't change 

2) If I consolidate all the footage into a library on a large HDD, I would have a backup and hopefully have no problems relinking but then I could not restore to a 2TB SSD as it would probably be too big. 

Any thoughts would be helpful. 

Thanks in advance, David 

David, it seems that most of your performance issues are likely due to having insufficiently fast enough throughput between your storage devices and your computer. Leaving files in place when importing is good, as it gives you more control over the location of the various types of media (e.g., original versus optimized, render, proxy, etc.).

For backup, you should be fine with a large HDD (preferably more than one copy, and at least one copy offsite). Trying to use an HDD for editing anything but the simplest projects is not going to provide enough data throughput.

If you are going to use optimized media in FCP, or proxies, you could leave your original media on a HDD and designate storage on an SSD or RAID (HDD or SSD) for FCP to use for render, optimized, proxy, and other temp media. You might consider that any "generated media" can be recreated and isn't something you'd want to keep for longer-term backups (or maybe you do).

What may be most important, if this is an ongoing or longer-term project, is to plan for your storage needs. If this is more than a hobby project, you might benefit from some detailed expert advice...

Cheers.

revival trainer wrote:

Hi Everyone,

Reaching out for advice on the best strategy for a) backing up
b) moving footage around without creating a relinking nightmare.

Background problem
- too much footage to fit on one SSD

I am working on a documentary which captures miracles a mission trip to 2 locations in Kenya.

As FCPX started generating optimised and render files, my second SSD filled up.
Creating proxies filled up my second drive - so I am working with original media on the SSD (stored outside the libraries).

---

        It seems to me that the issue is as much a problem of space as speed. I would be simply using Proxies and avoiding optimized files as they get so huge. If you wiped out the Generated files including the optimized ones  you would most likely find there was much more room on your drive. Your SSD would then be (hopefully) plenty big.
Personally I always bring my files into the Library. While many people prefer to keep the files outside the Library it has always been my experience that sooner or later I have broken links.  Using the Library does mean duplicating the files and I don't delete the files in their original folder structure which does require HD space but for that I use big drives. I have 24TB+ of storage and I'm just a guy at home making fun movies that nobody sees . www.youtube.com/@digibud/videos
      You DO need the Library (and files if not in the library) on a fast ssd, period.

---

FCPX Libraries
I am working in 2 libraries - one for each location.
The footage was not moved into the library - imported but left in place.
I imported footage into the respective library only.

I ended up using a slower HDD spinning drive as an overflow location when I generate a ProRes version of the first half of the film, which I bring into the edit of part II.

---

That ProRes version is the cause of the huge storage space problems.  For me, simply using Proxies is the solution if storage is an issue, which it often is with expensive SSDs.

_________________-

Challenge
It is getting tedious to have to render first 25 mins of film and import to second film each time I do an update.
I would prefer to have all in one library, but I am a little concerned after past spinning beachball problems that having too much footage/too many keywords etc. in one library could get messy, slow or difficult.

Questions

1) about consolidate projects in one library

Is it wise to consolidate the projects in one library – leaving Data in original folder structures.

Or is my concern that it might be too much justified?

Will I be fine if I move the project timeline from Location 1 (first half of the film) into the Library/project for Location 2.


Consolodating projects into a Library isn't a problem.  Even leaving the Data in a Finder folder isn't a problem other than losing links (which is, to me, a big issue). If you have enough fast storage it's all good but having enough storage on SSD for creating ProRes is very expensive and sometimes impossible when projects get huge so again, toss the ProRes and just use Proxies.
You can move the projects but of course you want to do that from within FCP...not by moving any files in the Finder.
__________________________-
Are you guys working on long films with many TBs in one library with no problem?

I rarely do but have done so in the past with no problems at all. I've never had a Library over 3TB as I use .... Proxies .
_____________________

If I put all the projects timelines into one library, will I end up with problems relinking because the original footage is spread across 2 SSDs and one HDD? Or will I be okay provided I have no proxies to relink and all my files have unique names.

Files do not need unique names. The world would end of that were the case. FCP will rename the link file to take that into account. Multiple cameras will regularly cause many files to have the exact same name. FCP will deal with that.  You can safely move projects from one library to another without concern of breaking links if done within FCP.

–––––––––––––––––––––

2) when it comes to backups

How should I back up all this material to avoid a relinking nightmare if I have to restore from that backup.
IMHO the only reason to NOT put the files into the Library is because you will want to use the footage in a different project  and you don't want to take up the space by copying the same files into a second library. In virtually all of my projects the files are unique to that project and not something that I will use in another project so for me it's a no brainer to bring my files into a Library and never have to worry about the nightmare of lost links.

___________________________

1) Should I be using something like Carbon Copy Cloner to backup volumes to own partitions so that the volume names don't change

I'm not sure that simply having the same name on a drive will prevent a loss of linking. The OS doesn't use your cute HD name...but I could be wrong on that count.

2) If I consolidate all the footage into a library on a large HDD, I would have a backup and hopefully have no problems relinking but then I could not restore to a 2TB SSD as it would probably be too big.
If you dump the ProRes and just use Proxies and import the files within the Library I'd bet you would be golden with no space issues and no dropped frames. Others will disagree and you'd have to spend some time to find out if I'm lying but it's a reasonable guess .

Any thoughts would be helpful.

Thanks in advance, David