So what is the best way to import footage in Final Cut Pro X? Chris Roberts shows us his battle tested methods for ingesting footage and managing media from camera cards. Some great FCPX tips not to be missed!

 

We’re all dealing with vast amounts of media these days. Even the simplest of editing jobs seems to generate a huge amount of footage which has to be managed, organised and backed up. Not just inside the Final Cut Pro Library using events, keywords, ratings and smart collections, but also the actual media files on the physical hard drives we’re working from.

Thankfully, in the large editing facilities I’ve worked in, this is often (though not always) taken on by the wonderfully talented support staff; though, even in managed environments such as these, the editor also has a responsibility for any additional media that gets added such as graphics, music, etc., so it doesn’t get lost and remains secure.

However, much of my work also takes place in self-contained, on-location setups, working on events for clients where I’m fully responsible for the media that passes through my system. Over the last few years I’ve edited highlight videos such as the one below:

 

This video was shot over several days at a huge trade event and you can only image the amount of media I was dealing with…

(In case you’re interested, it was around 400GB, from 5 different camera crews across four different days. Phew!)

This media not only needed to be ingested, checked and verified, organised, edited, sound mixed, colour corrected and shared to online platforms (deep breath…), but this valuable material we’d spent days - and a not inconsiderable amount of money - acquiring also needed to be backed up so it couldn’t be lost. Oh, and the client wanted to take another copy for their later use.

I’m sure this sounds familiar to many of you reading this. In the past I’ve found that any backup strategy tends to be left to the last. This usually means you then have to spend an inordinate amount of time watching a blue line creep across the screen as the files are painfully copied from one drive to another, usually adding hours to the end of the job before we can finally call it a wrap and head to the pub.

So, I’d like to share with you my media management and backup strategy. This has been something I’ve developed from these sorts of jobs. I’ve found it to be reliable and, once up and running, I can work confidently. It also means at the end of the job, I can simply unplug a drive and hand it to the client before breaking down my kit and heading to the pub for early doors.

 

The Setup

Ok, so what’s the setup?

 

 

Working on location, I use my 15” Retina MacBook Pro. It’s fully spec’d up with memory, GPU, etc. I also use the fastest media drives I can. As you can see from the photo above, my favoured drives for this are G-Tech’s Thunderbolt-equipped G-Raid drives. Needless to say, you really want the fastest drives to work from these days.

I also use a variety of USB 3.0 card readers for SD cards, compact flash, SxS and Sony’s latest XQD cards. Whilst SxS and XQD cards are themselves blindingly fast, if your cameraman is using more run-of-the-mill SD or CF cards, try to make sure they are the fastest possible: yes, your cameraman may have to pay a little more for them but speed counts.

You’ll also see that I’m using a CalDigit Thunderbolt Station, into which I’m plugging my card readers. I cannot recommend these devices enough (I have a Thunderbolt Station 2 on my studio setup) not only because they provide additional HDMI, ethernet and other ports without tying up any precious thunderbolt ports, but with three additional USB 3.0 ports, it means I’ve got spare USBs to plug in my full-sized keyboard and/or any thumb drives with graphics or music, without having to keep ejecting cards or hard drives to swap USB plugs over. An additional HDMI output is also useful for connecting a second monitor whilst keeping the MacBook Pro’s HDMI free for FCP’s A/V Output so the client can view our masterpieces on the big screen.

 

 

 

You’ll also see a number of other drives hanging around in these picture too. These are one drive for backing up my media and libraries and a second drive that the client wants to take away with them at the end of the edit. These don’t need to be big expensive drives like the G-RAID of course, though the larger the capacity and quicker the drive and connection, the better.

 

FCP X Library

For anyone who is unsure about media management (and it’s an important skill anyone using a computer should have to master), FCP X Libraries’ default settings work well whereby all the media is stored inside the library bundle itself, and I would generally recommend that this to be the easiest way of making sure your library’s media is secure. However, I prefer to work with a library that has external media. This makes managing backups much more efficient.

 

 

 

For example, in this instance you’ll see that the storage locations are set to a general “production” folder on the G-RAID. In the above example I’ve set up a “production” folder called “Spring Fair” in which I have stored the FCP Library, along with folders for Media, Camera Archives and Exports. It is this “production” folder which I’ll be backing up as I work. You’ll also notice that the cache for this library is stored externally from this folder. I’ll explain why a little further through the article, so please bear with me….

FWIW, I tend to leave the library to create its own backups in the default Final Cut Backups location as, in all honesty, I can only think of a couple of occasions when I’ve needed to recover a library from a backup.

 

Camera Archives

Right, so now the footage starts to flow in…

The first thing I do is make sure that any camera cards are clearly defined as whether they are still to be ingested or can be released back to the cameraman to be wiped and reused. My friend and colleague Simon Edwards introduced me to very low-tech system that seems to work:

 

 

Once a card has been given to me, the first thing I do is create a camera archive. I often think this is a deeply misunderstood process in FCP X as it not only creates a self-contained clone of the card but, as I understand, it also checks and verifies the integrity of the copied data. I create the archive, give it a name (adhering to the agreed naming convention for the client/job, of course) and add the archive to the Camera Archives folder on the main drive.

 

 

There are three main reasons I do this. The first reason is for backup; if I have any problem with the media I’m working with, I can always go back to the original camera archive and reimport. Steve and Mark at Macbreak Studio recently did a nice little feature on this.

The second reason though is for metadata. I don’t know what everyone else’s experiences are regarding this, but it always seems that every camera card I’ve been given was called some variant of “UNTITLED” or “NO NAME”. Now this isn’t a huge issue because, let’s be honest, who wants to have to worry about renaming their camera card every time they format it? True, but when it comes to managing which camera card a set of media is from, it’s useful to be able to narrow it down. So, by creating a camera archive from a card, I can give it a name (in a similar way we used to write the tape name on a label).

When I then import the media from the archive, the name of the archive magically finds its way into the metadata field of the media. This way, if I need to, I can trace the origins of this media back to a specific camera card, whilst also using that metadata in other ways to create smart collections, etc. In order for this metadata to be automatically added, you need to create the archive then import the media from the archive, otherwise you’ll have to add it yourself. Ticking the option to add the archive to your Import favourites is also another useful tip as it makes the archive easily accessible. Once you’ve imported the media from the archive, you can remove it from your favourites to keep the list nice and organised.

The third reason is speed. I can create an archive fairly quickly using Final Cut Pro’s background processes from the USB 3.0 card reader onto the fast G-RAID drive. Once on the G-RAID, it’s even quicker to import the media from the archive to my library, copying the media to the library’s media storage location. Again, FCP’s background processes mean the clips import instantaneously so I can start editing immediately, whilst the media copies in the background as I work.

It’s also worth remembering that you can create archives from multiple cards simultaneously and import from multiple camera archives at the same time. Whilst editing. Very cool, but another reason for having fast drives.

Once the camera archive is created, I then use the archive to import the media into the library proper, adding keywords, ratings, and editing as normal. Only once I’ve got an archive and the imported media safely in the library will I allow the card to be released back to the cameraman ready to be erased and reused.

One final point on archives: I always ask the cameramen to wipe the cards prior to reusing them; if they don’t then I’ll end up having to archive the full set of media on the card, including the stuff I’ve archived before, as there seems no way of selectively choosing what to archive on a card. Again, no biggie, but it just increases the overall amount of storage required.

 

Backing Up 

So, at the moment, everything is stored on the external G-RAID: library, media, cache, archives. We could have a discussion about creating the archives on a further drive and storing the media on a main editing drive as Apple recommend - and that’s certainly a way I’ve worked in the past. But for the moment, let’s take that as a given…

Of course, everyone should feel slightly uneasy about storing all our precious work on a single drive… Who knows what could happen to it. If that drive should be lost, stolen or simply fail, we’d be up a certain creek without a certain paddle.

There are many choices out there for backing up your files, but my favourite is ChronoSync by Econ Technologies. It’s a great program that can sync files back and forth, you can use it to clone your system and schedule syncs. Best of all, it’s only ever one price with all updates - including those to different versions - being free for all users. I’m also pleased to have shared this with FCP.co Editor-in-Chief Peter Wiggins who has also used it and mentioned it in passing in the following articles:

On the road with Final Cut Pro X and a Jellyfish, the new budget shared storage system

The top ten tips for video editing in hotel rooms

So how do I use this? Well, first thing is to set up a synchroniser document - effectively what do you want to backup and where you want it to backup to? In our case, I want to backup the entire “production” folder containing my FCP X library, media and camera archives, to another hard drive attached to my system.

 

 

Next, I use the scheduler to run a regular sync. Depending on what I’m syncing I tend to get the scheduler to run every 15 or 20 minutes. You can also set ChronoSync to run on particular days and times, etc, though I tend to set it for all day every day. It’s worth knowing that you can also get the scheduler to run only if both the source and target drive are available.

This is very handy especially at the end of the day when I often take my laptop away with me as I don’t need to disable the scheduler (and therefore remember to re-enable it the next day). Instead, the scheduler just pauses and then recommences when I mount both drives on the system again.

 

 

I set all of this up at the very beginning of the edit and, as I add camera archives, import media and edit throughout the day, ChronoSync is silently sitting in the background backing up the new files as they appear. This means that I have a complete copy of everything that’s only a few minutes from being backed up.

You can schedule as many syncs as you want, to whichever destinations you want. I don’t tend to run them all at the same time - you also want to be able to read and write to your drive at the same time. As I mentioned earlier, I’m using Thunderbolt-connected G-RAID drives which allows me to create camera archives, edit and sync to and from the same drive without any issues on performance. Slower drives may give you more issues. You can also store camera archives on one drive, media on another drive and the Library of your Macintosh HD and backup each of these locations to one or more separate heard drives.

So what about the cache files I hear you ask?

Well, the cache is interesting in itself. According to the Final Cut Pro X user guide, the cache stores render files, analysis files (generated by such actions a running stabilisation, optical flow, (Maybe that's in the Library - Editor) audio and colour balance analyses), thumbnail images for the filmstrips and audio waveforms in both the browser and timeline. The reason I don’t back up all of this is that the cache itself can become very large (particularly if you’re rendering a lot) and it’s presented as a single file.

ChronoSync performs incremental backups of files that are new or have been modified since the last backup, so I don’t want it to have to see that cache file as having changed and have to backup the whole cache again. That’s also the reason why I’m working predominantly with external media - once the file is backup up, unless it is altered it won’t be continually backed up. This makes the backup as efficient as possible when it occurs. The library bundle itself is not a problem as, because everything is stored externally, this is generally a manageable size and can be backed up or shared easily. If the library had been managed (i.e. all the media and cache inside the library bundle), every time a backup is scheduled, it would want to backup the whole library again!!

The only final thing to watch out for is if you do open up the library from the backup drive, it still thinks it’s referencing the media in the original location. You will need to redirect the storage locations to the media location on your backup and rebuild the cache, but otherwise you shouldn’t have a problem.

All I do now is make sure I’ve done a final, manual sync at the end of the day, put all the drives in a safe place, hand the backup drive back to the client and head off for a well earned pint.