How Do You Backup a 6 TB RAID used for FCPX?

Michael, your had created a thread asking how to backup your FCPX Events and Projects and I was wondering if you had ever got your head wrapped around a solution?

As far as backing up a single drive containing events and projects, I have been using Chronosync. I initially backed up the contents of my entire drive, and then in the evenings or first thing in the morning while I'm still reading the NYTimes, I'll use Chronosync to do my incremental backups. Works out great, if for some reason my working drive kicks the bucket, I can simply connect my backup (cloned) drive and pick up where I left off.

Actually, this is how I was backing up my FCP7 projects and when I was ready to start editing in FCPX (in the middle of a project as it were) I simply connected that drive to my iMac (my FCPX machine) and created an X project and simply referenced it to the original ProRes 422 files on the drive which I originally captured on the Mac Pro in FCP7. The cool thing was (is), if I wanted to, I could simply connect that drive back up to my Mac Pro (FCP7 machine) and continue editing my original FCP7 project. Then, connect it to the iMac, and edit in FCPX if desired. I'm not saying the FCP7 project files are being read by FCPX, I'm just saying since both projects live on the same drive along with the shared footage, I can edit in what ever version of FCP I want to edit in. I just may continue to capture my ProRes 422 files in FCP7 since Im still capturing from tape and FCPX is somewhat limited as to the file naming scheme.

Anyway, I didn't want to hijack your original thread, so I thought it would be best to start a new once along the subject of backing up a large RAID versus a single drive. As you know, I'm saving up for a Thunderbolt RAID. I've got my eye on the Promise Pegasus R6x1 TB RAID. So, now I've got to figure out a backup scheme for when I start using the Pegasus RAID for all of my combined events and projects, versus a single drive with usually just 1 Event and associated projects.

So my predicament is since the sum of all of my future Events and Projects on the RAID will be too large to backup to a single drive (as I'm currently doing) and I'm obviously not going to have a 2nd Promise RAID to do a clone type backup, just how do I accomplish backups from a 6 TB RAID? Should I just be concerned with backing up the footage, then if worse comes to worse and my RAID takes a dump and I can't rebuild it, I'll then at least have the footage backed up but what a PITA it would be to have to rebuild my FCPX events and projects from scratch? I don't much like that idea. So then, how do I accomplish a full backup of my events and projects on a large RAID to include daily incremental backups to those files that have been added or changed everyday? I'm not sure how to go about doing this with Chronosync especially since my Events and Projects summed up will be too large to backup to a single drive.

So how do others here, accomplish RAID backups?

Really the only solution I can think of is to Digital Tape Back up. Or just pull the drives and replace them. If it's more worried about losing footage because of a hard drive crash then make sure you are using RAID 5 or 6.

If you have 6TB of stuff, you should be getting paid by your clients enough to have an LTO tape system, or a second 6TB RAID. If not, how are you turning a profit when something goes horribly wrong? Just saying, a sound, profitable business must have emergency plans in place. Fees to customers must pay for enough hard drive space.

BenB wrote: If you have 6TB of stuff, you should be getting paid by your clients enough to have an LTO tape system, or a second 6TB RAID. If not, how are you turning a profit when something goes horribly wrong? Just saying, a sound, profitable business must have emergency plans in place. Fees to customers must pay for enough hard drive space.

Im certain this thread is getting more complicated then it needs to be. I just want to know how to backup a RAID.

I'm currently editing FCPX on an iMac via dual bay eSATA enclosure setup as a JBOD. From everything I've been reading, I would be better off purchasing a 6 drive Thunderbolt RAID versus a 4 drive RAID and configuring it as a RAID 5. But I also read that I shouldn't consider that to be an effective backup plan, that I would still need to backup the RAID.

I've also read that it's a waste to get a 4 drive RAID and configuring it as a RAID 5, that's why I'm looking at a 6 drive RAID. However, maybe I would be better off to get a 4 drive RAID and configuring it as a RAID 0 and getting a 2nd four drive RAID to setup as my daily backup drive?

Actually, I suppose there is no reason the backup RAID has to be a Thunderbolt RAID, right? Maybe I would be better off getting a cheaper Firewire 800 RAID for the backup drive. I just don't understand the procedure in backing up a RAID. Can I use any backup application like Carbon Copy or Chronosync to take care of my daily backup chores? Do I set up incremental backups the same way as I would if it was a single drive?

I don't normally have 6 TB of active "stuff" but I would like to have that option I suppose once I purchase the RAID. I mainly do weddings, and up to this point (prior to getting the Pegasus Thunderbolt RAID) I dedicate each wedding to it's own 1 terabyte SATA drive which sets in that dual bay eSATA enclosure I showed you the other day. This makes it easy to backup daily changes to a cloned drive which I keep plugged in to my Mac Pro's 5 bay eSATA enclosure set up as a JBOD (the Mac Pro doesn't have FCPX on it). But when I start putting multiple events on the Pegasus RAID, well, I'm trying to imagine what the easiest and most complete backup workflow would be. I really don't want to mess with a tape backup system.

My clients could care less how I edit their wedding video, whether I use a USB drive or a RAID it doesn't make any difference to them. And, they are certainly not going to pay extra for me to purchase backup equipment, so I guess you can consider the backup procedures for my own peace of mind, knowing if something goes wrong, that I don't lose all of my captured events and that I can fall back to my backup and not have to re-create the events.

There's no a single thing wrong with making a 4 drive RAID a RAID 5, not at all. Theoretically the more drives you have, the faster, because the data read/writes are being done in smaller chunks across each drive. But then you reach the connection and drive throughput limits. But there's nothing wrong with a 4 drive RAID system. And RAID 5 is the best for 4 or more drives in a video editing system. Best balance between redundancy, speed, space efficiency.

I'd personally never use a JBOD RAID for video work.

RAID 5 is redundant, but only allows for one drive to go bad at a time for hot swap and rebuild of a single drive. If two go out, you're screwed out of all your data.

Two ways to backup, to another RAID, or to LTO tape backup. A duplicate RAID restores faster, but LTO is longer life and safer. Check out Quantum for LTO solutions.

4 drives as RAID 0 is ridiculous, when you have that much space to work with. At least use RAID 1 and have redundancy. And with that much space, I'd go with the better speed/efficiency/safety balance of RAID 5.

How the RAID is attached to the computer only matters in that the attachment cable influences overall performance speed, nothing else. You can work off a T'bolt RAID, and use an frekin' USB 1.0 RAID to back it up if you want. You can work on a RAID 5 and use a RAID 1 unit to back it up. USB will take forever backing up your data, but if it's done at night, who cares? It's cheaper!

To a computers, a RAID array is just one single hard drive, period. Use any backup software you like. I prefer Carbon Copy Cloner because it's full featured, is written by a very reliable developer, and has a long history of stability and integrity.

And why to video folks keep using JBOD RAID?

So, you have a T'bolt RAID, put all your work on it, that's the best and fastest workflow solution.

Back it up with another RAID who's format and connection type doesn't matter, it's just another drive you'll read/write data with. Set up CCC to backup the T'bolt drive to your backup RAID at 1am every night. Oh, and you want to set your screen to go to sleep, but not the drives or the computers. Simple, easy, low maintenance, so tape as you stated.

One last thing I'd do, is get a copy of Disk Warrior, run it on all drives once a month to rebuild the directories.
Final Cut Pro X System Maintenance.pdf

BenB wrote: There's no a single thing wrong with making a 4 drive RAID a RAID 5, not at all. Theoretically the more drives you have, the faster, because the data read/writes are being done in smaller chunks across each drive. But then you reach the connection and drive throughput limits. But there's nothing wrong with a 4 drive RAID system. And RAID 5 is the best for 4 or more drives in a video editing system. Best balance between redundancy, speed, space efficiency.

One last thing I'd do, is get a copy of Disk Warrior, run it on all drives once a month to rebuild the directories.
Final Cut Pro X System Maintenance.pdf

Okay, I'm convenced. I think I'll probably go with a 4x1 Pegasus RAID then configured with RAID 5. I"m sure that's going to be faster than I need since I only have to deal with 3 data streams when I'm editing in multicam.

I do have Disk Warrior, but I need to upgrade to the latest version for Lion. Maybe I should just wait until Mountain Lion comes out, because I'm sure they will want another $50 to upgrade it again.

2 last points. First, a 6 drive T'bolt RAID will be faster than a 4 drive T'bolt RAID. And you're using more than only 3 "data" streams. Each video is a video stream, and it's also an audio stream. Text adds video streams, transitions add video streams, etc. Until it's Rendered, which simply writes a single video stream, and down mixes audio. Be sure you're doing your weddings in Compound Clips and assembling those into a Project that is set for Stereo audio. It'll help a tiny bit to improve the performance.

Okay, will do. Thanks for your input Ben.

RonPriest wrote:
I do have Disk Warrior, but I need to upgrade to the latest version for Lion. Maybe I should just wait until Mountain Lion comes out, because I'm sure they will want another $50 to upgrade it again.

FWIW I paid for an upgrade to DiskWarrior when Leopard came out. Aloft have given me free upgrades ever since. As Mountain Lion is like the Snow Leopard of upgrades I'm sure you'll be fine with version 4.

This program is far to valuable to have sitting there unusable.

Seanus wrote:

RonPriest wrote:
FWIW I paid for an upgrade to DiskWarrior when Leopard came out. Aloft have given me free upgrades ever since. As Mountain Lion is like the Snow Leopard of upgrades I'm sure you'll be fine with version 4.

This program is far to valuable to have sitting there unusable.

I'll check into it. I currently have version 4.2 and it's up to 4.4. On their product Support page, it does say free disc updater available for Version 4.0 - 4.3.

Seanus wrote:

RonPriest wrote:
This program is far to valuable to have sitting there unusable.

Thanks for the reminder about Disk Warrior guys, I haven't preformed any preventive maintenance on my external drives for a while so I just ran it and fixed all of my directories on my external drives. Sent them a message to see if I can run Version 4.2 on my Lion system drives.

4.2 will not boot for Lion. From Alsoft support, "DiskWarrior 4 version 4.4 is completely compatible with Mac OS X 10.7 Lion." I had to update to 4.4 when I updated to Lion, and it's just a good idea to stick with their latest version to take advantage of the improvements they continue to push out.

Disk Warrior's point updates have always been free. From 4.0 to the current 4.4, all been free download updates. But you need to have your ORIGINAL Disk Warrior DVD to burn a new updated boot DVD for it.

Okay then. I do have my original 4.0 DVD and I just downloaded the free updater, so I'll see if I can update and burn a new copy. I was just reading about that on the upgrade procedure and I'm a bit confused about the statement below which was extracted from the 4.4 Update Read Me.rtf file that came with the 4.4 Disk Update.app. It sounds like the new updated disc it creates will not be able to start up a computer with Lion, since the original version 4.0 disc I received from them was before Lion was released. And it appears it wont be able to start up my new iMac.


"You can use your updated disc just as you used your original disc to start up your Mac. Please note that your new DiskWarrior disc will contain the latest version of DiskWarrior, but will still contain the same version of Mac OS X as your original DiskWarrior disc. Alsoft cannot update the version of Mac OS X on your disc with this updater application. This means that the disc created by this updater will not be able to start up any Macs that your original DiskWarrior disc is unable to start up. If you have purchased a new Mac that requires a later version of Mac OS X than the copy of Mac OS X on your original DiskWarrior disc, you'll need to purchase an update disc from Alsoft."

So is that what it's saying, that I won't be able to boot up any of my computers which have Lion (which is all of my computers)? Sounds like they want me to purchase an updated disk with Lion OS on it.

My original is 4.1 or close to that, and my update boots Lion just fine.

If the DW disc contains a copy of OSX that your computer CAN boot from all will be well. Brand new Macs may refuse to boot from a previous version of OSX - THEN you'll need an upgrade disc from Alsoft.

Ian.

ianstubbs wrote: If the DW disc contains a copy of OSX that your computer CAN boot from all will be well. Brand new Macs may refuse to boot from a previous version of OSX - THEN you'll need an upgrade disc from Alsoft.

Ian.

After creating the new DW DVD, I was able to boot up my 2006 Mac Pro containing Lion with the disc by holding the C key down until I got to the "start up screen" but was not able to boot up my 2011 MacBook Pro using the disc. I didn't try it with the new iMac.

So, apparently my 2011 MBP isn't capable of booting up from the OS that was on the original disc, which I assume would be Snow Leopard, since the original DW dicc was version 4.2. But, that doesn't make much sense because Snow Leopard was on that MBP when I bought it, and has since been upgraded to Lion. After installing the DW application itself on the internal hard drives of the MBP and iMac, DW will work in Lion running on external drives. but that doesn't do me much good in preventive maintenance for the OS drive on those 2 computers.

I've sent an email to Alsoft Tech Support asking them if I need a new disc from them.

BTW, what exactly is the "start up screen" is it the Apple logo or the login screen? I held the C key down on my Mac Pro for about 3 hours until I got the DW screen (okay, maybe it was just 2 hours) but I could never get the MBP to boot up with the DW disc after holding the C key down for a good 10 min past the point of getting the Apple logo.

It's the Apple logo screen.

Yes, the newer Macs have unique firmware that require specific OS X updates to read them. If your DVD doesn't fully boot a Mac, you'll need to purchase an update DVD that will have the latest OS X updates that will be compatible with the latest Mac model firmware.

Side Note Trivia:
"Technically" there is not one single start up screen, just for some IT trivia. When hit the power button, a specific process starts, when you hear the chime, another process starts, gray screen and another, spinning wheel another, Apple logo another, blue screen another, then the desktop builds for the final boot processes. If you know what each of these represents, you can trouble shoot boot problems pretty easily by where it hangs up. Not that it matters, just thought I'd throw out that trivia.

Making a clone of the entire disk is no longer necessary with the 10.0.3 update as Apple has brought back the much needed Media Reconnect feature. Just make sure you have a copy of your media backed up somewhere (either on inexpensive USB hard drives or LTO if you have the cash). For final cut, the only important backups you need to make are you project folder and the event file in the event folder (not the media).

I'd never "clone" a drive as a backup. Just do a simple backup of the FCP Events, FCP Projects, and FCP Archives folders. You never needed to clone a whole disk as a backup.

If a drive goes back, you can restore everything quickly and easily. I don't think anyone said to "clone" a drive as a backup method.

fcpguru wrote: ... For final cut, the only important backups you need to make are you project folder and the event file in the event folder (not the media).

Yes, I see what your saying. To make things easier in case one had to fall back on those backups, since the Event folder (minus the source clips) isn't that much larger, one might as well keep daily backups of the entire Event Folder and Project folders minus the original source files (if they are backed up on another drive).

However, I'm trying to think of an easy way to do it, since the source files are going to be entangled in the Event folder unless they are alias files, but still you have to set your backup software to not include those source files (or alias link files). I'm not sure about Carbon Copy, I haven't used it, but I'm fairly proficient with using Chronosync but I don't see there being an easy automated process available in Chronosync to exclude the original source files especially when they can show up in multiple Event folders depending upon the number of different events you have on any given drive. There's probably a way to do it, but I'm just not as proficient as I think I am. Sounds like I may need to learn how to write scripts?

Someone needs to invent an easy configurable backup application for FCPX. One that works like Event Manager X allowing you to use checkboxes to backup only specific folders and subfolders. Or does Event Manager X already do that? I haven't used it yet, so I may be speaking out of ignorance here, I've been known to do that once or twice. :dry: