iMac 5K: What's Your External Hard Drive For 4K

Hi,
I'm about to purchase the following maxed out iMac 5K. Would like to know what external hard drive RAID system you are using for 4K editing. I'm not editing in RAW or RED. Thanks, Ray
Hardware:
4.0GHz Quad-core Intel Core i7, Turbo Boost up to 4.4GHz
32GB 1600MHz DDR3 SDRAM - 4x8GB
1TB Flash Drive (SSD)
AMD Radeon R9 M295X 4GB GDDR5

Software:
Final Cut Pro X
Motion 5
Compressor

Please..... Anyone :whistle:

I use the Promise P2, but Otherworld Computing, G-Tech, others make good RAIDs.

Thanks for the reply.... I've been looking at the Pegasus R6 12TB and the R8 24TB external drives. www.promiseworks.com/Pegasus-RAID-Thunde...e8pcQCFQuMaQodtGoA1g
I would like to run in RAID 5 which to my understanding requires 3 drives minimum. With that being said, in theory, I believe the R6 would be double in speed. With the R8 24TB (8x3TB) being double in speed with two more drives above the R6. Please correct me if I'm wrong, I believe that drive size doesn't matter for speed, it's the headroom meaning more drives. With that being said, is it really worth an extra $1100.00 for the R8 over the R6? If Read/Write speeds are minimal between both, I'd rather stick the $1100.00 into a seperate storage drive. Please let me know your thoughts and opinions. Thanks, Ray

More drives, better speed, to a point, yes, there is somewhat of a limit. I'm on a Pegasus 2 R4, which I got when they used to sell it empty. My understanding is that they don't anymore. I put four 4TB drives in it, and I get fantastic speed. I edit BMD 4K and pocket cinema 1080 and GoPro pretty much daily, and I'm super happy with my performance. I'm also on a Mac Pro tube, D700 GPUs, 3 GHz CPUs. I've no complaints at all. I don't think you'd humanly notice any performance different editing video with an R6 or an R8. Spend the money on a cheap backup unit for your RAID. That's what we did, got an OWC cheap unit to nightly backup the P2's we have. Remember, RAID's are meant for speed first and foremost. Secondary is the redundancy, which only gives you a very slight (and I mean really slight) corrupted drive buffer. That redundancy is NOT meant as data backup.

Hey, do the sell the R6 with 4TB drives? Man, that'd generate some major heat, though.

Hi Ben,
Thanks again for your help! I'm really starting to lean towards the Pegasus R6 12TB after reading your reply. I need to find out how much 4K video I would be able to place in the FCPX library for editing on a 12TB drive versus the R8 24TB drive. As I stated, I do not edit in RAW or RED therefore, that should help tremendously. My equipment only consists of;
Sony PXW-X70 XDCAM with 4K upgrade coming soon.
Canon EOS 7D
GoPro Hero 2
There is a Pegasus R6 24TB RAID System with a nice list price of $3600.00. :ohmy:
www.promiseworks.com/Pegasus-RAID-Thunde...e8pcQCFQuMaQodtGoA1g
Thanks again, Ray

Ben is right, you won't notice any editing performance difference between an an R6 and R8. It is vital to have a separate backup unit for your RAID.

Always keep in mind, this is not the "benchmark olympics". You don't get more points for ever-faster I/O. Increasing performance of any element only helps until that element is no longer the bottleneck.

The highest bitrate of the PXW-X70 is 50 megabit/sec, which equates to about 6.25 megabytes/sec. You could edit several camera-native HD streams on a Pegasus R4 without hitting the I/O limit.

If you transcode to optimized media (ProRes 422) the I/O load will be higher but the CPU load lower. I don't know what the per-stream I/O load is for 4k in ProRes 422. ProRes HD is about 18 megabytes/sec, so 4k would be about 4x this or 72 MB/sec. In theory a P4 could do several 4k streams of 4k in ProRes 422, although I've never tested that. But you only have one 4k camera, so you won't normally be editing multiple 4k streams unless you did a multiscreen.

For editing in camera-native H.264, this reduces the I/O load greatly so there's normally no compelling need for extreme I/O capability.

Hi Joema,
Thanks for the reply. I usually edit anywhere from 3 to 8 hours of video at any given time. My final edit for mastered output is usually from 1 to 3 hours. These times are the extreme for what I do. With that being said, I do like using effects, chroma keying, transitions, pictures, music, etc.. Coming from Sony Vegas Pro there are times I used anywhere from 4 to 5 timelines during editing. I want enough speed, but more importantly, more than enough headroom without choking while I'm editing. Basically, I want a smooth fairly fast editing experiance while using FCP X, Motion, Compressor and LightRoom. I also would like my RAID hard drive to hold/handle at least 30 hours (in total) of optimized media in my libraries. I do have a couple questions and please correct me if I'm wrong;

1) Would it be correct to say that 72MB/sec for 4k would come out to just under 4 hours of video per 1TB straight out of the camcorder?

2) As for my storage drive/s; In your opinion, should it be setup in RAID or simply save my original media on a couple seperate and/or partioned hard drives?

Your real world experiance is very much appreciated. Thanks again, Ray

Storage Drives, they're not doing work, they can be anything, USB 1.0 if you want. The only issue would be the time it takes for a backup to run. But keep the P2 in its native RAID 5 configuration, they seem to be happiest that way.

Here's Apple's white paper on ProRes and the various flavors.
www.apple.com/final-cut-pro/docs/Apple_ProRes_White_Paper.pdf

RayPaula wrote: Hi Joema,
Thanks for the reply. I usually edit anywhere from 3 to 8 hours of video at any given time. My final edit for mastered output is usually from 1 to 3 hours...I also would like my RAID hard drive to hold/handle at least 30 hours (in total) of optimized media in my libraries.

According to the FCP X white paper Ben posted, a MacBook Pro can handle nine 4k streams at 24 fps. The on-disk storage requirement for 4k ProRes 422 is 226 gigabytes/hour at 24 fps. Using Proxy media reduces this to 70 gigabytes/hr.

However you do not have to transcode to ProRes 422, a top-spec iMac is able to edit camera-native 4k. That hugely reduces the on-disk storage requirement also reduces the I/O rates. Camera native 4k from your PXW-X70 is 50 megabit/sec, which equates to about 6.25 megabytes/sec. That would equate to about 180 GB for 8 hr.

However you must also add a large "fudge factor" because FCP X consumes all kinds of space for rendering files, optical flow, etc. These are easy to see using the 3rd party utility Final Cut Library Manager: www.arcticwhiteness.com/finalcutlibrarymanager/

You may want to experiment with different methods using a smaller data set and and see what works well before committing to a specific workflow.

In general you want

(1) A robust Thunderbolt RAID system of adequate size and bandwidth, but not dramatically more than your projected realistic needs. Buying ever more space and ever higher I/O rates don't help overall performance unless that is the bottleneck. The above numbers show even a Pegasus2 R4 4-drive RAID5 should be able to handle your needs from an I/O rate standpoint.

(2) An adequately fast and large backup drive for the entire system, inc'l the RAID array. This need not be an identical RAID array but it should be pretty fast, else backup and potential restore of that large data volume will take too long. A lower-cost way is a Thunderbolt 2-or-3 drive RAID0 array.

For really important work the best practice is have two different backup methods. E.g, a constantly-connected drive or array for Time Machine, then a periodically-connected drive or array for Carbon Copy Cloner backups.

You will be dealing with a lot of data so maybe the R6 is a better choice, but I wouldn't go any higher than that. Note the "name plate" capacity after RAID5 formatting will be somewhat less. E.g, my 8TB R4 after formatting only has about 6TB available. You also have other RAID options such as RAID10 which have different performance/space/redundancy tradeoffs.

RayPaula wrote: ...As for my storage drive/s; In your opinion, should it be setup in RAID or simply save my original media on a couple seperate and/or partioned hard drives?

This is a matter of preference. I personally don't like partitioned drives because space growth is unpredictable and it places a "fence" which can get in the way. Also the more drives within a RAID volume, the faster the I/O.

Thanks..... I've learned so much from you folks here at fcp.co! I will definately leave my RAID hard drives as a whole without partioning them off. In regards to external storage drive/s, I won't be concerned about them being configured in RAID. I will more than likely save my original media on two seperate drives in case of drive failure (a backup of a backup). As I stated earlier, there are times that I edit on four to five timelines due to using effects, chroma keying, transitions......etc. I suppose you could call it close to four to five streams of video. On one hand, I'm leaning towards just the R6 12TB which seems to be enough speed, but may be too close to the headroom I may need. On the other hand, the R8 24TB would give me double the headroom and possibly a little more speed. With that being said, the R8 24TB would be a solid way to go without ever having to concern myself about either speed/headroom. I'm thinking out loud along with convincing myself in what direction I need to go in (LOL). It basically depends on how much I want to break the bank..... Thanks again, if anyone has any more thoughts and/or opinions please chime in. Ray

Hey, when you're throwing those capacity numbers around, let's be clear that those are drive size numbers, not usable storage space numbers. With RAID 5 you will lose the capacity of 1 drive.

In my R4,4x4TB drives = 16 TB.
RAID 5 only leaves me 12 TB of actual storage space.

So if you pay extra for 12 TB (6x2 TB) that will only give you 10 TB of actual storage, so think about it. Performance vs capacity vs price.

I went the route I did so I could use the four 4 TB of drives out of my 2008 Mac Pro. Saved money, and very happy with it. And these are some of the first 4 TB drive that hit the market back then. I'm sure my days are numbered. Which is why I do nightly backups.

Just food for thought.

Hi Ben,
I figured I would lose some usable storage, just didn't know how much. Thanks for clearing it up for me. I'm 99.9% sure that I'll be going with the R8 (8x3TB) drives = 24TB. This should give me ample space as well as speed for my 4K video editing. I've always believed in getting it right the first time, if not, it usually ends up costing you more to do so. I'm also a firm believer of backing up a back up with two other external drives. Thanks again for all your help and if you think of anything else please let me know. I sincerely appreciate everones help here at fcp.co. Thanks again, Ray