Libraries and Xsan

How are Xsan connected FCPX editors handling Library locations? In olden times (FCP7 and earlier) we would store the master project folder on the SAN but drag the project being worked on to the local drive. At the end of the day we'd copy it back- essentially a manual check in, check out process.

Now with X, we're opening Libraries right from the Xsan and working that way, but occasionally our edit systems become very unresponsive and start beachballing after every task. After a while (10-20 minutes) behavior seems to return to normal. Is that just bugginess in FCPX or is the traffic to and from the Xsan causing a slowdown? Our throughput is good according to Blackmagic Speed Test.

I've toyed with the idea of dragging the active Library to the local drive, but some Libraries are huge even though we keep the imported media outside of them. Not being able to modify the location of the render files is a big part of that.

Thanks,
Bill

We use NFS over gigabit ethernet at our place, so we can't compare Apples to Apples (pun intended.) But we have seen any slowdowns like this before. We are running managed media in fact, not external.

My first thought is render files, but it would just be specifically libraries giving you trouble. Is it just one library giving you the issues? This sounds to me like some kind of intermittent process that is running, perhaps on your XSAN? Are you checking the transfer logs on the XSAN to see if you can locate a pattern?

Have you checked to see if the timing of the FCPX automatic backups are all all related?

BTW. Ronny is the guy who could really help you. He runs a XSAN at a decent sized facility. Perhaps he will chime in.

Hi Bill,

We always keep our Libraries local to prevent traffic bottlenecks with our XSAN (16 editors). This slowdown is not related to bugginess in FCP X but to the continuous data traffic that goes on between the local FCP X bays and their Library databases.

It's hard to explain a complete workflow in a forum post, but ours is quite similar to what you can see in this PDF on pages 42 to 49: www.10dot1.co.uk/content/FCPXInASharedEnvironment_FINAL.pdf

I agree that being able to keep transcoded media outside of the Library would be another big step-up for FCP X collaborative work, and I have a feeling Apple will be going this way.

-Ronny

Thanks guys. Very useful info.

It seems like going local is the best plan of attack. Ronny, are you using the internal storage of the Mac or an attached drive? We have new Mac Pros with 512GB of internal SSD but I've been leery of using the boot drive to house media, even if it's just during the edit.

Bill

ronny courtens wrote: This slowdown is not related to bugginess in FCP X but to the continuous data traffic that goes on between the local FCP X bays and their Library databases.

What about the fact that Bill's issues were somewhat intermittent? I understand why you would have bottleneck issues with 16 editors opposed to our shop with just 2 editors (and we are not editing constantly).

Wouldn't there be a possible solution to distribute the Libraries on different XSAN locations/volumes (is that possible?), or do you feel it is overloading the fiber network itself?

The tiny packets are being constantly read and written with the FCP databases are just killer on the network. It seems someone out there would have a solution for it. Maybe putting the libraries on a shared SSD rather than spinning media. (I'm just thinking out loud now.)

@Bill,

Yes we use attached storage (Promise TB2 RAID6) on all our bays (full-spec 2013 iMacs and 2013 MPs). I know the new MP's SSDs are terrifyingly fast but I still don't want to run any media from any system drives. Call me old-fashioned

@Tim

The traffic overload pretty much depends on how much traffic is generated at the same time by different editors. That's why the issues can seem intermittent.

Yes you can create dedicated pools on the XSAN to separate the Libraries from the actual media. We do this for productions in which editors often need to share Libraries, in which case collaborating from a central Library is easier. It is slower than working from locally copied Libraries on TB RAIDs but still faster than constantly sharing the local Libraries over the XSAN.

I have a feeling we may expect some further changes in the FCP X Library structure to facilitate better media management in shared storage environments. In the coming months I will be counseling the setup of a new FCP X collaborative system for a large European production house. We may try out some new ideas in this concept. I will keep you guys updated on our results.

-Ronny

Much appreciated as always guys. I'll let you know how things go on my end after some testing with local Libraries.

Cheers,
Bill