So this FCPX lag that creeps into the UI with complex projects...I decided to measure it

BenB wrote: H.264 can be taxing on any system. A large project should be using Proxy, or better yet, Optimized media. It will save you tons of time through the editing process, as well as during the export process.

This is definitely my next step. But - in theory - should I expect H.264 editing cause sluggishness in the UI such as slow typing in titles illustrated in the video I posted?

- edit: also I did bring the project over to a 12 core New Mac Pro with dual D700 GFX cards, Maximum processor and 32GB Ram with no improvement in UI performance. I'd imagine if this was related to how taxing H.264 is for the system that testing it on that machine would show some improvement.

Hi,
I just started a thread in the Apple forums about extremely sluggish behaviour trying to colour correct clips in multicam and compound clips before I spotted this thread. The lag is so great that trying to move a slider is almost impossible either by dragging or with the keyboard. And now that you've pointed out editing title text is so slow, I can vouch for that too! Takes about a second for each letter to type!

The particular projects are, I guess, reasonably complex: there are a few hundred clips in the library, a mix of H.264 (from a Canon 550d) and AVCHD of about 500GB in total. They are multicam projects of a 2 day music festival, a separate project for each act but accessing a common event library. I've made the media external to the library rather than managed. I should point out that it doesn't make the slightest difference whether I edit in proxy or with the original media.

The libraries and media are stored on an external 4TB Lacie Thunderbolt disc and I've connected it to two different MBPs; one is a mid-2012 modified with a 256GB SSD system disc and 16GB RAM and the other is a 2013 Retina with a 512GB SSD and 16GB RAM. Same result on both.

Great to hear from you @filrmonic. It sounds like you and I are in the same boat. Can I ask you if you upgraded through from 10.0.9 to 10.1 on this project.

TGC wrote: Are you doing a lot of duplicating as snapshot? I found on a recent project that this slowed my entire library hugely.

Some, but no I wouldn't say a lot.

Hi,
Yes, this project was started last year in 10.0.9 before changing it to 10.1 and I really can't recall it being like this!

filrmonic wrote: Hi,
Yes, this project was started last year in 10.0.9 before changing it to 10.1 and I really can't recall it being like this!

OK. Me too. After much trouble shooting - and after your comment of Proxy/Optimized media not helping I am very close to being convinced that it is something that affects complex events/projects that were upgraded to libraries.

The only thing that makes me hesitate is that if you watch my original video UI slow-down is evident (albeit far less dramatically) the more complex a project gets despite being created in FCPX 10.1.1

I would love for someone who is not experiencing these issues to try the title test on a ~30 minute project timeline and confirm that it is not as laggy as I am experiencing. That would confirm that FCPX is capable of complex projects without sluggishness creeping into the UI and I could put it down to something in the upgrade process. Honestly if that is the cause I would quite happy with that outcome.

My initial "work-around" on the colour correction has been to correct one clip on the main timeline (which seems to respond fine) and copy/paste to the other associated clips. Not fun, but it works so far.

If they're multicam clips try doing the correction in the multicam editor not in the project timeline.

Tom Wolsky wrote: Yes. That's because when copying to a library on the same drive FCP uses hard links so the media is not duplicated on the disk. You have to delete both libraries to free the drive space. Great feature in 10.1.

Actually...that's a little worrying now that I think about it. Generally I prefer to keep my Media bundled with my libraries (managed). On page 409 the FCPX manual states:

"When you copy an item from one library to another, the corresponding files are duplicated on the disk that contains the receiving library."

So is FCPX duplicating, or creating a hard link?

...and if it's creating a hard link, what happens if I delete or move the library that I have copied the clips from?

EDIT: I think I have a grasp on this. It seems to me that a hard link is a file that has a number of "instances" assigned to it. So only when the count of those instances is reduced to zero is the media actually deleted. Is this correct Tom? If so...that's pretty clever!

It's something magic but when FCPX duplicate 500GB media on the same disk that's done 2x500= 500 GB !
It's very fast because it's hard link and if you delete the first library, medias are in the new one.

That's right. The media is never in the library, accept as being on the library directory. It's physical location on the drive never changes until all uses of the media have been deleted. Then the physical space on the hard drive will become available to be overwritten.

Hey Doug,

Hard links are quite ingenuous things. To understand how they work you have to consider how data are stored on your drives. There are no "Libraries" or "Media" on your drives, there only are bits of data that can be spread anywhere across your drives. Your system keeps track of these data and puts them together as "Libraries" or "Media belonging to a particular Library" in your UI.

When you duplicate a Library from one drive onto a different drive your system will gather all the bits of data for this Library that are stored on drive 1 and copy these data onto drive 2. As these are different drives the data will be effectively duplicated and the data on drive 2 will take the same amount of drive space as the original data on drive 1.

But when you duplicate a Library onto the same drive your system gets clever. It says: okay I already have all the data for this duplicate Library stored on this drive, so why would I waste drive space and make duplicates of data that I already have? So instead of making duplicates of the data that make up the original Library it just links both the original and the duplicated Library to the same data on your dive. Both your original Library and your duplicate Library share exactly the same bits of data stored on your drive. As long as you don't delete both these Libraries your data won't be erased.

- Ronny

Tom Wolsky wrote: If they're multicam clips try doing the correction in the multicam editor not in the project timeline.

Hi Tom,
I'm already doing that, thanks, and that's where it's slow. I've got a few ideas to try when I get back to editing later this morning.
Phil

Thanks for that explanation Ronny and Tom - really made my day learning that

So I have done a couple of experiments.

First I created a new multicam clip from the same source clips as before (the original multicam clip had been created in 10.0.9) within the original library (containing 100s of event clips). No change; editing in the multcam clip editor was as slow as before with the colour correction sliders.

Secondly I created a new library and copied the clips into it (actually copied the media, not just the links I believe). I created a new multi cam clip from these. Worked perfectly! The colour correction sliders were fully responsive.

I can only conclude, as has been suggested, that there is a problem with increasing event complexity.

I am using the free app Memory Clean - and that does seem to help to clear up RAM usage...

- Sam

You could also conclude that upgrading the original library is the cause of the sluggishness since your test that worked perfectly was created in a fresh library.

I only mention this because I am down to two possibilities: something happened in the update to cause sluggishness or FCPX is innately incapable of complex projects without slowing the UI.

I very much want it to be the former and not the latter.

Can anyone confirm they are working with complex projects in large complex libraries without a slowing UI? A simple test would be to add a basic title to an empty project in an empty library and see how it responds when typing, then open a complex project in a complex library and see how that responds when typing.

I just noticed something very interesting.

When grouped my clips together as a second storyline, playback was smoother there was a definite improvement.

Can anyone else confirm if the same thing happens on your systems.

-

Just noticed something else. I copied my project into a new Library then deleted all the keyword collections and almost had no dropped frames.

Doug Suiter wrote: You could also conclude that upgrading the original library is the cause of the sluggishness since your test that worked perfectly was created in a fresh library.

Can anyone confirm they are working with complex projects in large complex libraries without a slowing UI? A simple test would be to add a basic title to an empty project in an empty library and see how it responds when typing, then open a complex project in a complex library and see how that responds when typing.

I've had really bad sluggishness with large projects full of titles, when upgraded from 10.0.9 to 10.1. But currently working on a fresh library, 44min timeline with titles and yes, it thinks for a little longer every now and then when moving around, but there's none of the same sluggishness when adjusting a title etc. So pretty sure the upgrade mid-project from 10.0.9 is the reason.

Here's the timeline, complex enough I think to test this: Invalid consumer key/secret in configuration

I'd like to chime in and say that I too experience this sort of lag, but chalked it up to running an older iMac and working with h.264 footage. Maybe that is not entirely the case.

My system is a 2011 maxed out iMac (32gb ram, 2gb ati card, Pegasus r4 for storage). For me, proxy does help. Also working without certain effects (core melt lock and load, neat video, etc) but I would expect that since those are very power hungry stuff. But overall I do notice the system lagging more since the 10.1 update.

I am editing about 1.5 tb of h.264 media which create 7 lane multi cams (5 video angles, 2 audio - stereo and 8 channel). I'm often doing picture in picture with two multi cams (drum lessons yay!) - so I think my project is relatively complex and each lesson comes in at about 22 minutes. There are 60 lessons. This project / library was created in 10.1. I feel like FCPX should be able to handle it, but I don't gripe probably because I remember working in fcp7 and how this would take days

I hope someone from apple is silently watching and taking this in - thanks for doing all that legwork!