TOPIC:

Joe, Brave Browser and other Chromium-based browsers should be fine. From chromeisbad.com:

Now what browser should I use?

Safari is good and it's already on your Mac. It's fast and efficient. If you need a Chromium-based browser, try Brave , Opera , or Vivaldi . (Brave and Vivaldi both use an open source library called Sparkle for updates which makes an issue like this impossible.) Firefox has pretty noticeable pointer input latency which (I, the author) am pretty nitpicky about, but other than that it's fine. (Mozilla are a bunch of short-sighted dopes for firing the Servo team. If the Servo team regroups, I'd be inclined to recommend anything they make down the road).

I've noticed occasional delays, similar to what Erik has described, at times, regardless of the complexity or size of the Project. It's intermittent for me.

I am starting to believe, more and more, that there may be some OS-level changes that have been adversely affecting FCP for the past few years (i.e., not necessarily due to code within FCP itself). FCP (and Resolve?) may be using various frameworks/APIs from macOS that potentially could affect FCP responsiveness and performance. In other words, changes to macOS could negatively impact FCP.

If not due to macOS issues, I agree with you that FCP would seem to be suffering from less than optimal performance/responsiveness.

A few days ago, I was surveying my plugins in FCP (built-in and 3rd party) and when skimming over some effects (to see a preview), even built-in ones, FCP would become unresponsive briefly. This is with a very simple test timeline/Project.

DaveM wrote:

...I am starting to believe, more and more, that there may be some OS-level changes that have been adversely affecting FCP for the past few years (i.e., not necessarily due to code within FCP itself). FCP (and Resolve?) may be using various frameworks/APIs from macOS that potentially could affect FCP responsiveness and performance. In other words, changes to macOS could negatively impact FCP....

Dave, thanks. I agree this is a possibility. That said, there have been some recent performance improvements in specific cases, during which I was testing the same library. E.g, Tangier had a complex library where on my M1 Ultra running Monterey 12.6.4, it was unusable. Just constant long-duration beachballs. The library wasn't damaged because it was rebuilt numerous times from XML.

Then with some Ventura upgrades it got significantly better, although it's still slow. That is all running FCP 10.6.5 on the same machine.

Yet -- this could have taken place amidst a broader long-term backdrop of worsening MacOS performance in some areas.

Like you, my gut feel is there have been some long-term OS-level changes that have adversely impacted FCP performance, but I can't prove it.

In software development and support, certain items which are easily benchmarked get close attention, such as export times. One little slip up in that area and Max Yuryev will be showing graphs and charts of the downturn. Other items which are more difficult to benchmark may get less attention, or even have performance regressions. This could especially be so for certain types of complex timelines.

The current FCP code base and object design are highly dependent on single-thread performance, especially when the main thread is in the TLKit (TimeLine Kit) framework, which frequently calls the MacOS Core Animation framework. Yet you would not necessarily see this on simple timelines.

However, on complex timelines, the single run thread must navigate a highly recursive convoluted path as it fetches many different attributes per clip. Maybe certain code paths within Core Animation have gotten slower. I wish I had Instruments trace data from past versions of FCP and MacOS, but because of version dependencies, it is difficult to obtain.

When FCP was first written, likely many of those frameworks and the underlying MacOS frameworks were not designed to be thread-safe and reentrant. Whatever single-threaded "lean" code path then existed was obviously very fast, even on older machines. If the current executed code path has become less efficient, they can't just add more threads. The entire architecture is built around the current threading model and internal object model.

Thanks for your thoughts, Joe. I never heard of Max Yuryev until you mentioned him. May have to check out some of his videos.

No doubt the issue with FCP responsiveness and performance is quite complex. It's difficult enough to try to figure things out with respect to just FCP and macOS, but then any other application could have some daemon running in the background that periodically "messes" with the system, such that FCP has to "wait" on other processes.

Until around 10 years ago, I used to have a dedicated machine devoted just to editing. Time Machine was turned off, and no extras besides some plugins were installed. No e-mail program running in the background, web browsers, etc. All of those things might affect how well FCP works...

Addendum:
To be clear, I rarely have any significant issue with FCP. As much as there _may be_ some underlying issues regarding performance and responsiveness with FCP, in most cases it is "user error" that creates problems...

Oh boy, so it's looking like this is a real FCPX issue? Given they *still* haven't fixed that incredibly irritating browser bug (the one where no matter how many times you trash prefs, the browser enlarges clip sizes and scrolls to the bottom on starting the application - I truly, truly *hate* this), I am now wondering if it's time to consider leaving FCPX altogether...

*postscript - on re-read of recent posts I guess it's a combo of Mac OS + FCP to blame, and maybe the problem timeline that @joema has been helping me look at is a unique case. I hope it is.

I am starting a new feature this time with 4k footage, and although it may be less complex edit-wise, I am worried about it becoming unworkable when I have 30+ mins in my timeline. I considered an expensive Mac upgrade at one point but looking at the evidence, the problem is hardly how powerful or limited my iMac pro may be, so spending $$$ on a new machine would not solve anything.

Slaughter wrote:

...I am starting a new feature this time with 4k footage, and although it may be less complex edit-wise, I am worried about it becoming unworkable when I have 30+ mins in my timeline. I considered an expensive Mac upgrade at one point but looking at the evidence, the problem is hardly how powerful or limited my iMac pro may be, so spending $$ on a new machine would not solve anything.

After further scrutiny of your project and the generated XML, there are a few errors in how media files are referenced. By coincidence this is the same issue as discussed in this thread:

 fcp.co/forum/4-final-cut-pro-x-fcpx/3804...or-format-xml-import 

There are four files in your library which do not have file extensions. I list the XML below. That is the cause of the errors when importing the project XML. I don't know what caused this. In the other thread, a possible factor was Plural Eyes was used to sync and generated the XML when the media was imported, but we don't know for sure.

src="file:///Volumes/FCPX%20RESOURCES%20SSD%202TB/ALL%20NOISES%20BY%20THE%20POLICE/ALL%
20NOISES%20BY%20THE%20POLICE%20-%20SOURCE%20MEDIA/1980/1980%20Hotel%20Principe%20CB003">

src="file:///Volumes/FCPX%20RESOURCES%20SSD%202TB/ALL%20NOISES%20BY%20THE%20POLICE/ALL
%20NOISES%20-%20Extras/ZENYATTA%2040TH/ZM%20FACES%20triangle%201">

src="file:///Volumes/FCPX%20RESOURCES%20SSD%202TB/ALL%20NOISES%20BY%20THE%20POLICE/ALL
%20NOISES%20BY%20THE%20POLICE%20%20SOURCE%20MEDIA/1980/1980%20PHOTOS/
1980%20PAN%20PHOTOS%20Sting%20LP%20Mann%2001">

src="file:///Volumes/FCPX%20RESOURCES%20SSD%202TB/ALL%20NOISES%20BY%20THE%20POLICE/ALL
%20NOISES%20BY%20THE%20POLICE%20%20SOURCE%20MEDIA/1980/1980%20Magazines/
1980%20MAG%201981%20Rolling%20Stone%20Bottle">

Re upgrading to a new Mac, I had a 10-core Vega 64 iMac Pro for several years, and it was a good machine. However my M1 Max MacBook Pro is much faster, and my M1 Ultra Mac Studio is faster still on certain workflows.

One of our most experienced people here is TangierC who recently encountered a similar performance problem on a similar complex timeline, and upgraded from a 12-core "Trash Can" Mac Pro to (I think) an M2 Max MacBook Pro 16. He is going to give us an update when he has time. That would be a good data point.

A problem with damaged or misdefined data like the above four files is you can't tell if the observed anomalies are due to that or something else. I did numerous experiments with your timeline (albeit with no media) and it is really slow, even on my M1 Ultra machine. I cut down the timeline from the original size to about 30 min and it was considerably faster. At that size I didn't get any XML errors upon import/export, so maybe I exluded those. But if you try to drag/drop a large group of clips, that is very laggy.

I have seen this over the past year or so from several other users with large, complex timelines. In some cases (like Tangier's original timeline) he had 6,000 markers and when that was removed it got better. But it was still sluggish. In cases like yours, it really should not be sluggish at 30-60 min length -- doing anything. This assumes the lack of media isn't causing an artificial slowdown.

I have some work to do the next week but when I have time I'll study this more. Sorry about the problems.

@joema I've fixed the file extensions on those files you identified.
FYI it made absolutely no difference....

@joema

Appreciate you have other things to attend to.
Any more ideas or thoughts on this? I am going to have to come back to this feature and I am dreading it. I am considering how I might once and for all rid this project of its gremlins. Do you think any of the following could work:

Copying all assets to a different SSD, starting a new library there, going into the "old" library > project, grouping everything on the timeline and copying it over to a clean new project in the new library.

One thing I dread about this process is relinking files. I've found that relinking files - when there are many of them to relink - causes FCPX to crap itself when you select too many at once. So you have to sort of guess at what is a "safe" proportion of files to select to relink and hope that you don't freeze the application. It's nerve-wracking.

I'd only do this if that process would not carry over any deeper information which we now know to be loaded with what is effectively corruption of some sort.

Basically, I've gotta be able to work on this but when it takes up to 45 mins between mouse clicks, it's not doable unless I can find a way of completely and utterly extricating the project from the problems it has been plagued with despite render and cache deletion and everything else I've tried.

Slaughter wrote:

...Copying all assets to a different SSD, starting a new library there, going into the "old" library > project, grouping everything on the timeline and copying it over to a clean new project in the new library.

One thing I dread about this process is relinking files. I've found that relinking files - when there are many of them to relink - causes FCPX to crap itself when you select too many at once. So you have to sort of guess at what is a "safe" proportion of files to select to relink and hope that you don't freeze the application. It's nerve-wracking...

I have some deadline projects to deliver the next few days, after that I'll have more time to help you. Let me give some quick tips and comments:

- Did you ever get your files renamed on disk and from FCP's standpoint? What is the status of that effort?

- Anytime you copy a clip or project to another library, it should be within a "transfer event". DO NOT copy the bare clip or project by itself.

(1) In the source library: create a new event.
(2) Copy (do not move) the clip or project to that event.
(3) Copy that event to the destination library
(4) In the destination library if you open that project it should automatically connect to the disk files, if on the same disk.
(5) If the destination library and media are on a new volume, relink will be required. If the filenames are unique this will work perfectly, no matter if the destination folder structure is different. If the folder structure is exactly the same, it might work even with duplicate filenames. If the folder structure is different and filenames are not unique, that is a big problem.

Slaughter wrote:

...I'd only do this if that process would not carry over any deeper information which we now know to be loaded with what is effectively corruption of some sort...

Unfortunately that is unknowable.

Slaughter wrote:

...Basically, I've gotta be able to work on this but when it takes up to 45 mins between mouse clicks, it's not doable unless I can find a way of completely and utterly extricating the project from the problems it has been plagued with despite render and cache deletion and everything else I've tried.

The past few weeks, DaveM and I have gotten a better idea of some causes of some performance problems like yours. If you have lots of compound clips and you do several project snapshots, that can create performance problems. This is worsened if you have nested compound clips, IOW a compound inside a compound.

It is further worsened if you make a snapshot of a snapshot. IOW instead of making a project snapshot then continue editing the original project, you take snapshot A, then start editing snapshot A, then later make snapshot B from snapshot A, then start editing snapshot B, etc. That will cause exponential performance degradation. Have you ever done anything remotely like that? How many compound clips does your timeline have, and have you ever made a project snapshot?

Did you ever get your files renamed on disk and from FCP's standpoint? What is the status of that effort?

That's a different library / project and the answer is yes and no. I have left the files which I have started editing with as they are, but I have renamed every single other file that was yet to be used and which are now coming into the project. So far, so good.

Have you ever done anything remotely like that? How many compound clips does your timeline have, and have you ever made a project snapshot?

Possibly once. I have made snapshots of the original version which was edited in 720p, but they were treated as backups and I kept editing the original. More recently, in the lead up to this unhappy situation, I upgraded the whole project to 1080p by grouping the whole timeline, copying that giant compound clip and pasting it into the new 1080p project. There a lot of compound clips. One reason for this is because FCPX does this annoying thing where if you add a transition such as a fade to the start of a clip which you have applied a ken burns effect to, the darned thing *pauses* the ken burns effect until after the transitional fade. You have to group the clip and *then* apply the fade. There are probably dozens of those. There are plenty of groups within groups, yes. Largely because of how oddly FCPX handles things like fade transitions outside of the timeline but for a range of other reasons. Most of them to do with a less than efficient handling of my timeline. It is likely problematic that I do all of my editing on top of the timeline. I'm endlessly worried about having to redo previous versions of edits, so I have the whole thing spread out on top of the timeline all the time so I can see all of my options and all of my workings right in front of me. I don't like seeing a single clip width timeline - too much hidden info. Still, the doom problems here only emerged after the upgrade to latest FCPX + Monterey + change from 720p to 1080p project. In any case, there are plenty of places in the chain, there, where things could have become corrupted, I suppose. I have done plenty of cleanups of the library - only the assets being used are in the library at all, and I regularly trash render builds etc.

Slaughter wrote:

I upgraded the whole project to 1080p by grouping the whole timeline, copying that giant compound clip and pasting it into the new 1080p project.... the doom problems here only emerged after the upgrade to latest FCPX + Monterey + change from 720p to 1080p project...

Please export that project XML, and separately export an XML of the entire library, ZIP them separately and upload to me at this location. I will examine them: www.dropbox.com/request/8fWpy4IApBI1F83OsFn2

You have an XML of this one, but I'll do it again along with one for the library!

[done!]

I examined the project XML closely, loaded it in a new library and did various tests. The FCP undocumented project repair feature located and fixed several problems. See this before/after diff made with Beyond Compare:

www.dropbox.com/s/pi56x1x4i89w05o/AllNoiseFixes.pdf?dl=0

However even after that, it was very sluggish and prone to long "beachballs", esp. when drag/drop of several clips -- even with all effects removed. I then used the XCode Instruments tool to capture and examine the execution profile during a beachball period when I dragged/dropped some timeline clips.

It showed that only the UI thread in FCP was active, and that thread was deep within TLKit (TimeLine Kit), a framework private to FCP. The beachball happened because the UI thread (responsible for updating the UI) was stuck in a convoluted call path fetching and checking "Layout" metadata associated with the timeline. I don't know why it takes so long. Here is an XCode Instruments trace I took during a beachball period when doing drag/drop of several clips on the timeline:

photos.smugmug.com/photos/i-3sbjMQV/0/0c.../X4/i-3sbjMQV-X4.jpg

Dave and I recently studied a slowdown associated with multiple snapshots on timelines containing many compound clips. That produced a unique signature in the SQL tables (millions of rows, an extremely high number of parent/child references, and a spindump of the hung process showed it was stuck in the SQLite thread.

I examined your SQL tables for the "All Noise" project and did not see any of that. This implies this slowdown has another cause.

I then successively trimmed down the timeline, and at each point it got faster. The initial 1 hr 29 min timeline contained 4,659 items and 3,498 clips. You have more objects in that one timeline than many people have in their entire library. But is *that* the reason it's slow? If so that implies if it were smaller and simpler it would be lightning fast. Yet that is *not* what I observed.

It got faster but it still had anomalous behavior. E.g, even when trimmed to 11 minutes, 565 total objects and 562 clips, no Fx and no transitions, it would still beachball for 5 sec when drag/dropping a group of clips.

Pursuing this further will take time. I've got to work on some other items this weekend, but I should be able to resume work on this sometime next week.

@joema I appreciate this very much. It's an archive based doco project, with loads of montages, hence the complexity. Even if the outcome was a workaround in which the project is split into 3 or even 4 components for ease of use, this would be, comparatively speaking, a godsend - but I can see that this is likely not the solution at this stage. I'll stay tuned....

@joema
Not that I expect anything as you're simply being generous here, but wanted to check if any progress?
It's diabolical here... I've been waiting 2 minutes for a *clip to be deleted from the timeline*.....haha!
Good times...
PS a fun associated problem is random clips turning black in my timeline. I had previously thought this was due to crashes, but it's not. It keeps happening every time I do work on the project. When I see them, I have to take a deep breath and the delete all render files, all proxy / optimal media from the project AND the library, wait for THAT to be done, QUIT the application and then start all over again. In THAT session the clips are no longer black, but after I stop work and come back to it later (with a safe, functional quit in between), I have to go through this process all over again.
The other oddity is that often the dreaded beachball will only stop beachballing when I cross over to Brave browser (essentially Chrome) and hit play then pause on a youtube video. I have tried dealing with this process with no browsers running and the beachball just hangs for minutes at a time. *With* that browser running and youtube play / pause, the beachball usually is triggered out of activity. Damned weird.
It's so awfully corrupt and unreliable.

@ Slaughter
From some of your other posts, you mentioned having tried to use Chrome to upload an XML to @fc_fox's webapp. If so, you need to remove Chrome (see chromeisbad.com). It very likely, or almost certainly, does all sorts of bad things. FYI, Chromium, Brave, and others are not "essentially Chrome". The other apps are based on Chromium, more so, and don't have the "nasties" that come with Chrome.

Have you tried using a brand-new user account and doing some tests there? Or, even better, install macOS and only FCP on an external SSD (newly formatted) and test things there.

It is likely you have some assorted cache and other temporary files associated with FCP that could be corrupted/bad. This stuff doesn't go away when you delete preferences or re-install FCP. Bad HW (mouse, USB hub, cables, keyboard) could possibly be contributing to your issues, as well as 3rd party applications and tools.

Only a small number of other people have reported issues "similar" to yours. If this stuff was widespread, this and other forums would be flooded with posts.

Thanks.

> having tried to use Chrome to upload an XML to @fc_fox's webapp

The problem has since been established to be the size of the XML file and the script / upload process not liking that. The script has been successfully read by that process. It wasn't a browser issue.

Since reading your post I have completely uninstalled Chrome (not that I used it much - it was basically a backup / test browser). It has made absolutely no difference to how this project jams up. Most strangely, there is a definite function that Brave and youtube pages serve. I literally can't do anything in this project unless I run Brave with a youtube window and hit play/pause in youtube to unjam the beachball. What the?

> Have you tried using a brand-new user account and doing some tests there? Or, even better, install macOS and only FCP on an external SSD (newly formatted) and test things there.

No. Because this project is literally the only project this beachball and black clips thing happens with. Therefore the problem is this project, not my system or its interaction with FCPX. If they were the problem, why does absolutely no other project, including projects within the exact same library on the exact same SSD, behave similarly? Process of exclusion. @joema has also established that the XML for this project is demonstrative of various as yet unsolved oddities. So I doubt very, very, very much that spending hours recreating mac os on a brand new external drive etc etc will solve anything.

> It is likely you have some assorted cache and other temporary files associated with FCP that could be corrupted/bad. This stuff doesn't go away when you delete preferences or re-install FCP. Bad HW (mouse, USB hub, cables, keyboard) could possibly be contributing to your issues, as well as 3rd party applications and tools.

Hmmmm the evidence, again, is no. If it were so, then why is this the only project which behaves in this way - essentially, 99.9% of my projects are just fine and dandy with my version of FCP, OS, mouse, cables, keyboard etc. The one, single, solitary instance of this beachball problem exists in this project alone. So it can't logically be any of those things.

> If this stuff was widespread, this and other forums would be flooded with posts.

Agreed. Unlike the party time joy of FCPX randomly deciding to create "untitled" libraries and messing up clip view settings in the browser at startup (which does not resolve even after trashing prefs or loading a saved workspace), this specific - *project* - specific beachballing and black clip issue - is an exclusive one. @joema has been super helpful in starting to isolate the problem as existing deep in the XML of this project and hopefully more comes to light which I can address.

Slaughter wrote:

@joema
Not that I expect anything as you're simply being generous here, but wanted to check if any progress?...

I had to edit a video the past few days, hopefully I can look at your issue tomorrow.

Slaughter wrote:

Thanks.

> If this stuff was widespread, this and other forums would be flooded with posts.

Agreed. Unlike the party time joy of FCPX randomly deciding to create "untitled" libraries and messing up clip view settings in the browser at startup (which does not resolve even after trashing prefs or loading a saved workspace), this specific - *project* - specific beachballing and black clip issue - is an exclusive one. @joema has been super helpful in starting to isolate the problem as existing deep in the XML of this project and hopefully more comes to light which I can address.

Most of my comments probably better apply to your issue with clip view settings and UI quirkiness.


---


However, in terms of this particular Library or Project, there are several things that could be affecting the behavior, including "less effective" use of Project snapshots, Compound clips, etc. Certain plugins or 3rd party plugins (or applications, such as Chrome) may also have an effect. Some or all of this has been mentioned by Joe.

To be clear, various 3rd-party applications and tools could be adversely affecting how well FCP works. This is applicable to your larger Project precisely because macOS "slowdowns" or bottlenecks caused by these other things may only become relevant when you have a large Project in FCP. The amount of RAM in your system could be a factor. Having Time Machine or Spotlight trying to do things where previously connected drives are now missing can gobble up tons of system resources. Various background processes can cause temporary "stalling" in macOS UI behavior that is greatly magnified for larger or more complex Projects in FCP.

In other words, just because your other projects don't experience this kind of drop in performance is possibly not relevant. If this current project is significantly larger, or more complex, than most or all of the other projects you mention, then other issues mentioned above _are_ relevant.

There are things you could do yourself to further try to identify potential issues. This would then tend to suggest using a "clean setup" to do testing, in order to avoid possible contribution to the issue by things besides FCP and macOS.

Did you delete FCP's preferences and reinstall FCP after removing Chrome? Since Chrome and things like "CleanMyMac" can "destroy" parts of macOS, it's often useful to reinstall macOS after removing Chrome.

There are other possible things that may only be triggered by larger Projects in FCP, though I've worked on a handful of 2 hour plus long Projects without significant slowdowns. I also only use ProRes and ProRes proxies as FCP is most efficient using the ProRes codec (and what the codec was made for, at least originally).

If this is a serious problem, you might consider finding a local workflow FCP expert to analyze things in more detail, particularly if this is a paid gig. Analyzing a lone XML file is better than nothing, but it _is_ a limited way to try to figure out an issue. XML files don't capture the full extent of what's in an actual Library, for example, nor can they account for other factors.

Just some additional thoughts...

Cheers.

DaveM wrote:

... various 3rd-party applications and tools could be adversely affecting how well FCP works. This is applicable to your larger Project precisely because macOS "slowdowns" or bottlenecks caused by these other things may only become relevant when you have a large Project in FCP..

Those items have caused many problems for lots of customers, but when I load his project (minus any data) it is also slow on my machine. That makes me think it's something unique to the project, not something external.

DaveM wrote:

...In other words, just because your other projects don't experience this kind of drop in performance is possibly not relevant. If this current project is significantly larger, or more complex, than most or all of the other projects you mention, then other issues mentioned above _are_ relevant...

Excellent point. I'd be interested if he could roughly assess the size and complexity of the other projects which don't experience the problem. When he says "only this project has the problem", is it literally that? Or is it "this project and all other versions of it?" E.g, I'm looking at his library, and I see 21 projects, some quite small, and some very large and complex.

It is only one literal project (named "All Noises") that has the problem after the FCP update, or is "project" a loose term which means all versions of the "All Noises" project? Some of those have visual differences but they are large and complex. Knowing this specifics will help steer our troubleshooting.

In very general terms, his "AllNoises" project superficially has similar characteristics to other "slow" projects I've examined, including some from TangierC. It is quite large (about 90 min) with lots of connected clips. Of course, nothing wrong with that per se.

The library I have contains 5,803 items and 21 projects. Breakdown by type: 2907 video clips, 1372 compound clips, 253 audio clips, and 2624 stills. Of those 2624 stills, 2435 are JPG, 180 are Photoshop PSD, 7 are TIFF.

The "All Noises 4.0" project is 1 hr 26 min long, 1080p/29.97. It contains 4678 items, 3499 video clips, 950 secondary storylines, 1136 cross-dissolves, 436 compound clips, and 17 audio clips.

I need to review everything done so far and get a fresh look at it. I will post any info here. Dave if you have time, feel free to help or comment on any of the above. As soon as I re-verify the best representative library and project, I'll post a link to that.

Slaughter, if you could clarify the above items that would help. If any of this is redundant to answers you've already given, sorry about that.