BTW Apple has a command-line video quality assessment tool called AVQT. It is freely available but currently it only checks for compression and scaling artifacts, not dropped or duplicated frames or cadence issues. I have requested they enhance this in future versions to handle those cases, but even if they do it would take a while.
FCP 10.6.2 and 10.6.3 may produce stutter in render results
"The latest versions of FCP 10.6.2 and 10.6.3 may produce stutter (uneven movement of scene/objects) in rendered/exported video when a temporal video effect (such as Neat Video) is applied. The problem seems to originate in FCP itself, as it serves incorrect frames (coming in an incorrect order) to the video effect. Then this incorrect order of frames is visible in the final results as stuttering movement of the scene/objects. We are in contact with Apple to investigate the issue from both sides and find a solution, which will likely require an update to FCP itself."
When using Neat Video 5.5.2, I am seeing the same problem you reported, including both dropped/duplicate and mis-ordered frames. This problem is apparently not unique to Neat Video but so far I can only reproduce it using that plugin. You are not apparently using that so it shows it's a broader issue.
In my case I used 4k/23.98 8-bit ProRes 4:2:2 captured from a Sony A7RIII to an Atomos Ninja V. All media is on the internal Mac Studio SSD. It happens when exporting to ProRes 422 but is worse when exporting using the Computer, Better Quality H264 preset.
As you initially reported it seems somewhat random and does not happen (as much or at all) when render dots show (IOW, clip contains Fx but is unrendered in the timeline). It happens when rendering to cache with CTRL+R or exporting the clip (rendered or unrendered).
I tried the same clip using temporal noise reduction in Resolve Studio 17.4.6, exporting to both ProRes 422 and H264 and it doesn't happen.
I will try to find a replication scenario that doesn't use Neat Video, also will test Neat Video on the M1 Max using the same clip and export settings to see if there's a difference in problem frequency.
The Neat Video posting says the problem happens when using a temporal effect, IOW Fx which examine a multi-frame window. Besides Neat Video that would include things like Digital Anarchy's Flicker Free, Imagenomic Portraiture, Optical Flow smoothing, etc. It might also include some common transitions.
In the Neat Video forum, a couple of people mentioned "freeze frames" rendering out, which would explain the weird cadence in duplicate frames seen in Redifer's test clip (as a separate issue from the frame re-ordering).
A workaround is to go back to FCP 10.6.1, and if you're using Neat Video, to use the standard version of the plugin instead of the "SR" version for Intel Macs.
It would be nice to have a "test case" we could use to test things when FCP 10.6.4 comes out (so most of us can wait to see if things are fixed)...
Thanks joema for taking time to help figure this out!
The problem is apparently not limited to Neat Video. I don't think Redifer is using it, but would like confirmation of that.
I wonder if the problem has been seen on FCP 10.6.2 and 10.6.3 on *all* Apple Silicon machines, or only/mainly the M1 Max and Ultra. If the problem is related to multi-thread synchronization across the multiple accelerators, it would be more common on M1 Ultra, less on M1 Max and maybe wouldn't happen on M1.
If we could turn off the accelerators that would be a good test. The MacOS Video ToolBox framework allows turning it off but that is not exposed in the FCP product.
It might be possible to test that by using uncompressed 10-bit 4:2:2 for input and output codecs, plus setting the project render properties to uncompressed 10-bit 4:2:2. I will try that tomorrow.
I do not have Neat Video, however I think Neat has stumbled onto something within FCP that triggers this. I think if Apple fixes FCP so Neat works fine, then it shouldn't give me any issues. Hopefully Apple doesn't just tell Neat that it's their problem to fix because I don't believe that's the case here.
I do believe it is likely a ProRes acceleration issue going by the limited amount of info we have now. It only ever happens to my ProRes clips.
I do not have Neat Video, however I think Neat has stumbled onto something within FCP that triggers this. I think if Apple fixes FCP so Neat works fine, then it shouldn't give me any issues...
You are absolutely correct -- it's not isolated to Neat Video. I can now reproduce it without Neat Video, simply using the built-in Cube transition into a retimed clip -- as you previously described.
This is very serious -- not only does it cause dropped/duplicated frames, some in the wrong order, it is also causing "torn" frames. See attached. I will do more testing to isolate whether it's only M1 Ultra or also M1 Max (I have both), then get someone from my team with the regular M1 to run a test. Then I'll file a bug and follow up with a phone call to Pro Apps support.
joema, have you tried this on an Intel Mac? From the Neat Video forum, and their known issues post, it isn't totally clear on whether this can happen on Intel Macs, too...I have a 2019 Mac Pro with an AfterBurner card, so I could do a test if you want (and you don't have that configuration to test yourself). Glad to help...
Good point -- based on current reports on this forum I tentatively assumed it was unique to Apple Silicon, but we don't know that. I have a 2019 i9 MacBook Pro 16 on Monterey 12.4 and a 2017 i7 iMac 27 on Catalina, so I can test that later today. Hopefully by then I may have a more self-contained scenario I could forward to others and avoid wasting their time trying to reproduce it.
However it's not that hard to reproduce. Just take an approx. 1 minute or 2 minute 4k/23.98 or 4k/29.97 ProRes 422 clip, cut it in about 4 to 6 segments, retime each one to varying degrees, e.g, 80%, 150%, etc, and put the built-in "cube" transition between each one. Then export it using the Computer, 4k H264, Better Quality preset. Likely other presets will also work but that's what I've used so far. Play it in Quicktime at 1x and look for any jump or hitch in the playback. After seeing that, use JKL commands to single-step forward and backward over that region to examine more closely.
If you can't reproduce it put the ProRes clip in a custom project at double the frame rate, e.g, 59.94 fps for a 29.97 fps clip and also do the above steps. That is how Redifer first noticed it.
Thanks for the offer about testing with the Mac Pro and Afterburner card. The AB card only accelerates ProRes decoding, not encoding, but maybe that is a factor. If this also happens on Intel that is even worse.
Thanks Joe - I tried this test with my M1 Mini at 23.98 and 59.94, with both output files coming through without reordering or tearing.
Note that my test 4K 23.98 ProRes clip was not a camera file, but instead a client deliverable composited in AE and finished in FCPX.
There was one anomaly though. The browser persisted in displaying 23.98 as the project frame rate even after I changed it, while the inspector's Info pane displayed correctly, and the output file is 59.94 as expected.
EDIT: I moved the test over to an M1 Max MBP, with the same result - no tearing or reordering. The browser now shows the correct frame rate on both machines. Also forgot to note earlier that I applied Optical Flow to every other edit in all cases.
...I moved the test over to an M1 Max MBP, with the same result - no tearing or reordering....
Thanks for doing that. The original clips I used had been transcoded to ProRes from BRAW, I need to study that. Besides Neat Video, there is some other "x" factor or combination of factors that's causing it. I'll continue working on this as a high priority item. Even if it only happens 0.1% of the time, I'm concerned my team's ongoing post production, transcoding, etc, might be subtly poisoning the downstream data.
For casual viewers it's hard to detect during streaming playback since there are often dropped frames. Even for those looking at playback of a local file, if it has been retimed or rate conformed, there may be an expected uneven frame cadence, but any duplicated frames should follow a regular, unchanging pattern. The clip Redifer posted was severe and I've seen it that bad (or worse) if using Neat Video. However for milder cases it's difficult pick out the truly erroneous cases from the normal frame cadence variation and little micro-lags that systems exhibit.
...The ProRes clip that got jittery in this one has a Cube transition into a retimed clip, though optical flow is not applied to that particular one. Audio is not attached to either clip...
Can you give me any additional details on this aspect? Was it an Atomos 4k/29.97 ProRes clip in a 59.94 1080p timeline? What were the specifics on the retimed clip, was 80%, 120% or what?
I am struggling to achieve a reliable replication on normal clips without using Neat Video. As you reported it seems intermittent. The cube transition and retiming makes it more likely but it's still elusive.
Do you have any idea about the settings on the Panasonic camera? Are there any remaining in-camera clips you could examine for frame rate, shutter speed, etc. Can you send me a Panasonic camera clip from that sequence, even short one is OK. I just need to check all the metadata. The ProRes transcoded BRAW clips I have make it happen frequently and that camera was set to 23.98 frame rate, 1/60th shutter but 50 fps sensor scan rate, then the clips were retimed in Resolve to 199.80% to sync with the audio.
The clip was recorded on an Atomos Ninja V at 29.97 4K ProRes 422HQ. I don't believe camera settings make any difference as the Atomos doesn't record metadata. But the shutter speed was either 1/30 or 1/60. Nothing lower, higher, or inbetween. This was placed into a 1920x1080 59.94fps timeline. A cube transition of 46 frames was used (nothing changed in the transition's settings other than the duration). It transitioned into an HEVC clip of a videogame title screen that I had slowed to 4%. There is no movement in this title screen, but it doesn't stay up very long, so I just stretched it willy-nilly.
The ProRes 422HQ clip was edited to remove some dead space where my hand is off screen. These where then combined into a compound clip. I then used the effect that Ken Burns himself coded into FCP with his own hands to have a gentle push into the scene that wouldn't be disrupted by the edit. I then applied a Color Board, Color Curves and Hue/Saturation Curves to the compound clip (I generally don't use LUTs as they put me further away from my desired starting point than I'd like, and the VLOG LUT sucks).
I got the jitters upon my first export to h.264 for eventual upload to Youtube. They were gone on my second export and don't seem to be showing up any more now. I never render before export.
The clip was recorded on an Atomos Ninja V at 29.97 4K ProRes 422HQ....This was placed into a 1920x1080 59.94fps timeline. A cube transition of 46 frames was used (nothing changed in the transition's settings other than the duration). It transitioned into an HEVC clip of a videogame title screen that I had slowed to 4%. There is no movement in this title screen, but it doesn't stay up very long, so I just stretched it willy-nilly...
Thank you *very* much for all that detail. I'm on a shoot this weekend but will get back to this investigation early next week. Neat Video support thinks it is not isolated to Apple Silicon; they've seen it on a Xeon Mac (either Mac Pro or iMac Pro). On the transcoded ProRes I have from BRAW I can easily reproduce it using Hawaiki Keyer, no Neat Video. There is a theoretical downstream QC risk for FCP (or maybe Compressor) workflows. So there are two issues: (1) Spotting and avoiding the problem on current tasks and (2) Checking previously transcoded and exported material since 10.6.2 which might have been compromised.
Since many manifestations are mild (an occasional dropped/duplicated frame) it is very difficult and tedious to spot. The major broadcasters have sophisticated QC validation tools for submitted and distributed media some of which look for duplicated/delete frames and cadence errors, but those are either fully custom or very expensive. Reliable detection of duplicated frames is an ongoing area of academic research so it's an inherently difficult task.
I haven't seen it mentioned so I thought I would throw it out there. Do you have "Background render" turned on in your playback preferences?
I was experiencing a lot of similar problems with a complex motion template that I created. I found that Final Cut's caching is prone to getting messed up, and causing out of order frames as described. It usually happened when I was moving stuff that Final Cut didn't completely like around in the timeline. I think it was trying to use previously cached stuff that was now in a different location, and not doing it very well. It was a pain to fix too, because it was hard to purge the misrendered stuff. I actually had to go into the library bundle and manually delete cached things. (after making a backup). "Delete Generated Clip Files" didn't fix it for me. A good way to check if all caching on a timeline is purged is to look for the "unrendered dots" at the top of the timeline. If you see the dots, you know you have gotten rid of the cached stuff.
The good news for me, is that once I disabled the default "background render" I found that Final Cut is amazing at playing back complex multilayered comps un-rendered. I now want to see those little "unrendered" dots to be appearing at the top of my timelines, because when they don't, I often run into problems. With this strategy, I will sometimes run into certain clips that don't want to play back smoothly. When I hit one of those, I use "Final Cut's "Transcode Media" feature to convert it to pro-res and it then plays back fine (even when moved and retrimmed in the timeline. Good luck!
I haven't seen it mentioned so I thought I would throw it out there. Do you have "Background render" turned on in your playback preferences?
I was experiencing a lot of similar problems with a complex motion template that I created. I found that Final Cut's caching is prone to getting messed up, and causing out of order frames as described....
Thanks for that suggestion, interesting you experienced out of order frames. Deleting the render cache (maybe even manually as you described) and turning off background rendering is a good troubleshooting approach.
However for the OP's problem (which I can reproduce to a degree) it's likely not corrupt render cache. It can happen even if nothing is rendered and you simply export.
Now I'm starting to wonder if this issue extends beyond Final Cut Pro and into the Mac OS itself. Sometimes I use Quicktime to capture video from a Magewell USB Capture Gen 2. I capture in ProRes 422. I have had a few issues with the resulting captures, but it's been rare. The issues are the same jittery playback of the captures. Of course once it's been captured that way, there's no fixing it and the video must be re-captured.
....starting to wonder if this issue extends beyond Final Cut Pro and into the Mac OS itself. Sometimes I use Quicktime to capture video from a Magewell USB Capture Gen 2. I capture in ProRes 422. I have had a few issues with the resulting captures, but it's been rare. The issues are the same jittery playback of the captures...
I've reproduced it a few times but not the bad manifestation, just several dropped/repeated frames. It is very elusive. I tried the items you mentioned, Ken Burns, HEVC, 46-frame cube transition, compound clip, etc. So far that hasn't made it happen regularly, but that's not unexpected since you didn't change anything yet are not currently seeing it. It makes sense to try things like that.
I also have the same Magewell card, use it frequently for live Zoom calls on my M1 Ultra, but not for ProRes capture. With streaming live video there are so many sources of lag I'm not sure I'd recognize this unique problem it, even if it happened.
There is clearly a real problem here and it's not isolated to Neat Video. It's just very difficult to pin down. I will keep working on it. Neat Video suspects it may not be an FCP issue per se (even though it apparently started in 10.6.2) but in MacOS. There were definitely some performance-related changes in 10.6.2 and possibly some corresponding changes in MacOS, because Apple gave various reviewers pre-release versions of those for Mac Studio testing. In software development it is easy to adversely impact reliability when trying to improve performance.
I'm seeing a possibly unrelated behavior whereby FCP alters the cadence of a retimed clip if built-in noise reduction is applied. E.g, 23.98 clip in a 59.94 timeline and retimed using "automatic speed". Without NR, both FCP playback and (for the ProRes 422 exported clip) Quicktime and VLC play each frame, as expected, IOW cadence is 1-1-1-1.
However if the built-in FCP video noise reduction effect is applied, it changes frame cadence to 3-2-3-2, IOW each frame is duplicated 3 times or 2 times. That is the expected cadence for 3/2 pulldown when 23.98 fps material is retimed for 59.94 distribution. However there is no pulldown for "automatic speed", yet somehow the FCP NR effect is misinterpreting that. This is not unique to 10.6.3 on Apple Silicon, it also happens with 10.5.2 on Intel and Catalina. Neat Video on FCP 10.6.3 does not do this and Resolve Studio 17.4.6 built-in temporal/spatial noise reduction does not do that on the same clips and retiming scenario.
Since the "Jittery playback" problem that began with 10.6.2 happens more easily with Neat Video I was trying to reproduce it with the built-in FCP video NR, but that uncovered the unfixed bug from 2018. Now I need to file that with Apple before continuing with the main jitter problem.