TOPIC:

This bug became apparent while investigating the "Jitter / frames out of order" problem described by Redifer in fcp.co/forum/4-final-cut-pro-x-fcpx/3687...-frames-out-of-order

Background: the Jitter/Out of Order problem can be difficult to reproduce but happens readily with Neat Video. To avoid any misconception by Apple it's a plugin bug, I tried to reproduce it with the built-in video noise reduction affect which led to a superficially similar behavior. It is almost certainly unrelated since this one has existed since 10.5.2 on Catalina. However (like all bugs that produce incorrect results) it is potentially serious if it evades detection by QC and is distributed or (worse) saved as the only archival version of an encoded project.

Title: FCP video noise reduction produces incorrect frame cadence on retimed rate conformed clip

Filed 7-7-22 using MacOS Feedback Assistant app, bug #FB10578973.
This was also reported on 7-6-22 as Apple Support Case 101743180767.

If the FCP video noise reduction (NR) effect is applied to a retimed clip which has been rate conformed, it can produce incorrect frame cadence. E.g, a 29.97 clip retimed using “automatic speed” (80% in this case) in a 23.98 project should have 1:1 cadence. IOW frames ABCDEFG should play in order ABCDEFG. However if NR is applied, the frame order becomes ACDDEGHH, with some frames erroneously dropped and other frames duplicated.

The NR effect produces numerous different cadence errors which vary depending on the clip frame rate, project frame rate and retiming amount. This problem has existed at least since 10.5.2 and continues through 10.6.3 on both Intel and Apple Silicon. It does not seem related to codec but I tested mostly on ProRes 422. The timeline should be rendered to cache with and without NR to produce consistent behavior. If the timeline is not rendered, in the problem cases the frame cadence may unpredictably change during playback.

It does not happen from only rate conforming or only retiming. The clip must be rate conformed and retimed. It only happens with some combinations of clip frame rates and project frame rates if the clips are retimed and NR is applied. It does not seem sensitive to amount of retiming. For a given scenario, behavior is consistent and 100% reproducible, not intermittent (assuming a cached timeline).

Each combination of clip/project frame rates and retiming is expected to produce a specific cadence with varying perceptual smoothness. In some problem cases when that proper cadence is incorrectly altered by NR, it may be difficult to visually detect. In other cases it is very apparent. I list below some of the test scenarios which show the problem. For brevity “24” means 23.98, “30” means 29.97, and “60” means 59.94. I used alphabetic frame notation, e.g, frames 1,2,3,4 are denoted as ABCD. This notation provides clearer representation.

- 30 in 24, rate conformed but not retimed: OK with NR applied.
- 30 in 24, rate conformed, retimed to 80%, no NR: plays smoothly with 1:1 cadence; frames ABCDE play as ABCDE.
- 30 in 24, rate conformed, retimed to 80%, NR: plays roughly; ABCDEFGH becomes ACDDEGHH.
- 30 in 24, rate conformed, retimed to 90%, no NR: ABCDEFGHIJK becomes ABCDEFGHJ (expected cadence).
- 30 in 24, rate conformed, retimed to 90%, NR: ABCDEFGHI becomes ABDDEFIIJ (erroneous cadence).

- 24 in 30, auto speed no NR: expected 1:1 cadence, ABCDE remains ABCDE.
- 24 in 30, auto speed, NR: ABCDEFGHIJK becomes ABCDDFHIJJL (Verified also on Intel 10.5.2 and Catalina 10.15.7).
- 24 in 30, retimed to 180%, no NR: ABCDEFGHIJKL becomes ABDEGHIKLN (expected cadence).
- 24 in 30, retimed to 180%, NR: ABCDEFGHIJKL becomes ACEFFIJLN (erroneous cadence).

- 60 in 24: All tested retiming and NR combinations worked OK for 60 in 24.

- 30 in 60, automatic speed, no NR: expected 1:1 cadence, ABCDE remains ABCDE.
- 30 in 60, automatic speed, NR: cadence becomes 2:2, ABCDE becomes AABBCCDD (Verified also on Intel 10.5.2 and Catalina 10.15.7).
- 30 in 60, retimed to 150%, no NR: ABCDEFGHIJK becomes ABCCDEFFGHII (expected cadence).
- 30 in 60, retimed to 150%, NR: ABCDEFGHIJKLMN becomes AACCDDFFGGI (erroneous cadence).

- 60 in 30: All tested retiming and NR combinations worked OK for 60 in 30.

TO REPRODUCE:

1- Configure FCP Preferences>Playback>Background Render to OFF.
2- Import to FCP any 1080p 23.98 ProRes 422 clip which displays smooth, consistent horizontal panning.
3- Create a custom 1080p project with 29.97 frame rate.
4- Add clip to project.
5- Retime clip to 80% using Modify>Retime>Automatic Speed.
6- Add built-in timecode effect to clip.
7- Modify timecode effect to display frames, not HMSF. This makes it easier to see cadence behavior.
8- Render clip to cache with CTRL+R
9- Apply build-in FCP video noise reduction effect to clip.
10- Render to cache with CTRL+R.
11- Uncheck NR effect and play clip to observe smoothness and frame cadence.
12- Enable NR effect and play clip. The playback will be rough due to cadence errors.
13- To examine other problematic cases, create additional projects and use similar 29.97 and 59.94 clips.

Thank you very much joema for the in-depth testing and reporting.