Alex Gollner takes a look at Apple's recently granted patent that details how multiple editors could work on the same timeline. He's done the detective work discovering what might become true collaborative editing.

 

On July 1, 2014 Apple was awarded a video editing patent that they applied for in August 2009. This patent gives a rare insight into Apple’s corporate thinking two years before Final Cut Pro X was launched. Although Apple will have moved on, it influenced some Final Cut Pro X features and hints at a future direction for Apple’s video editing applications.

The patent (US no. 8,769,421 “Graphical user interface for a media-editing application with a segmented timeline) adds organisational structure to edits. As well as dealing with how video and audio clips are combined at the level of individual scenes, Apple suggests it would be useful for software to enable editors to consider productions at the act, episode and series levels. 

The patent is illustrated using drawings of the Final Cut Pro 7 user interface with an added ‘story outline’ panel reminiscent of Final Cut Pro X’s Timeline Index:

(Click for larger images)

 

 

The patent shows how story segments could be navigated to, created, selected, reordered, merged, deleted or combined into ‘acts’. Segment manipulation could be done by clicking and dragging in both segment headers above the timeline and in the story outline panel. 

The problem that the Magnetic Timeline solves

Most of the 92 page document shows how editing applications will need to deal with edge conditions - how clips at segment boundaries are handled when they need to overlap due to changes at the segment level. Obviously the patent writers don’t believe this is a useful idea if audio or video clips cannot cross segment or act boundaries.

 

An example of a ‘segment boundary problem’ included in the patent is describing the various ways to handle clips clashing with each other in this case: The user moves Segment B so it appears before Segment A. As Clip 2 in Layer 1 extends past the end of Segment B, how should its extra duration interact with Clip 1 in Segment A?

Solutions include trimming Clip 2 so it ends at the Segment B boundary, trimming it so it ends when Clip 1 in Segment A starts, or extending the duration of Segment B to allow for the extra duration of Clip 2. Another option shown is to make it possible for clips to overlay each other in the same layer - a solution reminiscent of the Magnetic Timeline in iMovie and Final Cut Pro X:

 

Productions as databases instead of lists

An earlier version of this patent was submitted in April 2009, which shows that Apple have been considering the problem of moving sections of a timeline a long time before Final Cut Pro X was introduced in 2011. Its implication is that instead of representing stories as one-dimensional timelines (with a few parallel lines represented by tracks), it is better to think of stories stored in two-dimensional hierarchies:

 

Simple databases can’t store the same clip being in two segments. Relational databases can. They can also store multiple representations of the same segment. In the example given in the patent there are three versions of Segment 3:

 

These alternate versions of parts of the timeline have been implemented as Auditions in Final Cut Pro X (although to have multiple versions of more than one clip in an Audition, you need to use compound clips in the Audition).

If structure is encoded into video editing projects, editing applications will be able to support multiuser editing. Individual editors could work at the segment, ‘supersegment’ or project level. The UI could show when a segment has been checked out by another editor - perhaps using a ‘guard’ metaphor. Hidden within earlier versions of Final Cut Pro X was text that referred to guards and story segments:

/* Guards */
“FFGuardConflictDescription” = “One or more of changes are protected by a guard owned by a different user. “;
“FFSetGuardOnTrack” = “Change Guard on track”;
“FFGuardAddUser” = “Add Myself to Guard”;
“FFSetGuardOnSequence” = “Change Guard on sequence”;
“FFSetGuardOnStorySegment” = “Change Guard on story segment”;

Multi-user editing is explicitly mentioned in the patent:

Another use for the versioning feature (and segments generally) is the ability for multiple editors to collaborate on a project. For instance, an editor might want to extract a copy of a segment (Version 1) and send that segment to a second editor. The second editor can then work on the segment (add or remove clips, modify transitions, etc.) in order to create a new version of that segment (Version 2). The editors (or another editor) can then determine which version of the segment to use in the final presentation.

Segment rules as production templates

The patent also explains how structural rules could be set up in different ways dependent on the nature of the production:

Different attributes and functionalities are applied differently to different levels of the hierarchy in some embodiments. In the case of the media-editing application, different attributes and functionalities are applied differently to media clips, segments (e.g., scenes), and supersegments (e.g., acts), and any other levels of hierarchy. For instance, media clips and segments may be displayed in the timeline, while supersegments and any higher levels of hierarchy are only displayed in the outline window. A media clip may be split among two or more segments, while each segment must be fully contained within one supersegment in some embodiments. Various other functionalities and attributes (e.g., how the items may be edited) are applied differently to the different levels in some embodiments as well. 

[…] Segments can also be used to impose timing requirements, such as the time between commercial breaks for a television show. Along this line, some embodiments include libraries of preset groups of segments based on timing, story arc, etc.

A step on the way to video literacy tools

Video editors and multimedia designers have been used to using layered timelines to tell stories for over 25 years. This patent tells us that back in 2009 Apple were considering whether a new organising principle was needed: an early step to make video literacy tools easier to grasp by more people.

Most people think that the editor’s job is ‘to cut out the bad bits’ in individual scenes. Many are surprised to discover that editors commonly change and improve storytelling by changing story structure. As many film and TV makers consider that structure is very important when it comes to telling stories, I think it is a good idea for video editing software to recognise story structure. 

Structure applies to feature films, TV shows, groups of corporate videos on an intranet, legal video depositions, architects’ video proposals or open-ended weekly web series. The more video applications can have these structures encoded in their projects, the better the tools they’ll be able to provide to a wider range of people all over the world.

 

Alex Gollner is a freelance editor who makes plugins for Final Cut Pro X and writes about Final Cut and post-production on his blog at Alex4D. You can also follow Alex on Twitter.