apple segmented-timeline

Alex Gollner takes a look at Apple's recently granted patent that details how multiple editors could work on the same timeline. He's done the detective work discovering what might become true collaborative editing.


On July 1, 2014 Apple was awarded a video editing patent that they applied for in August 2009. This patent gives a rare insight into Apple’s corporate thinking two years before Final Cut Pro X was launched. Although Apple will have moved on, it influenced some Final Cut Pro X features and hints at a future direction for Apple’s video editing applications.

The patent (US no. 8,769,421 “Graphical user interface for a media-editing application with a segmented timeline) adds organisational structure to edits. As well as dealing with how video and audio clips are combined at the level of individual scenes, Apple suggests it would be useful for software to enable editors to consider productions at the act, episode and series levels. 

The patent is illustrated using drawings of the Final Cut Pro 7 user interface with an added ‘story outline’ panel reminiscent of Final Cut Pro X’s Timeline Index:

(Click for larger images)



The patent shows how story segments could be navigated to, created, selected, reordered, merged, deleted or combined into ‘acts’. Segment manipulation could be done by clicking and dragging in both segment headers above the timeline and in the story outline panel. 

The problem that the Magnetic Timeline solves

Most of the 92 page document shows how editing applications will need to deal with edge conditions - how clips at segment boundaries are handled when they need to overlap due to changes at the segment level. Obviously the patent writers don’t believe this is a useful idea if audio or video clips cannot cross segment or act boundaries.



An example of a ‘segment boundary problem’ included in the patent is describing the various ways to handle clips clashing with each other in this case: The user moves Segment B so it appears before Segment A. As Clip 2 in Layer 1 extends past the end of Segment B, how should its extra duration interact with Clip 1 in Segment A?

Solutions include trimming Clip 2 so it ends at the Segment B boundary, trimming it so it ends when Clip 1 in Segment A starts, or extending the duration of Segment B to allow for the extra duration of Clip 2. Another option shown is to make it possible for clips to overlay each other in the same layer - a solution reminiscent of the Magnetic Timeline in iMovie and Final Cut Pro X:



Productions as databases instead of lists

An earlier version of this patent was submitted in April 2009, which shows that Apple have been considering the problem of moving sections of a timeline a long time before Final Cut Pro X was introduced in 2011. Its implication is that instead of representing stories as one-dimensional timelines (with a few parallel lines represented by tracks), it is better to think of stories stored in two-dimensional hierarchies:



Simple databases can’t store the same clip being in two segments. Relational databases can. They can also store multiple representations of the same segment. In the example given in the patent there are three versions of Segment 3:



These alternate versions of parts of the timeline have been implemented as Auditions in Final Cut Pro X (although to have multiple versions of more than one clip in an Audition, you need to use compound clips in the Audition).

If structure is encoded into video editing projects, editing applications will be able to support multiuser editing. Individual editors could work at the segment, ‘supersegment’ or project level. The UI could show when a segment has been checked out by another editor - perhaps using a ‘guard’ metaphor. Hidden within earlier versions of Final Cut Pro X was text that referred to guards and story segments:

/* Guards */
“FFGuardConflictDescription” = “One or more of changes are protected by a guard owned by a different user. “;
“FFSetGuardOnTrack” = “Change Guard on track”;
“FFGuardAddUser” = “Add Myself to Guard”;
“FFSetGuardOnSequence” = “Change Guard on sequence”;
“FFSetGuardOnStorySegment” = “Change Guard on story segment”;

Multi-user editing is explicitly mentioned in the patent:

Another use for the versioning feature (and segments generally) is the ability for multiple editors to collaborate on a project. For instance, an editor might want to extract a copy of a segment (Version 1) and send that segment to a second editor. The second editor can then work on the segment (add or remove clips, modify transitions, etc.) in order to create a new version of that segment (Version 2). The editors (or another editor) can then determine which version of the segment to use in the final presentation.

Segment rules as production templates

The patent also explains how structural rules could be set up in different ways dependent on the nature of the production:

Different attributes and functionalities are applied differently to different levels of the hierarchy in some embodiments. In the case of the media-editing application, different attributes and functionalities are applied differently to media clips, segments (e.g., scenes), and supersegments (e.g., acts), and any other levels of hierarchy. For instance, media clips and segments may be displayed in the timeline, while supersegments and any higher levels of hierarchy are only displayed in the outline window. A media clip may be split among two or more segments, while each segment must be fully contained within one supersegment in some embodiments. Various other functionalities and attributes (e.g., how the items may be edited) are applied differently to the different levels in some embodiments as well. 

[…] Segments can also be used to impose timing requirements, such as the time between commercial breaks for a television show. Along this line, some embodiments include libraries of preset groups of segments based on timing, story arc, etc.

A step on the way to video literacy tools

Video editors and multimedia designers have been used to using layered timelines to tell stories for over 25 years. This patent tells us that back in 2009 Apple were considering whether a new organising principle was needed: an early step to make video literacy tools easier to grasp by more people.

Most people think that the editor’s job is ‘to cut out the bad bits’ in individual scenes. Many are surprised to discover that editors commonly change and improve storytelling by changing story structure. As many film and TV makers consider that structure is very important when it comes to telling stories, I think it is a good idea for video editing software to recognise story structure. 

Structure applies to feature films, TV shows, groups of corporate videos on an intranet, legal video depositions, architects’ video proposals or open-ended weekly web series. The more video applications can have these structures encoded in their projects, the better the tools they’ll be able to provide to a wider range of people all over the world.


alex gollnerAlex Gollner is a freelance editor who makes plugins for Final Cut Pro X and writes about Final Cut and post-production on his blog at Alex4D. You can also follow Alex on Twitter.







Written by
Top BloggerThought Leader

I am the Editor-in-Chief of FCP.co and have run the website since its inception ten years ago.

I have also worked as a broadcast and corporate editor for over 30 years, starting on one inch tape, working through many formats, right up to today's NLEs.

Under the name Idustrial Revolution, I have written and sold plugins for Final Cut Pro for 13 years.

I was made a Freeman of Lichfield through The Worshipful Company of Smiths (established 1601). Though I haven't yet tried to herd a flock of sheep through the city centre!

Current Editing

great house giveaway 2020

2020 has been busy, the beginning of the year was finishing off a new property series (cut on FCP) for Channel 4 called The Great House Giveaway. I also designed and built the majority of the graphics as Motion templates. It has been a great success and the shows grabbed more viewers in the 4pm weekday slot than any previous strand. It has been recommissioned by C4 for 60 episodes, including prime-time versions and five themed programmes. The shows have also been nominated for a 2021 BAFTA.

Tour de france 2020
Although both were postponed to later in the year, I worked again on ITV's coverage of the Tour de France and La Vuelta. 2020 was my 25th year of editing the TdF and my 20th year as lead editor. The Tour was the first broadcast show to adopt FCPX working for multiple editors on shared storage.


BBC snooker the crucible

BBC's Snooker has played a big part in my life, I've been editing tournament coverage since 1997. I'm proud to be part of a very creative team that has pioneered many new ideas and workflows that are now industry standard in sports' production. This is currently an Adobe Premiere edit.

amazon kindle BF

Covid cancelled some of the regular corporate events that I edit such as trade shows & events. I was lucky however to edit, from home, on projects for Amazon Kindle, Amazon Black Friday, Mastercard and very proud to have helped local charitable trust Kendall & Wall secure lottery funding.

As for software, my weapon of choice is Final Cut Pro and Motion, but I also have a good knowledge and broadcast credits with Adobe Premiere Pro, MOGRT design and Photoshop.

Plugin Design & Development

I'm the creative force behind Idustrial Revolution, one of the oldest Final Cut Pro plugin developers. It hosts a range of commercial and free plugins on the site. One free plugin was downloaded over a thousand times within 24 hours of release.

I also take on custom work, whether it is adapting an existing plugin for a special use or designing new plugins for clients from scratch. Having a good knowledge of editing allows me to build-in flexibility and more importantly, usability.


Now in its 10th year and 4th redesign, running FCP.co has given me knowledge on how to run a large CMS- you are currently reading my bio from the database! Although it sounds corny, I am pretty well up on social media trends & techniques, especially in the video sector. The recent Covid restrictions has enabled live FCP.co shows online. This involves managing a Zoom Webinar through Restream.io to YouTube and Facebook. 

The Future

I'm always open to new ideas and opportunities, so please get in touch at editor (at) fcp.co. I've judged film competitions, presented workflow techniques to international audiences and come up with ideas for TV shows and software programs!


Log in to comment