FCPX shines in handling 62TB of RED Epic Dragon 4K RAW footage, 10TB of BMCC ProRes footage, 1,2TB of external audio clips and over 34.000 exported files for a groundbreaking 3D augmented reality project. Ronny Courtens looks at The Forever Project, which ended up being post produced on Final Cut Pro X.
In 2014, after extensive testing of several options, Pollen Studio in the UK decided to trust FCP X on a massive 2-year project that involved native handling and fast throughput of dozens of Terabytes of raw 4K 3D media and tens of thousands of clips to be synced, edited and exported against strict deadlines. Two years later they haven’t regretted their choice for one bit.
This article shows the speed and stability of FCP X combined with the new MacPro in a complex post production workflow. But it’s not only about FCP X. Before anything else, this story will take you behind the scenes of an impressive and very worthy virtual reality project that is unique in the world. I hope you will enjoy the read.
For decades now, survivors of the Holocaust have shared their stories with tens of thousands of children and adults each year in the National Holocaust Centre and Museum in the UK, so that people will never forget one of the darkest moments in the history of mankind.
The reputed British national newspaper The Guardian wrote: “Listening to and meeting a Holocaust survivor in person has a profound impact on young people: how they think of others, how communities treat people, how they understand history.”
Unfortunately there are not many survivors alive anymore. In an effort to preserve this unique experience for the future, the National Holocaust Museum approached Bright White Ltd - an award-winning creative agency in the UK specialized in interactive digital technologies - to develop an augmented reality project that will allow future generations to continue to meet the survivors in the National Holocaust Centre, to interact with them and to ask them questions… even when all survivors will have passed away.
The Forever Project combines a massive database of high quality 4K 3D recordings of testimonies and responses of survivors, 3D computer animation and cutting-edge speech recognition and natural language processing technologies, to produce life-size photorealistic 3D laser 4K projections of the survivors that will interact with the audience, instantly matching audience questions with up to 1500 pre-recorded answers.
Thanks to this project, future audiences will be able to have seamless interactive “conversations” with people no longer alive. It’s a fantastic and highly ambitious venture, and Final Cut Pro X has played an important role in the production of it.
Filming and post production workflow
For the filming and the post production of the live recordings for this project, Bright White Ltd contracted Pollen Studio based in Yorkshire, UK. Pollen Studio has been creating multimedia for over 40 years, taking on ambitious projects around the UK, Europe and the World. We will let Tom Sefton from Pollen Studio take it from here.
The span and the ambition of this project is huge.
Not only did we need to record the personal story of each survivor in a highly efficient yet sympathetic way, but for every survivor we also needed to record the individual answers to over a thousand of different questions harvested from schoolchildren who have met the survivor in the last few years.
Every stereoscopic 4K Red RAW clip we recorded needed to be aligned, synced with external audio from our studio recording desk, edited so we only kept the responses, and exported with unique metadata tags applied to both streams to allow the speech processing engine to play back the correct answer for any possible question asked by the live audience. The project is one that we are immensely proud to have been involved with. It has been an nforgettable, challenging and mammoth 2 years.
How did you deal with the recording workflows?
After an intense period of research around 3D filming techniques, with the focus being the quality of the end projection, we decided that we would film the survivors in our studio with dual Red Epic Dragon cameras mounted on a beam splitter 3D rig so that convergence could be matched to the same distance as human eyes. The Red Epic Dragons utilized the full quality of the 6K sensor and recorded in RedRAW, but we filmed at 4K 25p to maintain reasonable file sizes and processing times throughout the project.
A Blackmagic Cinema Camera recording ProRes 1080p25 was used as a secondary camera to record profile closeups of the face.
Audio was captured into all 3 cameras, and we had separate audio recorded via Adobe Audition and our sound desk. 9 of the 10 recordings took place in our studio near York, which has multiple sound proofed rooms and a control room with multichannel audio recording.
Each week we recorded a survivor for as long as they were comfortable. The studio setup was maintained for over a year to ensure continuity - not all of the sessions were recorded in a linear fashion, so we had to know that the same shot would be replicated throughout the entire production.
The survivors told their personal stories for the cameras the same way they would do for a live audience at the Centre. For the Q&A, we recorded for a 5 full days, broken up into multiple sessions with each survivor. During each session the questions and answers were recorded without interruption with the 3 cameras, meaning that every session resulted in 120 minutes of rushes. Over 900 questions and answers were recorded on a week, every question was carefully tagged with a unique Question ID.
File sizes were a major consideration, and one of the main issues for the project to be a success. The data that we have captured and handled until now is enormous in size:
- r3d data: 62.3TB
- ProRes data: 9.8TB
- Audio: 1.2TB
- Project files and other reference images: 250GB
- Total size of project: 72TB+ (and growing)
- Total files exported: 34.420
Due to the nature of the project, we didn’t want to waste any of the time we could film for, and over 90% of the rushes were used in the final production. During filming, all media was backed up with Shotput Pro to a G-Tech Studio XL 64TB RAID set to RAID10. At the same time, we duplicated the media to another G-Tech Studio XL 48TB RAID set to RAID5 for added performance. As the project progressed, we purchased duplicate G-Tech Studio XL drives to extend our storage and working drive capacity.
Our biggest challenge however was to find a post production workflow that would allow us to process these massive amounts of media in a fast and reliable way. For each 40’ Q&A session we needed to clean up the master audio, sync the 3 camera angles (Red L, Red R and BMCC) to the master audio, edit the synced clips to only preserve the responses and then export the L, R and BMCC clips for each edit as individual files which had to be tagged with unique Question ID metadata.
Our company is most familiar with Adobe Premiere, so at first I tried to make the workflow split between Premiere CC and Resolve for editing and delivery. This did not work out as well as we had hoped. You can batch export in Premiere via dynamic link with media encoder. But the dynamic link server took forever on a timeline with 50 nested files to export. With over 3000 files to export and name correctly each week, this was a major issue. We also experienced repeated and heavy system crashes.
Resolve was quick for exporting but sluggish for editing at full resolution. We also struggled with the terrible audio linking to secondary sound files issue in version 11, and found an odd audio error being exhibited on some of the files that sounded like a clicking at the peak of the waveform. Blackmagic UK tried to analyze this so we could recover the sessions, but finally we started looking at other options.
And then FCP X came into the picture?
We had edited small projects with FCP X before, and found it quick and efficient. However, we had no idea how we would be able to use FCP X for this kind of operation. And this is when I discovered how helpful the FCP X community really is. I asked questions about possible workflows on some forums, and I immediately got encouraging reactions and advice from many different people.
After a brief e-mail conversation with you, I reached out to FCPWorks and Sam Mestman did a couple of quick screen sharing sessions with us until we knew enough to get started. Seeing what was possible in Final Cut Pro X it became clear that it is not just a capable professional tool, but one that could save us a lot of time. Which is extremely valuable with a project of this size. So we decided to go for it.
We used one Library for each survivor and inside that Library we created one Event for each session of each day that we worked with the survivor.
After each day of shooting we imported the footage for every session into its corresponding Event: the Red RAW Left and Right angles, the BMCC closeups and the master audio from our studio desk. When you deal with such a big project, adopting clear and consistent naming conventions for all your assets is an absolute necessity.
We did some fine tweaking of the 3D alignment on the r3d clips using the available tools in FCP X and then we created a multi-cam clip for every session, to combine all video clips with the clean audio.
FCP X handled raw media brilliantly, the only exception being that sometimes we had to wait 5 minutes for a 40 minute waveform to build. This was frustrating at first, but ultimately an easy pill to swallow when we saw how fast it was at exporting the media in the end.
The survivor as individual clips on the multi-cam timeline. Editing performance was excellent, particularly when considering the size of each library was beyond 7TB and the video files were predominantly 4K raw r3d files.
Admittedly, earlier versions of FCPX required restarts to regain performance, but this has stopped being as necessary. The ability to playback full resolution 4K raw rushes whilst editing is incredible, especially from a multi-cam project. Performance from the Mac Pro and the G-Tech Studio XL combo has been perfect.
As we needed to deliver every Red L, Red R and BMCC clip on the timeline as a separate file, we duplicated every edited session multi-cam project into individual Left, Right and BMCC multi-cam projects, each with a different Active Angle.
This ensured us that all edit points for each project were the same, so that we knew with absolute certainty that each exported file would have the same length, had the same start and finish frame, and was perfectly synchronized for 3D.
As you can instantly change the Active Angle of any or all clips of an edited multi-cam sequence in the FCP X Inspector, this was very easy to do. That’s is the beauty of the multi-cam function in FCP X, which has no equal in any other editing tool.
Once the editing tasks were completed we started exporting in batch from FCP X. As it is now, the application does not have a batch export feature. But again we got help from the community and we discovered a great export utility for FCP X called PrimariesExporter, which is fully integrated with FCP X and can be accessed from the FCP X Share menu.
PrimariesExporter allows you to batch export clips from an FCP X timeline as discrete files in the format of your choice and with metadata information. It also has an Excel export option, which we used for some other pieces of metadata to help with the question matching at the programming and interactive stage of the project.
Thomas Szabo, the developer of PrimariesExporter, was incredibly helpful. He even sent us beta versions of the software with minor improvements to help our workflow. And that’s another thing that makes the FCP X ecosystem so powerful: it is supported by highly skilled third-party developers who make dedicated utilities and plugins for specialized workflows. So if you have any specific questions or requests, you can get directly in touch with the people who have created the tools.
How did your setup perform with so many files to export?
For an r3d raw file of one hour in length with its LUT pre-loaded during production, linked to an external audio file, our 8 core Mac Pro was able to export to ProRes in better than realtime. As a test I edited one 40 minute segment in Premiere and then in Final Cut Pro X. With multi cam editing they took similar times. But when it came to export, FCP X with PrimariesExporter just booted it out in double fast time - then chopped the file up for me. Perfect!!
We used screen share so that the Mac Pros could run 24/7 for weeks on end. FCP X had no issues with any exports over a 2 year period. Not once did we have an export error, so after exporting tens of thousand of files this is an incredible testament. Although the 2013 Mac Pro has come in for some heavy criticism, we haven’t had anything to complain about in 2 years of constant use.
We have been able to deliver all files perfectly within the agreed deadlines. Bright White have been working with specialist programmers to create a dedicated speech analysis tool that can hear what someone asks the 3D projection, and then bring up the most relevant response based on the Question ID metadata tags we have applied to each clip. They also created photo-realistic animated 3D models of the survivors to handle the transition of movement - such as nodding, head tilting, gazing and other idle motion - between clips when the survivor is listening to the questions of the audience.
The complete technology behind this groundbreaking project is impossible to explain in a single article, but this short video will tell you a little more about the scope of it:
So what is your opinion now about FCP X, after having worked with it on this project for 2 years?
We have been delighted with the performance that we have found from FCP X, and it has become an integral part of our studio. We’ve started using FCP X for projects in full 6K resolution and some 360 films now, and even without a Red Rocket X we can maintain useable performance in realtime.
Over the 2 years we have worked on this production, FCP X hasn't cost us any time at all with render issues or corrupted projects. We were editing multi-cam projects really quickly and then overnighting our exports - really efficient. The multi-cam function is also very impressive.
The previous workflow was allowing me to edit and deliver around 1 days rushes per day with an overnight render for delivery in Resolve. FCP X has cut the editing process down by at least half, and I could be encoding whilst I was editing. That has been a massive win.
The Forever Project is continually being extended due to the successful delivery, and the amount of holocaust survivors around the UK and Europe who have a story that deserves to be preserved in this incredible way.
To end this article, I would like to invite you to watch this great short film that was made about the project and that will be shown in theaters around the UK. The film was directed by Malcolm Green, who has created many of the UK’s most famous and award-winning ad campaigns for some of the biggest clients in the world in the past 25 years, and it was edited in FCP X by Tom Baker:
Many thanks to Tom Sefton of Pollen Studio for having taken the time to write up his experiences with FCP X and for answering my many questions over the last months: www.pollenstudio.co.uk
Also a huge thank you to Chris Walker, Adam Stanning, Katie, Lara and Becky at Bright White.
And to Sarah Coward, Phil Lyons, David Brown and everybody at The National Holocaust Centre.
Finally, a huge show of respect and gratitude to the Holocaust survivors who have embarked on this project and who give their time telling their stories to tens of thousands of children and adults every year, hoping this will help us create a better world: Steven Frank, Rudi Oppenheimer, Arek Hersh, Renee Salt, Janine Webber, Martin Stern, Joan Salter, Steven Mendelsson, Mala Tribich, Kitty Hart-Moxon.
© 2016 FCP.CO/Ronny Courtens