fbpx
fcp.co logo transparent
fcp clapperboard

Control the Size of Final Cut Pro's Libraries with FCP Diet 2

Hi, I am Alex, I am an editor and a software developer and recently I released this macOS app, FCP Diet 2.

I am not here to do (self) promotion, I would simply like to tell you the story of how and why I developed this software.

 

The Final Cut Pro libraries

Final Cut Pro's libraries are a very interesting object: they are not just a file format (but inside them there is actually a complex folder structure), but they are the beating heart of any project created with this video editing software.

Final Cut Pro X, from its origins, was designed to offer high performance at the expense of disk space occupation. This means that Final Cut's libraries are often very large, tens if not hundreds of gigabytes, because they contain the original files, transcoded files, optimized videos, and rendered files.

fcpdiet2 fcp 01

 

Final Cut Pro, in simple terms, " doesn't throw anything away": it means that every render file that is produced, for every single frame, is not deleted, and therefore as you work and work on a project its Final Cut library becomes larger and larger.

I like to get my hands dirty

As a good programmer, I have always taken care of "opening" the Final Cut libraries and deleting by hand all the unnecessary and redundant files, an activity that I have always done especially before archiving the projects: the disk space, especially the backup one, has however a cost that must be managed wisely.

Two years ago, in just one afternoon, I made a very simple application, FCPX Diet (version 1), whose sole purpose was to automatically delete these files.

fcpdiet2 fcp 02

 

FCPX Diet 1, although it was developed quickly and without too much effort, turns out to be one of the most successful applications I have ever made.

They always say "eat your own food", and in fact since that day I've been using FCPX Diet 1 practically every day, to manage my video projects. 

It wasn't enough

Over the last few months, during our second Lockdown here in Italy, however, I felt that FCPX Diet 1 had some limitations, especially considering a very simple fact: Final Cut Pro library files are not actually the only part of a project I make, because every time I have to start a new production, I prepare a series of folders and subfolders that contain all the files that will be related to said production.

Consider, for example, the footage shot, stock videos, music, sound effects, speaker, animations and motion graphics, After Effects files,…

fcpdiet2 fcp 03

 

Each production is of course different from every other, but generally it is made up of many assets that eventually go into the Final Cut project (and consequently into the library). So I started thinking about an evolution of FCPX Diet 1 that would take into account this fact: the Final Cut Pro library is a central element, but it is not the only one within an entire production project.

For this reason I started to develop FCP Diet 2 (without the "X", since Final Cut Pro X also became simply Final Cut Pro with the transition to version 10.5), so that it was not only a natural evolution of the first version of the program, but could add all those features that are convenient if not necessary when trying to manage a medium-large project.

All of the software that I write essentially has one purpose: to automate the manual work that I personally do.

FCP Diet 2 does the same, not only going to delete all redundant files, but trying to optimize the whole folder, the whole video production project, also performing consolidation and "general cleaning" functions.

But how does it work?

FCP Diet 2 is really simple to use: you just have to drag the library or the folder of your project onto the main screen:

fcpdiet2 fcp 04 

 

Immediately all the files will be scanned, even the hidden ones, and finally a screen will appear showing all the information about the libraries found, with even an indication of the space you'll save.

fcpdiet2 fcp 05

 

At this point all you have to do is press the Diet button and wait for the processing to finish. It can take a few seconds or even several minutes, depending on the complexity of the folder and the number of files it contains.

At the end, FCP Diet 2 shows a message indicating how many gigabytes have been saved. Simple as that… 

Everything under control

I don't know about you, but when I start a production I try, as much as possible, to keep track of all the phases of the work and to have absolute control over all the media that will then flow into Final Cut Pro for the final edit.

For this reason, the fact that there may be assets that reside entirely within the libraries of Final Cut Pro scares me a bit: it's one thing if we're talking about footage, which is essentially static once it has been captured by the camera, but it's another thing altogether if we're talking about files that must comply with strict versioning rules.

I can and do work on videos that are fully animated, motion graphics, or even just have complex titles, such as callouts, highlighted keywords, lower thirds, and so on, video segments that I usually do in After Effects and can constantly be updated while keeping the same file name.

I always put After Effects render files in folders and subfolders (VFX, MG, Subs, Lower 3rds, ...) and when I make an updated version of a single render file I overwrite this file; well, if Final Cut Pro has these files "in the belly" of the library, in practice I'm going to replace a file that is not actually referenced by the editing program, so I do not have a single source of truth: I "look" at a file but Final Cut Pro "sees" another one.

fcpdiet2 fcp 06

 

FCP Diet 2 takes care of exactly this: if it discovers that the same file (with checksum calculation of course) is in the library and also outside, it deletes the file in the library and forces Final Cut Pro to reference the external file; in this way you have two advantages: first you save space, but above all there is no more possibility of confusion because the file is unique.

Bring in what is outside

Some time ago I made a video in which I had a very tight deadline, so for the music I brought in the timeline some songs that were located on an external disk (in which resides all my library of music), this caused Final Cut Pro to refer to a file that was on an external disk, removable.

This resulted in a big problem when I had to render the timeline and I had detached this disk: the nice "missing media" message.

fcpdiet2 fcp 07

 

FCP Diet 2 also solves this problem because when it realizes that Final Cut Pro refers to a file that is outside the folder that has been dragged, therefore "outside the project", it takes care in complete autonomy to copy all external files in order to consolidate everything.

A step forward

Since I've written a series of procedures for analyzing files and folders on the disk, I thought I'd go a step further and allow the app to perform other, slightly more "destructive" operations.

I added the possibility to check the entire dragged folder in order to highlight if there were any duplicate files, and to go and delete all copies except the original, adding however a reference ("alias") for the deleted files: the folder structure remains almost identical but the space saved, especially when it comes to uncompressed videos is not insignificant!

Finally I've also added the possibility to automatically delete files that are not used by Final Cut Pro, this is useful at the end, when you are going to archive a project.

Note: FCP Diet 2 doesn't delete these files but puts them in a folder so that the user can choose if and what to delete!

fcpdiet2 fcp 08

 

These two operations can also be done on only one category of media (video, photo, audio) and/or by specifying a minimum file size.

Long live H.264

Recently I've often worked producing videos using many stock assets, downloaded at the highest quality (in ProRes format). Once the video was finished, however, the project folder turned out to be huge, with dozens of gigabytes "wasted" on sources.

So I thought to include one last option, in this case "destructive" that would allow to automatically compress the ProRes format files into H.264.

fcpdiet2 fcp 08

 

I added a lot of options so that it is difficult to make a mistake and transcoding files by accident going to lose quality, allowing you to exclude files and folders that contain within the name a certain word.

A simpler explanation with a video

I could have simply posted this video, because it wraps up and explains in a few minutes everything I pointed out by typing all these words:

 

But I wanted to try to be more "human" and tell you my thought process, the road I took to get to make this new application.

In the end I'm not a developer, I'm primarily a filmmaker and animator (and this was very useful to make these promotional videos), the activity of developer is my second job, although it takes me a lot of time, energy and passion... 

Icon

I'm not a graphic designer, this I think is clear, but I also thought to update the icon of the application moving from that of the first version is that of the second:

fcpdiet2 fcp 10

 

In both cases I liked the idea of showing a "distortion" due to a slimming, after all the application is called Diet because its goal is to make slimming folders and libraries of Final Cut...

A machine is more efficient than I am

In the end, I have to be honest, FCP Diet 2 doesn't do anything that you can't do manually by going through the internal folder structure of the Final Cut Pro library. But it does everything automatically, quickly, and accurately.

I personally use it every day, even and especially on projects that are in progress: since the Lockdown started here in Italy, my main office is my home, and my main machine is a laptop with 512 GB disk size, so not particularly large; and especially the most important and urgent projects are on a shared folder on Google Drive.

This means that the space each project takes up has a burden not only in disk space occupancy but in synchronization time.

FCP Diet 2 helps me to keep the projects up to date, and with the least possible space occupation, so that when I can go to the office the folders that are on Google Drive are synchronized without loss of time.

Concluding remarks

I don't know what idea you may have about me after reading this more or less long article. My goal is not to sell you the application, I just wanted to communicate the enthusiasm I have in these days for being able to finish it and present it to the market.

If you're interested in it, you can click here to find out more features and download the free trial version, for seven days you can use FCP Diet 2 without limits and see if it's right for you.

What else to say? Thank you for giving me your attention!

 


Written by
BloggerExpert

I'm Alex Raccuglia, I studied computer engineering and then, the cases of life, I found myself to be first an editor and then a director of commercials and television and promotional videos

(https://player.vimeo.com/video/198863318).

I've always developed tools that I use internally to speed up work, improve my workflow and make myself more efficient.

About four years ago I opened my second business as a software developer selling these tools (after having cleaned up and improved them): Ulti.Media (https://ulti.media/) produces software tools that help audio and video professionals to be more efficient and faster in their daily work.

Log in to comment


peteramwiggins's Avatar
peteramwiggins replied the topic: #112614 10 Feb 2021 16:22
Looks like a great update! Thank you for going in to so much detail.
Ben Balser's Avatar
Ben Balser replied the topic: #112615 10 Feb 2021 16:55
This is a really nice update, thank! I'm a big user of FCP Library Manager, and used FCP Diet occasionally, also. Because sometimes I don't want to be overwhelmed by all my Libraries at once, the the one or two I'm currently working on. All the added functionality makes this even more valuable. I'll still use both apps, but I think I'll be using FCP Diet (2) more often now.
Shylock74's Avatar
Shylock74 replied the topic: #112622 10 Feb 2021 17:36
Hi Ben,I'm Alex Raccuglia, the author of the software.First of all I thank you both for the feedback and for the fact that you use my software, of this I am very, very happy.If I have to be honest, Library Manager has always been a software that I have loved since the beginning, the guys at Arctic Whiteness have done a real masterpiece, even and especially for the interface.Precisely for this reason I have never wanted to create an application that was in direct competition with them: they have done a nearly perfect work, it makes no sense to enter their own field.FCP Diet 2 is first of all an application designed for my workflow, and in my workflow, as I wrote in the article, the Final Cut library is one of the many bricks that make up an entire production, so in FCP Diet 2 I recommend to drag the entire project folder and not only the library, in this way the application will manage all this set of information in a more organic way.That said, I invite you to try the application for seven days so you can get your own idea, and if you want to give me feedback, I'm always happy when I receive suggestions or requests because then I can continuously improve my products.Thank you!
Ben Balser's Avatar
Ben Balser replied the topic: #112645 11 Feb 2021 13:50
Alex, like I said, I use both side by side, depending on what I need at the moment. I think both products should be on every FCP editing system. Already purchased and love the version 2!
Shylock74's Avatar
Shylock74 replied the topic: #112646 11 Feb 2021 13:52
Thanks, man. ❤️
Vasili13's Avatar
Vasili13 replied the topic: #112686 13 Feb 2021 19:00
A lacking feature I desperately would like to see is a project trimmer, where you can trim original source footage and add a definable number of frames pre/post each edit in each project, this would by far further decrease library sizes.

Final Cut Library Manager doesn’ t do this, and Worx4 X has too many limitations for it to be useful. Old FCP Classic and Resolve does this, but I still prefer FCP. I would pay a lot for this. Anyone else?
joema's Avatar
joema replied the topic: #112688 14 Feb 2021 10:58

A lacking feature I desperately would like to see is a project trimmer, where you can trim original source footage and add a definable number of frames pre/post each edit in each project, this would by far further decrease library sizes...

This is very much needed. The problem is it is risky and difficult to implement and test with extreme reliability.

There are hundreds of different compressed Long GOP formats. Most of the video frames are not even present;  They are pseudo-frames, and must be mathematically reconstituted for trimming. When you see video with pixelated green frames at the start of a clip, this usually means a Long GOP format was trimmed or encoded incorrectly.

Historically, "no encode" trimming tools have been somewhat unreliable. Quicktime Player 10 can trim some Long GOP (e.g. H.264) formats, so why can't FCP? Because QT10 does one file at a time, and at the user's specific direction. The user typically checks the single result immediately.

By contrast a hypothetical FCP "consolidate with trim" feature might do 10,000 files in a batch, which contain dozens of different Long GOP formats of varying and unknown pedigree.

Resolve does this, but will refuse to handle certain Long GOP formats. What if you did 10,000 files, it trimmed 9,900 OK, quietly rejected 80, then corrupted 20, then you deleted the source thinking all your data was safely trimmed? And what if those mis-managed files were critical to your project? Almost no user can afford to visually check 10,000 files.

It is not a difficult feature to implement but it would be a huge burden to test and maintain. I really think that's why Apple has thus far not done it. But there is no question it is a needed feature and would be very valuable.
Vasili13's Avatar
Vasili13 replied the topic: #112699 15 Feb 2021 08:28
“ This is very much needed. The problem is it is risky ...”

If enough people ask for it, Apple or a developer will create it.

I don’t agree that this is technically any more difficult to do, than what FCP already does with its advanced library structure/format. Think about all the complex stuff a FCPXML file can already describe, in/out points, FX, speed changes, filters, describing in/out points, ReEncoding and updating newly reEncoded clips to reflect new in/out points is no more difficult than what a FCP library/FCPXML can already do, even with long GOP flavors.

This feature can ask the user what do do with certain difficult or yet to be supported formats, I really don’t see this to be a technical/programmatic problem that is any more difficult that what FCP is already doing.

I am willing to support a developer financially who is capable of creating this.
Ben Balser's Avatar
Ben Balser replied the topic: #112710 15 Feb 2021 23:44
XML files only refer to what exists and doesn't physically change anything.
What you're asking for physically truncates and changes files.
Huge difference, apples to onions comparison.

But as to simply truncating video files, yes, I agree, there's no technical reason for FCP to not do this already, as FCP 7 did it just fine with Long-GOP clips. I don't buy the Long-GOP is an issue argument at all. It's done daily by all sorts of software with accuracy.
joema's Avatar
joema replied the topic: #112715 16 Feb 2021 13:10

...I don’t agree that this is technically any more difficult to do, than what FCP already does with its advanced library structure/format. Think about all the complex stuff a FCPXML file can already describe, in/out points, FX, speed changes, filters, describing in/out points, ReEncoding and updating newly reEncoded clips to reflect new in/out points is no more difficult than what a FCP library/FCPXML can already do, even with long GOP flavors...

First, I fully agree the feature is badly needed -- even more today with physically remote workflows enforced by the pandemic. IMO it should be one of the highest-priority enhancements for FCP. In fact I'm now having to test Resolve as a workaround for this deficiency.

FCP7 had this feature in Media Manager, but it was dropped in FCPX. There were many reports of it not working properly on FCP7 in various situations. When speculating why FCPX did not include consolidate with trim, industry expert Jeff Greenberg said "I suspect this has to do with long gop codecs; I think this makes (for apple) media management easier/safer/more bulletproof in an age of sourcing from long gop codecs."

Likewise Resolve's 16.2.3 Media Manager recently dropped the "move" option, which was equivalent to FCP7's "Use Existing" option. Resolve still supports a "copy with trim" feature, but the result clips must be re-linked, which some people report can be problematic.

It's true that FCP must handle those same clips when exporting the timeline. And in fact trimmed timeline clips are available right now if using the free Frame.io utility, as described by Ripple Training. But it cannot "trim with extra handles":


Since 10.4.4, FCP can also batch export clips or selected ranges directly from the Event Browser, again without extra handles, and it doesn't support multicam clips:


But both of those options are strictly export, not media management. The clips should export OK, but if not there is no UI-driven expectation of a library or project using the newly-encoded clips. Your original media is still there.

FCP could hypothetically do what Resolve now does, which is consolidate with trim plus user-defined handles (IOW a copy not a move), while leaving the original media behind. You might do that to send a trimmed project to a remote collaborator. If the project had ten 5-second sound bites from ten different 30-minute multicam interviews, it would be a huge space savings. In that case if the recipient reported a problem, you'd still have the original clips.

Another reason you'd do that is to jettison the data load of the original large clips, and keep only the trimmed library or project. Presumably the original untrimmed media would be deleted. In that case if some problem was later found among (potentially) thousands of transcoded clips, your data would be compromised.

I tend to agree with Jeff Greenberg on a possible reason for Apple's reticence on this. In the FCP7 (and before) era, a lot of media was tape-captured to an All-Intraframe codec like DVCPro. The tape capture process produced a few large files using a limited number of capture codecs, typically All-I. The long length made trimming more important but it was inherently safer to trim an All-I codec than a complex Long GOP codec, esp. if the trimming process either itself discards the original media or incentivizes the user to do that. We just went through a phase whereby editing certain HEVC codecs would crash or destabilize FCP, apparently fixed in 10.4.10. There is no telling what that would have done to a library if consolidate with trim existed.

Regardless of those concerns, it is still a vital feature and I hope Apple can expedite delivery of this. 
Ben Balser's Avatar
Ben Balser replied the topic: #112718 16 Feb 2021 14:21
Trim with 5 or 10 second handles, problem solved for long-gop codecs, simple. And in the last years of FCP7 we were getting a lot of long-gop codecs from new cameras at the time. The function never gave me issues, most of my colleagues and students had very few issues with it. I still say it could be done safely. But FCP is still after all these years missing some vital features and yes, it's cheer leaders come up with clever ways to dismiss them. Sorry, this is my honest opinion. I love FCP, but I don't see any mechanical/technical reason for it to still be so lacking.