Alex takes us on a tour of his new app BeatMark Pro, designed to make editing to cues on music a lot easier.

Hi everyone, I'm Alex Raccuglia and I'm a software developer as well as a director and editor of promotional videos and TV commercials.

I'm here today to introduce you to the latest application I've made: BeatMark Pro, whose payoff is "Music-driven video editing for Final Cut Pro".

So it's an app that goes along with Final Cut Pro to automate the workflow of making music-driven video editing.

BeatMark Pro is the third, completely rewritten version of BeatMark, the most popular and successful app I've made with my little software house: Ulti.Media.

BeatMark Pro has two great strengths, in my opinion:

1. It allows you to cut the music to the desired duration virtually automatically;

2. It allows you to fully automate the editing process: just drag and drop the clips and the app will position them according to the beats of the music!

But let's go step by step…

 

The origins

The very first version of BeatMark, even before being a commercial product, was developed seven or eight years ago as an internal tool.

The goal was simple: to put markers at the beats of a track, a song, on audio files so that editing in music time would be easier and faster in Final Cut Pro.

When I started selling BeatMark X (version 1.0), it was immediately a "success", not in the sense that I became filthy rich, but in the sense that it was a product that was really needed: even today no one has written an application that competes with BeatMark for Final Cut Pro (and I wonder why).

 

The workflow was simple: you drag a music file onto the application, and it, once it has chosen frames per second, prepares an FCPXML file with the audio file in the timeline with the markers that mark the beats of the music (the FCPXML files are then " processed " by Final Cut Pro to create events and/or projects).

Simple and almost painless.

It was one of the first applications I wrote, and, as time went on, I realized that it could be improved in a way that provided more control to the editor over the music, so that the track could arrive at Final Cut Pro with a draft cut already in good shape.

Two years ago, in February 2020, I launched BeatMark 2, which also offered the timeline with markers, and the ability to highlight the 4/4 (or 3/4 of the song) and even make simple cuts.

BeatMark 2 also allowed to export Final Cut timeline with placeholders placed at the beats: another thing that allowed to speed up a bit the workflow.

 

Why a new version?

BeatMark 2 was certainly the most successful application I have ever made, and also the one that has received, of course, more updates, meeting the needs of customers or potential customers, and especially taking into account all possible edge cases (various audio formats, different music genres, ...).

I can say that BeatMark 2 works practically perfectly in 90% of cases, misses a few beats in 5% of cases, and absolutely sucks the remaining 5%.

That's why I've always thought I should completely change the algorithm that identifies beats.

In the summer of 2021, during a walk with the dog (walking with the dog is my biggest source of inspiration) I thought that I could write a Machine Learning model to "teach" the program to more accurately recognize the beats in music; I also had in my head how I could implement the system.

There was only one problem: the training.

When you do artificial intelligence, you have to teach and build the model, providing lots of examples, in my case producing many beats in correspondence of the beats of the music on many tracks, on many songs.

 

 

I told myself that I could use the existing algorithm, the one of BeatMark 2, and then proceed to a refinement done by hand; however, always walking with the dog, it occurred to me that maybe I could make a different algorithm, always to do the training.

So I did, during the fall of 2021 I developed this new algorithm whose initial purpose was precisely to do training for artificial intelligence.

I realized that this algorithm had a great strength but also a weakness.

Strength: it adhered perfectly to the beats of the music, in a much more precise way than the BeatMark 2 algorithm.

Weakness: sometimes it would miss some beats, perhaps when there were "ghost" hits or forced pauses in the song.

So I thought that this algorithm was not enough, since the BeatMark 2 algorithm, even if it was not so precise, allowed to find the subdivision of the measures in 4/4 of the music.

Then, in early 2022, I made an edit for a promotional video (for a pharmaceutical company, a B2B video). I naturally used BeatMark 2 to find the beats of the music, and did my own editing.

At the end of the work, however, I looked at the timeline in Final Cut Pro, and I realized that I had followed so slavishly the 4/4 division, that the editing adhered too regularly to the time signature of the music, in practice I had made an absolutely boring editing, which did not use in any way the video to go to counterpoint to the music.

 

And I was also ashamed of myself: my past as a musician usually allowed me to use video as something that would amplify the pace of the music, creating a whole new rhythm together with the images.

To put it simply: in order to adhere to the 4/4 count of the music, I had lost connection with the music itself!

At this point I went back to look at the markers that had been put in automatically by this new algorithm: and I said to myself, "why don't we try to use these markers, without going to think that the music is in 4/4 or some measure, to create an edit from scratch in Final Cut Pro?"

And to do this test, I also wrote a little piece of software that would take a few dozen clips of stock footage to put at these markers.

I exported this timeline and the result was this:

 

And I said to myself: if this is the first experiment I've done with this algorithm... It doesn't even suck too much... I can think of making an app that instead of finding the music markers, finds a sensible way to make the clips fit the music!

That same day I started writing everything that would be the blueprint for a new app: BeatMark 3.

 

From the project to the app

In order to write this app I radically changed the way I make my software: for the first time an app of mine is document-based, so the work you do on a file is not lost, but is saved and can be resumed and modified at any time, later.

The app is completely written in SwiftUI, which is why it requires at least macOS 11 as an operating system. That's not a problem, though, because I decided to make the app compatible only with Final Cut Pro version 10.6 and later.

Making this app document-based means that you can use the same track with different edits, without having to redo all the work from scratch.

And, if I'm honest, the potential of this app compared to the previous version is so great that I decided to call it BeatMark Pro and not just BeatMark 3, and in a little while, if you keep reading, you'll find out why.

 

The Interface

The document screen is divided into four parts:

At the top there is the timeline with the audio file and the markers found, always in this area there is a section that allows you to display only some markers, for example when the algorithm finds too many of them.

Immediately below is the section that allows you to cut the music: one of the two great strengths of BeatMark Pro is the ability to cut the music automatically so that the duration of the song is practically what you need.

The third section is the one that allows you to specify the content of the timeline: you can export a Final Cut Pro timeline with just the song (cut to the required duration), the song with placeholders at the beats of the music or, and this is the second strong point of this app, you can drag and drop on the app many clips that will be edited automatically!

Finally, there's the big export button.

 

The workflow

Nothing could be simpler: open BeatMark Pro and create a new document (menu File > New), and you'll find yourself in front of this view,

 

At this point all you have to do is drag the music file onto the large central area.

 

And then press the button in the upper right corner (Analyze).

In the application's preferences you can also choose to perform the analysis immediately once the file is dragged.

At this point if you want to change the duration of the song you have to select the corresponding flag, which opens this section of the interface:

 

and type the required duration in the field (for example 60 seconds).

BeatMark Pro suggests several cut points, listening to them one by one, from the first that is the most likely (according to the internal analysis algorithm) you can understand which cut is the most "natural". Once you have chosen the cut point you can move on to the third step.

Here you can choose to export only the file with the markers, or the Final Cut Pro timeline, and in this timeline you can have only the (cut) file with the markers, this file with the markers and the placeholders or a series of clips already edited in correspondence of the markers.

Probably this last option is the most interesting because it allows you to have in very few moments a "draft" editing that can be a good starting point to refine and improve.

All you have to do is drag and drop the clips you want to edit automatically in this small area and BeatMark Pro will already have all the material you need to proceed.

Pressing on this big button (which also tells us how many clips have been loaded) you access the media management window that allows you to specify, for each single loaded clip, some additional characteristics, such as whether it should be used as the first or last clip, or even the type of Spatial Conform.

Also in this window you can delete clips.

Finally the export: pressing on this big button the FCPXML file is automatically generated and must be dragged on Final Cut Pro, which will ask in which library to open it:

 

At the end in Final Cut Pro, in the library we have chosen, there will be the audio file with the markers and eventually the project with the timeline and if we have chosen also clips, a first cut in time of music.

At this point all you have to do is proceed with the cleanup, refinement and color correction.

 

Changing the duration of music

How many times have you had to take a three-minute song, and try to shorten it to 30 or 60 seconds?

During my walks with the dog, I thought that since I was no longer simply considering the single beat, but also everything around it, in the sense of the instants of time that precede and follow it, I could find the greatest points of similarity among all the beats found.

This led me to think that it would be possible, not only, desirable, to find the cut points that would create the least interference or interruption of a piece of music.

Basically, once all the markers have been found, BeatMark Pro compares each individual marker to all the others to find how much the points in the song resemble each other.

In this way you can ask BeatMark Pro to shorten the music to a certain duration in terms of seconds, and BeatMark Pro will find all the more or less "optimal" solutions to change the duration of the song.

Once the duration has been entered in this drop-down menu, the 20 most "sensible" cuts will be proposed that lead to a duration of the song very close to the one requested.

Obviously the algorithm that identifies the potential cuts "thinks" according to computational parameters, what you have to do is to listen to the proposed cuts in order to identify the one that offers the best relationship between the actual duration of the song and the quality of the cut.

It is, however, a relatively short operation and, often, even fun! I often like to see how the algorithm finds cuts that I would never have thought of.

I think this is really the most important and disruptive feature of the app, and it really saves a lot of time, and since I wrote this program I have been using it practically all the time.

Of course, it must be said that Adobe, with the latest version of Premiere, has gone even further with an algorithm that makes use of artificial intelligence to change the duration of the song. Personally, I haven't had a chance to try it out in depth yet, but from what I've seen it's a great little masterpiece that, I regret to admit, is missing from the Final Cut Pro ecosystem.

My solution is of course less "elegant" or cutting edge, but it's a good second place, in my opinion!

Of course, you can give BeatMark Pro further guidance by going to select each marker, suggesting it force the cut on one of them.

Also in this section there is a small preference panel that allows you to specify the duration range (plus or minus tot seconds of difference from the target) or how far away at least from the beginning and end of the song should be the cut point.

 

 

Timeline

In the timeline export panel (I know that in the Final Cut Pro world we talk about project, but it's stronger than me, I'm boomer!) the first thing to do is to choose the project format: clicking on the settings icon opens this little window that allows you to set resolution and frames per second.

I've prepared some standard resolutions and also some less typical ones, but choosing "custom" you can set horizontal and vertical resolution, and the same goes for frames per second.

Further down you can set more advanced settings for the behavior of the clips in the timeline: going to specify the general behavior for spatial conforming, as well as the point where to make the in point: if from a 10 seconds clip the algorithm will take only one second, you need to understand and specify where this second starts from.

By default the choice is at the center point, but you can choose the initial point or let random chance decide.

Of course this is the most... "Ephemeral" option, because my advice is always to do three-point editing then directly in Final Cut Pro: BeatMark Pro does not produce final results, it simply gives you a kick start.

 

Editing pace

You can also choose the type of pace to use for the editing: there are currently four styles:

1. Flat (the pace is the same throughout the song)

2. Crescendo (the pace increases as the song nears the end, meaning that the clips will follow each other faster and faster as the music continues)

3. Standard Valley (The pace is very high at the beginning, then calms down in the middle of the song and then returns to high at the end)

4. Crescendo valley (like the standard valley, but the pace grows faster and faster as you get closer to the end)

But how does this automatic editing work?

The answer is simple: every time a beat is found in the music, it is associated not only with a marker, but also with a coefficient indicating how "powerful" this beat is.

The more powerful the beat is, the more likely it is that a cut, i.e. a new clip, will be added to it.

In practice, going back to my initial worry, namely that of making montages too "square" and adherent to the four quarters, with this system the editing adheres more to the moments of cut actually present in the song.

But what does this mean?

Suppose we have a three-minute song, and in these three minutes an average of 700-800 markers are found.

Suppose we cut the duration to 30 seconds, that means we will have 100-150 markers at our disposal.

If we drag and import 30 clips to do the automatic editing, of these 100-150 markers, only the 30 most relevant will be found, with coefficients that increase the power of the marker found in correspondence also with the chosen pace.

And therefore, in correspondence of these 30 markers, the clips will be positioned on the timeline.

We can still go to force the use or not use of a specific marker by clicking on it in the timeline of BeatMark Pro.

Last thing I like to add: BeatMark Pro allows you to import both video clips (in QuickTime and MPEG-4 formats) but also images (JPEG, PNG and TIFF).

 

Exporting the audio file

One small utility I've allowed myself (again, reminding you that I'm user zero of this app, and an integral part of my workflow, of what I do to bring home the bacon) is the ability to export the audio file, cut to the duration of your choice, in Wave format.

This might seem redundant, but we can often and willingly need these musics, cut to the required duration, even outside the realm of video editing, and still with a single file that is much more manageable and convenient (and even sendable to others) than a Final Cut Pro project.

 

BeatMark Pro is a document-based app

What does this mean?

It means that, unlike previous versions, all the work you do analyzing and defining the song is not lost once you close the app, because it is saved in a single document (one for each song).

It means that if you analyze a song, make choices about markers, editing, cutting, all those choices remain saved.

This can be really useful in case you choose to make a format: once set the cut points, the most relevant markers, those to keep and those to ignore, and also made the first assembly with all the clips, you can choose to duplicate the document, in this copy remove all the clips (and possibly keeping the first and / or last that often in the case of format, are constant, perhaps one of the two includes the title ...), load new clips, and proceed to export: a new opening sequence made in a few seconds instead of starting from scratch!

 

Future developments

By the time you read these lines most likely the initial version of BeatMark Pro will be out (then if you go in the about you'll read 3.0, because actually BeatMark Pro is BeatMark 3, but since I rewrote everything from scratch, it seemed sensible to change the name of the project.

But, as I told you, BeatMark Pro is primarily an application that I use every day for my work: even if it does not seem anyway my main job is to work on video post production, being a developer is a kind of "extra".

As a result I continue to have ideas and needs, and as soon as I can, I try to solve them with new algorithms and new functionalities.

To make you understand, right when I was making one of the tutorials I'm producing to better explain how this app works, I introduced the possibility to mark a marker by telling BeatMark Pro to force it as first or last.

One of the first things that will be released with an update (free of course) of the app will be another algorithm that allows you to identify the beats that belong to the main stream, the division pattern of the music, so, once again, allow you to identify the infamous 4/4 and consequently have an analysis more related to the music than to the editing.

Another thing will be the ability to mark a clip as " more important" so as to give it more space and make it stick to a cut point where it lasts longer in the edit.

And then, of course, I'm always available to listen to requests for new features and suggestions from both my customers and also from those who are simply interested in this product: you could do it yourself by going to comment below this article!

 

Some final considerations...

Writing these articles for me is always a big effort because I have to try to condense into "a few" words not only the work I've done, but also the thought process that led me to make a product this way, and then summarize it all in such a way that you can understand, actually, what this app does.

And I realize that, most likely, especially for this particular app, there's probably a lot more to say than I've managed to do here.

So I refer you to the BeatMark Pro web page, which is a bit more technical and less discursive, a page where you can also find several tutorials that I have made and will be making over the next few days, so as to actually explain not only how to use this app, but how to take advantage of its potential.

 

Small call to action

like all the apps I develop, BeatMark Pro is available as a free triall for seven days, fully functional, downloadable from our site: I really ask you not to buy it blindfolded, but first to try it, so you can understand if it's really for you.

What else to say? I thank you very much for the attention you have given me and I thank Peter opportunity that gives me the honor to write these articles ...

If you have anything to ask, ask, you can do it directly to me on Twitter, by This email address is being protected from spambots. You need JavaScript enabled to view it., or, and this is something that I would love, commenting on this article so that my answer is visible to all.

I'd like to tell you that I'm not afraid of criticism, but I'm human, so... Let's just say that criticism is what makes you grow and pushes you to get better and better!

Once again thank you and I hope you find this app interesting.