Offline / Online Workflow for 4K Sony FS7 II footage

Setting up our workflow for a feature documentary that was shot in 4k with different frame rates. Mostly 60p, but some in 23.98. I've heard people like to transcode outside of FCPX either using Adobe Media Encoder or DaVinci, so I ran a couple of tests using AME. I transcoded to 1920x1080 / retained source frame rate / retained the same number of audio channels...the audio was recorded directly into the camera with a boom so no syncing necessary. When I tried to relink in FCPX , it would not link back to the 4k footage because of the different audio channels....even though they were technically the same.... maybe Adobe Media Encoder altered them slightly..not sure!

I did another test and exported an XML from FCPX of a sequence with said test clip and Davinci was able to relink back to the 4k clip, but it retained the source frame rate...60p. Usually I will drop a 60p clip onto a 23.98 sequence and conform at 40% for slow motion footage...I'm not sure if relinking + finishing will be a nightmare in DaVinci if we have tons of 60p footage.

Would it be easier to transcode everything using FCPX, relink back to the 4k footage within the program and finish in Resolve from there? Or would I still run into issues regarding the slow motion footage in DaVinci?

Yes, you'd be better off creating Proxies inside FCPX and switching between which version you need to export with.

lizjperl wrote: ....feature documentary that was shot in 4k...I've heard people like to transcode outside of FCPX either using Adobe Media Encoder or DaVinci, so I ran a couple of tests using AME. I transcoded to 1920x1080 ...When I tried to relink in FCPX , it would not link back to the 4k footage because of the different audio channels....
Would it be easier to transcode everything using FCPX, relink back to the 4k footage within the program and finish in Resolve from there? Or would I still run into issues regarding the slow motion footage in DaVinci...

Who is advocating transcoding to proxy outside of FCPX? There are two reasons to not do this:

(1) In general FCPX transcodes faster than other software, sometimes 3x or 4x faster if media is H264. Using FCPX an iMac 27 can transcode H264 content to proxy much faster than a Mac Pro due to Quick Sync.

(2) FCPX is not currently designed to use proxies generated by other software. If someone advocates using other software for this they must not be very familiar with FCPX.

Re finishing in Resolve, I don't have experience with this but people do it frequently. However your source media has mixed frame rates which are conformed within the FCPX timeline. You then want to send that to Resolve via XML. According to Ripple Training you can do this provided you use the right import options. In the Resolve import dialog there is a "mixed frame rate" format pop up: www.rippletraining.com/articles/2015/11/...resolve-12-and-back/

If you are concerned Resolve is not fast enough to edit 4k with proxy, that is a possible issue. However Resolve 14 has received massive performance upgrades so you could evaluate this. Resolve also has proxy support so you could just generate the needed proxies there if required.

The FS7 can record in intra-frame 4k H264, called XAVC-I, or inter-frame "long GOP" H264, called XAVC-L. XAVC-L is much more compute-intensive to edit and definitely requires transcoding to proxy. XAVC-I is less compressed and editing performance is a lot smoother.

If I want to import FS7 footage + optimize the media the "leave in place" option is greyed out. I thought the recent update supported XAVC-L? I don't want to copy + optimize again...I know there was a workaround when we had the FS5 where we just dragged the clips into an event and bypassed the "copy" option, but I heard that is not recommended for large projects.

The folder structure on my drive: RAW MEDIA ---> REEL_DATE ---> XDROOT

lizjperl wrote: If I want to import FS7 footage + optimize the media the "leave in place" option is greyed out. I thought the recent update supported XAVC-L? I don't want to copy + optimize again...

Not just XAVC-L but no folder-organized media can be imported from disk with "leave files in place". You are correct it's considered poor practice to copy the video files outside the tree and import them in place. However I do this frequently with XAVC-S and so far haven't seen any problems.

Doing this with AVCHD will definitely cause performance problems. My experience with XAVC-L is limited, so you could test it yourself. Another approach is rewrap (not transcode) it externally using EditReady, then import with "leave files in place": www.divergentmedia.com/editready

I have only tested EditReady on short XAVC-L clips but it seemed to rewrap them to .MOV at very high speed. This is supposed to consolidate all needed metadata into the .MOV file plus it enables importing to FCPX with "leave files in place". I use that method on AVCHD and it's by far the fastest and most reliable way to handle that media.

However if you do this I suggest you test it and examine the results closely before committing to full-scale production. That is only wise with any workflow change.

With EditReady the rewrap is still taking up space and is similar to FCP copying the media. I did a test and it was fast, I just worry about storage

lizjperl wrote: With EditReady the rewrap is still taking up space and is similar to FCP copying the media. I did a test and it was fast, I just worry about storage

Rewrapping with EditReady enables "leave files in place" which potentially enables a small portable library that can be emailed to collaborators, if that's important for your workflow. If you are using proxy, by default those are within the library and it takes additional steps to keep them outside. Again this is an issue if you need a "lean library". Ripple Training has a good class on this called FCPX 10.3 Media Management: www.rippletraining.com/products/final-cu...-final-cut-pro-10-3/

If you trust the rewrapped results of EditReady as your archival copy, after the brief transient phase of rewrapping, your total media copies is one, not including backups. By contrast if FCPX imports XAVC-L, you have one copy in the library and one copy on the disk. In theory you could delete the original disk copy of the folder tree but most people want to retain that .

The other approach is copy the MXF files outside the tree and import in place. Although I do that with XAVC-S, I would be uncomfortable doing that with XAVC-L or any other tree-oriented format without very extensive testing. There is always the risk some metadata is missing that you only find out about later. In the case of AVCHD it causes performance anomalies due to excessive I/Os.

Just curious, why not use the normal FCPX import? Is there a reason for the need to put the media somewhere and "leave in place"? You can still have a lean library if you point the library to a media folder on your RAID where you keep your media, and FCPX should handle the importing/wrapping automatically. Do all your organization/tagging in FCPX and not in the finder. FYI I do prefer graphics/audio to be organized in the finder and imported "leave in place," but this shouldn't be necessary for shot media.

Also, there are many cameras that will split up long shots into separate MXF files, for example, and this importing process is crucial for rewrapping these separate clips into a correctly spanned file. If you import these "leave in place" you will lose their proper metadata and you may have shots broken up into separate files... a big headache.

Finally, if you import it the normal way, you can do the normal proxy workflow. Also, it should go right into DaVinci through XML for color grading.

As for the problem of having duped media, you SHOULD keep a copy of all your raw media on drives and NOT edit off of that (in fact, you should probably have a dupe of that too, at least before you copy it to LTO tape or some other long-term storage). Your imported media should go onto your fast RAIDS, and your raw media should be completely separate. I don't wanna tell anyone what they have to do, but this is a good practice.

So I already have 3 copies of all our raw footage backed up on separate drives...not fast RAIDS but 8tb G drives. I work for independent film directors moreso than production companies and often these indie productions don't have the budget for raid drives and unecessaey copies of our media. I tried EditReady but we ultimately don't want to rewrap TBs of footage and then have to transcode from there. Doesn't matter if FCPX does the rewrapping or EditReady, I can work with these files in Premiere without having to rewrap and can made proxy files in Davinci onto my dedicated offline drives.

I've heard about longer MXF files being split but haven't encountered that yet with our footage.

lizjperl wrote: ...indie productions don't have the budget for raid drives and unecessaey copies of our media. I tried EditReady but we ultimately don't want to rewrap TBs of footage and then have to transcode from there. Doesn't matter if FCPX does the rewrapping or EditReady, I can work with these files in Premiere without having to rewrap and can made proxy files in Davinci onto my dedicated offline drives.

I've heard about longer MXF files being split but haven't encountered that yet with our footage.

Unfortunately this is an area of FCPX that badly needs improvement, yet most people don't mention it. E.g, on fcpx.tv, it is not within the top 100 items requested for improvement: fcpx.tv/top.html That is the difference in perspective between a final editor and an assistant editor, or someone who works with low shooting ratios vs high shooting ratios. Except for Tom Wolsky's suggestion on proxies (thanks Tom!) there is nothing whatsoever on that list about storage management.

In the current product state there is no good answer for your situation. It is not unique to the FS7 camera and XAVC-L but also applies to all other tree-oriented camera media including AVCHD and XAVC-S. With all of those you cannot import with "leave files in place", but must either:

(1) Import from the folder tree and let FCPX rewrap the content
(2) Copy the video files outside the tree and import, accepting possible problems
(3) Rewrap externally with EditReady and import "in place"

The simplest approach is just import from a disk copy of the folder tree and let FCPX rewrap it. The folder tree could then be archived, so you don't need it on-line. This import method copies to the library, which makes it big. However you also apparently need proxies which also by default go in the library. To avoid this requires setting the library storage location outside the library before the import and proxy generation.

Option #2 is what I use for XAVC-S, and for the reasons you mention. I want the fastest possible import and don't want wasted drive space since my doc team may shoot 1 terabyte per day. However anytime you copy files outside the media folder it's taking a risk since you never know the behavior. It may appear to work well initially then problems could show up later. AVCHD is a good example of this which can cause performance problems if imported "in place". With XAVC-S I have tested this thoroughly and currently believe if is OK, but it's always possible there are problems I haven't yet found. With XAVC-L I'd be afraid to do that without extensive testing.

That leaves option #3 -- Rewrap with EditReady and import "in place". That is very fast but does require transient disk space for both rewrapped and original media files.

You are right Premiere doesn't require these steps -- it can import "in place" from the camera tree. However all products have pros and cons which we must accept. E.g, Premiere has very rudimentary metadata management and is about 4x slower at transcoding and exporting H264 than FCPX on the same Mac hardware.

I haven't tested it specifically for XAVC-L files shot on Sony FS7, but...

What I normally do for an FS7 XDROOT folder is to bypass the Import Media dialogue:

1. Open the XROOT folder in an OS Finder window
2. Sort by KIND
3. Drag the MXF (or whatever) files into my awaiting FCPX Event

*** Yes, I just did a quick test with FS7 footage.
(Linear PCM, XAVC Intra 422/10 Class 100)
By dragging the MXF files into the event from the Finder, FCPX did-in fact- just point to the camera clones without transcoding/rewrapping.

I've done that with an FS5 for short films but I've read in other forums people running into issues with that method. Not sure why, I've never had any issues but just worry for a longform project, so I'm trying to do everything *right* to avoid hiccups later.

Maybe I should have been clearer, but your raw footage should just be stored on normal drives and put away. The media you're using for editing should be on the fast RAID.

To the point about letting FCPX do the importing and rewrapping... the rewrapping process is pretty much automatic and takes almost no time. The actual time for this is the copying to the RAID that the importing process does. This is no different than copying media from your drives/cards to your RAID and then dragging them in "leave in place". Btw, the segmentation problem is no different in Premiere: if your camera creates segmented MXF files, you'll still need to import them properly with the media browser to span the files into one clip.

As to joema's note about letting FCPX import making your libraries big. This does NOT happen if you point your library to an external location such as a folder on your RAID. The import process will COPY the media to wherever your point the library. You can store BOTH the media and the created proxies externally, on a RAID and outside your library through an import operation.

In general it's ok to import stuff into the library if you're the only person working on the film, but for your workflow where you're going to DaVinci externally, you should not do this.

I reiterate that this is your best bet. Click on your Library in the browser, and then go to the inspector and where it says "Storage Location" click on the "Modify Settings" button. Here's where you can change where the library stores the media, cache, templates, and backups. You would typically select some location on your RAID for the media and cache. After you do that, the import process will copy your media to that location.

By the way, you CAN use externally created proxies in FCPX if you want to. It's media just like anything else. In your workflow, you would just work with these proxies all the way up to the end, and in DaVinci, you would just relink to the full rez files and finish it there (or round trip back into FCPX with the colored, rendered files).
But in your case I would just use FCPX's built-in proxy workflow. The main advantage is that you can switch back and forth easily.

I'm not as concerned about the time it takes to rewrap the footage and already pointed my library to an external location. I don't need any more copies of my original media-- that's the main issue. I'm erroneously using the word "rewrap" and "copy" interchangeably but basically I just don't want FCPX to copy anything. So if I already backed up 8tb of footage onto 3 drives, FCP needs to make a copy of that and then transcode from there. I do agree that pointing your library to a specific location is the best practice for me personally.

Regarding your original problem--technically the relinking should have worked as long as the audio was the same and only the picture size was different. Did you try to use Compressor, and have the audio setting as "enable audio pass-through" so that it copies over the audio in-tact? AME may be doing some other weird things like adding metadata and things like that.

Also, DaVinci not getting the proper speed change is odd. What version of DaVinci are you using? One thing to try is bumping down the XML version when exporting the XML from FCPX (previous version 1.5). Maybe use the previous version of DaVinci Resolve that's not in beta. I do know that I've had speed changes go through properly to DaVinci Resolve before.

Anyway, it sounds like you're committed to externally-created proxies because of disk space issues, so hopefully that works out!

haysoos123 wrote: Regarding your original problem--technically the relinking should have worked as long as the audio was the same and only the picture size was different....

I believe she created proxies using Adobe Media Encoder and tried to relink those to an FCPX project. I've heard of people trying that but since FCP X has no option to relink offline proxy media, this is very tricky.

joema wrote:

haysoos123 wrote: Regarding your original problem--technically the relinking should have worked as long as the audio was the same and only the picture size was different....

I believe she created proxies using Adobe Media Encoder and tried to relink those to an FCPX project. I've heard of people trying that but since FCP X has no option to relink offline proxy media, this is very tricky.

Well, I think she wouldn't be using the proxy mode of FCPX at all. She would just be using the externally created proxy media as if it were the real media, and then relinking later either in FCPX or DaVinci. This is the "traditional" way.

The recommended workflow is to import the 4k, and then have FCPX create the proxies, but she says she doesn't have the space for this. Now, there is a way to actually just delete the original media after creating proxies, and just be in proxy mode for the entire edit, but you'd need to space to import and then create the proxies in the first place.

joema wrote: ...I believe she created proxies using Adobe Media Encoder and tried to relink those to an FCPX project....

haysoos123 wrote: ..Well, I think she wouldn't be using the proxy mode of FCPX at all. She would just be using the externally created proxy media as if it were the real media, and then relinking later either in FCPX or DaVinci. This is the "traditional" way.

Re externally created proxies (ie created outside of FCPX) and later relinking to the original 4k in FCPX, I don't think this is traditional at all for FCPX. She was trying that but understandably ran into problems. For FCPX the traditional, recommended way (as you described) is use only the FCPX internal methods for creating proxies and relinking.

haysoos123 wrote: .... there is a way to actually just delete the original media after creating proxies, and just be in proxy mode for the entire edit, but you'd need to space to import and then create the proxies in the first place.

That is a good point and I am using that workflow on a large documentary right now. It is possible to use FCPX-created proxy-only editing where all downstream editors have only the proxies, then later relink that to the original 4k media. This is a tricky area because it's not directly supported in the FCPX UI and there are issues with drive volume names and reliability of relink if the drive volume name changes. There may also be additional issues if loading Event XMLs from other editors, which I am still investigating .

However if you stay to the exact procedure described by Ripple Training in their 10.3 Media Management class, proxy-only workflow can work: www.rippletraining.com/products/final-cu...-final-cut-pro-10-3/

haha yes I have space for proxies! just not the "rewrap" copies But you're saying once FCP rewraps into its own "original media" I can delete that? If I want to relink to the 4k in FCP how would I do that without the original media? Or it would have to rewrap again...or just link in DaVinci? I just want DaVinci to reflect my speed changes but haven't worked with recent versions of it. Colorists I've worked with usually have me bake slow motion footage for them, so baking tons of slomo 4k footage in FCP seemed easiest.

For my original problem, I tried AME + Davinci and chose an audio pass through but I think it has something to do with FS7's audio configuration because linking back in premiere didn't work either. It only linked back the video and one audio channel and not the other 3 channels. But I can just link back to the 4k sans audio in Davinci.