How to archive when originial is 422 ProRes?

When I record to my Atomos Shogun in ProRes 422, I don't need to create optimised files. This is great.

However when I have finished a project and delivered to output to the client, I want to archive my Library to slow NAS storage instead of fast SSD storage, which was used for cutting.

I don't NEED to have 422 original video, so what do you recommend? Can I convert the originals to H.264 and still maintain integrity? Proxies are obviously not the answer, because they still need the original media in case I need to do a re-cut later. Should I use Compressor or Handbrake to compress the 422 files?

What do you guys do? Or recommend?

BUMP

Lots of views, but not a single comment! Don't hold back guys...

H.264 may be a smaller file size, but it takes much more CPU effort to playback than ProRes. Because of the highly compressed "Long GOP" (Group of picture) scheme, in a nutshell, pretty much every frame in the "group" (usually 8-12 frames) must be read to play just 1 frame back, and over again and again for each frame. So though you save drive space, you lose performance in playback. Keep the ProRes for "slower" storage, for sure!

Plus, there's to reasonable way to replace all your ProRes originals with H.264 highly compressed files inside the Library. Again, I'd consider it a bad idea to do so, anyway.

Ok, so you gave a good reason against H.264, but what about other alternatives?

If I shoot in camera compressed file it looks pretty ok. I can then generate optimised ProRes 422 files, which can later be deleted. How about starting with ProRes 422 out of my recorder, but then generating a compressed file for archiving? 95% of the chance I will never go back to it anyway. Or should I just:

a) Export a ProRes master file and delete the rest of the project, or
b) Only save the used clips (in ProRes 422) and delete all unused and un-favourited content

I really don't want to have Peta Bytes of unnecessary video footage, on the off-chance that one day I may find something useful.

What do other people do?
1. Do you keep everything
2. Do you just keep the edited Master file
3. Do you keep only used + Favourite contents

"If I shoot in camera compressed file it looks pretty ok. I can then generate optimised ProRes 422 file..."

Why optimize footage that works just fine? No reason for that.

1. Yes.
2. Depends.
3. No, because I usually toss out what's not useful during my edit.

Clips, photos, audio, graphics that are just not going to be used, I toss out as I go along on my edit. Anything there when I'm done, I need, or have a reason to hang on to.

Archiving Libraries is going to take up drive space, yes. But if you're getting paid for this stuff, you should include the price of archiving in the client's costs. If you do this for fun, well, that's the price of digital video.

When you are lucky enough to have ProRes original camera stock, I believe that you should not go out of ProRes....

The solution would be to re-encode to a lighter ProRes even way down to "ProRes Proxy" instead of H264.
ProRes is very consistent and faithful in multi generation situations which is not the case for Other codecs.
The only other codec which is trying to be as good is Avid DnX family. But I won't comment on the ....

An hour of 24p HD format in PR422 is about 53Gb, PR422LT is 37 Gb PR422_proxy is 16 gb.

ProRes_proxy at HD29.97 is still a 45mb/s codec it is a bit low for broadcast but it has such a hi-fidelity and can still sustain multi generation Editing. H264 is a good camera or final delivery codec but it degrades real fast.

The goal of every Apple ProRes family member is to perfectly preserve the quality of the original image source, be it good or bad.

Thank you, Francois.

I found Apple ProRes 422 Proxy in Compressor. I presume that is the best way to make these?

This would give a saving of about 70% in storage, which is well worth taking.

PR Proxy sucks for color grading, and is only 1/4 the resolution (but same frame size) of the original. Just know that if you use it, you're losing image data. H.264 would give you better image quality for the file size.

Not to confuse FCPX 1/4 resolution proxies which happen to be ProRes_proxy codec... I know 45Mb/s is low bit rate and bigger than H264 but ProRes is a great codec. As for color grading ProRes 422 is OK but if you want range HQ is the way to go.
Anyway I think this white paper says it all:
www.apple.com/final-cut-pro/docs/Apple_ProRes_White_Paper.pdf

EDIT: I myself would only go down to ProRes_LT if the Original camera clips can be of future value in other production but archives that might not be useful Proxy is OK. An other solution could be Worx4 X : you could string out all the parts of your original shots and use this software to trim down ... then use PR422LT.

"ProRes 422 is OK but if you want range HQ is the way to go."

Nope, transcoding to HQ does not "add" anything to the image. You have what you shot, that's it. Increasing to a codec that his higher bit rate does not in any way make color grading better. No image data is added. That you can is an old myth. If you shot in H.264, you can transcode to ProRes 4444, but you still only have the image data shot in H.264, you've dated drive space for nothing.

If shot in ProRes 422, stick with it. Going down to ProRes LT would be OK if you're willing to lose image data. Depending on what was shot, with what camera, it may or may not make a difference.

I agree with you on the myth... I only wanted to specify that when a project needs a lot of work on the color grading it should be shot ProRes422HQ and higher and that PR422 was just ok for broadcast. Sorry if that came out wrong ...

422 actually is greater than what broadcast requires.

My understanding is that transcoding to ProRes 422 (aka create Optimized version) will not improve content, BUT will make editing faster by not having to reconstruct intermediate frames on the fly.

This has nothing to do with my original question, which relates to having the original recording in 422 from an external Atomos Shogun Flame recorder, and then archiving the project with a lesser quality compressed file.

pszilard wrote: …archiving the project with a lesser quality compressed file.

just to feed my curiosity: What for?

Archiving the final outcome (to avoid misunderstanding the word 'project') in some compressed format, to put it in the shelf - ok.
Archiving the final outcome (to avoid misunderstanding 'project') in 'master' format, in case you need parts of the movie in future projects - ok.
Archiving a project = in editable form - ok, if you know, you will do, but that demands all sources in 'best' format/native.

… but why archiving a project = editable BUT deteriorating the sources - … ? Why recording then in proRes anyhow?

I shoot usually on a Fuji X-T2, and record to a Shogun Flame.
Why?

1. I have unlimited recording time - for lectures and events, which can easily exceed 30mins per take
2. I don't need to create optimized footage when editing

Archiving a project (as in a Library for a job).
Why?

1. Once I have delivered an accepted end result to the customer, it is extremely low likely hood that we need to go back to the source
2. Source material is client related and not suitable for other future jobs. I.e. NOT "stock" type of footage
3. End product is likely to be used on intranet for employees, so does not require cinema quality
4. I thought it would be prudent, however, to have the source in "YouTube" quality in the off-chance that the client requests another cut, such as a "Highlights Summary"
5. I was trying to be cost effective, in archiving
6. I had an academic curiosity on what would be an acceptable process and what others might be doing

Looks like most people on this forum - that responded, anyway - prefer to keep full 422 source. I am good with that.

Hope this clarifies, where I am coming from...

okayyy…

still not there, 1) why archiving a Project (=everything, used or unused) and not a Master(=just the 'good' ones)… but anyhow. For final storing, any 2.5" drive offers 1TB/~70€ (lately saw one of those tiny beasts fitting 4TB! my goodness!)


and 6), academical curiosity = that forced me to ask.

So, I can not contribute to your Original question - after final reports are finished and archived in a Master format, and probably a highlight reel is done, I dump a years work library

… yep, makes me shiver every time to confirm "Do you want to delete 1.4TB…" - :woohoo:

Yep, why degrade the quality of footage if you're going to work on it later?
Why save something you will never use again?

pszilard wrote: My understanding is that transcoding to ProRes 422 (aka create Optimized version) will not improve content, BUT will make editing faster by not having to reconstruct intermediate frames on the fly.

True, but the files are bigger, so depending on where your bottleneck is, might cause other issues if your drives are not fast enough...

Ok, so why not set your Shogun to record ProResLT or even ProRes Proxy to begin with? It will still look better than the internal codec and save you space and you don't have to waste time converting the files for archiving...

Why not archive just the stuff in your project to the NAS and the rest of the unused dailies to offline archive hard drives? If its' really low likelihood that you'll want to use this footage again, then storing it offline (on either spinning media or LTO) is the most cost effective way clear up space on your edit and NAS drives. I would be loath to alter original camera files. If you really need to see them more regularly for reference, make lower bit rate H264 for reference, but keep the originals offline.

Ok folks. Thanks for the ideas and discussions. I have digested it all and as far as I am concerned, personally, I feel that we have exhausted all angles of this topic, so I shall bail out of this thread.

Feel free to keep chatting, if you wish to, and have a wonderful day!

Thanks again...