FORUMS: list search recent posts

Is Pro-Res still an option for archiving?

COW Forums : Archiving and Back-Up

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Robert Withers
Is Pro-Res still an option for archiving?
on Dec 18, 2018 at 3:49:30 pm
Last Edited By Robert Withers on Dec 18, 2018 at 3:51:12 pm

I posted the below a few days ago . . . no one reacted . . . is it too newb a question? Not a "problem" topic?
Now related, Apple will be deprecating many codecs for any post-Mojave OS's -- how will this affect archiving?
And I'm still wondering, if a camera codec can be a small file with good quality, is there a small file codec that can be used for archiving? Instead of large "mezzanine" codecs like ProRes and DNxHD than some use?

See https://larryjordan.com/blog/important-dont-lose-access-to-older-media/?utm...

Thanks for any thoughts.
Robert

H264 vs Other for archiving
by Robert Withers
on Dec 14, 2018 at 12:47:19 pm

Hi all,
I'm on a list for media archives and libraries and many don't seem to know a lot about digital codecs(!).
I don't know more than a little (;-)) I archive my projects in ProRes.
One question that comes up when people are trying to make files that are small but have good, recoverable quality . . . are there h265 or h265 codecs that will preserve quality as well as large files like ProRes or DNxHD or other ? Or other good codecs that are small files with recoverable quality?
I know files like ProRes and DNxHD are big to enable frame-accurate editing, but they are often converted from smaller camera files that must be of the highest quality, no? Is there a way to reverse this process?
There are long and thorough answers to this I'm sure but would be grateful for any rules of thumb or top of mind.
I bothers me that I don't know.
Thanks,
Robert

Robert Withers

Independent/personal/avant-garde cinema, New York City


Return to posts index

Pat Horridge
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 8:38:01 am

The common thinking is lossless compression of 2:1 and that's either the J2K propriety route in something like MXF or open source FFV1 in MKV wrapper.
It's pretty much accepted that ProRes makes no sense going forward and many are looking at migrating their ProRes archives to FFV1.

Pat Horridge
Broadcast & Post Consultant, Trainer, Avid Certified Instructor
Free online Tutorials at VET digital media academy online http://vimeo.com/channels/752951
pat@vet.co.uk



Return to posts index

Robert Withers
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 4:57:54 pm

Thank you, Pat.
I have heard of some of the things you mention but don't really understand them. I will research MXF and FFV1. And whether my NLEs will export to them or if there is software to convert my ProRes to them.
Best,
Robet

Robert Withers

Independent/personal/avant-garde cinema, New York City


Return to posts index


Pat Horridge
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 5:01:21 pm

FFMPEG is a command line app that will convert to FFV1 in an MKV wrapper.
You can get FFMPEG for mac or pc.

Pat Horridge
Broadcast & Post Consultant, Trainer, Avid Certified Instructor
Free online Tutorials at VET digital media academy online http://vimeo.com/channels/752951
pat@vet.co.uk



Return to posts index

Robert Withers
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 7:04:04 pm

Thanks, I thought it was an app. Larry Jordan wrote about it. Now I need to learn about the FFV1 codec (is that right?)and the MXF and MKV wrappers as options for archiving. I have never used them. I wonder if NLEs will convert to them. I will research.
Thanks for your pointers!
Best,
Robert

Robert Withers

Independent/personal/avant-garde cinema, New York City


Return to posts index

Pat Horridge
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 7:06:59 pm

You won't get direct NLE support.
Export as SAS and convert outside.
. MOV is legacy now. So for FFV1 I'd look at MKV.

Pat Horridge
Broadcast & Post Consultant, Trainer, Avid Certified Instructor
Free online Tutorials at VET digital media academy online http://vimeo.com/channels/752951
pat@vet.co.uk



Return to posts index


Robert Withers
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 7:46:33 pm

Here's something I found about FFV1. Does anyone in production use it for archiving? Can a file be restored or recovered with standard equipment and software?

The reduced file sizes produced via FFV1 are exciting, but there are some downsides. Although FFV1 is open-source, the files will not play using standard video software on Mac and Windows, nor can FFV1 be utilized within commercially-available digitization hardware and software (only via terminal command). This is because no major company (Apple, Microsoft, Adobe, Blackmagic, etc.) has adopted the codec, or announced plans to do so. Any file format that does not eventually achieve widespread adoption and universal playback capability within the broadcasting and filmmaking communities, has a higher risk of long-term obsolescence, and lack of engineering support. https://blogs.library.duke.edu/bitstreams/2015/05/08/the-pros-and-cons-of-f...

That was from 2015. Has anything changed since then?
Thanks,
Robert

Robert Withers

Independent/personal/avant-garde cinema, New York City


Return to posts index

Pat Horridge
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 7:56:46 pm

Yes it has. There has been big move away from. Propriety solutions.
VLC player supports it as do a number of others. But you are right it stands outside current main stream solutions.
But think a second about the wisdom of following a propriety code solution and long term support. Maybe like ProRes and QuickTime.
This is exactly why the drive to open source is growing.

Pat Horridge
Broadcast & Post Consultant, Trainer, Avid Certified Instructor
Free online Tutorials at VET digital media academy online http://vimeo.com/channels/752951
pat@vet.co.uk



Return to posts index

Eric Strand
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 9:55:51 pm

Pat makes a good point about the move away from proprietary solutions; it makes logical and safe sense, in theory.

However, I will say he is the first person I've seen to say "it's pretty much accepted that ProRes makes no sense going forward"

As you pointed out none of the major players/companies/etc support FFV1. If you want to be absolutely safe, then you can archive in BOTH ProRes and FFV1, yes it takes up more space, but there's always going to be a tradeoff somewhere.

@ericstrand11


Return to posts index


Pat Horridge
Re: Is Pro-Res still an option for archiving?
on Dec 19, 2018 at 9:58:59 pm

If you start with ProRes then FFV1 doesn't gain you much in the short term. But ProRes isn't lossless. It's around 5:1 compression. FFV1 is lossless. For archiving a 2:1 compression that's lossless is a massive plus.

Pat Horridge
Broadcast & Post Consultant, Trainer, Avid Certified Instructor
Free online Tutorials at VET digital media academy online http://vimeo.com/channels/752951
pat@vet.co.uk



Return to posts index

Robert Withers
Re: Is Pro-Res still an option for archiving?
on Dec 20, 2018 at 6:21:27 pm

Thanks Eric. My initial speculation was about the possibility of archiving to a compressed format like cameras use instead of large-file mezzanine codecs like Pro-Res. So archiving to 2 codecs might be a good idea but is it possible to archive to a compressed format with smaller file sizes? We often convert high-quality compressed camera codecs to large files for editing -- is it possible to go back the other way?
Best,
Robert

Robert Withers

Independent/personal/avant-garde cinema, New York City


Return to posts index

Pat Horridge
Re: Is Pro-Res still an option for archiving?
on Dec 20, 2018 at 6:27:40 pm

Yes you can go the other way but unless it's lossless you are compromising quality. Will that be visible today? Maybe not but will it suffer when uprezzed in 6 years time? Too bad if it does as you have made that choice.
Also you have issues with concatenation of codecs if you use another long GOP codec as a archive method.

Pat Horridge
Broadcast & Post Consultant, Trainer, Avid Certified Instructor
Free online Tutorials at VET digital media academy online http://vimeo.com/channels/752951
pat@vet.co.uk



Return to posts index


Robert Withers
Re: Is Pro-Res still an option for archiving?
on Dec 20, 2018 at 7:15:17 pm

Thanks, Pat,
This is interesting. So we can go from a compressed camera codec to a mezzanine codec without (unacceptable) loss of quality but we can't reverse this to go back to a camera codec literally or another more compressed codec without loss. "Concatenation of codecs" is poetry but I don't understand the issue . Or at least what the issue would be if you went from an intra codec like ProRes back to a long-GOP codec. I do read that ProRes is itself a lossy compression system. Hmmn.
Cheers,
Robert

Robert Withers

Independent/personal/avant-garde cinema, New York City


Return to posts index

Eric Strand
Re: Is Pro-Res still an option for archiving?
on Dec 20, 2018 at 7:34:07 pm

ProRes is lossy technically, but it's visually lossless; thousands of people around the world are using and archiving with it.

Camera files that are "small" use Long GOP compression and probably 4:2:0 color subsampling. Using an ALL-I codec means you're back in the range of ProRes data rates, in which case just store and archive in ProRes and FFV1. ProRes so you can keep using it for at least the foreseeable future; FFV1 because it's open source.

@ericstrand11


Return to posts index

Pat Horridge
Re: Is Pro-Res still an option for archiving?
on Dec 20, 2018 at 7:17:34 pm

Your camera codec will have introduced some losses based on the day rate etc. But you can't do much about that (some use external recorders to avoid compressed codecs)

Pat Horridge
Broadcast & Post Consultant, Trainer, Avid Certified Instructor
Free online Tutorials at VET digital media academy online http://vimeo.com/channels/752951
pat@vet.co.uk



Return to posts index


Peter Malling
Re: Is Pro-Res still an option for archiving?
on Apr 27, 2019 at 9:20:32 am

@Robert Withers, I found this old thread and I'm curious to hear what you ended up doing?

I would like to archive my 200h super8/mini-dv/vhs video in a format that's also suitable for editing, as encoding to a new codec takes many hours for just one videotape on my rather outdated computer. And also just for simplicity. I don't get the idea of archiving in both ProRes AND FFV1 - that brings up the total storage to a level where you could just as well save it all in an uncompressed format.

All together I've been surprised by the large file sizes even for compressed files with these lossless or near-lossless codecs. When starting digitizing I thought H.264 would be just fine, and that's also what most home movie digitizers use. But it would be a pity to realize that in 5-10 years valuable info has been lost that could have been used for new technology that may appear, to utilize the old footage in new ways. I expect that AI-based tools that can bring old footage up to perfect quality will appear in few years.


Return to posts index

Robert Withers
Re: Is Pro-Res still an option for archiving?
on Apr 27, 2019 at 6:30:08 pm

Hi Peter,
I haven't done anything, since I don't have any special needs outside of maintaining my own backups.

However, there are interesting discussions with some controversy that relate to the Apple ProRes codec and maybe some risks of proprietary codecs.

Here are two things:
!st from Larry Jordan,
Earlier today, Richard W. sent me this question: "Something came up that I [need explained.] ...The auto-update on my MacBook Air Mojave prompted me to update iMovie from version 10.1.1 to 10.1.11 with the curious message: 'Detects media files that may be incompatible with future versions of macOS and converts them to a compatible format.... I really hate the idea of any software converting my media files without my clear understanding and approval so I didn't update this iMovie software... I wonder if you know anything about this?"

Yes, and it is critically important that you understand the answer. Apple has deprecated all codecs based on QuickTime 7. If you upgrade to the next version of macOS AFTER Mojave, media using these codecs WILL NOT PLAY and CANNOT be converted! In other words, if you ignore this warning and upgrade, you are screwed. (Here's an article that explains what is happening.)

What Apple has done, before the next OS upgrade, is provide a conversion utility in Final Cut, iMovie, Compressor, and Motion that recognizes these out-of-date codecs and converts them into something more future-proof; specifically ProRes 422. What this feature does is recognize when you've opened media which will soon be obsolete and gives you the ability to convert it.

This conversion is NOT automatic, nor behind-the-scenes. You have to click OK for the conversion to occur. However, if you DON'T convert your media, you will not be able to upgrade to the next version of macOS. (Here are two articles illustrating this conversion process in Final Cut Pro X and Compressor. Compressor is more flexible.)

Please, to prevent future problems, read my background article above and these newer ones showing how this process works. Because, once you upgrade macOS, you can't go back and you can't play your older media. Also, if you are debating what to do, please turn OFF automatic updating of your system. Here's an article that explains how. This will prevent unexpected, and unpleasant, surprises.

2nd -- I got complex responses to posting this on a list of archivists. Some disagreed with Larry and one posted this link: https://support.apple.com/en-us/HT209000

My solution until the dust clears: keep a machine that runs Mojave for future needs.

Good luck!

Robert

Robert Withers

Independent/personal/avant-garde cinema, New York City


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2019 CreativeCOW.net All Rights Reserved
[TOP]