Ex1 rendering times idea
Six months ago I bought a Sony Ex1 and since then have struggled on with impossibly long rendering times. An average finished edit to output to DVD would be about one and a half hours long and would have many layers and effects. Moves and changes to just one five second clip for e.g which needs 5 minutes of rendering with each adjustment of colour or length or speed or contrast or anything can build into an hour quite easily when all the other layers have to be adjusted and juggled individually too so you can imagine how long a one and a half hour video can take to complete. - Yes Weeks.
Basically my life is disappearing down a black hole called "render time" and any way to cut that seems good.
So I tried a few experiments today and noticed in one test that if I convert my xdcam files to pro res files,( which I did in MPEG streamclip ) the Video processing section of the sequence settings default to a 10 bit render. ( which it says provides a higher quality result when applying multiple effects or when the original material has more than 8 bits of precision, but takes longer to render.)
When I tested a short clip with a magic bullet effect applied it rendered in 15 minutes in 10 bit but only took 7 minutes to render in 8 bit.
Obviously I want the best render at the end of the day but that is one big time difference. So I am thinking if I convert my files to pro res and render them in 8 bit YUV it will be quicker to put everything together and at the end with everything in place just trash the render files and re-render everything in high precision YUV to get the best quality. - Which can happen overnight whilst I am sleeping so no time lost.
Is this just a stupid idea with no merits from a very tired brain and is it going to create another problem somewhere else. Is there even going to be a noticeable difference between 8 bit or 10 bit or 16 bit.
I can't go on with these render times anymore, it's seriously curtailing my creativity and the quality of work I would really like to be able to provide to my clients, not to mention keeping me on the poverty line and I am getting desperate.
Any advice would be appreciated.
I'm going to guess you use Final Cut Pro
I edit in XDCAM EX time lines but have renders set to ProRes. To me that's the best combination. There's no GOP confirm during editing since the renders are ProRes.
Firstly yes you are right i am using FCP sorry for not explaining that.
A few months ago I was doing exactly what you advise then after one particular problem of a lengthy render decided to output the finished edit as a reference movie to write to DVD. However I found I could not make a reference file and was advised of the problem which was that the render control needed to be the "same as the sequence codec". I have therefore been putting the compressor setting in the sequence settings on "pro res" and the render control on "same as sequence codec"
If I do as you say I will have to output the finished edit as a self contained movie which at the last attempt to do this said it was going to take twenty seven hours for a one and a half hour edit. (approx)
I thought therefore if I convert all the footage to pro res first then edit in a pro res timeline it would cut out a lot of the rendering which occurs when editing xdcam footage in a pro res time line. I can then make a reference movie to output to DVD
I am not great on the technical stuff so if I am talking rubbish please let me know.
I notice in the xdcam ex timeline setup you mention the video processing is in 8 bit which is not the high precision yuv that is available in the video processing options in the sequence settings. Which brings me back to one part of my original question. Do you know if there is really a noticable difference on this.?
If not then is my idea on my original post a valid one.
[brian paterson] "I thought therefore if I convert all the footage to pro res first then edit in a pro res timeline it would cut out a lot of the rendering which occurs when editing xdcam footage in a pro res time line."
The time it takes to render all the source material to Pro Res is likely to exceed the time it takes to render just the final 90 minutes to ProRes.
Me edits aren't FX heavy but I've output 90 minutes in about 3 hours on MacPro.
You could use a nanoFlash (MPEG2 I Frame I believe) or KiProMini (ProRes) when you shoot.
Can you tell me what difference your suggestion would make. I know nothing of these although having just googled the latter it seems to be a device for getting pro res files directly to fcp. The problem for me lies in what happens after that. It's at that point it all slows up and I don't know what the answer is.
I am running the laters version of fcp on an iMac 2.8 Ghz intel core 2 Duo with 4GB Ram. Would a faster computer be of any significant help.
An 8 second clip slowed down to 50 percent with lock and load some colour correcting, magic bullet and maybe a vignette takes around 11 minutes to render. If I then decide to make any changes it takes another 11 minutes if I decide to cut the length that's another 11 minutes to re-render and so on. - That's over half an hour for eight seconds and the more I refine the edit the longer it takes hence an hour and a half video taking weeks to produce. The manufacturers of these cameras and computers would really do themselves a favour if they concentrated more on something that worked quickly and efficiently rather than changing their products constantly. I don't know how anyone can be expected to make a living from the sort of work I do which is wedding videography when it all takes so long. Unless it's all just a chop and shunt.
Maybe if I downconverted to SD and gave up on the quality it might be quicker. All in all very dissapointing.
Maybe Sony should make all this clear in their advertising.
[brian paterson] "I know nothing of these although having just googled the latter it seems to be a device for getting pro res files directly to fcp."
The slowdown in export is due to GOP based codec. The aforementioned devices allow one to record directly to i Frame. They are not about getting anything into FCP so much as shooting in I Frame to begin with.
[brian paterson] "I am running the laters version of fcp on an iMac 2.8 Ghz intel core 2 Duo with 4GB Ram. Would a faster computer be of any significant help."
It's going to be slow for exporting from a GOP based codec that must have the GOP structure reconformed. You really need a 4 cores or more to work with EX or any GOP based codec in a timeline. I've tried working on a Core2Duo MacBook Pro and export is painfully slow on that as well.
You really have to weigh time vs money. Converting everything to ProRes after the shoot is time consuming and storage consuming especially on a slower computer. The alternatives are shoot I Frame (aforementioned devices) or us a faster computer. Some even do both.
Shoot I Frame to work on 2 core systems.
Use 8 core systems and then just encode the final master to ProRes if needed (and sometimes it's not).
You really need to think about your entire workflow. Speed is certainly important to me. I use CompressHD for H.264 export on desktop and MXO2 with MAX on laptop. Laptop is OK for SHORT FORM EX editing only. I wouldn't dream of doing 90 minute projects on a two core system. It's just not the right match.
Thanks for the explanation I can see I am going to need a whole pile of money if I am going to crack this one. At a quick guess looking on the web I would say another three thousand pounds. Out of the question of course as the work dosen't pay well enough. (Still paying for the ex1.)
Catch twenty two again. Need the money for equipment to save time but can't earn the money for equipment when it all takes so much time.
Isn't the simplest solution to import HD-SDI by playing back in the camera (or deck) through a capture card (Blackmagic Deck Link is cheap, although I use AJA Kona Lh) as if it were baseband video, into whatever ProRes codec you desire. The slight loss of time by importing in real time vs. as files often is made up on the export side. You usually have to log and/or view your material anyway so why not do it on import?
But John, you can't put a card in an iMac which is what Brian has.
Sorry,my bad. I guess then the cheapest solution is an AJA IO Express at about $800US.
These things are all new to me so I need to find out more about it and how it operates. It's great that you have given me your advise and have been so helpful.
I will do some research into this and hopefully it will provide me with a solution to my problem.
With Kind Regards
Doesn't that still need PCIe slot or Express port?
Basically it's the fundamental problem of expandability and why even the new iMac 4 Core, which otherwise might be close to MacPro 4 core, is hampered.
These is why many people want a low end MacPro because sometimes the connectivity can overcome limits weaker CPU. Of course since Apple is a hardware company (remember their business model is software sells hardware) they're not likely to go that route. Heck look at what they've done with Express port and MacBook Pro (Express port gone from 15") and what they attempted to do by removing firewire as well from MacBook for a time.
I very much appreciate the business situation you're in. ROI has to be the primary concern of any business these days. I've run into similar business decisions myself.
It's tough to be boxed into that corner. One may have to consider how to handle that sort of business problem. It's worthy of a business forum post. There are few of that kind in that forum though. You'd need to consider whether your price point is too low to fund the gear you need (and clients should be paying for) or maybe it's not bringing in enough work to have the margin for growth.
It seems the solutions are:
New 6 core MacPro
Convergent Design NanoFlash
AJA KiPro Mini
Render over night
Painful I know.
Are your clients really in a hurry or is this just an issue of convenience?
As long as you can play EX sequence set to ProRes fx renders there may be ways to play out to create a screening copy.
BTW throw in the demand for H.264 delivery and you're thrown down even further on the speed since there's no way to add CompressHD or MXO2 with MAX which require PCIe slot or Express port. Elgato H.264 Turbo only needs USB but the quality generally won't match MAX.
None of my clients have complained ( as yet) about the length of time I take and they all love the finished product. But this is a new business for me and have only done half a dozen videos this year.
Before the Ex1 and even before high def ( when this was a hobby) I remember the joy of the edit when ideas came quickly and the manipulation of the clips matched the pace of the creative juices flowing from the freshness of it all.
Now I fall asleep whilst waiting for things to render and I have to sit here 14 hours a day seven days a week watching my life dissappear. (I have recently taken up the guitar again which I practice whilst waiting for things to complete so that's a plus.)
But part of my transition into the professional world is my development of a more grand style of editing with many layers and dreamy transitions, all of which of course are very proccessor intensive.
To have it happen more instantaneously and to be able to do other things in my life would be good and so I need to find a quicker way of doing all of this, but times are tough and mony is scarce- both for me and my clients who are always on the point of not comissioning me at all because they are struggling to find the money.
If it were not for the fact that I have spent a considerable amount of money and several years upping my game to give a professional product then I might throw it all in and go back to creating art for a living.
Ideally if I could speed it all up I could do both.
Love the quality of the Ex1 - Hate all the problems and high additional costs that are coming with it.
Thanks for all you help and advice.
With Kind Regards
I long for the old days of on-line, A-B roll editing in $600.00/hr tape suites with the latest ADO and Super Hi-Band! :)
Daniel, I go further back than that. A & B roll editing FILM ! No problem with rendering or expensive edit suites. Cut all the footage into clips, actually see what you have, splice it all as you want it, synced up with your mag film audio. We did need a lab though to fuse it all together.
[Michael Slowe] "We did need a lab though to fuse it all together."
Labs had such slow render times.
Thanks Craig, I'll be using that one!
Have you tried "Search Posts"? Enlightenment may be there.
Actually, I was trying to make a facetious comment (in ten words or less) which was much better said here:
Alas, I failed on both counts.