FORUMS: list search recent posts

Uploading 20 Gb. of raw footage?

COW Forums : Apple Final Cut Pro Legacy

<< PREVIOUS   •   FAQ   •   VIEW ALL   •   PRINT   •   NEXT >>
Paulo Jan
Uploading 20 Gb. of raw footage?
on Feb 16, 2011 at 10:45:24 pm

Hello:

Sorry for posting this "OMG I'm in an urgent situation please help me" question, but... I'm in an urgent situation. It turns out that we've been tasked with recording a series of interviews (here in Spain) and sending them to the other side of the pond for a TV special to be broadcasted on Sunday. The raw material has to be in the client's servers by Friday morning at the latest, and we are recording in DVCPRO 50. They have asked us for the camera originals, so that the client can do the editing there. Now:

-Considering that DVCPRO 50 is 50 Mbit/sec., i.e., 21 Gb. per hour.
-That the complete interview footage will be about 1 hour (or even more).
-That the typical ADSL connection around here offers only 512 Kbps. upload speed, that is, about 50 Kb./sec.

Just by doing a bit of math, you quickly reach the conclusion that this simply isn't workable (leaving the computer uploading for the entire night would allow only 1.7 Gb.). However, I know that other people in the industry FTP files to clients, coworkers, etc., constantly, so my question is:

-Have you ever been in the need of doing something like this? I.e., not just uploading a H264 for client approval, or a Photoshop file, but 20 Gb. of raw footage.
-If so, what kind of Internet connection are you using?

Basically, I want to know if what we've been tasked with doing is feasible at all, with the resources that we have (a regular ADSL line, no dedicated network).

Thanks.

Paulo.


Return to posts index


Dave LaRonde
Re: Uploading 20 Gb. of raw footage?
on Feb 16, 2011 at 10:54:53 pm

Rent satellite time and have the client record at the other end. Not exactly camera-original, I fear, but it could be close in quality. Or air-freight the tapes/cards RIGHT NOW after they've been duplicated for safety. If you can't find the bandwidth to make the transfer, they're viable alternatives on such a tight deadline.

I'd like to know what the client was thinking in the first place. The term "magic wands" comes to mind.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index


Shane Ross
Re: Uploading 20 Gb. of raw footage?
on Feb 16, 2011 at 11:24:16 pm

There's always FedEx...but it might be spendy...

Shane

GETTING ORGANIZED WITH FINAL CUT PRO DVD...don't miss it.
Read my blog, Little Frog in High Def


Return to posts index


Paulo Jan
Re: Uploading 20 Gb. of raw footage?
on Feb 16, 2011 at 11:41:48 pm

[Dave LaRonde] "I'd like to know what the client was thinking in the first place. The term "magic wands" comes to mind."

It's more like "OMG teh Internet is awesome!!! We can communicate and send files to peoples all around the world!!1!!11!" and getting used to that mindset, without realizing that some things are still too huge to be sent online. (One of the people involved in the whole scheme is a sound engineer, and he works regularly from Spain with studios in L.A.; of course, sound files aren't the same than what we're trying here... :-) ).

I'll ask my producers tomorrow to research both the satellite link and the Fedex options. (And just from a merely academic point of view, I'm now curious now about what the people in the film/FX industry do: all those stories about studios in London or Australia working with editors and producers in Hollywood, FTPing each other files... Of course, I guess they have dedicated networks).

Thanks for your quick replies (and I meant it: thanks *a lot*).

Paulo.


Return to posts index

Dave LaRonde
Re: Uploading 20 Gb. of raw footage?
on Feb 16, 2011 at 11:56:58 pm

[Paulo Jan] "I guess they have dedicated networks"

Perhaps not dedicated networks, but very close to it. And expensive!




[Paulo Jan] "One of the people involved in the whole scheme is a sound engineer, and he works regularly from Spain with studios in L.A.; of course, sound files aren't the same"

Let's do a little thinking. An hour of studio-quality, 96kHz, 32-bit audio might have a file size 2.5 GB? Does that sound about right?

Tiny, isn't it?

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index


Eric Peters
Re: Uploading 20 Gb. of raw footage?
on Feb 16, 2011 at 11:26:06 pm

In the past, I have delivered hourlong ProRes-encoded episode masters via the web, but only by relying on a combination of (a) a speedy internet connection (T3) and (b) WAN-acceleration via Signiant web client provided by the network I was delivering to. So, it is definitely possible and can be a feasible means of delivery given the technology necessary for the job, but even then, the upload took hours.

Unfortunately, in your case, I think Dave and Shane have covered the only reasonable options... Either satellite feed or some form of air-freight/courier/FedEx delivery.

Eric Peters
ProMAX Systems
eric.peters@promax.com



Return to posts index


Ron Pestes
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 12:09:13 am

I would use FedEx or UPS Next Day for reliability. UPS is usually cheaper. I think uploading that much data leaves too many chances for failure.

Apple Certified Master Pro FCS 2
Sony EX-3
MacBook Pro


Return to posts index


Rafael Amador
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 1:20:00 am

Carps!!.
The only problem is that they want the "original files", if not I would convert that to an H264/HighProfile (422) and upload it. In the other side transcode it to 8b Uncompressed to be edited.
That is just one hour footage, and at some 10Mbps (great for SD) can be uploaded really fast with your home internet.
rafael

http://www.nagavideo.com


Return to posts index


John Christie
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 5:09:18 am

I'll second Rafael's suggestion.

I had a request from a broadcaster for a few shots to cut into some promo material ASAP. As a test, I transcoded some HD ProRes footage to h.264 (using the default h.264 setting in compressor) and then back to ProRes. Apart from a small gamma shift, you had to look very hard to see any artifacts int eh second generation ProRes file. The broadcaster on the other side of the country was happy to get h.264 files that day rather than waiting for Fedex.

Cheers

John C


Return to posts index


Paulo Jan
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 12:15:00 pm

Yes, this.

I thought of that idea (encoding to H264) after I went to bed last night, and today I've been doing some tests. It's workable... I think. The only problems I've seen so far are:

-The default H264 encoding preset in Compressor outputs a progressive file (it deinterlaces automatically), which is a no-no in our case.
-File sizes aren't as small as I expected. With the default H264 preset, the resulting file is around half the size of the original, which means that it's still 10 Gb.
-Also, there's the gamma shift.

We've also been asking around and found some companies with faster Internet connections that can give us a hand, etc., so I think we're out of the woods. Thanks again to everyone who responded.


Return to posts index

Rafael Amador
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 1:12:37 pm

[Paulo Jan] "-The default H264 encoding preset in Compressor outputs a progressive file (it deinterlaces automatically), which is a no-no in our case."
Paulo,
You are a Pro. Stop using "Default presets" and learn how to customize them.

[Paulo Jan] "-File sizes aren't as small as I expected. With the default H264 preset, the resulting file is around half the size of the original, which means that it's still 10 Gb."
Limit the data rate and ALWAYS set Key-Frames Auto.
With a 10Mbps (10.000Kbps), your file will be reduced to 4GBs.
Being SD, you can still reduce it.

I would recommend you trying "HandBrake' for the H264.
- Supports Interlaced.
- Supports H264/High Profile.
- Accepts DVCPro50 sources (No prores).
rafael

http://www.nagavideo.com


Return to posts index

Paulo Jan
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 2:32:48 pm

[Rafael Amador] "Paulo,
You are a Pro. Stop using "Default presets" and learn how to customize them.
"


Oh, I did that :-). Problem is, the interlaced version takes more space than the progressive one. With the default preset, the resulting file takes 50% of the original. With the default preset, modified to disable frame control, it takes about 60% of the original.


Paulo.


Return to posts index


Rafael Amador
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 4:05:15 pm

[Paulo Jan] "Oh, I did that :-). Problem is, the interlaced version takes more space than the progressive one. With the default preset, the resulting file takes 50% of the original. With the default preset, modified to disable frame control, it takes about 60% of the original."
Paulo,
From Interlaced to progressive can not be any big difference in file sizes. On both you are dealing with the same number of pixels per frame, so you need the same data to write the info.
The key for the file size is on the data-rate and the k-frames you set on the H264 compressor.
rafael

http://www.nagavideo.com


Return to posts index

Paulo Jan
Re: Uploading 20 Gb. of raw footage?
on Feb 18, 2011 at 2:06:03 pm

I know it sounds strange, but as God is my witness: take a 1.4 Gb. test clip in DV. Encoded with the default H264 setting. Result: 683 Mb. Take the default setting, copy it and disable frame control, in order to prevent deinterlacing (and leave everything else the same). Result: a 854 Mb. file.


Return to posts index

Keith Pratt
Re: Uploading 20 Gb. of raw footage?
on Feb 19, 2011 at 1:56:50 am

Paulo Jan: "Encoded with the default H264 setting. Result: 683 Mb. Take the default setting, copy it and disable frame control, in order to prevent deinterlacing (and leave everything else the same). Result: a 854 Mb. file."

Is that because you're using the 'automatic' setting for data rate, and deinterlacing is cutting down on the resolution/detail the H.264 encoder will try to retain?


Return to posts index


Paulo Jan
Re: Uploading 20 Gb. of raw footage?
on Feb 19, 2011 at 8:50:16 pm

Maybe. The default setting for the data rate is "automatic", and I didn't touch it. However, my understanding was that the deinterlacing in Compressor didn't simply discard one field, as most crappy deinterlacers do, but did blend both fields to create the new image.


Paulo.


Return to posts index

Keith Pratt
Re: Uploading 20 Gb. of raw footage?
on Feb 20, 2011 at 12:34:06 am

Paulo Jan: "However, my understanding was that the deinterlacing in Compressor didn't simply discard one field, as most crappy deinterlacers do, but did blend both fields to create the new image."

It does interpolate, but I wasn't necessarily meaning detail in terms of image fidelity. If the compressor treats a pair of fields as a single frame, the combing is additional detail. If Compressor or Apple's version of H.264 is unable to compress fields separately, 'Automatic' will yield a larger file in its attempt to preserve the "detail".

If you don't want the file size to free-wheel, set a target file size.


Return to posts index

Rafael Amador
Re: Uploading 20 Gb. of raw footage?
on Feb 20, 2011 at 1:45:17 am

I agree with Keith.
Limit the data-rate, and always set K-frames AUTO.
The "Multi-pass" option is to let the compressor set an optimal number of K-frames.
If you limit the K-frames, you are screwing the "Multi-pass", affecting size and compromising quality.

[Paulo Jan] "Maybe. The default setting for the data rate is "automatic", and I didn't touch it. However, my understanding was that the deinterlacing in Compressor didn't simply discard one field, as most crappy deinterlacers do, but did blend both fields to create the new image."
Whatever the method of de-interlacing, a Progressive frame has the same number than an Interlaced frame (two fields).
What is obvious is that, although the stream is passing the same amount of pixels/sec, the Interlaced stream is passing DOUBLE amount of PICTURES/sec. So the compressor may be adding more (double?) amount of k-frames/sec.
That would explain the different file size.
rafael

http://www.nagavideo.com


Return to posts index

David Roth Weiss
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 5:23:58 pm

[Paulo Jan] "I thought of that idea (encoding to H264) after I went to bed last night, and today I've been doing some tests. It's workable... "

Not a good idea!

1) h.264 encoding is very slow on your side.

2) h.264 is not an editing codec and will need to be transcoded to another codec on your client's end.

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
http://www.drwfilms.com

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


A forum host of Creative COW's Business & Marketing and Apple Final Cut Pro forums. Formerly host of the Apple Final Cut Basics, Indie Film & Documentary, and Film History & Appreciations forums.


Return to posts index

Ken Jones
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 3:04:59 am

I had a hard drive with about 200GB of P2 files sent over from Norway a few months ago. I asked them to send it the fastest way possible. They sent it using DHL's fastest, most expensive service and it still took 3 days. It sat in U.S. customs for over 24 hours.

I am used to FedEx Priority Overnight with delivery the next day before before 10:30 AM so I was quite surprised when it took THREE DAYS from Europe!


Return to posts index

Rafael Amador
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 3:36:57 am

[Ken Jones] "I am used to FedEx Priority Overnight with delivery the next day before before 10:30 AM so I was quite surprised when it took THREE DAYS from Europe!"
With a weekend in the middle that's 5 days.
rafael

http://www.nagavideo.com


Return to posts index

Gary Askham
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 10:02:50 am

If you have a nearby Post Production facility they should have a DigiDelivery service (or something similar).

Give them a hard-drive and they should deal with the rest (at a cost).

------------------------
FCP and Avid Technical Support
Air Post Production
Shoreditch - London


Return to posts index

Shane Ross
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 10:04:27 am

Yeah, I deal with digital delivery, and we typically upload between 2GB and 13GB. 13GB takes us a good 14 hours. But the speed is dependent on your internet conection. ISDN will be tons faster than cable or DSL. and there are degrees of service.

Shane

GETTING ORGANIZED WITH FINAL CUT PRO DVD...don't miss it.
Read my blog, Little Frog in High Def


Return to posts index

Andree Franks
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 10:39:13 am

IDSN be faster then DSL???
WOW in our Country we make fun of People that still use ISDN...

;)



Return to posts index

Dave LaRonde
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 3:43:08 pm

I still stick with my first suggestion; forget file delivery via the web. Go rent some satellite time.

An hour's worth of video is across the ocean in an hour's time.

Dave LaRonde
Sr. Promotion Producer
KCRG-TV (ABC) Cedar Rapids, IA


Return to posts index

Joey Burnham
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 7:21:39 pm

I agree with Dave. Satellite it or even maybe fiber feed if they are willing to pay.

You inquired earlier about how large studios are sending huge files to one another these days. Basically, you get what you pay for regarding internet service. We just upgraded to copper internet lines for $$$ but our upload speeds are about 2 MB a second. A gig only takes about 10 minutes or less to go up.

Joey


Return to posts index

David Roth Weiss
Re: Uploading 20 Gb. of raw footage?
on Feb 17, 2011 at 8:09:40 pm

[Joey Burnham] "Basically, you get what you pay for regarding internet service."

I completely agree... Trust me, when Spielberg wants to see his dailies, the studios pay, and he can see them anywhere.

It's all about what you're willing to pay for the luxury...

David Roth Weiss
Director/Editor/Colorist
David Weiss Productions, Inc.
Los Angeles
http://www.drwfilms.com

POST-PRODUCTION WITHOUT THE USUAL INSANITY ™


A forum host of Creative COW's Business & Marketing and Apple Final Cut Pro forums. Formerly host of the Apple Final Cut Basics, Indie Film & Documentary, and Film History & Appreciations forums.


Return to posts index

Mark Petereit
Re: Uploading 20 Gb. of raw footage?
on Feb 21, 2011 at 8:01:22 pm

Is bribing a MIS student at a local university out of the question? I would think a major university would have pretty fat pipes. Hungry students are universal.


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]