FORUMS: list search recent posts

Best compression for the web

COW Forums : Compression Techniques

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
Tiago Casais
Best compression for the web
on Jul 21, 2010 at 5:27:54 pm

Hello everyone,

I have a 1m:35s motion graphics video (pal widescreen square pixel) i have to publish on the web (streaming).
The video was created in AE and the plan is to render it from AE and then use Sorenson6 to compress it so it plays smoothly with best quality possible, as an flv. The final frame size will be half the 1050x576 original one from the pal preset.
I have some doubts regarding the best procedures/technics in order to obtain the best results possible. The web crew is allergic to megabytes and the client's server runs on coal! I need your help!
So here it goes:

Reduce frame size to half - should i be doing this in ae or squeeze ?

Reduce fps from 25 to 15 or 12 - is it any good at reducing file size maitaining playback?

Since the goal is an flv file, should i be rendering from AE in flv or in an uncompressed format such as AVI or MOV and then convert it to flv within squeeze ?

Compression always CBR (since its for streaming)?

Is a compressor like SSQ the best tool to reduce the file size ?And which is the best FLV codec ?

Any commonly known way to reduce noise while preparing the comp to render within AE?(some filter or some other ae feature i should know about?)


Any other pointers on this issue would be much appreciated. I've read/researched other posts in this forum but couldnt get much from them for my situation.
Im getting 7/9mbytes flv files, could this be considered heavy for web streaming playback?

Thanks in advance for your attention.
Best regards

Tiago


Return to posts index

Daniel Low
Re: Best compression for the web
on Jul 21, 2010 at 9:24:22 pm

Here's a good start
http://forums.creativecow.net/readpost/20/862224

1024x576 is PAL square pixel - not 1050x576

Do not reduce your frame rate.
Resize in AE
Export as ProRes if you have it or PhotoJPEG at 90% or uncompressed

[Tiago Casais] "Any commonly known way to reduce noise while preparing the comp to render within AE?(some filter or some other ae feature i should know about?)"

Noise in motion graphics? What do you mean?



__________________________________________________________________
Sent from my iPad Nano.


Return to posts index

Tiago Casais
Re: Best compression for the web
on Jul 22, 2010 at 1:34:29 pm

Hi, and thanks for your answer.

What i meant with the "noise" was if there are particular things you can do ( such as changes in the lights, be carefull with moving backgrounds, dont allow very dark areas, dont use gradients etc etc) so that your video maintains good quality when compressed, thus avoiding pixelation and artifacts in the final compressed version. I cant find any documentation on that.

Should i make the ae render in flv, or avi/mov and then transform to flv in the compressor ?
Why not change the frame rate ? I tried it and the playback seems fine.

note: palwidescreen square changed from 1024 to 1050 in cs4

cheers



Return to posts index


Daniel Low
Re: Best compression for the web
on Jul 22, 2010 at 1:49:29 pm

[Tiago Casais] "Should i make the ae render in flv, or avi/mov and then transform to flv in the compressor ?"

Make the AE render in AVI or mov - like I said in my previous post.

There is no need to change the frame rate - why would you want to?


__________________________________________________________________
Sent from my iPad Nano.


Return to posts index

Drew Jensen
Re: Best compression for the web
on Jul 22, 2010 at 7:08:41 pm

Q: Reduce frame size to half - should i be doing this in ae or squeeze ?
A: I haven't used squeeze so I will abstain from answering.

Q: Reduce fps from 25 to 15 or 12 - is it any good at reducing file size maitaining playback?
A: No frame rate reduction. Keep frame rate same as original for best results. File size shouldn't differ much if reducing.

Q: Since the goal is an flv file, should i be rendering from AE in flv or in an uncompressed format such as AVI or MOV and then convert it to flv within squeeze ?
A: Always keep in uncompressed (or near) format(avi,mov,etc) until putting to final codec,size,etc. NEVER re-compress files ESPECIALLY from FLV to FLV. That would be ridiculously crappy.

Q:Compression always CBR (since its for streaming)?
A: I actually use VBR, and it seems to work best for my particular product and method of delivery. I think most would say there is technically no wrong answer as long as it suits what YOU have and need to happen.

Q: Is a compressor like SSQ the best tool to reduce the file size ?
A: Have not used and will abstain from answering.

Q: And which is the best FLV codec ?
A: Confusion of codec and wrapper. FLV is a wrapper and the codec you usually see is On2 VP6, Sorenson Spark, etc. As far as the best... from what I've seen, heard and read, miles may vary so again, what works best for what you need. Try some variations.

Q: Any commonly known way to reduce noise while preparing the comp to render within AE?(some filter or some other ae feature i should know about?)
A: I'm with Daniel... ????

Q: Im getting 7/9mbytes flv files, could this be considered heavy for web streaming playback?
A: Not heavy, but on the upper end. What bitrate are you using? Codec? Audio bitrate? Etc... I could go on for quite a while.

Hope this is of some help.

-Drew
Just a beggar telling another beggar where to find bread.

Drew Jensen


Return to posts index

Craig Seeman
Re: Best compression for the web
on Jul 22, 2010 at 7:40:01 pm

I'd thought I'd give my slightly different answers and I applaud Drew for posting in a form Q&A which is much easier to handle than the OP.

[Drew Jensen] "Q: Reduce frame size to half - should i be doing this in ae or squeeze ?"

I too can't tell you which product but what you need to test is which does better scaling. One learns by doing a test.

[Drew Jensen] "Q: Reduce fps from 25 to 15 or 12 - is it any good at reducing file size maitaining playback?"

File size is primarily determined by duration and data rate. I'm not sure where the myth about frame rate (on it's own) as a method to determine file size began. It can impact quality and sometimes people will also drop data rate but I don't want to go there unless asked.

[Drew Jensen] "Q: Since the goal is an flv file, should i be rendering from AE in flv or in an uncompressed format such as AVI or MOV and then convert it to flv within squeeze ?"

Actually I'm going to return the volley with a question. Why is the goal FLV?
If the goal is Flash it's not necessarily FLV at all. Squeeze is a compression tool and it's going to be more capable for many reasons then AE. AE uncompressed in either AVI or MOV. Then Squeeze.

[Drew Jensen] "Q:Compression always CBR (since its for streaming)?"

Again I return this with a question. Do you know the difference between streaming and progressive download? Generally people should not ask questions based an assumptions which may well be wrong and the resultant answers not relevant.

If it's from a streaming server than CBR. If it's HTTP generally VBR is better.

[Drew Jensen] "Q: And which is the best FLV codec ?"

There are basically two FLV codecs. One is based on Spark (H.263) and the other is On2VP6. There's no reason to use Spark. On2VP6 is only useful when there's a need for Alpha Channel support or targeting some very very old computers. H.264 is the codec that should be used today and it generally can be wrapped in .f4v, .mp4, .mov (but there are certainly others that work). Flash H.264 support is extension agnostic.

[Drew Jensen] "Q: Any commonly known way to reduce noise while preparing the comp to render within AE?("

You got noise? That's usually from live video sources. How you reduce it depends on the processing filters in your compression app. It can range from using Black Restore to Noise Reduction and sometimes Gamma can even help because it depends on where the noise is.

[Drew Jensen] "Q: Im getting 7/9mbytes flv files, could this be considered heavy for web streaming playback?"

Incomplete question. Are those file sizes or data rates? 9 mega bytes a second! 9 mega bits a second is extremely high. 9 mega bytes a second wouldn't even work on a Blu-ray disc. That's like 72 mega bits a second.




Return to posts index


Drew Jensen
Re: Best compression for the web
on Jul 22, 2010 at 7:46:45 pm

"File size is primarily determined by duration and data rate. I'm not sure where the myth about frame rate (on it's own) as a method to determine file size began. It can impact quality and sometimes people will also drop data rate but I don't want to go there unless asked."

I've heard, again heard that it may differ slightly. By slightly I mean we're talking about overall size of a few KB because of amount of keyframes. I don't know if I buy it or not. But like you said, it's the duration and data rate.

Drew Jensen


Return to posts index

Craig Seeman
Re: Best compression for the web
on Jul 22, 2010 at 8:47:57 pm

The story behind the misconception of frame rate vs size as I know it.

If you decrease the frame rate you have more bits per frame. At that point you haven't changed the file size but you've increased the quality of each frame and the cost of temporal motion.

If your goal is not to increase quality but to decrease size, with a lower frame rate you can lower the bit rate to maintain quality. The problem is that it's not a proportional decrease in bit rate.

If you've cut the frame rate in half you've also just increased the amount of change from frame to frame. If it's talking heads it's still not too much change. If it's something with more action there's a heck of a lot more change. The result is that depending on the content, you may not really be able to lower the bit rate that much.

Again the actual change in data rate you can get away with is dependent on content.

A vaguely remember hearing in a past life that you could lower the data rate from about 10% to about 30% (depending on the content) to maintain the same quality when cutting the frame rate in half.

This was critical back in the days when people would view video with dial-up connections and/or servers had very very very limited space for files.

Some compression apps, depending on the codec and the app, will do the "maintaining" of quality for you so if you were to lower the frame rate and target a quality settings it would lower the data rate (and hence the file size) for you.

So the short answer is you might save 10%-30% in file size if you were to maintain frame quality when cutting the frame rate in half.

You might even find a few local news sites still doing this because they serve a rural or low income (dial-up) region or they have such high volumes of material (most of it talking head) and want to save server space. I've found that the last few of those sites I personally know of have changed that around two years ago.

For various reasons most sites have no interest in serving those still on dial up (certainly worth a brief separate topic). These days even people on slower broadband can handle full frame rates at reasonable frame sizes.

For example, typical live stream video on Livestream's free live service might be 448x336 25fps at 500kbps and that looks fine. A typical 480x360 YouTube video is running at full frame rate and looks fine as well.

Except in the most extreme circumstances in which you must reach dial up users and have limited server space, there's really no reason to drop the frame rate. There's very little file size to save and the temporal motion loss is just not worth it.



Return to posts index

Daniel Low
Re: Best compression for the web
on Jul 22, 2010 at 9:07:02 pm

Dropping the frame rate was also a requirement when the playback device simply couldn't cope with a full frame rate due to lack of processor power. This is simply not the case anymore.

__________________________________________________________________
Sent from my iPad Nano.


Return to posts index


Daniel Low
Re: Best compression for the web
on Jul 22, 2010 at 9:03:12 pm

[Drew Jensen] "Q:Compression always CBR (since its for streaming)?
A: I actually use VBR, and it seems to work best for my particular product and method of delivery. I think most would say there is technically no wrong answer as long as it suits what YOU have and need to happen."


Are you sure you're talking about streaming, not progressive download? - RTSP requires CBR, VBR would almost certain fail to deliver.

[Drew Jensen] "Q: Im getting 7/9mbytes flv files, could this be considered heavy for web streaming playback?
A: Not heavy, but on the upper end. What bitrate are you using? Codec? Audio bitrate? Etc... I could go on for quite a while."


Utter confusion here! '7-9Mbytes' refers to a file size (without a per second or /s at the end). File size has no relevance if the file is being streamed, that is if you are using the word 'streaming' in the true sense of the terminology. If the file is being delivered as a progressive download otherwise known as HTTP stream then it has a relevance.
As such, if it is a 9MB file delivered over HTTP then it is a very low demand delivery. If it's a 9MB per second file delivered of RTSP - true streaming then it's an impossibly heavy delivery 9MB/s = 72Mbits per second.






__________________________________________________________________
Sent from my iPad Nano.


Return to posts index

Drew Jensen
Re: Best compression for the web
on Jul 22, 2010 at 9:15:50 pm

My bad, I forgot to mention that important aspect. Yeah, I use it for progressive. As for the second part, I assumed the size, which I have to teach myself constantly that that is not a good thing to do.

Drew Jensen


Return to posts index

Tiago Casais
Re: Best compression for the web
on Jul 23, 2010 at 10:48:45 am

Guys,
Thank you very much for your replies. I got some good notions from them that helped clear things up.
Still the mystery of the unhappy client remains unrevealed...

It all started because the client reported that , when viewing the video on his website (the 9megabyte sized, 586x320 framesized and 1m:35s long motion graphic presentation, delivered in streaming) it would start, and then stop a couple of times during playback; you know, as if it was waiting for the loading, and then continue until the end (always stopping and going).

My first two ideas were:
1) decrease file size
2)compress it with CBR (since i'd read that if you're streaming it, you must use cbr).

Yet you say that 9megabytes would present any problems, either in streaming or http video. So file size is not the issue ? Maybe the VBR vs CBR ? Any ideias?

note: i decreased the fps from 25 to 15 (still in AE) and i got a some reduction in file size but i also changed some other aspects so i really couldnt say its a magic potion. I guess compression is a "try, change and re-try" thing.

note 02: What i was trying to say with the noise was that the image gets pretty sloppy with compression. Pixelated. Not captured video noise. Could this be helped with compression filters ?
Sometimes it gets pixelated and then smooth out, to get pixelated again and so on - could this be related to the amount of keyframes we set in when compressing ?

cheers

Tiago



Return to posts index


Daniel Low
Re: Best compression for the web
on Jul 23, 2010 at 10:58:16 am

You really need to tell us how this file is really, truly, actually being delivered - either from a Flash Streaming Server OR from a web server using HTTP.

Then you need to tell us what the datarate of that file is. NOT its file size.

As I said in my last post if the file is 9 MegaBytes per second then most people will have a problem viewing it over the internet, streamed or downloaded.

__________________________________________________________________
Sent from my iPad Nano.


Return to posts index

Tiago Casais
Re: Best compression for the web
on Jul 23, 2010 at 2:15:39 pm

Its being delivered from a local web server, not the flash server.


800 MB avi uncompressed (original file)

compression: 250 kbps
keyframe every 300 frames
On2VP6 Pro 2 pass CBR

Got an 9MB FLV.



Return to posts index

Craig Seeman
Re: Best compression for the web
on Jul 23, 2010 at 2:35:55 pm

[Tiago Casais] "compression: 250 kbps
keyframe every 300 frames
On2VP6 Pro 2 pass CBR

Got an 9MB FLV."


and your frame size
586x320

That's a pretty low data rate given your frame size. You have two choices. Go to a smaller frame size or increase the data rate to get rid of pixilation.

If the viewer is seeing pauses at that data rate on a CBR file, there's either an issue with the server or he's got some serious issues on his computer or internet connection IMHO. Maybe he should empty his browser cache as well at progressive download is buffered.

Back to quality, I'd use H.264, not On2VP6. H.264 works fine in flash on modern computers with recent versions of Flash going back a couple of years. I'd recommend 2 Pass VBR encode so bits can be allocated to the parts of the content that need it.


Again if he's getting pauses on a 250kbps CBR file from an HTTP server there's problems that may not be related to the file.

What's his internet speed? Even a 384kbps slow DSL connection should handle what's being served though. BTW the speed his ISP tells him he has may not be reality.

Reality (speed)
http://www.speedtest.net/
Reality (ping and stability)
http://www.pingtest.net/





Return to posts index


Tiago Casais
Re: Best compression for the web
on Jul 23, 2010 at 3:45:50 pm

Im sorry, he was havind this issues with a VBR compressed file that was used earlier. The CBR compressed file im talking about is the new version that replaced the troubled one. I still havent got any reply on that matter so I cant say for sure if the "VBR to CBR" change was of any good at all. I also cant say anything about the clients connection at this moment, but im sure its not the best one since i never had any troubles viewing the flv at any time.

You say i should use VBR, and im sure that would improve quality, but isnt that the primary suspect for the stop/go problem the client had ?
Im still not sure about the relation between CBR/VBR vS streaming/http progressive(?). Can you give me some hints on when to use one or the other ?
Also, squeeze doesnt allow using h264 for flv delivery. only for .mov. Any other compression software you recommend?
cheers
Tiago


Return to posts index

Drew Jensen
Re: Best compression for the web
on Jul 23, 2010 at 3:57:06 pm

Tell him to get faster internet connection :) In all seriousness, it could be his connection. Also check the server. Is this your server? A friends host server? An actual company server? If so, big company or small? At my work we use a CDN just for the reason of delivering our videos. The CDN has server locations almost evenly spaced out over the US in major cities so the latency is reduced. I might start to sniff out if it's a delivery issue because it sounds like you may have fulfilled your obligation.

Drew Jensen


Return to posts index

Craig Seeman
Re: Best compression for the web
on Jul 23, 2010 at 4:43:55 pm

[Tiago Casais] "You say i should use VBR, and im sure that would improve quality, but isnt that the primary suspect for the stop/go problem the client had ? "

It could be if the client has a very slow or unreliable connection. The parts of the file that use more bits will take longer to buffer. If the buffer takes longer to fill at a given point, their playback will pause. It's quite possible that a 250kbps file might have peaks that are several times that. In a professional compression utility you can constrain those peaks though. You REALLY need to know the target. It's a MAJOR factor in your choices as a compressionist.

As someone who advises my clients, if I had a client with the experience your client is having, I'd have them run the aforementioned tests. If they don't like the results they should contact their ISP and if the results aren't satisfactory, they should switch ISPs. This is assuming video is important to them as it seems it is.

Imagine driving a car that stalls every 2 minutes. Now if you don't really need to drive a car that often maybe you don't care. If you do need to drive the car as part of your business you either get the car fixed or get another one.

If every time they ask someone to make a video file for them the provider (you in this case) has to spend 10 hours on what should be a one hour job, you'd stop wasting your time with him as well every other provider of video files. If his car can't get him to the business meeting on time then his business will go under. And yours will too if you have to spend 10 hours on a job that bills one hour.

[Tiago Casais] "Im still not sure about the relation between CBR/VBR vS streaming/http progressive(?). Can you give me some hints on when to use one or the other ?"

Streaming servers need Constant Bit Rates. HTTP Progressive Download can handle Variable Bit Rates as long as the speed is fast enough that buffering happens faster than playback. VBR is generally better quality so fast action or transitions or motion FX that need more data for quality, can get those bits it needs.

[Tiago Casais] "Also, squeeze doesnt allow using h264 for flv delivery. only for .mov. Any other compression software you "

NO compression program should allow H.264 flv delivery. Adobe engineers recommend against it. Your REALLY should read the preceding posts. I already stated this but you seem to need it repeated.

Flash H.264 is EXTENSION AGNOSTIC. .mp4, mov, .f4v (and many other extensions) ALL WORK IN FLASH.

.flv is for Sorenson Spark (H.263) or On2VP6. Spark is long dead. On2VP6 is still good for Alpha Channel support but is not as efficient (quality at a given data rate) as H.264 on modern computers.

Squeeze uses .f4v for H.264 Flash but .mp4 and .mov will work just as well.



Return to posts index

Tiago Casais
Re: Best compression for the web
on Jul 23, 2010 at 5:46:14 pm

Indeed i had much better results in the past using h264 than on2vp6. I just didnt know that i could deliver a mov to be integrated in the flash player.

Anyways thank you so much for your detailed explanations, they were very helpful.

cheers to all

Tiago


Return to posts index

Drew Jensen
Re: Best compression for the web
on Jul 23, 2010 at 6:33:52 pm

If the web guys are not happy with large sizes, be careful of using .mov or .mp4, they tend to be larger, and can be quite larger than a .f4v counterpart. But, from my experience, quality is the trade off as usual.

Drew Jensen


Return to posts index

Craig Seeman
Re: Best compression for the web
on Jul 23, 2010 at 6:43:11 pm

Data rate and duration roughly equal file size holds true across codecs. There can me differences due to metadata (.mov hinted metadata for example). The distance from my house to your house remains the same whether by car or by train, whether by miles or kilometers. The wrapper itself has little impact on file size.



Return to posts index

Daniel Low
Re: Best compression for the web
on Jul 23, 2010 at 8:31:06 pm

Huh? What?
With H.264 as the CODEC in all, mov, MP4 and F4V will only vary in file size ever so slightly. Like the different between a sheet of paper and a sheet of thin card.

The quality will be identical.

Drew, some of what you say is good advice, the other stuff is crap. Sorry to be brutally honest.

__________________________________________________________________
Sent from my iPad Nano.


Return to posts index

Drew Jensen
Re: Best compression for the web
on Jul 23, 2010 at 9:10:57 pm

No offense taken. I'm just trying to learn anything I can, some good info and some bad but with comments like yours, it slowly helps me weed out the bad.

Then I must pose a question also. I just rendered a video in 3 different formats .mp4, f4v, and .mov. The .mp4 and .f4v were basically same size (1.6MB & 1.7MB) but the .mov was 10.3MB. Is it because of the default setting of .mov and it's bitrate? I mean, when exporting the .mov, I get quality settings of (low,med,high,etc..), that is basically dummy settings for bitrate... correct?

Drew Jensen


Return to posts index

Craig Seeman
Re: Best compression for the web
on Jul 23, 2010 at 9:23:29 pm

[Drew Jensen] "I just rendered a video in 3 different formats .mp4, f4v, and .mov. The .mp4 and .f4v were basically same size (1.6MB & 1.7MB) but the .mov was 10.3MB. Is it because of the default setting of .mov and it's bitrate? I mean, when exporting the .mov, I get quality settings of (low,med,high,etc..), that is basically dummy settings for bitrate... correct?"


It's all meaningless without details. If all you're getting is "low, med, high" there's no way to compare. If it's a "Quality" setting, then the encoder is making decision based on content and that bears no relation at all to what another thing with settings is doing.

Defaults are meaningless because the settings may not match. After all, Nigel Tufnel's goes to 11.



Return to posts index

Tiago Casais
Re: Best compression for the web
on Jul 26, 2010 at 11:33:25 am

Hello guys,

Craig mentioned the "hinted" metadata on the .mov. I was wondering what that was? Sorenson gives you the option of "hinted" or "not hinted" in the compression settings. Hinted being associated with "streaming" - actually the choice is between "streaming-hinted" and "non streaming -not hinted" Can someone explain to me what that is exacly? In what way will it affect the video?

Another thing i meant to ask: sometimes a compressed video gets pixelated ate scene change and then gets better after a second or two. Im guessing this has to do with the "keyframing at scene change" option. Is that so?

cheers

Tiago


Return to posts index

<< PREVIOUS   •   VIEW ALL   •   PRINT   •   NEXT >>
© 2017 CreativeCOW.net All Rights Reserved
[TOP]