Maximizing Youtube SD quality for super 8 footage

[Peter Woodford]

Hi folks,

 

I'm a still a novice at Super 8 filming but I'm continuing to plug away and trying to improve my skills.

 

The brick wall I've hit recently is the SD compression artifacts on Youtube vids. My theory could be totally wrong but it really seems as though its settings are fouled up by film grain and the compression ends up being very destructive--very blotchy/blocky. I've tried uploading the maximum quality file (since the videos are short) but it really doesn't seem to help much. I find the greens particularly bad:

 

 

Here's the Vimeo version for comparison:

 

 

Just wondering how others here have dealt with this-- does anyone treat the video slightly differently when it's destined for Youtube? I actually really like and embrace grain but if that's what causes the blotchiness, I'd consider having a separate version for Youtube (since the vast majority of people will see the SD version there).

[Josh Gladstone]

Youtube doesn't like grain. They might use lower bitrates than vimeo at standard definitions, which might account for the difference, but grain always looks bad when heavily compressed. That being said, I still prefer youtube over vimeo because youtube allows for 2k resolutions and higher bitrates at 2k. And when played back at 2k, the stuff looks pretty good. Plus it's free and has unlimited hd uploads. So youtube wins for me. I usually compress to 50,000kbps for 1080p (which youtube recommends), and 100,000kbps for 2k. Might be a bit of overkill.

 

Semi-relatedly, If you're going to send links around, you can add "&hd=1" to the end of youtube urls to force 720p playback.

Edited December 12, 2013 by Josh Gladstone

[Peter Woodford]

Thanks Josh, do you happen to know if there's a way to force 720 playback on playlists? I experimented and it seemed to ignore the tags. I can't seem to find any master Youtube url hack list that's up-to-date.

The other thing that's a bit of a disappointment is that the 'related videos'/social component of the site will almost always lead to people seeing the lo-fi versions.

[Will Montgomery]

Ads on YouTube absolutely kill me. I can't stand YouTube at all. Vimeo allows you to replace a video with a new version, add credits, ect. It is set up for the filmmaking community. Only advantage of YouTube is the larger audience.

 

At a recent conference in Austin Philip Bloom said YouTube is a necessary evil because of its reach but he much preferred Vimeo.

[Josh Gladstone]

Will, there are no ads on youtube if you chose not to monazite your video. And I admit that Vimeo has lots of features that are better than youtube. But for it to be really usable, you have to pay a monthly fee. And even then, youtube still supports higher resolutions and better picture quality, and plays back on more devices all for free.

 

Peter, actually you can and you can't. First, I want to take back the "hd=1" trick, as that was apparently recently deprecated in the most recent youtube player api, unfortunately. The newest player automatically plays back based on the size of the container it's in. So if you embed it in a webpage and its container is small, it won't play in HD by default. If the container is large, it will. So, if you want to, you can use the url for a youtube popup window, which makes the container the size of the browser window. If your browser window is large enough, it'll automatically playback in 720p, or even 1080p. All you have to do is change the "watch" in the youtube url to "watch_popup", e.g. youtube.com/watch_popup?v=P2xnHyzlNqs&list=PL6E414828ACB06EE9

It seems that's the best you can do right now. Kinda blows, but it is what it is.

Edited December 13, 2013 by Josh Gladstone

[Phil Rhodes]

Bear in mind when you're watching something at, what, 480x360, the subsampled colour channels are only 240x180. So, if you don't have high luminance contrast associated with high chroma contrast, it will look mushy.

 

I remember when they started talking about "Youtube HD" one NAB, everyone was saying "Yes, well, lovely, but Youtube SD would be a start."

 

P

[Will Montgomery]

Will, there are no ads on youtube if you chose not to monazite your video. And I admit that Vimeo has lots of features that are better than youtube. But for it to be really usable, you have to pay a monthly fee. And even then, youtube still supports higher resolutions and better picture quality, and plays back on more devices all for free.

Interesting. Vimeo was down last week late at night for uploading so I used YouTube. The CEO of my company needed to view it and it wouldn't play via YouTube on his iPad so I had to upload to Vimeo the next day where it played flawlessly. Must be a setting on YouTube that I don't know about like disabling ads.

 

Higher resolutions than HD? How many monitors are capable of that? 99% of anyone viewing videos online couldn't see anything higher than 1920x1080. Better picture quality? Vimeo's compression blows away YouTube in every test I've seen, especially for grainy film (although no compression handles it well enough for me.)

 

The free Vimeo service has plenty of power for hobbyists and $59.95 per year isn't too much for me to pay for the advantages of Vimeo since it's what I do for a living.

 

They are just different services geared to much different audiences.

[Josh Gladstone]

Yes, higher than HD. So if you were to compare a 2k or 4k version of a video on Youtube, to the same thing uploaded to Vimeo, the Youtube version would look better, even on a 1080p monitor. Especially since Vimeo doesn't even allow your videos to playback at 1080p unless you pay for plus. And even if you do, you don't even get unlimited uploading! 5gb a week? That's not even enough for 15 minutes of video at 50,000 kbps. (Although to be fair, they do recommend 20,000 kbps for 1080p video, which really makes me wonder if it's higher quality than Youtube, as they suggest 50,000, but I digress).

 

So I dunno man, to me it's not about the audiences, hardly anyone watches my junk anyway. To me it's about just putting stuff out there in the highest quality way possible. Youtube definitely has its issues, but I think it does that better.

Edited December 13, 2013 by Josh Gladstone

[Andries Molenaar]

Yes, higher than HD. So if you were to compare a 2k or 4k version of a video on Youtube, to the same thing uploaded to Vimeo, the Youtube version would look better, even on a 1080p monitor. Especially since Vimeo doesn't even allow your videos to playback at 1080p unless you pay for plus. And even if you do, you don't even get unlimited uploading! 5gb a week? That's not even enough for 15 minutes of video at 50,000 kbps. (Although to be fair, they do recommend 20,000 kbps for 1080p video, which really makes me wonder if it's higher quality than Youtube, as they suggest 50,000, but I digress).

 

So I dunno man, to me it's not about the audiences, hardly anyone watches my junk anyway. To me it's about just putting stuff out there in the highest quality way possible. Youtube definitely has its issues, but I think it does that better.

Are you aware what these 50 or 20 mbps speeds are? In the US most would be glad to have real 10 mbps downstream.

 

I am on 50 mbps fibre from an top provider and I can tell you that nothing from youtube which is listed as HD 720 or 1080 actually plays as anything near that quality. Not on my computer and certainly not on my real Full HD TV. The only real good HD sources are Blu-Ray and Sat tuner. Broadcaststreams from Internet providers don't come anywhere near HD

[Peter Woodford]

Thanks for the responses folks. I'm personally OK with either Vimeo or Youtube's HD quality. My suspicion, however, is that Youtube's HD option is too hidden away for the average viewer to notice. So if someone finds a vid by 'related videos' or by the search engine, I suspect they're most likely going to see it in SD.

 

I've noticed some people on this board use Neat Video. I would much prefer the natural look of the grain at HD resolution but I'm curious if a reduced-noise vid would survive the Youtube SD grinder less scathed?

 

Phil's comment was very interesting but way over my head. I really have to read up on this stuff!

 

Also, I did come across a partial solution for the lack of HD playlists in Youtube--I learned that Vimeo lets you create 'albums'.

[Carl Looper]

The way in which compression works is that grain tends to be flagged for greater compression - not intentionally as such but as part of the image optimisation process - however the removal is not as good as a dedicated grain remover (such as Neat Video). A compressor is looking for coherencies in both space and time and prioritises such. Grain has no coherency so it ends up being flagged for compression more than the other information. The problem with grain is that it doesn't compress very well (indeed pure grain doesn't compress at all). A compressor can end up converting it into blocky mud rather than removing it. It's better to remove grain using a dedicated degrainer than have it end up as this blocky mud. So degraining prior to compression is a good idea. One can do experiments to see how much grain to remove relative to a given compressor. So for YouTube one might apply more grain removal than if prepping for vimeo. Only by doing tests can one find what would be a balance.

 

Keep in mind that the reason one wants to keep grain is because degraining will tend to otherwise soften the image. But a dedicated degrainer will soften the image less than a video compressor, so one should use the degrainer - because otherwise the compressor will do it and do a worse job of it. So it's just a question of removing that which the compressor would otherwise try to remove (and in doing so make it worse).

 

Hope that makes some sort of sense.

 

C

[Phil Rhodes]

 

 

The only real good HD sources are Blu-Ray and Sat tuner. Broadcaststreams from Internet providers don't come anywhere near HD

 

This is not always the case; some blu-ray and satellite streams are MPEG-2, and almost all use less careful encoding than online servers.

[Perry Paolantonio]

Are you aware what these 50 or 20 mbps speeds are? In the US most would be glad to have real 10 mbps downstream.

 

Unless I'm mistaken, Josh is referring to the bit rate of the UPLOADED file, not the bitrate of the file you view on YouTube, after they've compressed it. The quality of the uploaded file makes a massive difference in the final quality of the encodes that YouTube makes. We no longer upload H.264 files if we can help it, we upload ProRes or ProRes HQ. The files are significantly bigger going up and the uploads take forever. But the results are far and away better than when we upload high bit rate H.264 (plus it takes less time to make a ProRes file than an H.264 for us, so it's usually a wash).

 

Experiment with it - upload the same movie as H.264 and ProRes and you should see the difference pretty clearly. It makes perfect sense too - I mean, we wouldn't make a Blu-ray from an already highly compressed file format and expect it to look good, because it's being re-compressed. YouTube is the same concept, but with more extreme compression.

 

Garbage in, Garbage out...

[Andries Molenaar]

The bitrate in mbps is for the linespeed. 50 and 20 are hispeeds which are not commoin in the US and neither in many other.s

 

Possibly you can upload a raw-like file. But the planned download of 50 or 20 remains unlikely. And as stated 720 or 1080 maybe listed but the quality is never anything properly filled 720 or 1080.

[Perry Paolantonio]

The bitrate in mbps is for the linespeed. 50 and 20 are hispeeds which are not commoin in the US and neither in many other.s

 

Again, (and Josh can clear this up), I believe you and I are talking about different things here, Andries.

 

When one uploads a video to YouTube or Vimeo, that is NOT the file one downloads. Those services re-compress the video to a variety of bit rates, so they can fine-tune the playback depending on network speed issues (check out this video for a good explanation of the concept: http://www.gammaraydigital.com/blog/how-youtube-handles-variable-network-speeds). You have zero control over the bit rate of the resulting file, that's entirely handled by YouTube or Vimeo, and it's totally automated encoding. The only thing you have control over is the quality of the source file you give them. The higher the quality, the better the resulting streaming file.

 

If you upload a 1080p file that's compressed to H.264 at 10Mbps, and you upload that same file as ProRes (100+Mbps), you will see that the streamed file looks a lot better with the version that was uploaded as ProRes. (I'm just using ProRes as an example here - it could be another high quality format). But the point is that you want to upload the *least compromised* file you possibly can, since you don't have control over the encoding parameters of the file that's actually streamed. One way of doing this is uploading a file that's got a very high bit rate, which theoretically has less compression artifacting, though that really depends on the type of compression in the file you're uploading. A better solution is to get as close to Uncompressed as is practical, but that's tough given slow upload speeds on most internet connections. ProRes is a decent middle ground, which is why we use it for this.

 

We're not talking about the internet connection speed of the end viewer here at all, it's about the bit rate of the file provided to the streaming service. Higher is usually better.

[Josh Gladstone]

Yes, thank you Perry, I have certainly been talking about encoding at 50,000kbps and uploading that file to Youtube. Youtube then re-encodes that into various formats. So, in theory, the higher quality file you upload, the higher the quality of the re-encodes (although I have seen some anecdotal posts that there's a limit on this). I really doubt Youtube ever serves 50,000kbps files.

 

Andries, I don't know what you're talking about that "720 or 1080 maybe listed but the quality is never anything properly filled 720 or 1080." Go to keepvid.com or anywhere else that lets you directly download the files Youtube streams, download a 1080p or 720p file and check the resolution for yourself. I just downloaded a 720p version of one of my videos and according to VLC, it's a 1280x720, 24fps h264 video playing back at between 2500 and 3500kbps. Youtube isn't lying about resolution.

[Perry Paolantonio]

I'm sure there is a practical limit, yes. While we do all of our feature film Blu-ray encoding work from Uncompressed 10 bit files we've done tests on ProRes HQ and 4444 files made from the same HDCAM SR tapes, and we haven't found any difference in quality between the two. ProRes HQ is about 1/10 the size of uncompressed 10 bit files. If you're talking about uploading a large clip to YouTube, then ProRes is a good starting point and is more reasonable than Uncompressed, given the relatively slow upload speeds most of us have. And I don't think you'd see any difference in the final product.

 

A couple months ago we stopped uploading H.264 files to YouTube and switched to ProRes HQ. Even with H.264 files that looked great on the desktop at high bit rates, we could clearly see the quality difference vs. a ProRes or Uncompressed test clip once YouTube recompressed them.

[Carl Looper]

In the case of Super8, it's more than just uploading a high quality file.

 

Certainly doing the prep to ProRes (etc) is definitely recommended, for the reasons stated, but this won't actually solve the interference problem that grain causes. One needs to also do a little grain removal - how much to do can only be solved by testing - too little and the remote compressor does a lousy job with your source - too much and the remote compressor does an excellent job, but useful detail is lost (the image looks too soft).

 

The compressor is only interested in image information that is consistent from one frame to the next (movement being tracked by motion vectors). Anything that is inconsistent from one frame to the next (such as grain) eats into the compressor's bandwidth budget and can muck up the result - because the compressor doesn't know it's grain - it tries to retain it in the belief it's actual image information - and the total image suffers - ending up looking softer or muddier than it otherwise might, ie. not as high res looking as one might otherwise imagine given the pixel dimensions.

 

This is only of real concern for small format film sources where the grain is more magnified.

 

The grain is actually important information (statistically useful information), but online compressors are completely ill equipped to deal with it - as they are set for low quality compression (in terms of compression amount rather than pixel dimensions).

 

C

[Carl Looper]

Better than grain removal, for Super8, is grain integration using Super Resolution processing of HDR scans - which is able to re-localise the information otherwise distributively (statistically) encoded in the grain. Human perception of grainy source works along lines like this - we can easily see "through" the grain - but codecs can't. Unlike our eyes and brain they are not designed to read grainy sources very well.

 

C

[Andries Molenaar]

Yes, thank you Perry, I have certainly been talking about encoding at 50,000kbps and uploading that file to Youtube. Youtube then re-encodes that into various formats. So, in theory, the higher quality file you upload, the higher the quality of the re-encodes (although I have seen some anecdotal posts that there's a limit on this). I really doubt Youtube ever serves 50,000kbps files.

 

Andries, I don't know what you're talking about that "720 or 1080 maybe listed but the quality is never anything properly filled 720 or 1080." Go to keepvid.com or anywhere else that lets you directly download the files Youtube streams, download a 1080p or 720p file and check the resolution for yourself. I just downloaded a 720p version of one of my videos and according to VLC, it's a 1280x720, 24fps h264 video playing back at between 2500 and 3500kbps. Youtube isn't lying about resolution.

The may claim and label their streaminfo as 720 or 1080 or 1080p.

The content is definitely not what they say it is. It is totally impossible to stream 1080p in full resolution.

I.e. they compress with loss and just expand it at the receiving end.

 

Watch Full HD TV on a proper set and watch the demo-inside, a blue-ray with recent content (not upscaled old digital stuff) or watch a proper HD broadcaster like BBC 1 or 2 ARD ZDF with a game of rugby or soccer.

[Will Montgomery]

The best thing would be to offer prints of your films that you could mail to each potential viewer, maybe with a projector too, so they can see it in it's celluloid beauty. Just saying'...

[Perry Paolantonio]

The may claim and label their streaminfo as 720 or 1080 or 1080p.

The content is definitely not what they say it is. It is totally impossible to stream 1080p in full resolution.

I.e. they compress with loss and just expand it at the receiving end.

 

 

You're talking about different issues here. You can make a 1080p file that's easily streamable on the slowest of broadband connections. It won't look good, but it'll still be 1920x1080 pixels. Yes, it will have lots of compression. But it will still be 1080p, because that number refers to nothing other than the dimensions of the image. "1080p" is not a measure of quality, it's a measure of size.

 

That said, it's possible to make 1080p look quite good on broadband connections even at relatively low bit rates. At home I don't have cable TV but I do have a typical 20mbps cable modem connection to the internet and a Roku box that can stream 1080p video from a variety of sources. From Netflix (I believe just under 6mbps) it usually looks quite good on a 46" screen from normal viewing distances (about 10 feet away), as long as there aren't network issues that cause the highly compressed versions to kick in. From Amazon it's hit or miss - sometimes it looks great, sometimes the encodes are unwatchable. From Hulu Plus it's usually pretty bad with lots of network problems and outright stalling.

 

OTA broadcast tends to look pretty great compared to Cable TV, because a single channel has a lot more bandwidth to play in - ATSC MPEG2 is just under 20Mbps, and that means that most content will look good. On Cable TV, you have hundreds of channels competing for the same small pipe, so they all have to be more compressed than the signal that the local broadcaster is sending out over the air.

 

H.265 is going to up the ante significantly, by the way. I'd say within a year or two we should see much more widespread adoption of that format, now that encoders are starting to appear. The picture quality is pretty amazing even at very low bit rates.

 

And for what it's worth, bit rate is not in and of itself a measure of quality either - it's a measure of how much data there is. But every codec is different (10Mbps HD MPEG2 looks pretty awful, but 10Mbps HD AVC can look really good - the files are identical sizes, but the compression is different).

 

-perry

[Josh Gladstone]

The may claim and label their streaminfo as 720 or 1080 or 1080p.

The content is definitely not what they say it is. It is totally impossible to stream 1080p in full resolution.

I.e. they compress with loss and just expand it at the receiving end.

 

Watch Full HD TV on a proper set and watch the demo-inside, a blue-ray with recent content (not upscaled old digital stuff) or watch a proper HD broadcaster like BBC 1 or 2 ARD ZDF with a game of rugby or soccer.

 

I really don't want to get into it more, but Andries, I think you're confused about how compression works, and that you're conflating bit rate and resolution. YouTube is absolutely streaming 720p, 1080p, even up to 2K files. H.264 compression does not work by reducing the size of the picture area and then blowing it back up in the end. If you're interested in how it works, here's a link to the Wikipedia article.

 

And that stuff you're watching broadcast on BBC or on a blu-ray is absolutely compressed as well, just at higher bit rates.

 

Edit: Perry, I wrote this before your response got posted. But yes, I agree with you. Actually, if I had seen your post first, I probably wouldn't have written anything myself.

Edited December 17, 2013 by Josh Gladstone

[David Cunningham]

I tried the Prores thing as perry suggested and definitely had better results. But. The grain is still annoying compressed. It's true. There really is only so much you can do with the bit rates they are using.

[Andries Molenaar]

 

I really don't want to get into it more, but Andries, I think you're confused about how compression works, and that you're conflating bit rate and resolution. YouTube is absolutely streaming 720p, 1080p, even up to 2K files. H.264 compression does not work by reducing the size of the picture area and then blowing it back up in the end. If you're interested in how it works, here's a link to the Wikipedia article.

 

And that stuff you're watching broadcast on BBC or on a blu-ray is absolutely compressed as well, just at higher bit rates.

 

Edit: Perry, I wrote this before your response got posted. But yes, I agree with you. Actually, if I had seen your post first, I probably wouldn't have written anything myself.

I know what I am talking about. I am educated in technical IT and have worked many years in IT.

 

Merely calling it 1080p and feeding it with a highly compressed stream is not the way it works.

You cannot have proper 1080p from a 6mbps stream. It just like upscaled DVD of 10 years ago.