HD formats and compression, what is a good "medium quality format"?

[George Peters]

OK, so I am confused about all the different HD formats... it was so much easier in the SD days when all you had was (basically, digitally) three options.

 

 

1- Uncompressed. Best quality and the best option for professionals wanting the best. Overkill for consumers, a bit more tricky to edit.

2- DV compressed. Good quality. Easy to edit. A good option for semipro and for consumers.

3- Mpeg2 compressed. Not as good image quality as DV, a bad option for editing. Mostly a compression for finalizing the work to a DVD.

 

 

 

OK, lets look at HD.

 

1- Uncompressed 4:4:4. Best quality, huge files. Only for Pros. Hard to edit?

2- ? What is the HD compression comparable to SD-DVcompression?

3- H264. Not as good image quality due to compression, not a good option for editing. The compression to use when finishing on Bluray.

 

 

 

So What do you guys think should be the "DV comparable" compression in HD (number 2 in my list)?

 

AVCHD? Hell no, it is compressed a lot.

HDV? Hell no, is feels more compressed than DV.

HDCAM in 3:1:1?

HDCAM SR in 4:2:2?

Uncompressed HD 4:2:2? (is there such a format?)

DVCPro HD in 4:2:2?

 

What are all these formats, and wich one is "the closest" and comparable to DV compression in SD work? Are there any other "medium quality HD formats" I have failed to mention? Something with only I-frame compression?

 

I think I am going nuts...

[Phil Rhodes]

DVCPRO-HD or HDCAM, probably. People say the former looks better but I think that's just 'cos Varicam is a really good camera.

 

Cutting uncompressed isn't actually that hard; it is easier in many ways because it obviates or simplifies a lot of technical issues. The disks to do it are no longer particularly expensive.

 

P

[Saul Rodgar]

DVCPRO-HD or HDCAM, probably. People say the former looks better but I think that's just 'cos Varicam is a really good camera.

 

Cutting uncompressed isn't actually that hard; it is easier in many ways because it obviates or simplifies a lot of technical issues. The disks to do it are no longer particularly expensive.

 

P

 

I agree on both counts.

 

I would add that handling uncompressed material puts more stress on the computer's processor IF a lot of rendering (heavy fx, compositing, color correction, image manipulation, etc.) is needed. Unless one has (relatively) expensive uncompressed, real-time-rendering, PCI card-based HD video output hardware, the processor and/ or RAM will max out -and those processing-intensive tasks will progress slowly, a best. Render farms and other solutions exist but not for cheap, to my knowledge. Proxy previewing with heavy image manipulation is not a very good solution either, in my experience.

 

Blackmagic Design claims render-free real time HD/SD effects on their $3000 Multibridge hardware, but I have not used it so I can't personally endorse it:

 

http://www.blackmagic-design.com/products/multibridge/

 

Panasonic's AVC Intra HD codec is supposed to be the best compromise in file size/ relative ease of handling vs stunning quality. Have not used it either.

Edited August 23, 2008 by Saul Rodgar

[Scott Fritzshall]

MiniDV isn't even "medium quality," it's pretty low quality. Don't forget you also had analog formats like BetaSP and compressed professional formats like DBeta. MiniDV was really consumer-level stuff, and the HD equivalent is probably HDV or AVCHD.

 

If you're stuck on tape formats, then DVCProHD or HDCam are probably middle-of-the-road formats. Kinda professonal but also with fairly high compression. But why limit yourself to tape formats? Nowadays you've got an essentially infinite array of data options to choose from. If you're shooting on a particular camera that uses a particular format, then you're going to be stuck with that format, but if you're recording as data or shooting film and getting a scan or datacine, then you can pretty much store it however you'd like. ProRes seems like a good compromise actually; fairly lightweight but also decently high quality.

 

The thing though is that this area has opened up so much that you've really got to specify your parameters before asking what to use, since depending on what you're doing you may have infinite options.

[Stephen Price]

Is there any tape format out there, that is uncompressed? If a format has 4:4:4 colour sub-sampling, then that means it is compresed, yes? What are the true uncompressed formats?

 

Thanks

 

Steve

[John Brawley]

Is there any tape format out there, that is uncompressed? If a format has 4:4:4 colour sub-sampling, then that means it is compresed, yes? What are the true uncompressed formats?

 

Thanks

 

Steve

 

 

Yep. one of these baby's.....

 

Bank's use them to store data...320GB a tape if i recall.....2k for this is pretty cheap.....about the same as you pay for a tape....

 

jb

[Phil Rhodes]

Hi,

 

D6 is 4:2:2, so it isn't really "uncompressed", for a certain value of "uncompressed."

 

I would add that handling uncompressed material puts more stress on the computer's processor IF a lot of rendering

 

Woah, ah, hold on there... think about it. If I'm doing a colour correction on (say) DVCPRO-HD footage, I have to unwind the DVCPRO-HD codec (a nontrivial task), do the colour correction (variable, but can be quite stressful), and then recompress it to DVCPRO-HD (which is a targeted-fit DCT operation over n blocks, and is downright hard work.) As compression codecs get cleverer, this only gets more and more difficult to do; actually, HDV is tougher to handle in this respect than DVCPRO-HD of equivalent resolution, because the HDV uses a codec that requires a lot more messing about to encode.

 

Compare this to an uncompressed sequence of identical dimensions; all I have to do is load it into memory and I can immediately start poking at it. OK, so your uncompressed sequence is more likely to be 10-bit and 4:4:4, which makes it tougher, but that's a choice and these are determinably quality increases, as opposed to compression, which you pay CPU time in order to have it degrade your image.

 

Absolutely the only place uncompressed is hard work is on disk or tape, and that's not exactly the biggest deal anymore. Everywhere else it makes things easier.

 

If a format has 4:4:4 colour sub-sampling, then that means it is compresed, yes? What are the true uncompressed formats?

 

No, 4:4:4 means that no colour information is thrown away; arguably, colour subsampling is a form of compression. Google will explain.

 

There is to my knowledge no tape format capable of storing full, uncompressed, unsubsampled 4:4:4 1920x1080 10-bit RGB frames at 24fps. The closest you will get is HDCAM-SR, a very expensive format which is about 2:1 MPEG-4 compressed (which is pretty good). The only devices capable of doing the full monty are disks.

 

P

[Walter Graff]

There is far more to the aesthetic quality of a picture than a set of numbers. Just as in the old days, it was difficult to tell the difference between DV footage and betacam. Betacam had slightly better numbers but Dv pushed the luminance values which tricked your eye. All of the HD formats give and take to create a picture. The key is most eliminate info that will allow you to appreciate the picture while still making data that makes it work for the format. I could show you footage on a monitor from all the HD formats and you'd have a tough time telling me which is which. I find too many people are concerned with numbers these days and little more. Any of the HD formats you mention make a great picture. But the only thing that makes a great picture is a person, not a camera. So worry more about "how to" than "how big" the numbers.

[Saul Rodgar]

Woah, ah, hold on there... think about it. If I'm doing a colour correction on (say) DVCPRO-HD footage, I have to unwind the DVCPRO-HD codec (a nontrivial task), do the colour correction (variable, but can be quite stressful), and then recompress it to DVCPRO-HD (which is a targeted-fit DCT operation over n blocks, and is downright hard work.) As compression codecs get cleverer, this only gets more and more difficult to do; actually, HDV is tougher to handle in this respect than DVCPRO-HD of equivalent resolution, because the HDV uses a codec that requires a lot more messing about to encode.

 

Not necessarily in the case of native (DVCPRO HD, HDV, etc) codec editing, as Final Cut Pro handles it:

 

"It is possible to edit compressed HD on a laptop that would not be able to edit uncompressed HD because of computing power and storage requirements . . . The number of available video streams and real-time effects increases as host processing speeds increase."

 

Taken from the FCP user manual that came bundled up with my copy of the software.

[George Peters]

MiniDV was really consumer-level stuff, and the HD equivalent is probably HDV or AVCHD.

 

But both HDV and AVCHD are mprg compressed with only a few I-frames, MiniDV is all I-frames.

Somehow the idea of having non-I frames in a clip I am about to edit feels... wrong.

 

If you're stuck on tape formats, then DVCProHD or HDCam are probably middle-of-the-road formats. Kinda professonal but also with fairly high compression. But why limit yourself to tape formats?

 

No need to. I am going to have some film telecined and I am searching for "the HD-format" to get it telecined to. I don´t have any HD camcorder, so I have no need for a tape. Once I have decided for the format, then the search for a transfer house that can treansfer to hard drives in "my HD-format" begins.

 

Nowadays you've got an essentially infinite array of data options to choose from.

 

And that is the whole problem. Why all these formats? What format to choose?

Where is the editing-fiendly all I-frame compressed "medium quality HD-format"?

I don´t have the disk arrays needed to go for uncompressed HD, it feels like overkill for me. And I don´t want HDV, it feels almost worse than miniDV due to the mpeg compression with long GOP structure...

[Michael Belanger]

You're looking for something around 100 to 150 mbps for editing then...

 

DVC Pro HD

Apple ProRes 422

Cineform

HD-CAM (not SR)

 

Your transfer house should be able to output to any of these. Part of the decision will be what editing program you'll be using.

[Scott Fritzshall]

But both HDV and AVCHD are mprg compressed with only a few I-frames, MiniDV is all I-frames.

Somehow the idea of having non-I frames in a clip I am about to edit feels... wrong.

I'm with you on that, but again, miniDV was always a low-end/consumer format, designed to get good image quality for minimal bandwidth. HDV and AVCHD are the successors of that paradigm. With HD you've now got up to 6x more pixels to store, and for low end systems they wanted to stuff that into as little bandwidth as possible, so the compression went up.

 

No need to. I am going to have some film telecined and I am searching for "the HD-format" to get it telecined to. I don´t have any HD camcorder, so I have no need for a tape. Once I have decided for the format, then the search for a transfer house that can treansfer to hard drives in "my HD-format" begins.

Well if you get it telecined, then you do have a need for a tape. Telecines output a video signal to a tape deck. The tape is then digitized to hard drive. If you want to go straight to hard drive, then you need either a datacine or a scanner. You'll have to talk with the post house about how they're giving it to you; a lot of the time I think they've got it pretty much set up already. I got some film scanned about a year ago and received it in .dpx format, which was perfect. At that point if you want something else you can convert it on your own.

 

And that is the whole problem. Why all these formats? What format to choose?

Where is the editing-fiendly all I-frame compressed "medium quality HD-format"?

I don´t have the disk arrays needed to go for uncompressed HD, it feels like overkill for me. And I don´t want HDV, it feels almost worse than miniDV due to the mpeg compression with long GOP structure...

Well look, if you want "editing friendly," then get an offline transfer to miniDV or DBeta, do your offline editing, and then get your selects re-transferred to HD and do a nice high quality finish. You haven't really been specific about what you're doing or any of your parameters, but this is a good general workflow. Keep in mind that you can always transcode uncompressed material into different formats; if the post house can only give you uncompressed but you want to work in ProRes, there's nothing preventing you from making that conversion on your own. Also HDV isn't a production format; no one is going to give you a transfer to that anyway. DVCProHD is an i-frame only "medium quality" tape format, if you want to go with that as well. You can get a transfer to that and then ingest it directly to your editing program.

[Phil Rhodes]

Not necessarily in the case of native (DVCPRO HD, HDV, etc) codec editing, as Final Cut Pro handles it

 

...by doing the DVCPRO-HD decode on the processor, using up processor time. What'd you think, it does it with magic fairy dust?

 

"It is possible to edit compressed HD on a laptop that would not be able to edit uncompressed HD because of computing power and storage requirements

 

Nothing to do with computing power and everything to do with storage requirements. Any reasonably recent desktop computer using PCIe internally is trivially capable of handling uncompressed HD at a hardware level; the only consideration is storage performance. Clearly it is difficult (though not always impossible) to interface HD capable storage to a laptop this is nothing to do with "computing power", whatever that lovely, friendly, easy-to-understand and typically meaningless Apple Mac phrase actually means.

 

It's about storage. Everything else is easier.

 

P

 

PS - And - point of order once more - despite being an entirely decent format, DVCPRO-HD is not 100mbps unless you're shooting 60p. Not that there's anything wrong with it, but it's pretty compressed.

[Robert Houllahan]

Well if you get it telecined, then you do have a need for a tape. Telecines output a video signal to a tape deck. The tape is then digitized to hard drive. If you want to go straight to hard drive, then you need either a datacine or a scanner.

 

 

All HD telecine systems use HD-SDI which is a video transport tat is common to decks and capture cards alike, you can capture a telecine's video output with a card like the AJA Kona-3 or Bluefish 444 at full uncompressed HD 444 video resolution.

 

-Rob-

[Michael Belanger]

PS - And - point of order once more - despite being an entirely decent format, DVCPRO-HD is not 100mbps unless you're shooting 60p. Not that there's anything wrong with it, but it's pretty compressed.

 

Ah, good catch. DVC Pro HD uses the same amount of data per frame so for 24p footage its more like 40 mbps.

[Scott Fritzshall]

All HD telecine systems use HD-SDI which is a video transport tat is common to decks and capture cards alike, you can capture a telecine's video output with a card like the AJA Kona-3 or Bluefish 444 at full uncompressed HD 444 video resolution.

 

-Rob-

That's true, I hadn't mentioned it because honestly I'm not sure how many places actually do this. Do you know of places that offer this as an option?

[Michael Most]

That's true, I hadn't mentioned it because honestly I'm not sure how many places actually do this. Do you know of places that offer this as an option?

 

Many facilities, including the one I work in (Cineworks, in Miami) can go directly from a telecine into either a capture card on a computer or a disk recorder that acts like a VTR, but actually takes in uncompressed video and writes it directly to file formats. Both methods are used to create files from video that has never gone to tape and never been compressed. The advantage of the disk recroder is the ability to use sync sound on material that is being recorded "shot by shot." When going directly to a capture card, we usually have to do "one edit per lab roll," because there is no way to control the capture with an edit controller - you simply do a "capture now" and stop it when the roll is complete. In either case, however, there is no videotape involved.