Shooting 10 bit uncompressed into disk array. How different than compressed?

[Guest razerfish]

Shooting 10 bit uncompressed seems like the big thing, or is about to be the big thing, right now.

 

What difference is there image-wise as compared to shooting the compressed into tape on camera's deck vs. this?

 

Has anyone done it or compared images? Can you see the difference with the naked eye?

 

I must confess that I'm excited about shooting uncompressed myself, but I've never seen the footage compared to the compressed footage that you get going from the camera to tape. [i'm talking about the Sony F900]

[Stephen Williams]

Can you see the difference with the naked eye?

 

Hi,

 

On a monitor on the set probably not. However color correct or key the material and the difference is clear.

 

Stephen

[Guest razerfish]

Hi,

 

On a monitor on the set probably not. However color correct or key the material and the difference is clear.

 

Stephen

 

Really? Can you elaborate a little? I ask this because I'm planning on shooting my first feature this year and want to shoot uncompressed into a hard disk array, but only if it really makes a difference.

[Phil Rhodes]

Hi,

 

I am trying to organise a test of exactly this scenario.

 

It depends what format you're talking about compressing to. HDV is pretty cripplingly compressed, even worse than DV, and your ability to do even the slightest postproduction work on it is pretty limited. HDCAM is... less bad, being six times the bandwidth. HDCAM-SR is either twelve or twenty-four times the bandwidth, and is thus... pretty good.

 

You can experiment on this by obtaining or shooting a raw digital photo file, and applying various levels of JPEG compression to it. Then, try fiddling with the Levels filter in Photoshop. This situation closely approximates the sort of issues you'll hit when colour grading compressed video.

 

Phil

[Guest razerfish]

Hi,

 

I am trying to organise a test of exactly this scenario.

 

It depends what format you're talking about compressing to. HDV is pretty cripplingly compressed, even worse than DV, and your ability to do even the slightest postproduction work on it is pretty limited. HDCAM is... less bad, being six times the bandwidth. HDCAM-SR is either twelve or twenty-four times the bandwidth, and is thus... pretty good.

 

You can experiment on this by obtaining or shooting a raw digital photo file, and applying various levels of JPEG compression to it. Then, try fiddling with the Levels filter in Photoshop. This situation closely approximates the sort of issues you'll hit when colour grading compressed video.

 

Phil

 

I'm not real technical, so forgive me if I don't ask the question properly.

 

Let's say I had a F900 and shot some material. Is that 4:2:2 ? Let's say I didn't compress it, worked with it as it came from the tape [i know the camera uses a codec to compress it to get it to tape].

 

What would 4:4:4 uncompressed look compared to this? I don't have a chance to test.

 

I'm just wondering how significantly better 10 bit uncompressed will be. I'm hoping it's really significant.

[Jamie Metzger]

I'm not real technical, so forgive me if I don't ask the question properly.

 

Let's say I had a F900 and shot some material. Is that 4:2:2 ? Let's say I didn't compress it, worked with it as it came from the tape [i know the camera uses a codec to compress it to get it to tape].

 

What would 4:4:4 uncompressed look compared to this? I don't have a chance to test.

 

I'm just wondering how significantly better 10 bit uncompressed will be. I'm hoping it's really significant.

 

Obviously any sort of compression, is image loss. So shooting 10 bit uncompressed is going to give you huge file sizes (seriously), but you won't loose that quality of shooting to DV (dv has 5:1 compression, then importing to your NLE system your loosing quality all over the place).

 

Good Idea if shooting to HDD, stay HDD all the way, don't go to tape and back to HDD.

[Guest razerfish]

Obviously any sort of compression, is image loss. So shooting 10 bit uncompressed is going to give you huge file sizes (seriously), but you won't loose that quality of shooting to DV (dv has 5:1 compression, then importing to your NLE system your loosing quality all over the place).

 

Good Idea if shooting to HDD, stay HDD all the way, don't go to tape and back to HDD.

 

Let me ask this another way. What would the difference be between, say, 10 bit uncompressed vs., say the very least compressed image? I'm trying to figure out what the difference in image quality will be between taking the image straight out uncompressed through fiber link to a hard disk array and edit it that way vs. the typical way -- camera to HD tape, then from deck to editing program.

 

I realize files will be bigger, but is there really that much difference in quality?

 

Has anyone done this uncompressed test yet?

[David Mullen ASC]

Compression is just one of those things that look worse on some types of subjects but is hardly noticeable on others. Compression is also something that doesn't affect the look of the original so much as it affects your ability to process that signal in post for color-correction and efx work.

 

Some compression can be very mild and not affect quality much, if at all. For example, the HDCAM-SR 4:4:4 recording has a 2:1 compression I think, or something close to that.

 

So the question of "is there a difference in quality?" depends on what you're doing to that recording. Will it get processed several times in the post chain, needing uncompressing and recompressing, etc.

[Phil Rhodes]

Hi,

 

Okay, I'll try to wax poetic and describe it for you:

 

> Let's say I had a F900 and shot some material. Is that 4:2:2

 

3:1:1, which is worse.

 

> Let's say I didn't compress it, worked with it as it came from the tape

 

Then you would be decompressing it as it came off the tape, and it would come out of the HDCAM VTR as an HD-SDI stream, uncompressed. Of course that is then an uncompressed image with all the errors and softness of compression baked into it.

 

> What would 4:4:4 uncompressed look compared to this?

 

Nicer. Sharper, finer detail, less chromatic aberration. More quiescent, less objectionable noise, particularly in the blacks.

 

> I don't have a chance to test.

 

If you are serious about using any of these systems it ought to be able to talk to a postproduction facility and actually go look at it.

 

> I'm just wondering how significantly better 10 bit uncompressed will be. I'm hoping it's really significant.

 

Probably not that signficiant when you're looking at the monitor on set; you may well go "Oh, what on earth was I worrying about" if you use something like HDCAM. The denouement will come in post when your colourist (possibly you, in FCP, or whatever) is gingerly manipulating the controls, trading off technical picture quality against the desired grading situation.

 

Left: Low compression; Right: High compression, γ=+2 in each case.

 

 

Notice mosquito noise in the black jacket and on the bricks in the right-hand image. This is JPEG compression, but it works very similarly to the way HDCAM, DV25, DV50 and other tape formats compress. Note that this is pretty exaggerated and the effects weren't nearly as bad until I applied γ, but it shows what you get if you start grading highly compressed images.

 

Phil

[David Cox]

Hi All,

 

We have done some tests comparing a 35mm transfer compressed and uncompressed. We transferred an identical HD (4:2:2) sequence to uncompressed files and to HD-CAM. We chose HD-CAM because, to be honest, if you look at the maths of what happens to the image in HD-CAM, it's amazing you can still see what the image is! We therefore felt that this was a good comparison to do.

 

We were very,very surprised to see that cutting between the uncompressed frame and the HD-CAM compressed frame showed very little difference. Even looking at it with engineers eyes it was very hard to see any change in the image at all. If I were being extremely critial, I would say there was a small amount less detail in the very highly detailed parts of the image - in this case leaves in a background tree. But really, it was next to nothing.

 

However, blue / green screen work is a nightmare on HD-CAM, because the colour channels are extra compressed and reduced in resolution.

 

So my advise based on these tests is - if you're shooting a "straight" movie (hmmm as in no special effects - gay cowboys as in Brokeback Mountain are fine!) professional level compression such as that in HD-CAM (or better) HD CAM SR will not provide you with any realistic problems. However, if you have blue / green screen comp shots, then you would be very well advised to avoid HD-CAM for at least these shots.

 

HDV is a different issue. I woudn't recommend it for anything except when it is either that or your film doesn't get made.

 

David Cox

Baraka Post Production

[Guest Kai.w]

I second what David Cox said. Keying HDCAM footage is rather difficult. The image looks quite good but as soon as you pull a matte you see compression hell. I felt like basic color correction operations also work ok (within certain limits), unless you start using selectives.

Uncompressed 10 Bit images were much much easier to key.

I did not work on HDCAM SR material yet, so I can't really judge whether that may be mild enough to work well during keying.

 

-k

[Keith Mottram]

Hi All,

 

We have done some tests comparing a 35mm transfer compressed and uncompressed. We transferred an identical HD (4:2:2) sequence to uncompressed files and to HD-CAM. We chose HD-CAM because, to be honest, if you look at the maths of what happens to the image in HD-CAM, it's amazing you can still see what the image is! We therefore felt that this was a good comparison to do.

 

We were very,very surprised to see that cutting between the uncompressed frame and the HD-CAM compressed frame showed very little difference. Even looking at it with engineers eyes it was very hard to see any change in the image at all. If I were being extremely critial, I would say there was a small amount less detail in the very highly detailed parts of the image - in this case leaves in a background tree. But really, it was next to nothing.

 

However, blue / green screen work is a nightmare on HD-CAM, because the colour channels are extra compressed and reduced in resolution.

 

So my advise based on these tests is - if you're shooting a "straight" movie (hmmm as in no special effects - gay cowboys as in Brokeback Mountain are fine!) professional level compression such as that in HD-CAM (or better) HD CAM SR will not provide you with any realistic problems. However, if you have blue / green screen comp shots, then you would be very well advised to avoid HD-CAM for at least these shots.

 

HDV is a different issue. I woudn't recommend it for anything except when it is either that or your film doesn't get made.

 

David Cox

Baraka Post Production

 

Bit confused about this test are you saying that you transfered 35mm to data and to hdcam tape? this is really different to shooting to HDcam (or data)- the results are very noticable in this respect. for me there is a huge difference between footage compressed into HDcam and footage sent to data, via HD camera- especially at the limits of the compression's range (highlights etc) regardless of post production. however if you've shot film, then a large percentage of these 'compressioin issues' are dealt with before you get to the tape / data stage. I assume this test was to compare images post film out? In which case I'd say you'd be pushed to find any difference in the footage between true 4:4:4 and marginally compressed 4:2:2 (single link SR for example), but I'd expect to see some problems in a straight comparison with HDcam.

 

keith

[David Cox]

Bit confused about this test are you saying that you transfered 35mm to data and to hdcam tape? this is really different to shooting to HDcam (or data)- the results are very noticable in this respect. for me there is a huge difference between footage compressed into HDcam and footage sent to data, via HD camera- especially at the limits of the compression's range (highlights etc) regardless of post production. however if you've shot film, then a large percentage of these 'compressioin issues' are dealt with before you get to the tape / data stage. I assume this test was to compare images post film out? In which case I'd say you'd be pushed to find any difference in the footage between true 4:4:4 and marginally compressed 4:2:2 (single link SR for example), but I'd expect to see some problems in a straight comparison with HDcam.

 

keith

 

The test we did was to evaluate the difference to a known image between 10 bit uncompressed 4:2:2 and HD-CAM tape compression. I think the original question was about the difference between recording an uncompressed HD video signal to disk against recording it to HD-CAM, and so I think the above test answers that question. However, if we were to compare raw (RGB) data from a digital cinema camera to disk as data, rather than a video signal, then there would most certainly by some differences. Apart from the latitude difference you mentioned, there is also the fact that 4:4:4 images have twice the colour resolution as 4:2:2 images. Although this provides a surprisingly small amount of perceived difference in definition, it does make the difference for keying, as used in blue/green screen comps or as Kai.W pointed out, when you need to select a colour for grading.

[Paddy Eason]

We are in the middle of some testing of just this.

 

We ran an Arri D20, shooting 4:4:4 into both the SRW-1 deck at HQ quality (2:1 compression) and also the s.two disk array (no compression) simultaneously. Very shortly I will be having a look to see whether the compresson of the SR deck has any visible implications for grading, keying etc.

 

We are trying to work out a complele pipeline for our production (med budget vfx-heavy feature) using either the SRW1 or the s.two. Or both. It's certainly not obvious how to run all this through with complete safety, redundancy, convenience etc. Actually, it's a complete headache. I am really hankering after 35mm again!

 

Can anyone show me a tried-and-tested pipeline for SR > final print? :)

 

P.

[Keith Mottram]

We are in the middle of some testing of just this.

 

We ran an Arri D20, shooting 4:4:4 into both the SRW-1 deck at HQ quality (2:1 compression) and also the s.two disk array (no compression) simultaneously. Very shortly I will be having a look to see whether the compresson of the SR deck has any visible implications for grading, keying etc.

 

We are trying to work out a complele pipeline for our production (med budget vfx-heavy feature) using either the SRW1 or the s.two. Or both. It's certainly not obvious how to run all this through with complete safety, redundancy, convenience etc. Actually, it's a complete headache. I am really hankering after 35mm again!

 

Can anyone show me a tried-and-tested pipeline for SR > final print? :)

 

P.

 

Really dont understand your complications with this setup. If your going down the SR route, then simply think of it as any other tape route. You offline, online, pass shots to FX, reimport, then do a DI and master. If it is an fx heavy feature then I'd suggest onlining with a FCP 4:4:4 Blackmagic system, you can then export sections as QT's and break them down to DPX files via framelink for your FX team. reverse the DPX to QT for reimporting then do a final grade either in your preffered DI system- output final DPX string then send to Arri laser. This works. So do many other options. This is obviously very simplified and there are always other factors to take into account in any post workflow. Send me an email if you want to discuss it further.

 

Keith

[Paddy Eason]

Really dont understand your complications with this setup.

ah, now, you've made me feel reaally stupid already... ;)

 

The problem is, yes, it's tape. Most of us already involved on this feature have spent decades avoiding anything with the word "video" in, and consequently our brains lock up when someone says "tape" or "timecode" or "deck". And this stuff ain't obvious, in terms of detailed workflow - you can ask the people sweeping up the mess on several features that are around in post right now... But your tech hints are actually quite revealing and reassuring (some of the same lines we have been thinking on). I may well email you - thanks for the offer!

 

P

[Guest peter orland]

We transferred an identical HD (4:2:2) sequence to uncompressed files and to HD-CAM...

 

We were very,very surprised to see that cutting between the uncompressed frame and the HD-CAM compressed frame showed very little difference. Even looking at it with engineers eyes it was very hard to see any change in the image at all...

 

However, blue / green screen work is a nightmare on HD-CAM, because the colour channels are extra compressed and reduced in resolution. So my advise based on these tests is... ...if you have blue / green screen comp shots, then you would be very well advised to avoid HD-CAM for at least these shots.

 

David Cox

 

I hope you don't mind me resurrecting an old thread.

 

In regard to capturing on HD-CAM (tape)for post FX and Keying. Would transferring the selected clips into a 10 bit uncompressed sequence achieve similar results to capturing uncompressed in the first instance?

 

Thanks.

[Phil Rhodes]

Hi,

 

The AJA boards (Kona is the Mac incarnation, Xena is the Windows) support recording directly to DPX sequence, but unless you are getting involved in old-school post houses, AVI or Quicktime is likely to be easier to handle.

 

The post should be easy; it really comes down to how you are shepherding your data about. You do not want to be walking around with Firewire drives, that rapidly becomes very tiresome. Connecting things together with el-cheapo Gigabit Ethernet cards is an entirely reasonable way to do it, but you still have to think about the sheer volume of data.

 

That said storage is cheap as chips at the moment; you could do worse than keep your circle takes on a RAID and stick the rest out to [insert digital tape format of choice].

 

Any more detail than this is really fine-print engineering issues; anyone doing this seriously will need someone with the requisite background in general computing to manage that stuff.

 

And keep watching Showreel magazine; I'm preparing an article on a low cost HD record system.

 

Phil

[Guest peter orland]

Not meaning to be disrespectful at all Phil but, is your post in response to my question?

 

"In regard to capturing on HD-CAM (tape)for post FX and Keying. Would transferring the selected clips into a 10 bit uncompressed sequence achieve similar results to capturing uncompressed in the first instance."

 

Because if it is, I'm afraid I haven't got a clue what you mean.

[Stephen Williams]

"In regard to capturing on HD-CAM (tape)for post FX and Keying. Would transferring the selected clips into a 10 bit uncompressed sequence achieve similar results to capturing uncompressed in the first instance."

 

Hi,

 

No.

 

HDCAM has already down sampled and then compressed. The colour and luminance information are lost for ever.

 

Stephen

[Michael Maier]

But I think it would at least avoid farther degradation. Specially if it will undergo heavy color correction or compositing in post. At least it would not degrade more than the original footage or if using the original tapes.

Edited July 28, 2006 by Michael Maier

[Stephen Williams]

But I think it would at least avoid farther degradation. Specially if it will undergo heavy color correction or compositing in post. At least it would not degrade more than the original footage or if using the original tapes.

 

Hi,

 

HDCAM @ 3:1:1, 8 bit is not suited IMO for projects that require 'heavy color correction' or compositing. That is why people record uncompressed 10 bit 4:4:4 where possible.

 

Stephen

[Phil Rhodes]

Hi,

 

My post was partly in response to your specific question, and partly in regard of other comments made in this thread.

 

Phil

[Keith Mottram]

Hi,

 

No.

 

HDCAM has already down sampled and then compressed. The colour and luminance information are lost for ever.

 

Stephen

 

Stephen, there is a bit of yes and no on this one. obviously you cannot get back what isn't recorded, but digitising at 10 bit means that you hold onto the maximum that is available via SDI. As you are in effect outputting a signal, rather than directly moving the data onto your NLE there are advantages to this. I am currently conforming HDcam footage at 10bit due to this and whilst the difference is admitedly small there is a noticable difference when it comes to the footage posted in the colour correction programme (in my case Final Touch HD). Restricting yourself to a working 8 bit space is not preferable and if there was no point in working in a higher color space than the shot footage, then there would be no point in working in floating point.

 

Keith

[Stephen Williams]

Stephen, there is a bit of yes and no on this one. obviously you cannot get back what isn't recorded, but digitising at 10 bit means that you hold onto the maximum that is available via SDI. As you are in effect outputting a signal, rather than directly moving the data onto your NLE there are advantages to this. I am currently conforming HDcam footage at 10bit due to this and whilst the difference is admitedly small there is a noticable difference when it comes to the footage posted in the colour correction programme (in my case Final Touch HD). Restricting yourself to a working 8 bit space is not preferable and if there was no point in working in a higher color space than the shot footage, then there would be no point in working in floating point.

 

Keith

 

Keith,

 

Colour correction at upto 32 bit floating point would help I agree. The main limiting factor is the 8 bit 3:1:1 compressed HDCAM tape. Having recently seen a Viper Film Streem output graded on a Matrix, I have seen the advantage of uncompressed 4:4:4.

 

Stephen