Some questions regarding testing with test charts

Dennis Schaller · July 12, 2013

Hi,

I didnt know where else to put it, so i hope this is ok.

I have some questions regarding camera testing with test charts. We actually did some tests yesterday (and i guess we did a lot of mistakes, since that was our first time and noone had a plan :wacko:).

We had a test room in a big rental house, so the setup was ok from the start. Two tungsten lights on both sides of the charts, I checked to light them evenly so that shouldve been fine. We white balanced a C500 so the white was right at the middle in the vectorscope (3100K), so that shouldve been fine too.

We used the following 3 test charts:

I suppose we did some mistakes in some of the tests though so I'd like to know which of our results are usable.

Cameras we tested: Nikon D3100 (Low-Budget DSLR with 1080p video), Canon 7D, Canon C500, Red One. (and for fun a GoPro Hero2 and 3, but i guess those results are unusable anyways because the charts were too small due to the fisheye)

Lenses: Cheap standard lenses for the DSLRs (18-55mm Nikon, 18-135 Canon), Arri/Zeiss High Speed 85mm T1.3 for the C500 and the One.

1. For colortesting i guess all you have to do is get the white balance right (if your scene is lit well enough), right? So we shot that chart with all of our cameras without watching the framing (shouldve done that?) AND we actually didnt look at a waveformmonitor to get the white to clip and the black to be black (that could be a problem right?)... DSLRs with some of the standard picture profiles, C500 in log mode, One delivers Raw ofcourse.

Now if I open the files, crop the image so all you see is the chart, I think i have some nice values at the vectorscope (but are the usable?). Also can I just add saturation/tweak the master gamma to make the vectorscope be easier to read or will this distort the colors? (as you can imagine its quite centered in the C500's log picture)

2. For testing the dynamic range right, you have to set the white in the middle just to clip and the black to be at the bottom of the waveform, I know that. But what can you actually tell from a 9-stop dynamic range chart when all the cameras easily capture that DR? Does it even make sence? (We set our exposure wrong on those shots, so I guess theyre unusable anyway)

3. I just wanted to use the third chart to show lineskipping in the DSLRs images (because there are some diagonal lines ofcourse), but I guess it would be better to get some bigger diagonal lines, because the images I got out of the test, I think aren't sharp enough to show line skipping.

Another thing about this chart: I guess its important to frame it to fill the whole image right? (Because you cant compare anything otherwise?)

In my results the C500 easily resolves all of the lines possible. Then we had to move the tripod because of the One's crop factor in 2K mode. So we roughly doubled the distance to the chart and then recorded a shot with the One (I guess changing the distance is a mistake for that test, even though you have the same framing?), the results are quite bad, it resolves all the lines up to 6... Then with the DSLRs we had quite different focal lengths (which could be a mistake too?), so we moved again to get the same framing. In my results the 7D shows some nice moiree (at least that result is usable I guess) and resolves the lines up to 5/6 to be distinctive diagonal lines.

The D3100 is so hard to focus that those results are unusable anyways I guess.

Ok so now what can I get out of my test shots without relying on wrong results?

Questions collected:

1. Do you need to watch the framing when using a color chart?

2. Do you need to expose white/black to the maximum when using a color chart?

3. Can you tweak saturation/master gamma without distorting the results of a color chart shot?

4. Can you get any results out of shooting a 9-stop DR chart when your camera resolves those anyways? (Except for results like how the ISO changes your grays or something like that)

5. Do you have to watch the framing when using a resolution/multiburst chart? (I guess so)

6. Do you have to keep the same distance when using a resolution/multiburst chart?

7. Do different focal lengths make the results of a resolution/multiburst chart unusable?

8. When testing rolling shutter, if you change the distance but your framing stays the same (due to focal length and/or crop factor), does it change anything? (It shouldn't because the image on the sensor will be the same size and rolling shutter just depends on the readout time of the sensor right? Or does the increasing distance reduce rolling shutter even though your framing stays the same?)

Wow that's quite a lot, I hope someone takes the time to answer, thanks in advance.

David Mullen ASC · July 12, 2013

One solution for the 11-stop grey scale limitation is to shoot it normal, then maybe 4-stops over and 4-stops under... Between the three exposures you can see where it falls apart.

I think line resolution charts have to fill the sensor area horizontally or vertically depending what you are measuring. But as long as you are consistent in your framing between the cameras, at least you can make comparisons even if the measurements are inaccurate.

You should have shot the Red One in 4K or 4.5K (or 4K HD mode, which 3840 across)... No one shoots in cropped 2K mode except for high frame rates.

David Mullen ASC · July 12, 2013

Keep in mind that some of these cameras add a certain amount of edge enhancement... And sometimes aliasing looks like extra detail when it's not.

Jason Outenreath · July 12, 2013

sometimes aliasing looks like extra detail when it's not.

What is the tell of such aliasing? I have a difficult time sometimes seeing the edge enhancement in the image. Thanks.

David Mullen ASC · July 12, 2013

It's sometimes hard to separate lines created by moire from actual lines.

Dennis Schaller · July 13, 2013

Yeah I already thought about that too... It looks like thats the case on the 7D but the lines are too small to tell exactly.

We shot the One in 2K because thats how we shot our film (we needed Slowmo), so we wanted the results to match our settings.

There are still some open questions though (and I really dont want to write about faulty test results in my bachelorthesis):

Ok so now what can I get out of my test shots without relying on wrong results?

Questions collected:

1. Do you need to expose white/black to the maximum when using a color chart?

I guess the answer is: If you don't expose black/white to the max, the colors wont be as saturated, but the direction they're showing on the vectorscope stay the same?

2. Can you tweak saturation/master gamma without distorting the results of a color chart shot?

You should right? Because saturation/master gamma settings influence every color the same?

3. Do you have to keep the same distance when using a resolution/multiburst chart? Do different focal lengths make the results of a resolution/multiburst chart unusable?

My answer would be: As long as the image that is projected on the sensor has the same size, the distance doesnt matter? (But will a long focal length/same focal length with crop factor resolute the image as good as if you're closer to the chart?)

4. When testing rolling shutter, if you change the distance but your framing stays the same (due to focal length and/or crop factor), does it change anything? (It shouldn't because the image on the sensor will be the same size and rolling shutter just depends on the readout time of the sensor right? Or does the increasing distance reduce rolling shutter even though your framing stays the same?)

Would be really great if you could answer those too David, thanks for the answers you already gave, it's always good for me to have some profound answers, because I'm really looking into all that technology stuff and it's really fun to think about that stuff in a way it all makes sence :)

David Mullen ASC · July 13, 2013

Read this article for some ideas regarding exposing a grey scale for different gammas:

http://blog.abelcine.com/2012/10/05/working-with-canon-log/

The thing to keep in mind is that a log image should not look "normal" in brightness on a monitor and even on a waveform, it looks odd (and each manufacturer's version of log is different... When I first looked at the Alexa's Log-C on a waveform, I thought something was odd because the 11-step grey scale was so compressed into the lower middle of the waveform, because the camera was recording more like 14.5-stops, so 11-steps took up less of the waveform range. I was used to the white square hitting 70% with PanaLog, but with Log-C it was more like 60%.)