How is dynamic range actually measured?

[Keith Walters]

I've been working on a simple device that would in theory allow you to make a quick and unambiguous measurement of the dynamic range of a video camera.

 

One version of this is a box with a row 15 white LEDS, which have been carefully set up so that the brightness of each LED is exactly 50% of the brightness of its neighbour to the left.

 

If the leftmost (brightest) LED is regarded as 0 stops attenuation, then each LED represents another stop darker, until you get to 14 stops (1/16,384th as bright as the first LED)

 

The "works" of the LEDs are clearly visible through their epoxy cases, so a reasonable indication of the saturation point of whatever sensor is used, would be when you could no longer image the YAG crystal that produces the white light.

 

At the moment I don't have the facilities to accurately read down to such low light levels, but indirect measurements indicate I have the attenuation levels approximately right.

 

The problem I have is that no camera I can get hold of can handle more than about 9 stops! That is, when I adjust the iris so that I can just clearly see the inside of the brightest LED, anything below 9 stops just doesn't appear on the monitor screen!

 

So what am I doing wrong? Surely the definition of "dynamic range" refers to a camera's ability to handle extremes of brightness in a scene.

 

How do other people measure this?

[John Brawley]

The problem I have is that no camera I can get hold of can handle more than about 9 stops! That is, when I adjust the iris so that I can just clearly see the inside of the brightest LED, anything below 9 stops just doesn't appear on the monitor screen!

 

So what am I doing wrong? Surely the definition of "dynamic range" refers to a camera's ability to handle extremes of brightness in a scene.

 

How do other people measure this?

 

 

Keith it sounds like a brilliant device.

 

And 9 stops sounds about right to me...

 

jb

[Max Jacoby]

Keith

 

What cameras did you test?

[Walter Graff]

Keith, a grey scale and waveform scope will do the same thing as what you are trying to do and a lot easier.

 

This should help explain lattitude

 

http://www.reel-stream.com/DynamicRangeAndLatitude.pdf

[Keith Walters]

Keith

 

What cameras did you test?

 

Camera that are supposed to have more dynamic range than that.

I'm sorry, but given my (and others') past experiences with this sort of thing on this forum and elsewhere,

that is all the information I am prepared to divulge on that subject at this point.

 

All I am prepared to say about my preliminary test results is that shooting my test setup with an ordinary still camera with 100ASA film,

the results were broadly in line with what is generally expected of film capture. It was only a rough "range finding" exercise;

All I did was shoot the LEDS with a renge of iris settings and selected the one that clearly captured the most of them.

 

My aim is not produce test reports, only test equipment.

I only want to determine if my methodology is correct before I spend any serious money on this project.

[Keith Walters]

Keith, a grey scale and waveform scope will do the same thing as what you are trying to do and a lot easier.

 

This should help explain lattitude

 

http://www.reel-stream.com/DynamicRangeAndLatitude.pdf

I am perfectly well acquainted with that technique, since I spent many years maintaining broadcast equipment.

In my experience, making accurate reflective gray scale charrts with more than an 8-stop range is extremely

difficult. The ones I used to use had strips of black and dark grey fabric glued on for the darkest bars.

 

The backlit glass plate types were better, but they cost a fortune and are cumbersome and fragile,

and I seriously doubt they could produce a 15-stop range anyway.

 

I want to design something that is

 

*Cheap

*Portable

*Rugged

*Verifiable without needing expensive equipment.

[Daniel Sheehy]

I am perfectly well acquainted with that technique, since I spent many years maintaining broadcast equipment.

In my experience, making accurate reflective gray scale charrts with more than an 8-stop range is extremely

difficult. The ones I used to use had strips of black and dark grey fabric glued on for the darkest bars.

 

The backlit glass plate types were better, but they cost a fortune and are cumbersome and fragile,

and I seriously doubt they could produce a 15-stop range anyway.

 

I want to design something that is

 

*Cheap

*Portable

*Rugged

*Verifiable without needing expensive equipment.

The only issue I could see with that is the fact that the phosphors in a white LED degrade eventually.

[Keith Walters]

The only issue I could see with that is the fact that the phosphors in a white LED degrade eventually.

Yes, but I have the means for detecting and compensating for that.

The other possibility is to put a series of ND bands on a fluorescent tube, although I don't think ordinary ND gels are accurate enough..

 

I have an old 19th century physics textbook that tell you how to make precision smoked glass filters using a kerosene lamp flame. I would have to build a sort of rotisserie affair to rotate the lamp while it is running and do continuous measurements until the correct density was achieved.

 

Apparently smoked glass filters are still highly regarded for their stability and accuracy, although they are very delicate. (I made some so I could record the transit of Venus a few years back, and they worked extremely well).

[John Sprung]

The only issue I could see with that is the fact that the phosphors in a white LED degrade eventually.

Maybe go with tricolor LED's and match them to white? Do they have better long term stability?

 

 

 

 

-- J.S.

[Keith Walters]

Maybe go with tricolor LED's and match them to white? Do they have better long term stability?

 

-- J.S.

I have never been able to get satisfactory results from that approach. The result always has distinct patches of red, green and blue.

 

In any event, there is no particular reason why the light has to be white, except that people will invariably think it should be! Ultra-bright Red or Green LEDS should serve just as well.

 

As far as degradation of the phosphors goes in white LEDS, I doubt that would be an issue, since they would only be activated for short periods.

 

It is a real problem in some cases, though. Some time back I bought some pink LEDS for a custom bathroom installation. I'm not sure exactly how the pink colour is obtained, but they all faded to a yellowish-white after a couple of months :lol:

 

Overall, I have found blue LEDS make the best night lights. They seem to have an indefinite life, unlike white ones which don't even seem to last as long as ordinary light bulbs. Some parts of my house looks like the set of Blade Runner if you have to get up in the middle of the night! People don't realize where the light is coming from, they just assume it's moonlight.

[John Sprung]

Funny you should mention that. We have this blue LED clock that's so damn bright I see what it lights using cone rather than rod vision. It led me to figure out that there's about a stop and a half of difference between 12:58 and 1:11. ;-)

 

 

 

 

-- J.S.

[Keith Walters]

Funny you should mention that. We have this blue LED clock that's so damn bright I see what it lights using cone rather than rod vision. It led me to figure out that there's about a stop and a half of difference between 12:58 and 1:11. ;-)

 

-- J.S.

I have a GE clock radio that's nearly 30 years old, with a vacuum fluorescent display (which works more or less on the same principles as a CRT TV set). That is the only clock radio I have ever seen that has a brightness control you can turn down to almost nothing. In fact I was given it because its previous owner thought the display was worn out - it was just turned down! On full brightness it makes a handy night light, you can easily read by it!

What always amuses me is the number of DVD players, HiFi systems and suchlike that have a glaring blue "standby" light. It's really hard to watch TV with a blue searchlight glaring into your eyes all the time.

I have a cheap DVD player that not only has FOUR extra-bright blue LEDS that com on when it is switched to standby, but they flash as well!!

I would love to meet the peabrain responsible for designing THAT.

The other amusing thing is that with many electronic appliances, switching them to standby seems to do little more than disconnect the signal output and change the display to "standby". The difference in power consumption is so minute, you would wonder why they even bother!

[Keith Walters]

But getting back to the original subject, exactly how exactly are the figures for dynamic range determined?

From the figures that are routinely published, either I am doing it wrong or they are doing it wrong.

I'm afraid I have little time for notions of "accuracy by consensus".

[Walter Graff]

Funny you should mention that. We have this blue LED clock that's so damn bright I see what it lights using cone rather than rod vision. It led me to figure out that there's about a stop and a half of difference between 12:58 and 1:11. ;-)

 

Actually John, you are more using your rods far more than cones. In fact mostly all rods with blue in darkened rooms and no cones. Actually your eye has two mechanisms built into it that make you not see blue well, and not in focus. Your fovea centralis of your eye (the most sensitive color seeing part of the eye) has zero cones for blue and the macula actually filters blue out of your site all the time. This on top of the fact that you eye has 20:1 and 18:1 red and green cones to your blue cones. This means that you can not focus blue light well as your eye is not designed to besides the wavelengths being too long. You may have noticed that some ignorant designers use blue LEDs for signs like buses and marquees, but you can't read them because you can't focus on them. And you may have noticed that when a blue LED is in a room and well out of where you are looking, it is distracting. That is because most of your cones are on the periphery of the retina so blue is magnified on the edge of your sight. Basically we never needed to see much blue so we developed eyes that ignored it mostly. This on top of the fact that blue LEDS emit up to five times more light than other LED colors and you have a blue disater. The effect you are all talking about related to blue being so bright in darkened rooms is called Purkinjes Shift. It is because your rods are most sensitive to greenish-blue light so blue is harsh on the eyes in dark and low light situations. It goes back many millions of years when our family of ancestors developed better green-blue night vision because we developed our eyes and there ancestral abilities both under green leaf canopies and underwater. And talk about stupid, about the worst kind of light you can sleep with is a blue light as in LED. Blue light's effects on melatonin production or should I say the reduction in melatonin production are published all over peer review and yet because it looks cool and blue LEDs are cheap, these idiots are actually causing you to get a worse nights sleep by making blue clocks. Take your blue clock out of the room and you will get a better sleep.

 

WOW! Between my degree in astrophysics and my further study of color psychology, I actually learned something. :rolleyes:

[John Sprung]

Take your blue clock out of the room and you will get a better sleep.

It being the alarm clock, of course I'd sleep better -- and longer -- without it. ;-)

Seriously, though, I'll try covering it up.

 

I'm familiar with Perkinje's shift, photopic/cone vs scotopic/rod vision, peak sensitivity going from 555 nm down to 505 nm in dim light, and all that. I definitely do see the light from this clock as blue within about 10-15 feet. In the far corner of the room, though, it does get under the threshhold, and I see it fade to monochrome. I don't seem to have any trouble focusing in a blue-only envrionment. It's when you have a strong blue along with other colors that the problem arises -- hence those low rider tail light things with the little dot of blue in the middle of a red lens.

 

Thanks --

 

 

 

-- J.S.

[Dominic Case]

I have a cheap DVD player that not only has FOUR extra-bright blue LEDS that com on when it is switched to standby, but they flash as well!!

I would love to meet the peabrain responsible for designing THAT.

Straying further off-topic, but what the hell, this bugs me. I have an el-cheapo printer for my computer. (It cost less than a full set of replacement ink cartridges for my previous one.) It has one button. It's not really an on-off button because the printer turns itself on when you plug it in. Though if the printer is off, you can push it to turn it on. Following so far?

 

However, when the printer is on, the button really comes into its own. There are three functions.

• Push it briefly to put the printer into standby mode (when the yellow LED flashes for ever - how irritating). Push it a second time to return to "on". (Not "off".)
  
• Push it for ten seconds or so (the book says three) to turn the printer off. (But not from standby).
  
• Push it for more than a second and less than ten (or so) and it prints a test page. :blink: (Just what I need)
  

I try to turn the printer off every night to save power (minimal, but everything helps and I hate the flashing). I wasted so much paper trying to guess what the wretched thing will do, I learnt to take the paper out before attempting to turn it off. No good. It goes into "fault" mode, and won't do anything (unless I unplug it) until I feed it paper.

 

I think I see why it was so cheap.

 

BTW Keith, I'd say your pink LEDs were just white ones with pink glass. In our film darkrooms we use yellow (583nm) LEDs because they produce a highly visible light that sits between the peak red and green sensitivities of the film print emulsion layers and rod sensitivity to that wavelength, though not at the peak, is good. They still aren't "safelights" just safer than any other wavelength. And, unlike filters, they NEVER fade.

[Walter Graff]

BTW Keith, I'd say your pink LEDs were just white ones with pink glass.

 

 

Actually there are true pink LEDS available.

[Walter Graff]

In the far corner of the room, though, it does get under the threshhold, and I see it fade to monochrome. I don't seem to have any trouble focusing in a blue-only envrionment.

 

As I said in my post, I wouldn't doubt it. Mother nature actually doesn't want you to see blue in that type of environment and actually filters it out.

 

As for seeing the blue blur, simply go to a strip mall at night and try to read the store that has a blue lettered sign. You can't .In my town of Amherst, folks are pissed because the local theater bought a all blue LED marquee and no one can read it. And my local bus company bought half a fleet of buses with all blue LED signs and recently the community petitioned them to change them cause no one can read hte bus routes as the bus approaches.

[Evangelos Achillopoulos]

I?m just getting everyone back to the topic?

 

Keith this is a nice Idea? But the problem is that you can?t evaluate the noise in all the steps (as you point in your message) and with the waveform or a monitor you can?t see information that?s hidden below a point.

 

The correct method is, just to buy for 84$ a Danes Picta calibrated step chart like TS28D that has 28 steps of ND?s with ½ stop each reaching a range of 14 stops

 

The web site is : http://www.danes-picta.com/txt_PhotoDigital.htm

 

Or alternatively you can buy a Stouffer T4110 step chart with 41 steps of 1/3 stop a total of 13,6 stops for a bit more dollars?

 

The web site is : http://www.stouffer.net/TransPage.htm

 

Put the step chart to a black carton sheet in a way that no light passes from the edge of the film, so only the steps are visible, back light it with a diffuser in a darkened room in a way that the back light is not lighten the room or the lens of the camera. The level of light should be in a point that you can discriminate, by naked eye the darkest step from the carton black sheet.

 

Use a waveform to adjust the aperture in order to have at the top and just clipping the brightest step. Record few frames. Capture uncompressed 10bits few frames of the recorded footage, use software like Combustion or After Effects in 16bit mode to export a frame in TIFF 16bit format.

 

Then buy for 199$ a software that is a great tool for all of us that we dealing with photography or cinematography in order to assist us in measuring scientifically and evaluate our hardware. The software is the IMATEST. It measures system MTF, tonal response, noise, dynamic range analysis, veiling glare, chromatic aberration analysis, calculate distortion for 3rd and 5th order polynomial and many other aspects of what we do.

 

The web site is : http://www.imatest.com/docs/q13.html

 

The total cost of that set up is no more than 300$. And it?s quite scientifically accurate. And I thing its inexpensive compeered to the LED method?

 

This set up allowed me to do the following tests back in September 2007:

 

http://www.motionfx.gr/files/Latitude%20Ob...son%20Sep07.pdf

 

Check out the graphs and you will realize that a simple evaluation in a monitor or a waveform it?s not good enough, the approach that Mr. Norman Koren follows in Imatest is giving immediately an answer of the level of DR quality that the camera has. By splitting the results in High quality DR (noise level less than 0,1 of a stop for the given step that measures), to Mid High quality DR (noise level between 0,1 and 0,25 of a stop), Mid quality DR (0,25 to 0,5 of a stop) and Low quality DR (0,5 stops to 1 stop noise).

 

In the past I have propose the Imatest workflow to be a reference in our measurements, but it didn?t had much of acceptance my proposition, probably because will make some people to sad?

[Keith Walters]

Straying further off-topic, but what the hell, this bugs me. I have an el-cheapo printer for my computer. (It cost less than a full set of replacement ink cartridges for my previous one.)

I've done that on two occasions now. I bought an HP 1210 all-in-one printer-scanner-copier about four years ago for the then-unheard of price of $154. It still works fine, but on one occasion I needed new cartridges and Big W were out of them. But Tandy Electronics across the way were selling a complete HP printer with the same type of cartridges included for $56, considerably less than the price of two new cartridges alone! So I've never used the printer, and it would still cost me more to re-cartridge it than to buy another new one!

Then the Post Office were selling a later version of the 1210 for $79, again including cartridges, so I bought one as a spare! It can also be used as a standalone colour copier, which is amazing in itself.

 

Now, Dick Smith Electronics are selling an even more advanced version of the HP all-in-one for $69! I haven't really used my "spare" yet!

 

 

It has one button. It's not really an on-off button because the printer turns itself on when you plug it in. Though if the printer is off, you can push it to turn it on. Following so far?

However, when the printer is on, the button really comes into its own. There are three functions.

• Push it briefly to put the printer into standby mode (when the yellow LED flashes for ever - how irritating). Push it a second time to return to "on". (Not "off".)
  
• Push it for ten seconds or so (the book says three) to turn the printer off. (But not from standby).
  
• Push it for more than a second and less than ten (or so) and it prints a test page. :blink: (Just what I need)
  

 

A lot of this stuff is designed by recent engineering graduates in Chinese universities. Having to deal with this on a daily basis, the explanation soon became painfully apparent to me: most of them have grown up in pre-economic boom households where they never had VCRs, computers or DVD players! It's fairly obvious to us how something should (or should not) work, most of them simply haven't got a clue. This also explains why you get so many remote controls packed with sometimes idiotic functions that you would never use, or only ever use once, when installing the appliance! You can sometimes get ludicrously cheap TV sets (20" for $99), but you will nearly always find at that low price, the remote will have "behavioural problems". (Like one I recently bought that permanently changes the sound carrier frequency on the station you are tuned to at the accidental touch of a particular button!) If you prise open the remote and put some insulation tape under the offending buttons, you can often turn a problem "cheapie" into quite a good set.

 

 

 

BTW Keith, I'd say your pink LEDs were just white ones with pink glass.

 

No, the fluorescent crystal has a slightly pinkish appearance, but it's nothing like the colour it glows, and the epoxy envelope is colourless. It's not actually pink anyway, more of a magenta colour. Certainly I can get better results with gels taken from a sample book, but they also fade eventually.

 

In our film darkrooms we use yellow (583nm) LEDs because they produce a highly visible light that sits between the peak red and green sensitivities of the film print emulsion layers and rod sensitivity to that wavelength, though not at the peak, is good. They still aren't "safelights" just safer than any other wavelength. And, unlike filters, they NEVER fade.

Yes I know, I worked there for a while. Apparently they were paying a horrendous price for the LEDs from the USA, and then they found they were available from RS components for a fraction of the price!

[Keith Walters]

I?m just getting everyone back to the topic?

 

Keith this is a nice Idea? But the problem is that you can?t evaluate the noise in all the steps (as you point in your message) and with the waveform or a monitor you can?t see information that?s hidden below a point.

 

The correct method is, just to buy for 84$ a Danes Picta calibrated step chart like TS28D that has 28 steps of ND?s with ½ stop each reaching a range of 14 stops

 

The web site is : http://www.danes-picta.com/txt_PhotoDigital.htm

 

Or alternatively you can buy a Stouffer T4110 step chart with 41 steps of 1/3 stop a total of 13,6 stops for a bit more dollars?

 

The web site is : http://www.stouffer.net/TransPage.htm

 

Put the step chart to a black carton sheet in a way that no light passes from the edge of the film, so only the steps are visible, back light it with a diffuser in a darkened room in a way that the back light is not lighten the room or the lens of the camera. The level of light should be in a point that you can discriminate, by naked eye the darkest step from the carton black sheet.

 

Use a waveform to adjust the aperture in order to have at the top and just clipping the brightest step. Record few frames. Capture uncompressed 10bits few frames of the recorded footage, use software like Combustion or After Effects in 16bit mode to export a frame in TIFF 16bit format.

 

Then buy for 199$ a software that is a great tool for all of us that we dealing with photography or cinematography in order to assist us in measuring scientifically and evaluate our hardware. The software is the IMATEST. It measures system MTF, tonal response, noise, dynamic range analysis, veiling glare, chromatic aberration analysis, calculate distortion for 3rd and 5th order polynomial and many other aspects of what we do.

 

The web site is : http://www.imatest.com/docs/q13.html

 

The total cost of that set up is no more than 300$. And it?s quite scientifically accurate. And I thing its inexpensive compeered to the LED method?

 

This set up allowed me to do the following tests back in September 2007:

 

http://www.motionfx.gr/files/Latitude%20Ob...son%20Sep07.pdf

 

Check out the graphs and you will realize that a simple evaluation in a monitor or a waveform it?s not good enough, the approach that Mr. Norman Koren follows in Imatest is giving immediately an answer of the level of DR quality that the camera has. By splitting the results in High quality DR (noise level less than 0,1 of a stop for the given step that measures), to Mid High quality DR (noise level between 0,1 and 0,25 of a stop), Mid quality DR (0,25 to 0,5 of a stop) and Low quality DR (0,5 stops to 1 stop noise).

 

In the past I have propose the Imatest workflow to be a reference in our measurements, but it didn?t had much of acceptance my proposition, probably because will make some people to sad?

The trouble is, what you have described is apparently what everybody is doing now, but frankly, I don't believe the results they are getting (or claim to be getting). There is no way of knowing for sure that what is reflecting off the chart is actually what the manufacturer says it is supposed to be.

 

My idea is to make a test device, where I know with absolute scientific certainty that, for example, the LED marked "10 stops" is 1/1024th as bright as the one marked "0 stops".

 

There is also no way you could accurately view 12 or 14 stops of a linear grey scale on a waveform monitor. The only thing I am interested in is how many of the square YAG fluorescent crystals could be clearly seen on the monitor screen. Which is all that counts really!

 

I'm not interested in spending $300 either. $30 is more like it:-) Think of me as the Jim Jannard of test equipment :lol:

Edited February 14, 2008 by Keith Walters

[Evangelos Achillopoulos]

Keith probably I didn?t explain well or the language barrier?

 

The step charts are NOT REFLECTIVE and yes, you are correct, reflective test charts are totally wrong in DR measurement.

 

The step charts are TRANSMITIVE they don?t reflect the light?

 

Its like an array of Neutral Density filters that has being calibrated with a densitometer to have exact steps of 0,15D or 0,10D status A and they allow to pass the full visible light spectrum like normal camera mounted ND?s.

 

LED?s do NOT transmit a uniform full spectrum light. That?s very crucial in DR measurement because sensors are not having the same behavior in blue for instance, so if your LED?s are not calibrated with a Spectrophotometer and a Light Spectrum analyzer you can never be certain on the results. You have to have a flat spectrum light source like an array of diffused dido tungsten light as a BACK light (that?s because the step charts are NOT reflective) for the step charts.

 

More over the LED light is a point light it?s not an area light so it?s very difficult without a diffuser to make a judgment in a so few pixels about the noise?

 

In order to make your LED array so accurate in light spectrum and so diffused that you can measure accurately DR I thing that you will spend more than 1000 dollars.

 

I?m an electronic engineer and I know what am saying.

 

?There is also no way you could accurately view 12 or 14 stops of a linear grey scale on a waveform monitor. The only thing I am interested in is how many of the square YAG fluorescent crystals could be clearly seen on the monitor screen. Which is all that counts really!?

 

That?s why I suggested you to use that software in order to see the hidden info in that range. The waveform is needed only to adjust the aperture.

 

The Cinematography class cameras are having totally different ways in image recording than typical Broadcast cameras, for instance Varicam and F23 are recording Log encoded images that need special processing?

 

So monitor observation is good to a certain degree, certainly not to a Cinematography class?

[Evangelos Achillopoulos]

Probably you didn?t read the full contents of the links that I send

 

In the http://www.imatest.com/docs/q13.html

 

Go down to the middle of the page and read the Dynamic Range measurements?

[Walter Graff]

"I'm not interested in spending $300 either. $30 is more like it:-) Think of me as the Jim Jannard of test equipment :lol: "

 

So then it makes sense to market it as a $30 piece of test equipment, but after you buy all the LEDs it's really a $300 piece of test equipment. But you'll get a lot of folks thinking somehow it's a $30 piece of equipment :)

[Keith Walters]

"I'm not interested in spending $300 either. $30 is more like it:-) Think of me as the Jim Jannard of test equipment :lol: "

 

So then it makes sense to market it as a $30 piece of test equipment, but after you buy all the LEDs it's really a $300 piece of test equipment. But you'll get a lot of folks thinking somehow it's a $30 piece of equipment :)

Did I say I was going to market it?

All I was really looking for is an explanation of what, if anything, is wrong with my reasoning. "Other people don't do it that way" is not an explanation.

I jsut want to produce is a portable black box which has 15 little white lights on it, where I can verify that each one is putting out half as many photons as the one to the left of it.

I do not need absolute measurents, my scheme is to have the brightest LED supplied with its maximum rated current of 20 milliamps, the next with approximately 10mA, the next approximately 5mA and so on. To calibrate each LED, all I would need to do is compare its light output with that of its neighbour to the left through a precision 50% ND and adjust its current so that they match. There is no need for quantitive measurements, comparative measurements for equality are all that is required, so the accuracy of the photometer is not an issue.

However, I am not convinced that the dimmest LEDS can be made to work reliably with only 1.2 microamps, so the darkest ones may need some "help" from another ND filter.

I propose to mount the LEDS in a block of aluminium which will be thermostatically heated to 40ºC to minimize thermal drift.

The colour of the LEDs is not important. All this device will do is simulate a real-world dynamic range situation. All I am interested in measuring is how many of the LEDs can be successfuly imaged at the one time.

Other test equipment will not be necessary: all that will be determined is whether the LED can be imaged, or not.