Shooting to the right (and overexposing to reduce noise)

Two *very* interesting articles:

http://www.fredmiranda.com/forum/topic/76284

"Based on that observation, I thought that instead of taking a picture at 800, I can increase the ISO to 1600 and dial +1 stop compensation to overexpose the sensor by 1 stop, that is by a factor of 2. (Don't dial the compensation when you are in Manual, of course, just keep the settings the same). Then (I presume that shooting is in RAW) one can dial down the exposure by one stop during the RAW development. The expected signal to noise gain would be 2/1.25=1.6 compared to just shooting at 800."

http://luminous-landscape.com/tutorials/ex...ose-right.shtml

"A 12 bit image is capable of recording 4,096 (2^12) discrete tonal values. One would think that therefore each F/Stop of the 5 stop range would be able to record some 850 (4096 / 5) of these steps. But, alas, this is not the case. The way that it really works is that the first (brightest) stop's worth of data contains 2048 of these steps — fully half of those available.

Why? Because CCD and CMOS chips are linear devices. And, of course, each F/Stop records half of the light of the previous one, and therefore half the remaining data space available. This little table tells the tale."

The first articles is simply confused and wrong. If he had the exposure right in the first place going to +1 compensation would have clipped his highlights.

"Expose-right" is usually a good idea. You still have to consider all 3 color channels. Craig pointed out earlier that you can clip the blues even if your histogram does not have points pinned on the right edge.

IMHO Norman Koren has written the best article I've seen on this topic. Much better than the Luminous Landscape tutorial.

Yeah, I remember the blue clipping problem.

I know he would clip highlights at normal exposure + 1, but I thought his point was that in RAW, it doesn't clip. Or, perhaps he's in a dark environment, and wants to capture a dark image.

Who knows.

Shooting to the right certainly challenges the older assumption of "always underexpose with digital, since it's easy to clip the highlights by accident." Of course, that statement was usually applied to consumer digicams with lower dynamic range. But does it (still) apply at all to DSLR's when shooting RAW?

Without reading the articles yet (I'm supposed to be working right now) to me, it seems that shooting to the right is a way to try and minimize shadow noise, but at the risk of clipping highlights.

Unfortunately, underwater wideangle, we have a severe problem shooting with the sun in the frame for this very reason - clipping of the detail in the sunball.

So is shooting to the right appropriate for digital underwater photography? Can you even tell by looking at the histogram on most cameras?

Cheers

James

He should have increased the exposure one stop at ISO 800. Unless he' s limited by the lighting.

When the available light is the limiting factor, it may make sense to go to higher ISO rather than underexpose and push in post processing. In the first case you suffer a little higher amplifier noise, and in the second you have higher quantization noise and artifacts. If that's the point he was trying to make, he didn't say it very well.

Higher quantization noise due to the linear nature of the sensors?

The sensors are very linear, but the increased quantization noise is due to not taking full advantage of the range of the Analog to Digital converter. The range of the 12 bit A/D is 0 to 4095. If the highest number that comes out of the picture is 2047, and later you multiply everything by 2 in the raw converter or photoshop then 1/2 the intensity levels are not used or needs to be interpolated. Uping the ISO 2x increases the analog amplification by 2 and fills all 4096 levels at the cost of a little amplifier noise.

Herb: why would the highest number to come ouf of the A to D converter be 2047? Is that just a random number that you picked, or is it the highest you can get with 11 bits? Why did you pick 11 bits?

Cheers

James

Herb: why would the highest number to come ouf of the A to D converter be 2047?  Is that just a random number that you picked, or is it the highest you can get with 11 bits?  Why did you pick 11 bits?

 

Cheers

James

It's just an example. It's exactly one stop underexposed from a "perfect" exposure.

Hi James,

"exposing to the right" (right limit of the histogramm) is also good for UW. But only for images with small contrast. With high sunlight there is no sense or way to use this method.

If you have small contrast, 12bit sensor where 0 is black and 4095 white (4096 steps), an example:

- image contents dynamic range goes from 200 to 2000.

- let's asume from 0 to 100 is sensor noise.

the signal to noise ratio is not perfect as you only have 100 steps between noise to image content and throw away over 2000 steps of the sensors range.

now you set + exposure compensation until the highlight of the image content (was 2000) moves close to 4095 (but never beyond to prevent clipping).

now you have the following:

- image content from 2200 to 4000

- noise maintains 0 to 100

you have improved the signal to noise ratio for this certain image. the distance between noise and image content has raised from 100 steps to 2100.

if the image now looks to bright exposed you lower it in photoshop. But you also lower the noise so the noise to image content distance is still better than without the method.

I've just taken imaginary numbers.

I think on a tripod when shooting landscapes is different to UW hovering in front of a coral. So just if the capicity is left and no risk to clip highlights it makes sense.

I would also not gain ISO for this method as you get higher sensor noise drawback.

cheers,

Julian

People need to stop thinking simultaneously in linear and logarithmic terms. Michael Reichman did it and that's why his article so completely misleading. I'm no fan of LL and MR as I believe people who read there are likely to misunderstand how things really work. No greater example of that than this topic.

In the example given, noise values from 0-100 would represent a full seven stops of noise out of a possible 12 stops of range. Hardly representative of what actually happens. We should all be talking about what exposure is optimal and what happens when we fail to expose to that ideal. In the example given, underexposing yields a range from 0-2047 rather than 0-4095, or about 2000 discreet values less. That may be true but it's still only one stop and that stop is no more valuable than the rest. What you have is 4 stops of usable range instead of 5 (because the quoted noise is so high). Why don't the extra 2000 values really matter? Because the bulk of them are used to represent the detail that's already in the lower stops.

Each stop of light recorded is represented in the ADC as a bit. Each stop, like each bit, is independent of all other stops of light, so its best to talk about digital samples logarithmically rather than linearly (since that's how we talk about the light). If you want to talk about the extra stop of light "doubling the possible linear values" that's fine, but remember that the extra stop of light represents of doubling of all possble light intensities as well. There is no inconsistency here unless you foolishly think of light in stops and pixels as linear as Reichman does. Linear encoding is exactly, precisely what is required for the job---no more and no less than what is needed and absolutely nothing is wasted. To suggest otherwise is to misunderstand how this works. Remember that the image is converted to a color space later.

All modern cameras are capable of delivering 7 or more stops of dynamic range depending on the definition of acceptable noise. Assuming 12 bit converters, that means that noise traditionally would be confined to the lower 4 bits or so of the image. In reality its much more complicated than that, but that should give a better idea of the noise range (0-15 rather than 0-100). According to Thom, the noise profile of the D100 makes it not suitable for a traditional "expose to the right" technique, throwing yet another wrench in the works.

I think people should be "exposing to the right" underwater just like they should everywhere else (Thiom's concern with the D100 excepted). Trouble is that you need to expose up until one of the color channels reaches the right and that can be difficult underwater with today's cameras. For Kodak fans, ERI makes this even harder because we can't even be certain where "the right" is! Thanks for the link, Herb. Looks really good.

craig,

that is exactly why I did not mention any stop values and relations. Just commenting the "11bit" question and risk of overexposureing.

the analogue output of the photo element is converted into digits. I don't see an AD Converter precisely calibrated to f-stops in respect to switching to the next higher bit. Then the dynamic range would have to be exactly 8 stops. I guess you just wanted to express the logarithmic term. It is of course true that light values are handled logarithmicly (right word in english?).

As I have written, the noise and image content values were imaginary numbers, no measurement of sensors noise.

but craig, that brings up a question to me what you could answer:

I took my 10D and zoomed into a light. I did several shots in manual.

to difference between complete black to complete white was about 6 stops.

opening in Photoshop for simplyfing converted to 8 bit:

the values (BW)

black image: 0

3 stops above black (mid grey and middle of the histogram): 128

white (6 stops away from black): 255

Here's my question: does photoshop convert the numbers and does not show the true file information?

in spite of algorythmic f-stops in respect to light, the image data seems to be very linear handled. So far I was pretty happy that it is that way. Otherwise the complete left half of my 10D histogram would have a very very low dynamic digital resolution compared to the right side.

Please tell me what I get wrong here?

craig,

 

that is exactly why I did not mention any stop values and relations. Just commenting the "11bit" question and risk of overexposureing.

 

the analogue output of the photo element is converted into digits. I don't see an AD Converter precisely calibrated to f-stops in respect to switching to the next higher bit. Then the dynamic range would have to be exactly 8 stops. I guess you just wanted to express the logarithmic term. It is of course true that light values are handled logarithmicly (right word in english?).

 

As I have written, the noise and image content values were imaginary numbers, no measurement of sensors noise.

The converters are highly linear over their functional range, so assuming the photo sites are as well, 8 bits is 8 stops as you say. The correlation between linear bits and stops is (or at least should be) very high.

Regarding whether light is linear or logarithmic, light simply is what it is. Photographers treat it logarithmically because our exposure system works well that way. It's not important which way you look at it so much as making sure you don't change horses midstream. If we want to talk of light in stops (as we do) we need to talk of pixels in bits, not linear values. An extra stop is simply an extra bit of resolution. Michael Reichman's explanation of this could not have been more misleading or damaging in this regard. The total number of linear values added by adding a bit or resolution is completely irrelevant. The number of discreet values added is exactly the right amount needed to achieve that extra bit. Thinking that this is somehow important is wrong.

The original comment about overexposing a stop and using exposure correction in the raw converter is naive. This would work only if the camera manufacturer deliberately overrated the ISO. All manufacturers do this to some extent, Kodak far more than others, but it's hard to believe that a full stop can be had consistently without producing garbage except with the Kodak where it's known to be the case.

Cameras must consistently produce proper exposure over a wide range of white balance settings, so its safe to assume that we have some extra exposure lattitude when shootting with light well matched to the sensor as we do with strobes. Whether that gives us a full stop consistently is another matter, but remember that anything other than black backgrounds will not be well balanced light. I would ignore the advice of setting a blanket +1ev exposure compensation or adding a stop of ISO. I think that's a sure way to ruin a lot of underwater images.

<<In the example given, noise values from 0-100 would represent a full seven stops of noise out of a possible 12 stops of range. Hardly representative of what actually happens>>

sorry, I don't get it. You really want to say:

You say that the value 100 given in the example would cover 7 fstops out of 12.

That yould mean:

within the first bit (1st f-stop) you have a dynamic resolution of 2 steps (2^1).

within the 12th f-stop you would have 2048 steps (4096 minus 2048 steps used for the first 11 stops).

no, impossible. the value 100 (it of course is not the real sensor noise) out of 4096 gives 1/40 of the linear scale.

Assuming 12 fstops, it's just 0,3 fstops. never whole 7 fstops.

The lower 7 bits of a 12 bit word contain the values 0-127. If we assume that values from 0-100 are noise, then almost the entire linear range of the first 7 bits is consumed by noise (6 2/3 bits). Since each bit is a stop, this example has nearly 7 of the 12 possible stops consumed by noise, leaving on 5 1/3 stops of possible dynamic range left.

A computation like 100/4096 is meaningless because it is a linear computation and we are talking about logarithmic quantities. This is exactly my point. Pick one, linear or logarithmic, and stick with it. Since we like to talk about stops of light we need to stop talking about 100 values of this and 2000 values of that. Those concepts are meaningless in the context of stops. 0-100 is nearly 7 stops of noise, not 1/3 stop.

Keep in mind that JPG's and TIFF's are encoded in a color space and are NOT linear-encoded images. We are talking about the raw image coming from the sensor, not a cooked image with gamma applied to it.

7 bits are not 7 fstops. What makes you believe it?

Also a 7 bit word with every bit set to 1 is not 7 fstops. mixing up bits (lower or higher ones) and f-stops is wrong.

The sensor output is linear in relation to the output value (integer!), that means: 7 out of 12 give you 2047. Not 127.

You really think every f-stop is recorded with different dynamic resolution? going from 2 steps (1st fstop) to 2048 steps (within last fstop)?

the original question: the signal to noise ratio is gained when exposing to the right.

In linear encoding, 7 bits is 7 stops. Every bit represents a doubling of value and every stop represents a doubling of light. This is an undisputable truism.

Now, a 7 bit word can fully describe all possible discreet values over a 7 stop range where a 7 bit value of all ones is commonly the brightest (or darkest) possble value. A 7 bit word of all ones is not "7 fstops" because that's just nonsense.

The sensor output (actually the ADC output) is linear with respect to the light input. "7 out of 12" gives nothing because you have provided no context within which to interpret it. If you are referring to 7 bits out of 12, 7 bits yields 128 possible discreet values. 2048 discreet values requires 11 bits. This is asimple base2 to base10 numbering conversion.

I do not think "every f-stop is recorded with different dynamic resolution". First, they're called stops, not fstops. Second, Each stop is a doubling of light and its presence or absense is recorded by precisely one bit in the word. Nothing more, nothing less. Each and every stop, from brightest to darkest, gets exactly the same treatment. Take any subset of stops, say 4 arbitrary stops in the image, and that subset will be described by precisely 4 bits in the word regardless of which stops you choose. Every stop of light is orthoganal just as every bit is.

You seem to be confusing stops of light with "zones". Zones are tonal ranges of an image which are typically 1 stop apart, but zones and stops are not the same. The lowest stop of light represented in an image has exactly two values, 0 and 1. If it had any more values than that it would not be the lowest stop by definition! This is not true of zones and explains why there is a difference between the number of zones in an image and the dynamic range. This is another source of irritation I have with Reichman. He can't seem to be bothered to get his facts or his terminology consistently correct. Reichman claims that digital cameras provide roughly 5 stops of dynamic range. This is, of course, absurd, but I suspect he really means 5 zones which would be a far more reasonable claim. For someone who thinks so highly of his own knowledge and talent I expect better.

I did not hear or read about reichmann, just overflew this article. it's not the first one dealing with this topic. does he claim to have a lot of knowledge?

more bits give more steps from black to white. just the dynamic steps. but doubling the amount of light (next fstop) does not double the output value of the sensor? Thats what I have expirienced with my 10D: one fstop moves the histogram content to the right, in a liniear way. if I go one fstop further it moves about the same distance in histogram and gives the same linear raise in photoshop.

your theory: every fstop brighter would move the histogram content twice as far to the right as before. but it does not. Also the values in photoshop just move about the same amount.

And even if it's logarythmic value output, that would even more gain the advantage of using the right side of the histogram to keep noise down.

I don't know/think it's perfectly linear. But from 4096 steps covering 12 stops, I don't think the first stop in the histogram gives the value 1 (first bit set to 1) out of 4096. would be very bad shadow handling with just a few steps available.

You're right saying every fstop doubles the amount going onto the sensor, but I still don't think the stops have different distances on the cameras histogram display.

Craig,

sorry, in german it's "Blende". thought it is f-stop in english.

I think I am on the way to get it:

Reichmanns statement is some kind of curious as he focuses on the value (4096) which has a big distance from the low noise values.

But as you mentioned, the gain is not over the complete range up to the right. It's just the amount on top to reach the right side. Very true. I got it now. thanx a lot for your patience.

But handling this with stops I think is not very usefull, because it doesn't describe the noise and signal values from the sensor very good.

To make sure I got the second item: to go from the left of the histogram you could have to double the amount of light twice or more in order to reach the middle. To get further to the right end you have to double less times as before?

Maybe it was not very obvious to me when I tried with my camera. It gave a pretty linear impression.

I still think the linear contemplation makes sense as the image data and noise is what we receive with the file.

cheers,

Julian