Technical: one stop on gamma histogram

<p><br />Technical question about gamma in histogram, and one stop exposure difference.<br /><br />We know camera sensors are linear (RAW data).<br /><br />We know that one stop underexposure is 1/2 of the light intensity.<br /><br />So we know one stop underexposure will move any data represented by 255, down to be at midpoint, or half scale, which is at 128 (in linear RAW data).<br /><br />We know we never see linear, all RGB images and histograms are gamma encoded. So this gamma encoding shifts linear midpoint 128 at 50% to be shown at<br />(0.5 ^ 1/2.2) = 0.73, x255 = 187, or 73% of 255 full scale <br /><br />So the conclusion is that one stop down from 255 should be at 187, or 73% on histogram.<br /><br />I would never question any of that, it seems true by definition, but yet.... it does not happen. One stop underexposure (Nikon D300) does not move a white peak at right edge at 255 to be anywhere near 187 at the 3/4 point. One stop under full scale shifts no more than half of that much, if that. Crudely, perhaps to 88% or 90% instead of 73%.<br /><br />I cannot account for why not? There are additional things happening, like White Balance, but I would not expect any other effect nearly this large...<br /><br />Any insights to resolve this would be appreciated.</p>

<p>A quick test on a couple of my photos show 1 stop under shifted the histogram by about 75%, pretty close to the 73% you were looking for. Here are the photos<br>

<a href=" 1/644 sec</a><br>

<a href=" 1/322 sec</a></p>

<p>Wayne,</p>

 

<p>RAW converters do not apply a simple exponential gamma factor to the linear data. Instead, they

mimic a film curve with its shoulder and all.</p>

 

<p>See <a href="http://www.photo.net/canon-eos-digital-camera-forum/00W31g">this thread</a> for an

experiment I performed to address exactly that question.</p>

 

<p>Cheers,</p>

 

<p>b&</p>

<p>Ben, that curve in that linked thread looks very familiar. </p>

<p>Can't count the times I've had to apply that specific shaped curve to quite a few of my images whose scene's dynamic range pushes my camera sensor's capabilities.</p>

<p>Just thought I'ld add I had to remind an imaging scientist working in a lab who posted over at CambridgeInColour's forum trying to nail the linear behavior for precise repeatability in characterizing his Nikon DSLR that consumer grade DSLR's (meaning cheap by scientific lab research standards) are not precision scientific instruments like NASA uses costing over $10K.</p>

<p>And middle gray=50 Luminance in Lab readouts maps to 110 RGB gray in ProPhotoRGB output because of its 1.8 gamma encoding.</p>

 

<p>Tim,</p>

 

<p>Surely that was a typo? Both in Photoshop and according to Bruce Lindbloom’s calculator,

L* = 50 a* = 0 b* = 0 works out to R=G=B=100 (not 110) in ProPhoto.</p>

 

<p>Our DSLRs certainly aren’t up to NASA standards, but I’ve found them to be far

more than up to any job I’ve had to put them to. And I’d think the shutter would be a

much larger source of error than the sensor itself.</p>

 

<p>And if he’s looking for that kind of data out of a DSLR, he’s not going to want to use

any of the commercial RAW converters…he’s at least going to need to start with

something like DCRAW or LIBRAW, and probably wind up coding something himself.</p>

 

<p>Cheers,</p>

 

<p>b&</p>

<p>Thanks guys. I think it must be the camera making adjustments. I know Adobe ACR does a S-curve, but I was speaking of the camera LCD. Was speaking of a Nikon D300, but I just tried a D70S, and one stop down is just above 75%, more in agreement to the expected 73% (there is some error at my 255 end). But a D300 is more like 7/8 or 88% at one stop down. <br /><br />Thom's D300 guide has some info related to this. However, it seems more confused than I am though, not sure what it is saying. Page 67 (edition 2) shows a chart of values within stops, which says theoretically assumes the camera has exactly 7 stops of range. It starts at the 0 end, and it puts the center 4th stop centered on 128 (says 111 to 146). So that is not a linear assumption. I think it is not gamma either. It says this imaginary top 7th stop is 220 to 255. However, this chart obviously only divides 255 into 7 equal spans (36, 73, 110, 146, 183, 219, 255) , which I think does not represent stops, but he calls it stops.<br /><br />Page 69 shows a response curve said to be plotted from photographing a Kodak step chart. No units or coordinates to get hold of, but I doubt this can be linear, and it does in fact look similar to a gamma chart (a curve with midpoint boosted substantially). Says a bit unexpected that it is not perfectly flat as on previous Nikons ? I don't get much from that. No content containing detail. The words imply a boosted gamma curve however, and that is what I am seeing, on the D300, but not on the D70S.</p>

<p>Wayne,</p>

 

<p>I don’t think it should take you more than an hour or two to reverse-engineer what’s

going on and know exactly. Just (as I did) fill the viewfinder with a uniformly-lit gray card, shoot in

manual mode, put the bug wherever you want on the meter, and record the results.</p>

 

<p>One thing I observed, but didn’t mention in that overly-long post, was that the in-camera

histogram is off by about a third of a stop from the meter in the viewfinder. For example, when I place

the bug on the middle exposure setting, the histogram is usually slightly to the left of the middle line.

Brightening the exposure by a third of a stop will either center the on-camera histogram exactly or

nudge it very slightly to the right past center. However, that doesn’t hold throughout the

exposure range…put the bug on the leftmost marker and the histogram is to the right of the

corresponding line, not the left.</p>

 

<p>I’m now much less concerned about the in-camera histogram than I ever was before.

It’s good as a rough gauge, but not much good for truly evaluating exposure. It’s far too

unreliable and unpredictable. Where incident metering (including with a gray card) is appropriate,

that’s by far the best way to go. Spot metering and intentional placement in a particular zone

works superbly, too — with said zone being determined by an experiment such as the one I

performed.</p>

 

<p>If neither of those are suitable, your best bet is probably autoexposure (aperture or shutter priority

as appropriate) with the camera’s most intelligent entire-scene metering mode, perhaps assisted

by a subtle amount of Zen-assisted exposure compensation. But if the work is at a slow enough pace

to worry about exposure, it’s probably also slow enough to shoot manually and meter

properly.</p>

 

<p>Cheers,</p>

 

<p>b&</p>

<p>Whoops! My mistake on that 110RGB gray ProPhotoRGB comment. I just tested it in CS3 by creating 50L* in a Lab document and converting to ProPhotoRGB and it comes to 100RGB gray. Bruce made a mistake?! Who'da thought?</p>

<p>That explains why I was having so much trouble and why it looked so off profiling my camera using the X-rite 24CC where the RGB readouts in ACR's ProPhotoRGB middle gray patch under the yellow patch kept being over and under no matter how I adjusted the camera's exposure shooting the target. I was using the wrong reference numbers.</p>

<p>Thanks for the correction on that, Ben.</p>

<p>But I still say that our eyes don't follow a precise response due to its adaptive nature with regards to contrast, saturation and color temp. Machines like digital sensors and spectophotometers only measure and capture wavelengths without considering total adaptation.</p>

<p>>>For example, when I place the bug on the middle exposure setting, the histogram is usually slightly to the left of the middle line. Brightening the exposure by a third of a stop will either center the on-camera histogram exactly or nudge it very slightly to the right past center</p>

<p>That was how this started for me. An 18% gray card ought to come out at 117, a bit below histogram center (everything about this is variable of course). But nothing about a gray card is related to histogram midpoint. 18% is 18% (linear).</p>

<p>After gamma, then the histogram should show:</p>

<p>One stop down: (0.5 ^ 1/2.2) = 0.73, x255 = 187 73% of 255 full scale</p>

<p>18%: (0.18 ^ 1/2.2) = 0.46, x255 = 117 46% of 255 full scale.</p>

<p>Kodak 18% gray card instructions say if we meter from a gray card, we should increase exposure 1/2 stop (to the 12% value claimed by others, including Sekonic).</p>

<p>Some recalibrate their meter to make the gray card be centered. This makes 18% read as if it were 22% (which should move to center, and the 50% center moved to 187). This is the opposite direction, and a bit of underexposure, but a small error, less than 1/3 stop (except for that 1/2 stop too). </p>

<p>The gray card was popularized by Ansel Adam's Zone System in 1940. There were no digital histograms and few quantitative tools at that time, and of course, negative film had more latitude of exposure. Still handy, though, for what it is.</p>

<p> </p>

<p>Tim, our eyes, amazing as they are, are far from precision measuring instruments. Even aside from

the gross adaptive measures that let us stare at the mine entrance for a minute or so while shading our

eyes and see detail even on a sunny day, there are well-known small scale adaptations. There are

plenty of optical illusions based on the fact that two colors can look radically different (or surprisingly

similar) when in close proximity <i>v</i> adjacent <i>v</i> viewed sequentially, or other combinations.

And that’s in carefully-controlled laboratory environments!</p>

 

<p>Wayne, another complicating factor might be the color balance of the light. I’ve yet to test

this. It may (or may not) be that the relationship between the viewfinder meter, on-camera histogram, and final processed RAW data changes significantly from open shade to

candlelight.</p>

 

<p>Cheers,</p>

 

<p>b&</p>

<p>Just remember that the incamera histogram is controlled not only by the profile color space (in this case 2.2 gamma sRGB or AdobeRGB) as chosen but all the other saturation and contrast settings and Picture Styles. Trying to find a null setting for repeatability in characterizing exposure behavior with these variables is going to make using the incamera histogram pretty much useless for nailing precise middle gray.</p>

<p>When I shoot the X-rite 24CC and adjust the brightness slider in ACR three points or more the middle gray can be off as much as 5 RGB points and up.</p>

<p>Wayne,</p>

 

<p>Something else I should add. As measured from a spectrophotometer, an 18% Kodak gray card

has an L* value of 50 ±1. Centering the metering bug on that gray card resulted in an L* value

of 54. I’d have to underexpose (not overexpose, as you indicated the Kodak instructions state)

by a third to a half a stop to bring that in line.</p>

 

<p>This is all likely to differ from camera to camera, model to model, and manufacturer to

manufacturer. It’s all but guaranteed to differ from one RAW developer to the next.</p>

 

<p>Cheers,</p>

 

<p>b&</p>

<p>My first question I always ask myself in compelling discussions on this subject is "what is the end goal".</p>

<p>Is it repeatability in characterizing the response of your camera to any given scene shot?</p>

<p>Why try to characterize this overly sensitive electronic device we call a digital sensor? There's so many variables.</p>

<p>Is it so we can reduce the amount of post processing? (This is what I'm hoping will be achieved in discussions like this, but so far it never pans out.)</p>

<p>Characterizing the exposure behavior of a digital camera is not like characterizing a scanner which has a fixed light source, focal length, fixed lens, glass plate and uses transmissive and reflective photographic targets with specific sprectral qualities to measure its response.</p>

<p>I don't think using just a gray card is going to be enough on a DSLR.</p>

<p>Something else I forgot to mention is that ACR engineers embedded exposure offsets by as much as .5 (Exposure slider) in the default settings of some camera models.</p>

<p>Damn these unknowable variables! </p>

<p>Tim,</p>

 

<p>You’re absolutely right. Why are we doing this?</p>

 

<p>My own goal was to try to figure out how to expose a scene, after finally getting thoroughly fed up with

ETTR. And my experiment resolved that goal for me. I learned:</p>

 

<ul>

<li>When lighting conditions and practicalities permit, an incident exposure (either from an incident meter

or proper use of a gray card) is guaranteed to give good results.</li>

<li>

The in-camera meter in spot mode can be reliably used to determine the following (within the limits of

the meter’s field of view and the other usual caveats):

<ul>

<li>If the subject lies on the meter, detail will be retained.</li>

<li>Detail can be recovered from areas where the bug is a stop or two beyond the top of the meter,

but one should never place critical highlights beyond the top of the meter.</li>

<li>You have to go quite a ways below the bottom of the meter before something is rendered as

pure black, but shadows get dark, featureless, and prone to noise just below the bottom.</li>

<li>The meter can reliably be used to place objects in the scene at a particular tonality. That is,

something a stop and a half below the middle will be medium dark; something a stop and a half above

middle will be medium light.</li>

</ul>

</li>

<li>A scene with important highlights and shadows that are simultaneously above and below the tops and

bottoms of the meter will require HDR for a satisfactory rendering, or very careful exposure and non-trivial

post-production, or sacrificing either the highlights or shadows (or both).</li>

<li>If a proper incident meter reading results in either (but not both) highlights or shadows above or beyond the meter

limits, the above probably still applies.</li>

<li>Real-world scenes tend to be such that the shadows that block and the highlights that blow when

incident metering is used also have very little detail visible to the eye. Creative expression sometimes

demands that we capture those details anyway, which becomes a technical challenge.</li>

<li>Fixing the light is <em><strong>FAR</strong></em> more effective than futzing around with trying to find a better exposure.</li>

<li>The in-camera histogram isn’t too terribly useful, except as a gross tool to indicate problems

bad enough that they’re apparent even in the preview image.</li>

<li>ETTR really is a bunch of hooey. “Fix it in post” may be necessary, but it should never

be the goal.</li>

<li>Even if I plan on making substantial creative alterations in post-processing, starting with an exposure as close as possible to what I want in the final print is far more likely to result in a good print than trying to optimize the exposure to the theoretical properties of the sensor.</li>

<li>When in doubt, and whenever possible, go with an incident metering. When you can’t meter properly (for whatever reason), the camera’s most intelligent whole-frame autoexposure mode will probably get you “good enough” results.</li>

</ul>

 

<p>Cheers,</p>

 

<p>b&</p>