Different histograms for same image

<p>I thought there had to be a standard for histograms. The histogram on the camera (Canon 5D) LCD is nothing like the Digital Photo Professional (DPP) histogram for the same image file. When I expose the Ed Pierce Digital Calibration Target for a neutral exposure for the camera LCD, DPP produces a histogram of everything way to the right of center. In DPP the black (shadow) spike is even to the right of center, white (highlight) spike is shy of clipping and the gray is situated halfway between the shadow and highlight spikes. Furthermore, Adobe ACR shows that the image is underexposed. Seems to me the image file containing the color and exposue values is interpreted with no regards to any standard. With no standard, which one is the true histogram?</p>

<p>Please see attached DPP screen shot.</p><div></div>

<p>Please see attached ACR screen shot.</p><div></div>

<p>They all are.</p>

<p>A histogram, as it's currently used in DSLRs, is solely a representation of the <strong>processed</strong> image. It bears little relation to how much light the sensor sites actually captured—just the result of demosaicing that sensor data and processing it into an RGB image.</p>

<p>So the in-camera JPEG engine will generate a different histogram from DPP, which will generate a different histogram from ACR, which will generate a different histogram from Capture One... And they'll all change as you modify the image. Switch the camera from sRGB to Adobe RGB? That'll give you a different histogram too. Since there are computational costs to calculating a good histogram, the in-camera ones also tend to be a little on the loose side.</p>

<p>And your ACR image is underexposed because you left it on auto and ACR pulled it back almost 2 stops. Give that screen shot a good look again. (I missed that the first time myself.)</p>

<blockquote>

<p>I thought there had to be a standard for histograms. </p>

</blockquote>

<p>Not at all. Take Adobe Camera Raw. Open a raw and alter the RGB color space options (workflow options) and the histogram updates and changes based on that color space. The rendering controls which are proprietary for each converter plays a role too (as does the underling color space assumed for processing). The correct histogram is the one you are currently viewing in the software product of choice when you have a rendering you visually prefer. </p>

<p>Hi David. It may simply be that the scale of the two histograms is different (e.g. DDP -10 to +6EV in 2 EV increments, ACR -6 to +6EV), but it looks like the underlying data is also different. Start by checking if the following are the same (they will all affect the underlying data and therefore the histogram):</p>

<p>1) Color space (DPP appears to be in sRGB, ACR in aRGB)<br>

2) White balance<br>

3) Exposure et. al. (ACR appears to have changed them by default)</p>

<p>Those are just the first few parameters. Each raw converter will apply its own set of optimizations, some of which are under your control while some aren't, therefore no two raw converters will produce the same results and histogram. As a histogram simply displays the distribution of your data, no histogram is right or wrong, it just is.</p>

<p>If on the other hand your question is 'which histogram is more representative of the distribution of my image data as related to the capabilities of my camera's sensor', that's a much more complex and controversial topic.</p>

 

<p>I understand that altering/modifying the file will change the histogram but one would think software would process the UNALTERED straight out of the camera (source file) the same across the board.<br>

Seems to me, the result of demosaicing the sensor data for an image should yield "X" and not "A", "B" or "C" for the same file. The data did not change.<br>

So, viewing the DPP histograms searching for the properly exposed image is a challenge without the camera LCD. This is especially true if the image is no longer on a card to view in the camera and you are totally dependent on DPP. This just seems sooooooo wrong.</p>

<p>The major source of variation is a consequence of how the image is rendered. This is particularly true for raw file conversions. In short, the histogram represents the image you see, not the one on the chip. Rendering includes application of a particular color space, which affects the distribution of tonal values.</p>

<p>An histogram of a digital image, in it's simplest form, is a count of the number of pixels at each value of an 8-bit scale. There are minor variations on this scheme. The ordinate (vertical axis) is usually normallized so that the peak value does not exceed 100% (modal normalization). The values can also be normalized to represent a percentage of the total number of pixels (percentile normalization). This changes the shape of the curve, but not the distibution on the abscissa (horizontal axis), which is proportional to the exposure value.</p>

<p>The abscissa can be represented as exposure values (q.v., DPP), which may be subject to a transfer curve, with an arbitrary "mid" point. In that case, comparisions can only be made within DPP, not against histograms in ACR, Photoshop, or some other program.</p>

<p>The conversion from 16 bits/channel to 8 bits/channel is simple - truncate (ignore) the lower 8 bits.</p>

<p>Looking at your screen captures, DPP is set to RAW/sRGB no exposure compensation while ACR is set to AdobeRGB, exposure compensation -1.6 EV.</p>

<p>The histograms from both will never be equal, no way.</p>

<p>If you are interested in the histogram of the RAW data there was a tool called Rawnalize that allowed that. Unfortunately the author passed away and is no longer updated.</p>

<p>Another option, which is rather unfriendly, is to use dcRaw, which is a command line raw converter where you can specify white balance coefficients=1 for all channels and linear output (no gamma corrected). The resulting image will probably not be useful for editing, but it will show you the data as it was captured.</p>

<p>Yes, software does process unaltered data from the raw file the camera provides, but there is no histogram for a file that is just numbers until that software has rendered it into an actual image. There is no histogram information encoded into a raw file. What you see is the histogram of that processed image. The minute you can see an image on the screen, you're seeing a processed image, and each piece of software may not be doing that exactly the same way. You can make it more like the same way by altering the sliders or whatever controls exposure. Then you may work it into a similar histogram as you saw in the camera. But then, if that's the histogram you want, why bother with raw files?</p>

<p>You're right when you say "The data did not change" as it is provided in the raw file, but you have already changed how it is presented by opening it, because the software for doing that can't do it without having applied some default choices which affect what is done with that data.</p>

<p>There are some software programs which don't do as much for you automatically when you open the raw file, and so tend to allow you to really start from scratch and see what's going on with your raw data. Try Ufraw (or just the command line processor Dcraw), or Picture Window Pro 5.</p>

<blockquote>

<p>I understand that altering/modifying the file will change the histogram but one would think software would process the UNALTERED straight out of the camera (source file) the same across the board.</p>

</blockquote>

<p>The issue is, non of the images you see rendered are what the camera actually produces since you are seeing <em>output referred</em> not <em>scene referred</em> data (and again, the color space of a raw file is entirely up to the raw converter to assume). See: http://www.color.org/ICC_white_paper_20_Digital_photography_color_management_basics.pdf</p>

<blockquote>

<p>there is no histogram for a file that is just numbers</p>

</blockquote>

<p>Actually, a histogram is a graph showing the frequency of appearance of a particular value (number) in a set of numbers, so by definition all you need to build a histogram is a file with just numbers</p>

<p>As I posted previously, Rawnalize shows the histogram of the raw numbers, before any processing</p>

<p>For the record - here is the ACR screen shot with the Auto selections off.</p><div></div>

<p>article about histograms: http://www.ppmag.com/web-exclusives/2007/12/what-is-a-histogram-and-how-do.html</p>

<p>"Actually, a histogram is a graph showing the frequency of appearance of a particular value (number) in a set of numbers, so by definition all you need to build a histogram is a file with just numbers"</p>

<p>That's just arguing semantics. You know what I meant.</p>

<blockquote>

<p>That's just arguing semantics. You know what I meant.</p>

</blockquote>

<p>Sorry, I really don't want to argue. I just wanted to point out that you can obtain a histogram from RAW data before any processing or rendering into an actual image. And that resulting histogram could be very useful from the photographer perspective. That's what Rawnalize does.</p>

<p>One of the possible uses of such histogram, is that it is probably the best way to tell if a channel is overexposed or blown out</p>

 

<blockquote>

<p>I just wanted to point out that you can obtain a histogram from RAW data before any processing or rendering into an actual image. </p>

</blockquote>

<p>But such utilities still have to assume (assign) some color space assumption to that data. All converters have to as well. Raw is essentially Grayscale data who’s color space, based on the filters used and the illuminant assumed at capture are just that, some assumption and thus even that raw histogram, albeit far, far closer to the “truth” about the data compared to other raw converters is still not an absolute. At least in terms of color (luminance I suspect should be accurate). That said, seeing the histogram in this state is often only useful for other analysis and tells us little about what the data could be after rendering which I suspect is more important to the OP. </p>

 

<blockquote>

<p>One of the possible uses of such histogram, is that it is probably the best way to tell if a channel is overexposed or blown out.</p>

</blockquote>

<p>Agreed. </p>

<p>It seems to me that both histograms are showing you relatively the same thing? The ACR histogram with -0- adjustments is showing you your middle grey spike? The DPP is showing your blacks between -2 and -4, your neutral grey at -0- and your whites at +2. That is about as close to perfect as it gets? So I might so the DPP software is a bit more "accurate". But in any case, both are showing you essentially the same thing. I think what is confusing is the DPP scale of -10 to 4.0. </p>

<blockquote>

<p>But such utilities still have to assume (assign) some color space assumption to that data. </p>

</blockquote>

<p>No, that's the nice thing about them. The only "Processing" made is to separate the R,G,G,B channels according to the sensor arrangment.</p>

<p>Take for example a 12 bit depth Raw file, any given pixel (or sensel) will have a value from 0 to 2^12 = 4096. </p>

<p>The histogram is build according to this values (0 - 4096) and no assumption about color space has been made, just the raw values are used.</p>

<p>I believe that an additional in-camera (and Raw Converter) function that would show you when your raw data is blown would be VERY useful: in order not to confuse the issue let's not call it a histogram, let's call it an exposure meter or whatever. Being limited to sRGB and aRGB (both gamma corrected color spaces) to determine exposure in-camera with an ETTR approach can be quite misleading.</p>

<blockquote>

<p>such utilities still have to assume (assign) some color space assumption to that data. All converters have to as well</p>

</blockquote>

<p>I believe that for the purposes of an 'exposure meter' type histogram all you possibly would need is an estimated white point - and perhaps not even that.</p>

<p>Francisco wrote:</p>

<blockquote>

<p>The only "Processing" made is to separate the R,G,G,B channels according to the sensor arrangment.</p>

</blockquote>

<p>Yes, that would be the simplest approach. However, in order not to 'blow' the channels during the conversion to your working color space I believe that it would be useful to take the original raw data into XYZ space and display THAT histogram, for which you need a white point.</p>

<blockquote>

<p>However, in order not to 'blow' the channels during the conversion to your working color space </p>

</blockquote>

<p>Don't want to diverge to much from the topic of this thread, but "blowing the channels during the conversion to your working color space" it is probably the most common cause of blown channels and that can be solved in postprocessing if you know what you're doing. The most common case is "blown out red channel in images of flowers"</p>

<p>Going back to the OP</p>

<p>Histograms are usually displayed based on output image data. These affect a lot how the histogram looks:<br>

- Output Color space<br>

- White balance<br>

- Exposure compensation<br>

- Sharpening<br>

- Any adjustment in brightness, shadows, contrast<br>

- Highlight recovery</p>

<blockquote>

<p>No, that's the nice thing about them. The only "Processing" made is to separate the R,G,G,B channels according to the sensor arrangment.</p>

</blockquote>

<p>That’s <strong>luminance</strong> info, it can’t be <strong>color</strong> info without some assumption of the RGB primaries used (the color space). </p>

<blockquote>

<p>The histogram is build according to this values (0 - 4096) and no assumption about color space has been made, just the raw values are used.</p>

</blockquote>

<p>Agreed (again) it can provide data about luminance clipping but it can’t tell us anything about saturation clipping. Depending on how a histogram is being drawn, for example the Histo in Adobe products, you get <strong>both</strong> kinds of data (but that’s in the current assigned color space with the current output referred rendering). </p>

<p>Andrew,<br>

You're absolutelly right. It is luminance info only. </p>

<p>It may be that when you convert to a specific color space you get saturation clipping from a non-clipped raw channel (luminance), but this can be solved in post processing.</p>

<p>If you start with luminance clipped channels, then there is not much that can be done (well, there are a couple of things that can be done, but then it is not the original image anymore)</p>

<blockquote>

<p><em>"blowing the channels during the conversion to your working color space" it is probably the most common cause of blown channels and that can be solved in postprocessing if you know what you're doing.</em></p>

</blockquote>

<p>In my opinion, for an 'exposure meter' type of histogram it would be useful to bring image raw data through the matching functions, demosaicing into R*G*B* camera space and transforming to XYZ PCS - stopping short of converting into a working space just in case it were too small.</p>