I understand 8 vs 16-bit but why not 32 bits?

<p>I have been doing a lot of reading on raw processing and I am confused by what software tools really mean when they say they operate on 16-bit images. From what I can tell from reading Adobe's DNG specification as well as digging through the source code of several open source raw processing tools, the first thing the tool does when processing the pixel values is normalize the original 8, 12, or 16 bits into a 32-bit floating point number between 0.0 and 1.0. All "processing" from then on--including exposure, hue/sat/value, white balance etc--is done using 32-bit floating point values. When, if ever, does the software convert back to the 8 or 16-bit values? Why would they do this at all? Every time you convert from 32-bit floats back to 8 or 16 bit integers you're throwing information away. Why not just store the 32-bit values and work with those exclusively?<br>

Thanks,<br>

Andy</p>

<p>I haven't dug into any code, but I'm surprised that the internal representation would be a 32-bit float (vs. a 16-bit int). That would double the memory requirements for the image, and slow the processing down quite a bit (depending on the processor - some GPUs have very fast FP instructions).</p>

<p>Granted, memory is cheap these days, but images are getting larger very rapidly. A 16-bit color image is just over 100 Mb, and if you add some layers with alpha-blending, that number can quickly grow to a significant fraction of the available memory, even on a modern 64-bit OS.</p>

<p>For most non-iterative algorithms, the difference between a carefully crafted 16-bit integer implementation and one using floating-point is likely to be small. And the loss of fidelity in a round-trip from integer to FP and back to integer for algorithms that work better in FP is also pretty small (assuming one is careful in the implementation).</p>

<p>Can you post some links to the info. on use of floats in photo-editing s/w?</p>

<p>Andy; do you keep track of your daily cash spent to cents; dimes; dollars; 10 dollars; 100 bucks?</p>

<p>I could write down each day what spend in actual cash for stuff and just use up or down to the nearest 5 bucks and I would get a decent budget value. A dollar would be better; or even cents too.</p>

<p> If was buying a million gumballs and rounded down to zero; it might look like a zero cost; when at 1 cent each it is a real cost of 10 thousand bucks.</p>

<p>You can preach for everybody to use cents or 32bits; it just involves more scratchpad.</p>

<p>If one is dealing with a mess of items in cost below 5 bucks; using cents matters; so does working pixels in 32bits when the data is all bunched up at one end.</p>

<p>Some folks do not want to wait 5 minutes to get a balance of 123.03 versus 123 bucks at a garage sale; others get their rocks off with gobs of pennies and buying a used manure spreader to 5 decimal places in cost.</p>

<p>Early scanners I had were not even 8 bit; ie 4 and 6 bits in greyscale in the 1980's.</p>

<p>In engineering; one might have a tolerance stackup where one adds a mess of dimenisons and tolerances; you use the full significance to do the calculations; then give the results in practical truncated numbers that have significance.</p>

<p>Thus if I add a BB to a freight train; the BB's mass is lost in the noise of the trains known mass. If a BB weights 1/4 gram and the engine weights 100 metric tons; the total is 100,000,000.25 grams; ie absurd.</p>

<p>If I add Billion BB's; then the tide is turned; it adds 250 million grams; ie 250,000kg; ie 250 metric tons; ie huge,more than the train.</p>

<p>Thus folks use what matters in a calculation as far as places and bits.</p>

<p>A real been counter would take my daily cash expense slips and add them to cents.</p>

<p>Most printers barely run in 8 bit mode; let along 16; thus 32 is absurd.</p>

<p>Why do weather men say it is 77F instead of 77.23 F :)</p>

<p>Numbers are truncated when the number of usefull info is often BS; thus folks usually do not say they pushed tri-x to 1558 iso; they round it to 1/3 stop values; ie 1600, 1250, 800, 500, 400 etc.</p>

<p>Adding more bits or more digits does not automatically give one more relevance.</p>

 

<p>Another issue is that not all functions may be out in 32bit versus 16 bits; thus program's GUTS ( in the raw processing) might start in 32 and then go to 16 once you use a feature that is not *written yet* for 32 bits in the raw converter.</p>

<p>When Photoshopin 1994 went 16 bits back in version 2.5; it only worked in a few things.</p>

<p>Photoshop had 16 bits before layers.</p>

<p>Thus one scanner here was 12 bits; another 14 bits. You scanned at these modes with say an underexposed image; then used curves to work the pixels.. The the next thing you wanted was greyed out; thus you saved a 16bit worked copy and went down to 8bits to say do other image modifications.</p>

<p>It is very common to try to avoid roundoff errors; in light; BB's or cents.<br>

<br /> It is easy to at first use a big column for all then trucate at the end; ie folks did this 100 years in balancing ship design.<br>

You really want the sensors/input values to be the limit; not math roundoff errors.</p>

<p>Geoff, if you punch DNG specification into Google you'll find it pretty easily. This spec describes the raw conversion process in detail and the first step is to convert to single precision floating point number between 0.0 and 1.0. In reading it again, it could be that the process described is intended to be executed per pixel, which is also how I have seen it implemented in code. In that case you wouldn't really double the memory requirements because you convert a 16 bit pixel to 32 bits, do a bunch of manipulations in 32 bits, then convert back to 16 and move to the next pixel.</p>

 

<blockquote>

<p>... you convert a 16 bit pixel to 32 bits, do a bunch of manipulations in 32 bits, then convert back to 16 and move to the next pixel.</p>

</blockquote>

<p>I've seen (and written) lots of algorithms that do that kind processing. Sometimes it's easier/faster to accept the cost of two floating-point conversions in order to simplify the intermediate processing. For example, in an integer FFT (Fast Fourier Transform) implementation there is quite a bit of work done to detect and handle underflow and overflow in intermediate values; coding the algorithm using floating-point eliminates the effort (at the expense of moving around more data and accepting slower floating-point mathematical operations).</p>

<blockquote>

<p>... the first step [of the DNG process] is to convert to single precision floating point number between 0.0 and 1.0.</p>

</blockquote>

<p>I just scanned the <a href="http://www.adobe.com/products/dng/pdfs/dng_spec.pdf">DNG Spec</a> and I didn't see any reference to floating-point in the conversion section. I searched for "floating" and only found references in a couple of look-up tables.</p>

<p>Section 5, "Mapping Raw Values to Linear Reference Values", of the spec. mentions the range 0.0 to 1.0, but never specifies "single precision floating point". The "Rescaling" sub-section mentions, "The black subtracted values are then rescaled to map them to a <strong>logical</strong> 0.0 to 1.0 range" (my emphasis).<br /> I was actually a bit disappointed at the lack of specificity about the format of the pixel values. I know TIFF can support floating-point values (and DNG is a TIFF extension), but for the life of me I can't even find a good TIFF spec. that describes the allowable formats (and how they are identified in the file).<br /> I guess I'll have to dig into DCRaw's code (or maybe just a TIFF library) to get some more information...</p>

<p>Edit: I found it! The <a href="http://www.awaresystems.be/imaging/tiff/tifftags/sampleformat.html">TIFFTAG_SAMPLEFORMAT</a> tells the format of pixel values.</p>

<blockquote>

<p>I have been doing a lot of reading on raw processing and I am confused by what software tools really mean when they say they operate on 16-bit images.</p>

</blockquote>

<p><br /> Keep in mind that not all data provided to the software is actually 16-bit anyway (could be 10-bit, 12-bit etc). <br /> The high-bit representation in Photoshop has always been "15 1" bits (32767 (which is the total number of values that can be represented by 15 bits of precision) 1). This requires 16 bits of data to represent is called "16 bit". It is not an arbitrary decision on how to display this data, it is displaying an exact representation of the exact data Photoshop is using, just as 0-255 is displayed for 8 bit files.<br /> In terms of image processing, we want to send ideally, at least the best 8-bits of data to our output devices. Due to rounding errors and quantization when editing our images, working with more than 8-bits per color ensure we do send the best 8-bits per color to the output device. Some modern ink jet printers are now allowing us to send “full 16-bit” to the drivers, the benefits of this are difficult to see. None the less, if you have high bit data, might as well send it to the driver, there’s not speed hit and no harm. Perhaps in the future, we will see a visual benefit. <br /> This may also help:http://www.digitalphotopro.com/gear/imaging-tech/the-bit-depth-decision.html<br /> As others have said, I also don’t believe there is any 32 bit floating point mathematics used.</p>

<p>Marketing can be weird with bits too. Our ancient Canon 2700F scanner was called a 30bit color scanner; ie 10 bits for each channel. That was about 13 or 14 years ago. One could scan in regular 8 bits mode; or 10bits. In 10 bit it was really a 16bit file with only 10 used. Thus one got a file twice as big; but really with some more significance. Later different models went to 12; then 14 bits.</p>

<p> A later 1980's expensive fax machine I had photo-mode as 6 bits; and would output an actual image file to a PC via a link. It was like star trek in advancement compared to an earlier model that was 4 or 5 bits. I sent an 8x10 sharp photo to a buddy on a local Fax line; it took about 15 minutes for the wazoo 6 bit file</p>

 

<p>I would prefer 32-bit integer over 32-bit floating point. With linear encoding, individual photons would often be somewhere around the 16th bit, leaving plenty of extra precision without needing potentially slower floating point math.</p>

<p>For many purposes, 16-bit is enough even for editing.</p>

<p>What's the point of the added precision?<br>

The precision of the input is at best 14 to 16 bits, and often considerably worse due to factors like flare, motion blur, astigmatism, and so on. The precision of the output to monitor or printer is usually 8 bits.<br>

Your camera should theoretically be able to use the added bits to represent scenes of greater dynamic range. In fact they're generally used by the denoising algorithms to provide wiggle room to denoise more effectively. Your RAWs contain all the information from the sensor, and doubtless look far better than JPEGs to the eyes of angels; but to our eyes they seem rather drab and lifeless. The images of limited range have snap and sparkle.<br>

Your RAW is more useful than a JPEG because it allows you to choose the information you want to throw away. It lets you peel more detail out of highlights and shadows by compressing parts of the sensitometric curve and expanding others, creating combing or posterization that largely disappears when it's crunched to 8 bits for storage or output. Denoising algorithms like wiggle room, and so do image editors.</p>

<blockquote>

<p>Your camera should theoretically be able to use the added bits to represent scenes of greater dynamic range.</p>

</blockquote>

<p>Bit depth and dynamic range are two totally different items and subjects. You can have a 5 stop, 12 bit capture or an 8 stop 10 bit capture. </p>

<blockquote>

<p>Your RAW is more useful than a JPEG because it allows you to choose the information you want to throw away. It lets you peel more detail out of highlights and shadows by compressing parts of the sensitometric curve and expanding others, creating combing or posterization that largely disappears when it's crunched to 8 bits for storage or output. </p>

</blockquote>

<p>Yes, I’d agree that it allows you to choose the info you want to keep or throw away. But to be clear for lurkers, the camera has a fixed dynamic range. Shooting raw doesn’t provide any more (in fact, the in-camera JPEG is a direct conversion from the raw). What raw does provide, as you point out is far more control over rendering the fixed dynamic range of the capture. </p>

Section 5, "Mapping Raw Values to Linear Reference

Values", of the spec. mentions the range 0.0 to 1.0, but never

specifies "single precision floating point".

 

0.0 to 1.0 is inherently floating point. You're right though I assumed

single precision because 64 bit double precision seemed extreme.

<p>Andrew,<br>

To your comment on my first quote: yes, exactly, just what I was driving at. Theory and practice are two totally different items and subjects. In theory, you'd expect to derive 2 to the 8th tonal values from an 8-bit image and 2 to the 16th from a 16-bit image, and assuming equal separation of values the 16-bit image would have vastly greater dynamic range. In practice the system doesn't provide accuracy over that range, so the tonal range of the 16-bit image is considerably compressed, and instead the separation between values is reduced. <br>

To your comment on my second quote: not precisely. The high-bit sensor has a fixed dynamic range, but the conversion to 8-bit JPEG doesn't capture all of it, and the RAW file does. According to Thom Hogan, the effective increase in dynamic range is a stop and a half for the Nikon D300. When you need to pull detail from highlights and shadows, you're better off with the RAW.</p>

 

<p><em>In theory, you'd expect to derive 2 to the 8th tonal values from an 8-bit image and 2 to the 16th from a 16-bit image, and <strong>assuming equal separation of values </strong>the 16-bit image would have vastly greater dynamic range.</em></p>

<p>Therein lies your error (emphasis added). The total range of tonal values is from pure black to pure white. Whether the value is expressed in 8 bits or 16 bits merely expresses the number of steps in that range. As you increase the bit depth, the steps get smaller.</p>

<p>You might be able to extract a couple more steps at each end of the scale, given more resolution, but that depends more on the medium than the numbers. In practice you simply divide the steps you see rather than extend the range further to the extremes.</p>

<p>Raw images have a greater range because they are <em>raw </em>values subject to interpretation, not cooked as in JPEG and not simply because they have a greater bit depth.</p>

<p>32 bit floating point has only 7 significant digits, roughly equivalent to 24 bit fixed point. The added precision makes a noticeable difference in sound quality, particularly in classical music which is mostly soft. I'm not sure it matters in image presentation, due to the limitations of available media. Where fp operations do help is in making adjustments to the image (or sound) by elimination of most of the roundoff errors. These types of errors cause aliasing or posterization, seen as the comb effect in histograms.</p>

<blockquote>

<p>As you increase the bit depth, the steps get smaller.</p>

</blockquote>

<p>Exactly! The analogy that’s worked for years is this. Dynamic range is like the fixed length of a staircase, bit depth is the number of steps making up the staircase. </p>

<blockquote>

<p>Raw images have a greater range because they are <em>raw </em>values subject to interpretation, not cooked as in JPEG and not simply because they have a greater bit depth.</p>

</blockquote>

<p>Well said. Its all about the rendering. </p>

<p>The most useful feature of using floating point representations of digital data is that there is no overload point. An integer value has a fixed maximum an minimum which cannot be exceeded. You can't get blacker than black or whiter than white, so values which attempt to exceed these limits are simply truncated. With floating point, the decimal place moves (that's why it's called "floating point") so that the steps get larger or smaller as needed by factors of ten.</p>

<p>How is this useful? If you do a floating point calculation that results in an overload, that data is not lost when reverted to integer format. Rather the entire scale is shifted, compressed or expanded so that everything stays in range. Neat, eh?</p>

<blockquote><a href="../photodb/user?user_id=419409">Edward Ingold</a>

<p>... With floating point, the decimal place moves (that's why it's called "floating point") so that the steps get larger or smaller as needed by factors of ten. ...</p>

</blockquote>

<p>Just to pick a nit (because this has become something of a technical discussion it seems apropos), in almost all floating-point formats these days the exponent is a power of 2 (ex. <a href="http://en.wikipedia.org/wiki/IEEE_754-2008">IEEE-754</a>, the decimal formats don't even have common names - I don't remember ever seeing an implementation of one of the <em>decimalXX</em> formats).</p>

<p>Edward,<br /> The sentence preceding your first quote:<br /> <em>Theory and practice are two totally different items and subjects.</em><br /> Andrew Rodney seemed to have missed this point in his earlier correction, so I was essentially hammering him over the head with it<em>; </em>but the two of you seem to have resilient skulls that can withstand repeated hammering. It signals an upcoming parallel construction, of which the first part is your quote:<em></em><br /> <em><strong>In theory</strong>, you<strong>'d</strong> expect to derive 2 to the 8th tonal values from an 8-bit image and 2 to the 16th from a 16-bit image, and assuming equal separation of values the 16-bit image <strong>would</strong> have vastly greater dynamic range.</em><br /> Therein lies your misunderstanding (emphasis added). The word "would" is a modal form frequently used to signal supposition contrary to fact, and " 'd" is a contraction of it. You also seem to have some difficulty with the second part of the parallel construction:<br /> <em>In practice the system doesn't provide accuracy over that range, so the tonal range of the 16-bit image is considerably compressed, and instead the separation between values is reduced.</em><br /> This should not be a difficult concept. It is essentially what Andrew Rodney was saying and you were confirming later in this thread. <br /> I don't have a problem with questions of fact or resolving real contradictions, or having someone present informal ideas like "wiggle room" in more technical terms, as you did immediately above. However, when you contradict, correct, or ignore my statements and then turn around and confirm them in slightly different language, it's not helpful to anyone, merely confusing. Engage in intellectual disputation if you must, but first learn to read and understand.<br /> <em><br /></em></p>

<p>http://hdrphotos.info/dynamic-range-and-bit-depth/</p>

<blockquote>

<p>Bit depth and dynamic range are indeed <strong>separate</strong> concepts and there is <strong>no direct one to one relationship between them.</strong><br />The bit depth of a capturing or displaying device gives you an <strong>indication</strong> of its dynamic range capacity, i.e. the highest <a id="KonaLink0" href="http://hdrphotos.info/dynamic-range-and-bit-depth/" target="undefined">dynamic range</a> that the device <strong>would be capable of reproducing if all other constraints are eliminated</strong>. For instance, a bit-depth of 12 for a <a id="KonaLink1" href="http://hdrphotos.info/dynamic-range-and-bit-depth/" target="undefined">CCD</a> tells you that the maximum dynamic range of the sensor is 4096:1, but the captured dynamic range is likely to be much less once noise is taken into account (most 12-bit sensors have on average a dynamic range around 1,000:1 only).</p>

</blockquote>

<p>Andrew Rodney wrote:</p>

<blockquote>

<p>You can have a 5 stop, 12 bit capture or an 8 stop 10 bit capture.</p>

</blockquote>

<p>http://www.cambridgeincolour.com/tutorials/dynamic-range.htm</p>

<p>Most digital cameras use a 10 to 14-bit A/D converter, and so their <strong>theoretical</strong> maximum dynamic range is 10-14 stops. However, this high bit depth only helps minimize image posterization since total dynamic range is usually limited by noise levels. Similar to how a high bit depth image does not necessarily mean that image contains more colors, if a digital camera has a high precision A/D converter <strong>it does not necessarily mean it can record a greater dynamic range.</strong> In practice, the dynamic range of a digital camera does not even approach the A/D converter's theoretical maximum; 5-9 stops is generally all one can expect from the camera.</p>

<p>Yes, absolutely. Your facts are correct. We have no disagreement of substance. We were merely in need of a healthy dose of denoising.</p>