Question about Compression

[paul_brenner1]

<p>I just purchased a Canon S100, and its level of jpg compression (compared to my G9) raised some questions, and I did an experiment:</p>

<p>I took a jpg photo and a raw photo of the same scene. The jpg file size was 3.5 megs; the raw file size was 12.6 megs.</p>

<p>I then converted each file to tiff format; the two tiffs were the same size: 34.6 megs.</p>

<p>Then I converted each tiff back to jpg in CS4. The jpg-tiff-jpg was 5.5 megs; the raw-tiff-jpg was 7.7 megs.</p>

<p>Several conclusions:<br>

1. compression doesn't throw anything away permanently (else the tiffs would not be the same size)<br>

2. Converting a jpg to tiff then back to jpg in a photo program is a way to have a less compressed jpg.<br>

3. If 1. and 2. are correct, it's not clear why jpg-tiff-jpg file ends up somewhat smaller than a raw-tiff-jpg file.</p>

<p>Am I missing or mis-understanding a lot of stuff here?</p>

<p>Thanks,<br>

Paul</p>

 

[mendel_leisk]

<blockquote>

<p>1. compression doesn't throw anything away permanently (else the tiffs would not be the same size)</p>

</blockquote>

<p>Compression <em>does</em> throw stuff away, and when it's gone, it's gone. I may stand corrected, but I believe tiffs of the same pixel dimensions (and bit depth) are always identical in size, regardless of content.</p>

<blockquote>

<p>2. Converting a jpg to tiff then back to jpg in a photo program is a way to have a less compressed jpg.</p>

</blockquote>

<p>"Compression" is just a reorganizing of graphics, homogenizing of nearly identical areas, all with the aim of simplifying the information, and accordingly reducing the space requirement. Your 'round trip conversion doesn't achieve anything positive. Every edit of a file that started out as a jpeg only increases the corruption.</p>

[mark_sirota1]

<blockquote>

<p>I may stand corrected, but I believe tiffs of the same pixel dimensions (and bit depth) are always identical in size, regardless of content.</p>

</blockquote>

<p>You are correct, mostly. The exceptions would be (1) metadata, including things like the embedded preview which is JPEG-compressed and could therefore vary in size, and (2) compressed TIFFs. But the image data from two uncompressed TIFFs with identical bit depth and pixel count will indeed be the same -- specifically, each pixel takes 3 bytes (8 bits/channel) or 6 bytes (16 bits/channel).</p>

 

[ljwest]

<blockquote>

<p>1. compression doesn't throw anything away permanently (else the tiffs would not be the same size)</p>

</blockquote>

<p>As mentioned above, JPEG compression does throw away information. There are "lossless" compression methods, but I'm pretty certain JPEG isn't one of them. The (overly) simplified explanation of JPEG compression goes something like this: the compressor looks through the file, line by line, and when it encounters two or more pixels in a row of a particular color, instead of recording, the information for each pixel, it records that the next X pixels are a certain color. The lower the "quality" setting for a JPEG, the greater latitude there will be for the algorithm to determine that two pixels are the "same" color.</p>

<p>Try this in CS4: create a blank image, say 2000 pixels by 1000, and fill it with one color using the paint bucket. Save it as a PSD, then, using the "save for Web" command, save it as a JPEG and as a TIFF. Compare the file sizes. The JPEG should be really, really small. Now, do the same thing, but instead of filling it with one color, create a gradient from white to a color from one side of the image to the other. Now, using "save for web" save it as a JPEG with different "quality" settings (and differing names, of course). Look at each in turn with another editor or photo viewer. You should be able to see some "banding" in the gradient (it'll start to look like bars instead of a smooth transition from white to your chosen color) as you view the lower quality images.</p>

<blockquote>

<p><br /> 2. Converting a jpg to tiff then back to jpg in a photo program is a way to have a less compressed jpg.</p>

</blockquote>

<p>No. All you're doing is seeing the effects of different implementations (the camera's and Photoshop's) of the JPEG algorithm. When you save it as a TIFF, you are expanding the image to its full pixel size, and with an uncompressed TIFF, you are simply saving every single pixel. When you save it again as a JPEG, now you're using Photoshop's JPEG compressor, probably with different quality settings than the camera has. The JPEG expanded to a TIFF still has no more information in it than the original JPEG did, even though the saved file size is larger.</p>

 

<blockquote>

<p><br /> 3. If 1. and 2. are correct, it's not clear why jpg-tiff-jpg file ends up somewhat smaller than a raw-tiff-jpg file.</p>

</blockquote>

<p>RAW-TIFF is a 1:1 pixel for pixel conversion. The RAW file starts out with more information than the JPEG, and the TIFF retains all that information. Again, Photoshop's JPEG algorithm is probably better than, and may have a different value for a given "quality" than the camera does.</p>

<p>In the camera, when you record JPEGs, are you using the "fine" or finest setting?</p>

[dave_redmann]

<p>As others have said, JPEG is a "lossy" format, and uses a compression scheme that cannot* fully retrieve all of the original picture information. How much information is lost depends on the degree of compression and the nature of the original picture.</p>

<p>But one key point: I have to disagree with the statement, "RAW-TIFF is a 1:1 pixel for pixel conversion. . . . the TIFF retains all that information." If you notice, a raw file is much <em>smaller</em> than the corresponding TIFF (even an 8-bit TIFF, to say nothing of a 16-bit TIFF). The reason is that the Canons and most cameras use a Bayer filter over the sensor, so each raw pixel is <em>only</em> red, green, or blue (or more accurately what passes through those filters). The raw conversion process uses the information from nearby pixels of other colors to reconstruct full color information for each pixel. So <em>how</em> a raw file is converrted into a TIFF or JPEG absolutely affects what content you get, and different algorithms produce slightly different results. Some raw converters even give you raw conversion algorithm options. The TIFF file contains all of the data resulting from the raw conversion, at least if it has as much bit depth as the raw data (raw files are usually 12 or 14 bits per pixel, but sometimes 10 or 16, and TIFF files have either 8 or 16 bits per each color channel, R, G, and B, for each pixel).</p>

<p>*I say "cannot" for simplicity, but a truer answer may be "usually does not". Most JPEG's involve substantial compression, and some lossiness. However, the JPEG standard does not require any particular level of compression or degree of lossiness. Some software allows saving very high quality JPEG's. While a typical JPEG may achieve 10-to-1 compression, the best quality ones may only achieve 2-to-1 compression. Do such files at least sometimes contain <em>all</em> of the original information? Maybe so. In any case, you'd be hard pressed to tell the differences with a typical picture.</p>

 

[sarah_fox]

<p>In theory, TIFF file sizes can vary between images of the same pixel dimensions. However, it doesn't quite work that way in practice. TIFF is a lossless compression schema perhaps better suited for computer generated graphic files. In practice it ends up more like a bitmap file, as adjacent pixels would have to have EXACTLY the same values to be compressed together. That's unlikely/infrequent in a real-world photograph.</p>

<p>JPG files are lossy, as pointed out above. There are different levels of compression that can be used. The more compressed, the smaller the file, and the more lossy. A jpg conversion in-camera is not necessarily equivalent to a jpg conversion in-computer. It depends on the compression level settings.</p>

<p>The size of a jpg file is, IMO, a fair reflection of the amount of detail it contains. You can see this when you compare a jpg of a white wall vs. a jpg of an unfolded newspaper, for instance. Your "jpg-tiff-jpg" is smaller than your "raw-tiff-jpg," indicating it has less detail. All things considered, though, it's hard to tell much of anything from your tests, because part of what is encoded in the reconversion of tiff to jpg is the artifacts of the original jpg compression. Artifacts are those noisy/checkery bits you'll find around higher contrast lines and edges. You won't find those in raw or tiffs converted from the raw.</p>

<p>I quickly googled an example of jpg compression artifacts for you. Just scroll down a tiny bit:</p>

<p><a href="http://dpfwiw.com/exposure.htm#compression">http://dpfwiw.com/exposure.htm#compression</a></p>

<p> </p>

[jcoffin]

<p>Just to add a little more mud to the water: JPEG isn't really a single format -- it's a whole group of formats. The format that's used the most often is, indeed, lossy as others have noted. <br>

<br />There are also, however, two JPEG formats (lossless JPEG and JPEG-LS) that do lossless compression. Despite being lossless, both of these are still (usually) more effective than LZW- or ZIP-compressed TIFFs, for the simple reason that they use compression specifically designed for pictures (where ZIP and LZW are quite general-purpose compression algorithms being applied to picture data). There's also a "low loss" version of JPEG-LS that does lose a little information, but not very much. If I recall correctly, it's also not progressive like normal JPEG, so re-saving the data repeatedly won't increase losses.</p>

<p>JPEG-2000 also includes a wavelet based algorithm. One variant of this uses only integer math, giving precise transformations from input to output, so it can also be used for lossless encoding, if desired. It works pretty well for many real photographs, but not usually as well as JPEG-LS for synthetic pictures (i.e., things like charts and graphs).</p>

<p>For better or worse, however, none of these has ever been used much though. At least In my opinion, they've been victims of their predecessor's success -- the original JPEG was good enough that most people don't seem to see much (if any) reason to start using anything else. If anything, lower costs of hard drives and network bandwidth make them even less interesting over time.</p>

[paul_brenner1]

<p>Wow! A lot of very informative posts! Let me pose one more question in response to what I read above:</p>

<p>If a jpeg compression permanently throws information away (and a higher compression throws away more than less compression), and the tiff file is the same size from either (even from a jpeg and a raw file), then the tiff file must be "filling" in with artifacts, not "resurrecting" an un-compressed version if I convert a jpeg to tiff. So if I convert to tiff and back to jpeg, and the final jpeg file is larger than the original one, I'm fooling myself if I think the larger jpeg is higher quality.</p>

<p>It's interesting: I can delete a file and then recover the original file again. I might have thought that "compression" was similar; the detail wasn't permanently "gone". But that's apparently not true...</p>

<p>Thanks again,</p>

<p>Paul</p>

[sarah_fox]

<p>It's hard to answer your question. Yes, the TIFF conversion of your JPG is faithfully encoding all of the compression artifacts. Those artifacts might be construed as detail, but certainly not "good" detail. Then when you reconverted to JPG, the second JPG encoded all of the compression artifacts of the first JPG and added its own (artifacts on top of artifacts). As for one JPG being larger than another, I think it depends on settings. In the end, though, two JPG conversions with the same settings (e.g. your JPG->TIFF->JPG, vs. your RAW->JPG) probably reflect this relationship of larger file size meaning more detail. In other words, the first JPG compression eroded details, and that shows up in the second conversion. Again, though, it's hard to say for sure.</p>

[Stock-Photos]

<p><a href="http://www.scantips.com/basics09.html">http://www.scantips.com/basics09.html</a></p>

<p><a href="http://www.scantips.com/basics9t.html">http://www.scantips.com/basics9t.html</a></p>

<p><a href="http://www.scantips.com/basics9j.html">http://www.scantips.com/basics9j.html</a></p>

[paul_brenner1]

<p>Thanks again!</p>

<p>I realize that I can avoid some of this by simply shooting in raw!</p>

<p>I do wonder why the S100 has only two compression settings rather than three, as does the G series (the S100 doesn't have the "fine" setting).</p>

<p>Paul</p>