Nikon sRAW--what is wrong with it?

<p>Over the past week, I have been reading up on the Nikon sRAW format. I have seen a lot of discussion denouncing the sRAW format as not "true RAW" or a "glorified JPEG", <a href="http://www.rawdigger.com/howtouse/nikon-small-raw-internals">citing this article</a> as evidence. Even with my background in digital imaging and multimedia, there are details in that article which go clear over my head, such as encoding color data YCbCr vs RGB. I'm not a computer scientist; I won't dispute that which I don't understand.<br>

<br /> Yet--over my career, I have spent a lot of time on print and multimedia projects, which included ensuring that images were optimized to their output, whether to print or the Web. So I know how to tell if an image is blown out, noisy from high ISO or compression, or has a compromised gamut.<br>

<br /> Rather than simply conclude that sRAW would give me poor results, I decided to compare output from a D4, linked below. In my analysis, I compared a 12-bit full-size RAW against a 12-bit sRAW file. I then opened each file in ACR and modified numerous parameters (but in equal amounts for each file). Finally, the files were studied in Photoshop in the 16-bit LAB color space, where I compared the histogram, gamut, and color data between files--rather than simply trusting my eyes. After 4+ hours of studying these images, I could not <em>find</em> any proof that the sRAW either had less color or tonal data than the 12-bit full RAW, or responded differently to adjustments in ACR. Judging by the sRAW file’s response in ACR and the color/tone numbers, it appears to be <em>very similar</em> to RAW, albeit smaller size—the one difference I saw.<br>

<br /> Perhaps there is is data loss somewhere--but I don't see it with the tools I'm using. Nor am I denouncing the writer of the article, who is obviously very smart. I only suspect those who might find sRAW useful won't be making any noticeable compromises by using this reduced format--except of course in resolution.<br>

<br /> But try it out for yourself; here are two files to open in ACR and compare:<br /> <a href="http://updates.rawdigger.com/data/D4s/_DSC0151.NEF">_DSC0151.NEF FX (36x24) 4936 3288 12 Uncompressed 25 MB</a><br /> <a href="http://updates.rawdigger.com/data/D4s/_DSC0163.NEF">_DSC0163.NEF FX (36x24) 2464 1640 12 Small 13 MB</a></p>

<p>I think sRAW is great. I use it all the time with a couple of old Canon's, the 40D and 5D. Not sure about the point the articles are trying to make. Canon users uses it all the time without complaints.</p>

<p>Thanks Mark for your input. It's good to hear some hands-on experience! When I get my D810, I suspect I'll have use for sRAW too--and will push the format and see if I can break it.<br>

I am not disputing the findings in the article, only in practical terms I can't see it those files. This topic has gotten rather contentious in other forums--unsure why--so I'm glad we can discuss this amicably here.</p>

<p>Canon's implementation of sRAW isn't necessarily similar to Nikon's implementation, so frankly the fact that it works fine on a Canon has little to no meaning in this context.<br>

For me, the main thing with these sRAW files is that (from all I've cared to read *) they actually aren't much smaller than the normal losless compressed NEF files. So, they aren't saving storage space and as a result they also won't clear the buffer faster while the file is being written - then where is the advantage? I mean, I can see a clear advantage for a smaller RAW file (esp. on a D810), but the resulting file should be much smaller. If it isn't, much of the advantages are rather moot.</p>

<p>__<br>

(*) I spared myself the deeply technical articles, I understand what they say for the most part, but it's a camera, not a pixel-generator. It's about photos in the end, and if compression methods or resizing algoritms do not introduce visible degradation, any other discussion quickly becomes rather meaningless to me.</p>

<blockquote>

<p>For me, the main thing with these sRAW files is that (from all I've cared to read *) they actually aren't much smaller than the normal losless compressed NEF file</p>

</blockquote>

<p>I gleaned all I could from the above article, and that point caught my attention too.<br />Comparing a full-size 12-bit uncompressed D4 RAW @ 25MB against the 12-bit sRAW @ 13MB, one might conclude the savings in terms of memory is not that great. However, a <a href="http://www.nikonusa.com/en/About-Nikon/Press-Room/Press-Release/hwuezq15/The-Power-to-Create-the-Compelling%3A-The-Nikon-D810-is-the-HD-SLR-that-Delivers-Unmatched-Image-Quality-and-a-Truly-Cinematic-Experience.html">Nikon press release </a>states the following:</p>

<blockquote>

<p>For full workflow versatility, the D810 also gives users the option to shoot in full resolution 14-bit RAW/NEF file format or the new RAW Size Small format. This 12-bit file format is half the resolution and <em>approximately 1/4 the file size</em> of full RAW files</p>

</blockquote>

<p>While Nikon doesn't state if D810 sRAW is 12-bit uncompressed or uncompressed, generally speaking 1/2 the resolution produces a file 1/4 the size, or 36mpx reduced to 9mpx. With all things otherwise equal, we might arrive at the following estimates:</p>

<ul>

<li>If 12-bit uncompressed RAW in the D810 is 57MB, then uncompressed 12-bit sRAW would be 14.25MB. Or...</li>

<li>If 12-bit lossless compressed RAW in the D810 is 32.4MB, then lossless compressed 12-bit sRAW would be 8.1MB.</li>

</ul>

<p><br />Naturally, there will be a lot of speculation until we can get our hands on a D810 and generate image files for ourselves.</p>

<p> </p>

It is 12-bit compressed. The is according to Steve Heiner of Nikon in a pre-release discussion of the D810.

<blockquote>

<p>While Nikon doesn't state if D810 sRAW is 12-bit uncompressed or uncompressed, generally speaking 1/2 the resolution produces a file 1/4 the size, or 36mpx reduced to 9mpx. With all things otherwise equal, we might arrive at the following estimates:</p>

</blockquote>

<p>So far, the only Nikon DSLR with RAW small that is on the market is the D4S, as the D810 has not yet been released. My experience with RAW small on the D4S is that actual file size, in terms of bytes, is only a tiny bit smaller than 14-bit compressed RAW files. As a result, I don't see any point to shoot RAW small at all.</p>

<p>See my comments in this earlier discussion: http://www.photo.net/nikon-camera-forum/00cbEM<br>

On that thread, I also referenced a thread on DPReview that provides file size numbers: http://www.dpreview.com/forums/post/53407482</p>

<p>At least on the D4S, file size savings is almost negligible. Worse yet, since you cannot shoot RAW + RAW small, shooting RAW small is a capture-time decision and you are throwing away pixels (and details) that will never be recoverable.</p>

<blockquote>

<p>It is 12-bit compressed. The is according to Steve Heiner of Nikon in a pre-release discussion of the D810.</p>

</blockquote>

<p>That's interesting...perhaps the sRAW for the D810 will be around 8MB then? We'll see I guess.</p>

<blockquote>

<p>Worse yet, since you cannot shoot RAW + RAW small, shooting RAW small is a capture-time decision and you are throwing away pixels (and details) that will never be recoverable.</p>

</blockquote>

<p>With a 36 mpx sensor, you lose a lot of information by shooting in this mode. This feature intrigued me, but I honestly don't yet know when I'll use it. However, it might save myself if I find myself shooting somewhere without enough memory cards. Thanks for sharing your experience here!</p>

<p>On my computer, opening up a 36MP NEF (lossless compressed, about 45MB file) with ACR into PS CC takes 3 seconds. The time to open up a 200MB TIFF from HD into PS CC is 1.5 seconds. Thus the disk read time of the NEFis a small fraction of the time it takes to convert it from a raw image into an RGB image in memory. What the small NEF should do is considerably reduce the time it takes to calculate the RGB image (since the dimensions are cut in half) whereas the file size does not significantly contribute to the total time of opening up a NEF file into RAM on my computer.</p>

<p>This, I believe is the motivation of the small NEF implementation.</p>

<blockquote>This 12-bit file format is half the resolution and approximately 1/4 the file size of full RAW files</blockquote>

 

<p>That's interesting. The D810 <a href="http://www.nikon.es/tmp/EU/2419865273/3760176746/2327365364/27184057/1391280926/2780083465/688362553/2027325250/4291728192/4102963099.pdf">brochure</a> (no idea why Google found it on Nikon.es...) says "a quarter of the resolution and half the size").</p>

 

<blockquote>It is 12-bit compressed. The is according to Steve Heiner of Nikon in a pre-release discussion of the D810.</blockquote>

 

<p>The same brochure says 12-bit <i>uncompressed</i> only - borne out by the rawdigger analysis of the D4s. This is the real issue for me - Thom Hogan's <a href="http://www.dslrbodies.com/newsviews/additional-d810-data.html">article</a> says:</p>

 

<blockquote>Size for the sRaw 12-bit file is 27.9MB, while the 12-bit Compressed NEF file size is 29.2MB.</blockquote>

 

<p>I don't know whether that compression was the lossy or lossless version - and obviously it may depend on noise levels. Frankly, I normally shoot 14-bit lossless compressed, because I want the dynamic range, but I'd probably mostly be using small raw in low light anyway. But it appears to slow the camera down significantly, disable on-camera editing, and - because there's no compression scheme - loses most of its size advantage over applying compression to a full-resolution image.<br />

<br />

In terms of image quality, the first gotcha would be that the (already half-sized) small raw is encoding chroma with 4:2:2 downsampling - so high frequency colour edges are losing more information than the mere resolution drop would suggest. The same would be true if the "small raw" encoding had quartered the number of Bayer samples, but the image would be smaller. Otherwise, that article suggests 11 bits of data rather than 12 in some places - still more than you'll see in unmodified images, but I could believe a potential fractional difference compared with full 12-bit raw. (But I'm not in the vicinity of my Adobe-aware computer at the moment, so I can't check the linked raw files.)<br />

<br />

Anyway... I'd be completely tempted if there was some decent lossless compression applied and the file size was halved - and if it sped up the camera at all.</p>

 

<p>Um... from <a href="http://www.dpreview.com/forums/post/41045909">dpreview's fora</a>:</p>

 

<blockquote>Canon calls it sRAW, but it's not raw at all. It has been fully demosaiced and resampled in the camera. It is more similar to a 15-bit TIFF with light JPEG-style compression (specifically, 4:2:2 chroma subsampling).<br />

...

Nikon's "compressed raw" is far superior (and in fact one of the reasons I switched from Canon), but it can only match Canon's largest sRAW size -- it is way bigger than Canon's smaller sRAW sizes.</blockquote>

 

<p>Canon's sRAW format <a href="http://dougkerr.net/Pumpkin/articles/sRaw.pdf">appears</a> to be similar to Nikon's small raw - except that lossless compression is applied to the result. If you get another factor of two compression out of doing so (not unusual for most compressors, though I've not compared Nikon's) then the small raw formats suddenly look useful.</p>

<blockquote>

<p>The D810 <a href="http://www.nikon.es/tmp/EU/2419865273/3760176746/2327365364/27184057/1391280926/2780083465/688362553/2027325250/4291728192/4102963099.pdf" rel="nofollow" target="_blank">brochure</a> (no idea why Google found it on Nikon.es...) says "a quarter of the resolution and half the size").</p>

</blockquote>

<p>This may be where some of the confusion is originating. When Nikon says "half the size", I believe they mean <em>half the dimensions</em>, ie 7360 x 4912 (L) becomes 3680 x 2456 (S). And just like resolution, when you half the dimensions, the file size is <a href="http://web.forret.com/tools/megapixel.asp?width=3680&height=2456%20">roughly 1/4 the original</a>.</p>

<blockquote>

<p>small raw is encoding chroma with 4:2:2 downsampling - so high frequency colour edges are losing more information than the mere resolution drop would suggest</p>

</blockquote>

<p>So far I have not measured any difference in those files I linked, but when I get my D810, I'm going to see if I can 'break' the quality of sRAW format in my shooting situations--if only to satisfy my curiosity.</p>

<p>Kurt: small raw is certainly a quarter of the pixel count - half the linear dimension. If it's the same approach as the D4s's small raw - and, if only for the sake of converters, I have to think it's unlikely they'd have changed it yet, despite the bad press (Ilkka aside!) - they're encoding 24 bits per pixel (or, strictly, 48 bits as 2xY' + Cb + Cr per pixel pair). 12 bit raw uncompressed is literally 12 bits per pixel, because it's one channel at a time. So I'd expect 12-bit small raw to be half the byte count of uncompressed 12-bit full-size raw (a quarter the pixel count, but twice the size per pixel). That tallies with the 25MB and 13MB file sizes you linked to.<br />

<br />

The question is how close to that size a compressed full-resolution raw file (lossless or lossy) can get - whether Nikon's raw compression can achieve a 2:1 factor. I'm guessing it depends on the image, and sometimes could get quite close. But please report back when you get your D810! (I'm going to wait for the price to drop...)</p>

<blockquote>

<p>So I'd expect 12-bit small raw to be half the byte count of uncompressed 12-bit full-size raw (a quarter the pixel count, but twice the size per pixel). That tallies with the 25MB and 13MB file sizes you linked to.</p>

</blockquote>

<p>I do understand the implications of bit depth, whether expressed mathematically by 2^8, 2^12, 2^14 or 2^24. But I'm not going to delve into the minutiae of encoding on file size--I lack experience there. As you say--I should report back when I have the camera in hand and actual files to compare. I will report whatever I find--and happily learn/correct myself at that point. :)</p>

<p>You know that feeling of having posted something on the internet that was gibberish, and only realising the next day?</p>

 

<blockquote>In terms of image quality, the first gotcha would be that the (already half-sized) small raw is encoding chroma with 4:2:2 downsampling - so high frequency colour edges are losing more information than the mere resolution drop would suggest.</blockquote>

 

<p>Actually, you could consider the chroma resolution already to have been downsampled if you were to try to reconstruct an image just by quartering the number of samples in each channel. The vertical chroma resolution (which, according to the investigations in the article, are not downsampled) are actually as high as the original raw file, though the luma detail is still halved. That explains why the representation is twice the size it would be just by throwing away ¾ of the pixel samples. The actual quality loss is likely a combination of transformations and (possibly) gamma, any limitations in the camera's debayering approach, and the slightly reduced range of the representation - and the error is, of course, designed to be small. Apologies for the red herring (with pink edges).</p>

<p>OP: I think you want there to be a problem.</p>