Embedded jpg in the RAW file?

[Wouter Willemse]

<blockquote>

<p>The only answer I can think of is if the RAW editing data are not kept in a .xmp sidecar, but stored within the RAW it self. But, my Oly .ORFs are 13MB and the sidecars only 6KB - unless Nikon is not as smart as Oly and uses a larger font for metadata? ;)</p>

</blockquote>

<p>Thormod, the growth is solely and purely due to the much higher quality JPEG being embedded - there is no XMP sidecar with this program, and as stated, the solution is rather unique to Nikon, so there is little point in comparing to Olympus. As said, I used the open source command-line tool ExifTools to extract JPEGs, and verified myself. With all respect, it seems you still oppose the idea there is a JPEG embedded in there, but do reconsider what I wrote - by far the most likely explanation is a JPEG being embedded. Occam's Razor.<br>

Indeed I mixed up WIA and WIC - close relatives but not identical. I think your use of browser pertains to Windows Explorer, while I usually reserve that word for internet browsers (IE, FF etc.) only. Hence my other point.<br>

I'll drop you a PM for the other question.</p>

[thormod]

<p>I don't oppose the existence of a embedded jpg in a RAW file! I question it. My knowledge of RAW stems from the literature of Bruce Fraser, and I have credible information from three major camera manufacturers supporting my opinion.<br>

On the other hand, with all respect, I have some people on the internet claiming the opposite. Occam's Razor right back at you:)<br>

However, I am getting close to the answer. I now know how to "force" a colored block into a RAW file without impacting upon a embedded jpg. A single line was simple, a block requires some math to avoid destroying the raw data<br>

Wouter, I will await your PM, but in the mean time,</p>

<blockquote>

<p> there is no XMP sidecar with this program </p>

</blockquote>

<p>What program are you referencing? If it is a RAW editor, there are( to my knowledge) two options for keeping the etiting meta data. Either within the RAW file it self or a separate .xmp file. If the meta data is saved within the RAW file, I thought the RAW file would grow in size. Anyhoo, actually the statement about size in my answer was a joke. Did you miss my ridicoulous claim that the size might grow due to the use of a larger font! You do know that the meta data in the.xmp is ASCII text?<br>

<br />Tom</p>

[Wouter Willemse]

<p>I refer to CaptureNX2, to which I refered earlier. It saves into the raw file, but given the increase in filesize (4MB on a 12MP photo), it's not only metadata. Metadata is pure txt, 4MB of pure text would be quite a novel. I did send you the PM via the internal messaging system of this site, it'll give you the possibility to see for yourself.</p>

[thormod]

<p>Thanks Wouter, PM received. I have a feeling that this may convince me:)</p>

[thormod]

<p>It did and didn't:)</p>

<p>Wouter, thanks for the files. I have looked at them closely, but they do not answer whether the RAW file from the camera contains a embedded jpg or not.<br />Your image no.:2 (BW) proves the existence of a preview file embedded in the edited cnx2edit.nef.<br>

After installing the Nikon codec, I could view both files in windows Explorer. The cnx2edit.nef thumbnail was BW.<br />I then opened both files in Adobe ACR where both images were in color. Unquestionable proof that WinExplorer renders a thumbnail. from the cnx2edit.nef. Albeit strange that the thumbnail/preview does not look like the raw image.<br>

My conclusion is that you are right. A nef file may contain an embedded .jpg, but only if it is edited and saved using the CaptureNX2 software. (and any other Nikon SW I don't know about).<br>

I continue to believe that the RAW file, unedited, from the camera, does not contain an embedded .jpg.<br>

I continue to believe the reason the unedited RAW file renders an image in WinExplorer, and other, is the WIC and the Nikon Codec.<br>

It seems that you are all right. You'all, and the guy at Nikon.<br>

My error is not considering the possibility of a jpg being added during editing along with the editing meta data. Maybe even Oly does that, when using their SW. I use PS.<br>

Tom</p>

<p>Wow, major headache: Does ACR add a jpg to my edited .orfs?</p>

[Wouter Willemse]

<p>I am going to disagree once more, simple test:<br>

Run exiftools (<em>exiftools -b -JpegFromRaw input.raw > output.jpg</em>) on a system without ANY WIC codec installed (i.e. XP, Vista, 7 without additional download, or Linux): it creates a JPEG that looks identical to what you see in camera. If you have your camera set up to shoot monochrome, the extracted JPEG will be monochrome. So, obviously, it is not using dcraw or similar to 'create' a jpeg on the fly - it's using something the camera already put there. Maybe a more manufacterer agnostic approach would be "exiftool -a -b -W %d%f_%t%-c.%s -preview:all dir" (ref.: http://www.sno.phy.queensu.ca/~phil/exiftool/exiftool_pod.html). As said, I only have Nikon files to play with, and on those -JpgFromRaw works fine.<br>

Exiftools is open source, if you're a bit PERL-savvy you can dig into the code to figure out how it works. A nice place to start would be this: http://www.sno.phy.queensu.ca/~phil/exiftool/under.html.</p>

<p>You're probably aware, but for the completeness of discussion: there is no such thing as 'the raw image'; the raw data needs interpretation of some sort to render an image. There are different algoritms in different programs to do this (in camera, ACR, manufacterer software, CaptureOne, dcraw/ufraw, and so on - they all use different engines to render the final result). There is no single correct way to do this, and so it's a false assumption to believe one is a correct interpretation, and the other not. They're all equally valid, in their own ways.</p>

 

<blockquote>

<p>Does ACR add a jpg to my edited .orfs?</p>

</blockquote>

<p>No. ACR never changes the raw file. It keeps previews (=jpeg files) in its cache, and its edits in either xmp sidecars, or in a database (depending on the configuration you choose).<br>

If you use an Olympus WIC codec, your thumbnail in Windows Explorer should not change after editing the image with ACR, nor standard filedata (i.e. Date Last Modification) - hence, the file didn't change in any way.</p>

[Tim_Lookingbill]

<blockquote>

<p>No. ACR never changes the raw file. It keeps previews (=jpeg files) in its cache, and its edits in either xmp sidecars, or in a database (depending on the configuration you choose).</p>

</blockquote>

<p>I actually hunted down Bridge CS5's cached previews of my edited (with xmp sidecar) Pentax PEF Raw files in my OS X directory. Pretty easy to find in a logically named folder structure in my user's Library. </p>

<p>I copied one of the 1024 resolution versions to my desktop and opened in Photoshop and got the color management warning that it was embedded with sRGB profile. Adobe thinks of everything. </p><div></div>

[thormod]

<p>Thanks Wouter for you patience. I dug up an old XP laptop No WIC no codec. no MS codec pack. I can not see a thumbnail in WinExplorer, as expected.<br>

Had an old version of Exiftool on it, the one where you drop imaged om the desktop icon (the camel), with the -k -w txt switches. (Creates a txt file on the desktop with the Exif data). Started updating Exiftool and wife unplugged power supply during installation. The old suffering battery has about 10 sec charge and Laptop crashed. I get the option to boot in safe mode - which it doesn't:(<br>

Tim: Interesting. Found the cache but very strange. The thumbnails seem to have the same size as the small jpgs I load together with the RAW file. I used to save a 640x480 jpg together with the RAW until I heard of the WIC and Oly codec. One day, I saved larger jpgs, to do a QD (quick and dirty) comparison with a Canon lady. The thumbs from that shoot are larger.</p>

<p>I think it's about time to pull the plug on this thread. I declare my question for answered. Sheer probability indicates that I, Canon and Nikon are wrong. <br>

Thanks to everybody who have participated. I will at leisure continue investigating and should I find conclusive evidence to the existence of a thumb/preview in the unprocessed RAW file straight from the camera, I will return here and make a statement to that effect.<br>

Tom</p>

[thormod]

<p>This was quick.<br>

I have proven myself to be wrong. I hacked an Oly .orf RAW file with a Hex editor and created a block of junk data within the RAW data. dCraw renders the Raw data containing my junk, while ExifTool renders a preview unblemished. I consider this conclusive proof of a jpg thumbnail/preview in thee Olympus RAW file.<br>

I am very disappointed with Oly support. We will have words!<br>

<br>

Tom</p>

[thormod]

<p>Ok, final update. I have had words with Oly, and it turns out that Oly was right. A .Orf file does <strong>not</strong> contain a .jpg thumbnail/preview. The .Orf does contain a miniature of the "RAW" image in a proprietary format.<br>

The WIC and the Oly Codec allows third party rendition of the miniature, like WinExplorer.</p>

[thormod]

<p>I have to stop claiming "Final"!<br>

Therefore: New update. <br>

I am now 99% convinced (again) that the Oly Raw thumbnail is a .jpg.<br>

I have called Oly support, again, and they confirm it, with 99% certainty. </p>

[q.g._de_bakker]

The thing with proprietary file formats is that the one who invents it can do anything he likes. It would appear that many raw formats are based on the idea of a TIFF-like container. Because the OS doesn't know all these proprietary formats, it requires a little helper to be able to know and show what's in them.<br>That does however not mean that what is hidden in those file could not be a plain and universally readable format like JPG. The OS just doesn't know about it, nor where to find it.<br>That Windows Explorer needs a 'codec' to be able to show the embedded miniature says nothing about the nature of the miniature. If that miniature itself is indeed encoded in a well-known well documented way, and if the information how to find it inside th eraw file is available, any programmer would have no trouble at all to write a little programm to extract it. Would not be difficult, nor much work. Just find where it starts and ends inside the container, and perhaps prepare a separate header, and write it to disk.

[thormod]

<p>Quote: Just find where it starts and ends inside the container, and perhaps prepare a separate header, and write it to disk.Unquote.<br>

Thats what I did. Didn't even need to prepare a header. My problem was Oly supports claim that the thumbnail was not JPEG. They have changed their mind now.</p>