Are any corrections applied to RAW image data in-camera?

<p>I'm a very novice shooter, and I'm trying to set up a workflow for post-processing my images. Right now, I'm looking to get a definitive answer as to which (if any) optical distortion corrections are applied to RAW image data.<br /><br />Specifically, it has been my working assumption that, when shooting in RAW, it's the the actual light readings as measured by the sensor that are recorded--a true digital negative--and the whole point of shooting in RAW in the first place. Therefore, things like white balance, picture style, noise correction settings, peripheral illumination correction, red eye correction, etc., are either saved in the EXIF data so they can be applied in DPP/ViewNX/CaptureNX during post-production, or applied in-camera only to JPGs.<br /><br />I was recently shopping for a new camera, and the salesman at the store told me that I was wrong, and that all of those corrections are made to the actual RAW image data, and if I was correcting for them in post-production, then I was double-correcting for these things. I like to use DxO, if that matters.<br /><br />So I am looking to see if someone can give me a definitive answer as to which (if any) of the following corrections (specifically any optical distortion corrections) are applied to RAW image data (for both Canon and Nikon):</p>

<ol>

<li>Noise correction settings like high-ISO noise correction</li>

<li>White balance</li>

<li>Color space / Picture style (standard, vivid, neutral, etc.) </li>

<li>Red-eye reduction</li>

<li>Peripheral illumination (vignetting) correction (Canon)</li>

<li>Highlight tone priority (Canon)</li>

<li>Auto Lighting Optimizer (Canon)</li>

<li>Active D-lighting (Nikon)</li>

<li>Auto Distortion Control (Nikon)</li>

</ol>

<p>as well as any other applicable settings you can think of.</p>

<p>You're right. The EXIF data records what the camera recommends for an image, that is, what it would have done if it had generated a JPEG instead of saving the raw data. I don't know of any camera that applies distortion correction, white balance, peripheral illumination, etc. to raw files. In fact, one of the ways you can tell what corrections the camera applies is by comparing JPEGs to raw files. With the Olympus E-P1 and its 17mm pancake lens, for example, reviewers at first thought that the lens was soft around the edges until they looked at the raw files and realized the softness was caused by the camera automatically correcting some pretty severe barrel distortion.</p>

<p>If I recall correctly, Canon's Highlight Tone Priority causes the camera to underexpose by one stop as a defense against blown highlights, then applies a tone curve when generating a JPEG. For raw images you just get the underexposure.</p>

<p>It depends o nthe cameras and software you are using. Some software will read the EXIF data and apply pictures styles and other corrections. Other software packages will allow you the option Some cameras will apply the corrections to raw while others will not. I do know from my own experimenation that most of the picture setings on my 5D MkI are appllied to Raw files if I us CS3 software. I suspect Canons DPP software does the same but I am not sure. Many of the Canon corrections you list were didn't exist when I purchased my camera so I cannot provide you with any information about those. Olympus and panasonic m4/3 cameras store information about distortion and Vignetting in the lens and and the cameras apply the settings to the files. in Some of the eary reviews of m4/3 lenses the authors did a lot of work to find software that would not apply the corrections automatically. </p>

<p>So in short there are no hard rules. Some cameras manufactures apply the correction while others do not. Some software vendors apply the corrections, others don't, and then some give you the option. Unfortunately I haven't seen a comprehensive list showing what cameras and software manufactures do. Probably the best thing you can do is to perform your own experiments.</p>

<p>I just noticed this recent thread, <a href="00WlMH">Noise removal in long exposures with Canon EOS Rebel XS</a>.<br>

According to the respondents, noise removal is applied to both JPGs and RAWs for long exposures, so that's something additional to note.</p>

<p>Highlight Tone Priority (Canon) affects the RAW image</p>

<p>1) High ISO and long exposure noise reduction are applied to raw files in all cameras. They require information that is only available at shooting time. Long exposure noise reduction, especially, requires subtraction of a "dark frame" made just after the exposure.</p>

<p>2) Some cameras have per-color gain controls (typically anything that has at least four sensor readout channels: Nikon D2X, D3, Canon 7D, 1D III or IV, 1Ds (any version), anything with Digic IV, and apply RGB gains for the white balance set or determined by AWB calculations.</p>

<p>There's also bypassing of odd (hot, dead, and stuck) pixels. And something called "bias mapping" that maps all the pixels to a uniform gain. Without that, you have weird lines through the middle of FF sensors, and "hot" areas on sensors of any size.</p>

<p>5,9) Distortion and vignetting correction are not applied to raw files.</p>

<p>6, 7, 8) Tone mapping is not applied to raw files, but both Canon HTP and Nikon ADL do cause an exposure reduction, as others noted.</p>

<p>3,4) Everything else is just tags. No picture settings, on any camera brand.</p>

<p>FWIW: "RAW" is NOT a "digital negative" but an image already processed in-camera. Proof? Take different firmware versions for many a camera (lesser brands mostly, such as Sigma Foveon based products, some Nikons, Pentaxes, etc.) and you'll see that a picture shot with the same shooting parameters with cameras of the same model equipped with different firmware versions will look differently even if all image tages ("picture style", etc.) are disabled in post.</p>

<p>Don't mean to piggy-back on this discussion, but why are RAW and Large JPEG the same size comming out of the camera ? </p>

<p>Craig - my understanding is that M4/3 images (such as the EP-1) are not actually RAW. At the bottom of this page from Photozone you will see what they found. If you search their site you will see more on this M4/3 issue.</p>

<p>http://www.photozone.de/olympus--four-thirds-lens-tests/414-pana_1445_3556?start=1</p>

<blockquote>

<p>why are RAW and Large JPEG the same size</p>

</blockquote>

<p>They aren't the same size on my cameras. I just shot a picture with my 5D here in my office and the CR2 file was 15.1 and the large jpeg was 5.7 in size.</p>

<p>Michael, I'm not sure what your point is. I'm not surprised that given the circumstances you describe that you'd see variation in the RAW files.</p>

<p>As for "digital negative", I think the analogy is better than you suppose. In film, each negative image, so to speak, was very different in the negative <em>depending on the physical character of the film itself</em>. That is to say, that even before developing, there were characteristics of the film negative that were already set.<br>

Also, it's an <em>analogy</em>, for chrissakes, not a statement of exact correspondence in every way. It's not even a <em>homology</em>.*<br>

_____<br>

* When I was a Teaching Fellow, the answers of what we called the "white shoe boy" undergraduates to the exam question "compare analogy and homology" was always entirely predictable. :)</p>

<blockquote>

<p>FWIW: "RAW" is NOT a "digital negative" but an image already processed in-camera. Proof? Take different firmware versions for many a camera (lesser brands mostly, such as Sigma Foveon based products, some Nikons, Pentaxes, etc.) and you'll see that a picture shot with the same shooting parameters with cameras of the same model equipped with different firmware versions will look differently even if all image tages ("picture style", etc.) are disabled in post.</p>

 

</blockquote>

<p>Exactly the same differences occur when using different types of film. Negatives are still negatives, whether they are shot on low speed B&W film or high speed color film or even on Kodak or Ilford film of the same speed and type or even on two types of Kodak B&W film of the same speed. They are all starting points for the rest of the photographic process.</p>

<p>As Adams said (approximately), the negative is the score, the print is the performance. Applies to digital (RAW) as well as analog. You could even call JPEGs "digital negatives" when you print them, it's just that they have a more limited range of adjustment than RAW files.</p>

<blockquote>

<p>Craig - my understanding is that M4/3 images (such as the EP-1) are not actually RAW. At the bottom of this page from Photozone you will see what they found. If you search their site you will see more on this M4/3 issue.</p>

</blockquote>

<p>That's not quite what they said. Photozone is showing a difference between raw processors. The Panasonic raw files "contain unprocessed data", they say, but it also contains annotations concerning lens distortion, vignetting, CA, and who knows what else. ACR and Silkypix automatically applied those corrections without giving the user the option of ignoring them, but Raw Therapee did not. If the camera had already applied the corrections, Raw Therapee's image would have looked the same as the others. So the files really are raw.</p>

<blockquote>

<p>High ISO and long exposure noise reduction are applied to raw files in all cameras.</p>

</blockquote>

<p>Not true. Of these two, only long exposure NR affects RAW files.</p>

<blockquote>

<p>I do know from my own experimentation that most of the picture settings on my 5D MkI are applied to Raw files if I us CS3 software.</p>

 

</blockquote>

<p>No. CS3 <em>does not write to RAW files</em>, and - as has already been stated here - most in-camera settings are completely ignored by software that can't recognise them when converting from RAW, which means that they are not part of the RAW. Of the list provided by Frank, only HTP (and, I imagine, Active D-Lighting, the Nikon equivalent) and long exposure NR directly affect the RAW file.</p>

<p> </p>

<p>Craig, while semantically you are correct, most users will either use the software supplied with the camera or the most used aftermarket RAW processing software (ACR) In both cases they will actually correct the RAW image for lens effects and the user is unable to prevent this. When you are comparing the EP-1 JPEG to RAW what processing software are you using (i.e. is the RAW file corrected or uncorrected?)</p>

<blockquote>

<p>Craig, while semantically you are correct, most users will either use the software supplied with the camera or the most used aftermarket RAW processing software (ACR) In both cases they will actually correct the RAW image for lens effects and the user is unable to prevent this.</p>

</blockquote>

<p>Sure, but that's a defect (or at least a limitation) of the software they're using; it has nothing to do with the raw file itself. The raw file as it comes out of the camera has not been corrected for lens distortion, vignetting, etc. The original question was whether corrections are applied in-camera to the raw data, to which the answer is mostly no. Your comment is a good explanation for why people might think the corrections are applied in-camera, but they're still wrong.</p>

<p>My question was actually what software did you use to see the differences between the JPEg and RAW on the EP1. I would assume the RAW would look worse if the lens is corrected in the JPEG in camera and the RAW is not corrected. Did you use one of the RAW convertors that corrects or one that does not?</p>

<p>I didn't say I did any such thing; it's mentioned in photozone.de's review of the Olympus 17mm pancake lens:</p>

<p>"The M.Zuiko lens produces only a moderate degree of barrel distortion (~0.8%) which is only rarely noticeable in field conditions. However, this is only a part of the truth. Micro-4/3 images are auto-corrected according to the stored lens profile (lens firmware) - this is done either by the camera directly (JPEG) or by the more popular RAW-converters a la Photoshop. The "untouched RAW" results (thus the principal capability of the lens) are much worse with a barrel distortion as bad as ~4.5% which is extreme (poor) for a prime lens."</p>

<p>They don't say (on that page) exactly what software they were using to see the "untouched RAW", but the photozone.de page you linked to earlier in this discussion (the Panasonic 14-45mm review) says, "it is possible to use e.g. DCRaw in order to have a look at the uncorrected data." It also says that Raw Therapee did not apply corrections. So either DCRaw or Raw Therapee (probably UFRaw as well) can be used to see images from micro-4/3 cameras without automatic corrections.</p>

<blockquote>

<p>I do know from my own experimentation that most of the picture settings on my 5D MkI are applied to Raw files if I us CS3 software.<br>

No. CS3 <em>does not write to RAW files</em>, and - as has already been stated here - most in-camera settings are completely ignored by software that can't recognise them when converting from RAW, which means that they are not part of the RAW.</p>

</blockquote>

<p>Well then the camera is applying the picture settings to the raw file and as a result you cannot see the raw data with CS3. The raw data is probably in the file but simply not available to the user. So the answer to Frank F question. is still the same. </p>

 

<p>Stephen F wrote:</p>

<blockquote>

<p>Well then the camera is applying the picture settings to the raw file and as a result you cannot see the raw data with CS3. The raw data is probably in the file but simply not available to the user. So the answer to Frank F question. is still the same.</p>

</blockquote>

<p>Stephen, I think you might be seeing something that you are interpreting as ACR applying in camera settings from the exif data to the raw conversion, however that isn't the case.</p>

<p>ACR will read in the white balance setting when it converts the raw data, but the file handling of other settings isn't necessarily known by Adobe, so ACR doesn't generally attempt to read those (especially back in CS3). With CS5, ACR will now read lens information in the exif and can apply lens corrections automatically if appropriate profiles are available, but these can always be turned off and other settings are still generally not used by ACR.</p>

<p>Because raw formats from each manufacturer and each camera model are different and proprietary; and Adobe doesn't necessarily have access to the raw encoding format, ACR will read the white balance tag in the file and then convert the raw data according to the defaults you or Adobe have set (eg. exp 0, brightness 50, contrast 25, etc.).</p>

<p>No other in camera settings are generally applied during the raw conversion by ACR. Additionally, aside from processing during the analog to digital conversion (which may include noise reduction, etc.) other in camera settings are not applied to the RAW data in camera. In camera settings are written into the exif data and can be applied, mostly by the camera manufacturers raw conversion software, but you are always free to adjust the settings without affecting the original raw data.</p>

<p>So in relation to Frank F's question, it really depends. Some settings such as long exposure noise reduction are applied to the RAW data in camera (this requires 2 frames to be shot) and the white balance setting is read by ACR. During processing of the raw data, the camera processor might apply some adjustments to the raw data (eg. noise reduction), but mostly, raw data is undemosaiced information about the sensor response, together with the exif data.</p>

<p>Regards,</p>

<p>Peter</p>

<p>Craig I apologise for any misunderstanding - I was merely trying to get your thoughts on am m4/3 RAw conversion approach. I use a G1 with Fd lenses and found Silkypix to be quite poor, ACR is much better but I was wondering if you had seen better results from another package. The FD lenses I use are superior to the m4/3 lenses (at least to the one I have) and even using Silkypix give much better images. This is not a surprise as they are lenses like the 24F2, 85 F1.2, 135 f2, 50 f3.5 etc... What I was wondering was which RAW convertor works the best - I was assuming that ACR does not apply corrections when using "shoot without lens" but do not know this for a fact. The fact that poor lens performance of M4/3 lenses is corrected automatically and that this leads to image quality issues was something I was aware of.</p>

<blockquote>

<p>Stephen, I think you might be seeing something that you are interpreting as ACR applying in camera settings from the exif data to the raw conversion, however that isn't the case.</p>

</blockquote>

<p>Peter, I am not refering to white ballance. I am refering to two pictures taken of the same subject with the same exposure and the same white ballance and the same lighting. The only difference between them is that in one I set all the picture settingts (contrast, color saturation, sharpness, etc)to minimum and then take the second withthe all the picture settings to maximum. If you then open both pictures in CS3 you will see two distinctly different pictures. I did this experiment a number of times last year and the resuls were always the same. And there is no button I can press to cancel out those changes. There is a way to return the white ballance back to the As Shot setting but there is no way to undue to the color saturation, sharpness, and contrast settings don in the camera. I can adjust hese settings seperatly but but I would be just guessing on home much of an adjustment to make. </p>

<p>When I switched from film to digital I read a lot of post that stated that Raw was just the actual data from the sensor without any camera settnigs. (If that was true in the past it is definitely not today. Most software today show the raw with a lot of camera settings applied to the image. And in many cases you cannot undo those changes. Yes some software packages do allow you the option of undoing those changes but a lot don't</p>

<blockquote>

<p>So in relation to Frank F's question, it really depends.</p>

</blockquote>

<p>Which is exactly what I said in the second post on this thread.</p>

 

<p>Stephen F wrote:</p>

<blockquote>

<p>If you then open both pictures in CS3 you will see two distinctly different pictures. I did this experiment a number of times last year and the resuls were always the same. And there is no button I can press to cancel out those changes. There is a way to return the white ballance back to the As Shot setting but there is no way to undue to the color saturation, sharpness, and contrast settings don in the camera.</p>

</blockquote>

<p>Stephen, are you sure you weren't shooting jpeg, thinking you were shooting raw?</p>

<p>Two relevant quotes, one from Canon and one from Adobe:</p>

<p>Canon (http://www.usa.canon.com/content/picturestyle/file/index.html):</p>

<blockquote>

<p><em>Original RAW image data is never changed by use of Picture Styles or other controls in the camera or Canon software.</em></p>

</blockquote>

<p>Adobe (http://www.adobe.com/designcenter/dialogbox/why_shoot_raw.html):</p>

<blockquote>

<p><em>With raw, the only on-camera settings that have an effect on the captured pixels are the ISO speed, shutter speed, and aperture. Everything else is under your control when you convert the raw file.</em></p>

</blockquote>

<p>The second quote from the late Bruce Fraser isn't so current now, but is still generally true.</p>

<p>The quote from Canon very much applies to your 5D files.</p>

<p>That's why I think you must be seeing something that you think is one thing, but it's not. The in camera adjustments and picture styles relate to jpeg shooting, not the raw files.</p>

<p>Regards,</p>

<p>Peter</p>

<p><em>>Exactly the same differences occur when using different types of film.</em><br>

Not really: the same camera body (and other settings) with diferent firmware versions, might give vastly different RAW images. Thus the image is processed in-camera before being saved as a RAW file and that procesing may vary between different firmware versions and may, dramatically sometimes, affect the resulting RAW image (other settings notwithstanding.) If you wish to equate different versions of firmware with different films - be my guest, but that's a very cumbersome and imprecise way to think about it.</p>

<p>Canon cameras WILL apply Picture Styles to a RAW file's embedded JPEG preview. When you're looking at your pictures on your camera's LCD, you're looking at a jpeg preview, not a CR2. Even if you're shooting RAW only (no +jpeg), your camera will apply your chosen picture style to the image you see.</p>

<p>So, while your RAW file is uncorrupted by any in-camera processing, it may be a surprise when reviewing your RAW images only to find that they don't look like what you expected from the preview!</p>

<p>I don't know if this is the case for Nikon, Sony, or Pentax.</p>