More ISO 1600 Pixel Peeping -- D300, 5D, & D3

[tonybeach]

At DPR Wai-Shan Lam made a series of shots he took with these three cameras

available, so I downloaded them in order to look at the ISO performance of these

three cameras shot under nearly identical conditions and converted the RAW files

using the same RAW converter.

 

The original thread where these images appeared is here:

http://forums.dpreview.com/forums/read.asp?forum=1021&message=26361433 An

explanation of how the shots were taken is there; but I would add that

differences in exposure times appear to have been a result of perhaps Matrix

metering from the Nikons and Evaluative 35 zone metering from the 5D and

probably the slightly different framing. Whatever the reason, there is about a

1/3 of stop difference between the D3 and the 5D, and about 2/3 of stop

difference between the D3 and the D300. The converter may be playing a part in

the EC (exposure compensation) applied to the images, but it is somewhat

disconcerting to see that ECs did not correlate with the the predicted EVs

(exposure values) based on the different shutter speeds.

 

I used Capture One to convert these files and manually set WB off the wall in

the same spot for all three images (the WB target was on the upper right side

outside of the cropped area). I also adjusted EC to make all three images as

close to one another as possible; since the D300 supposedly had the middle EV, I

used it as the reference point for the other two camera's files. The resulting

discrepancy already commented on will require more investigation, which will be

forthcoming in another thread comparing the D3 and D300 using these files.

 

Capture One has a default NR setting for ISO 1600 files regardless of the

camera. If I disabled that completely the 5D had less chroma noise than the

D300 (although both had significant luminance noise) but the D3 simply looked

astounding. I could have applied less NR to the 5D image than the D300 image to

get similar results, but ended up using identical settings for both cameras so

as to not skew the results. I ended up setting the 5D and D300 NR to

approximately half of Capture One's default, and the D3 to approximately a third

of Capture One's default setting.

 

Quantifying noise in the D300 and 5D files indicates that the results are very

close, with a marginal advantage for the 5D (although not enough to consider

important IMO), but the D3 simply shines in this regard. The D3 file was so

impressive that it was hard to distinguish its ISO 1600 example from the ISO 200

examples from the other two cameras. Is the D3 really 3 stops better?

Answering this question merits a follow-up where I plan on comparing D300 and D3

files in both Capture One and NX at various ISO settings (and I will be

reviewing the EV issue at that time). For now, you can draw your own

conclusions by viewing the comparative crops for yourself (it's a 1.4 MB file).

 

http://photos.imageevent.com/tonybeach/mypicturesfolder/sharing//ISO%201600_D300-5D-D3_Comparison.jpg

[tonybeach]

Addendum:

 

The different EV should have been 2/3 of a stop between the D3 and the 5D, and 1/3 of a stop between the D3 and D300. At the end of the second paragraph in the OT these were accidentally reversed.

[t_n1]

Capture One has separate profiles for each camera, so I don't know how fair it would be to compare raw files converted in that program.

 

My experience differs from yours. I've owned the Canon 5D for awhile before purchasing the D300. I've found that the Canon 5D, using Lightroom, had at least 2/3 stop advantage in regards to noise. The details in the D300 at higher iso are also slightly 'mushed' compared to the details from the 5D. I suppose this is fair since we're comparing crop sensors versus full frame. Between iso 200 to 640, the Canon 5D had at least a full stop advantage.

 

I have not owned the D3 to make any qualified assessment.

[mmene]

Gordon Laing at Camera Labs says " For only slightly more than the Nikon D300 you could buy the Canon EOS 5D. The D300 trumps it in every respect bar one: the full-frame sensor. As you can see in our Nikon D300 Noise results, the full-frame Canon 5D may share the same resolution, but the bigger sensor allows it to out-perform both the D300 and Sony A700 at higher sensitivities. So if you?re looking for an upgrade in image quality and aren?t bothered about other features, the EOS 5D remains a contender, but be aware many are expecting a successor early in 2008. See our Canon EOS 5D review for more details."

http://www.cameralabs.com/reviews/Nikon_D300/verdict.shtml

 

You may visit also http://www.imaging-resource.com/IMCOMP/COMPS01.HTM

at I imaging resource to see the difference between D5 and D300 in image quality. (By the way if you download images of D300 and D200 at

1600 iso and manipulate them at PS CS3, that I use, there is no difference at all between D300 and D200).

 

So PR of Nikon is very interesting, because I believe also in the general superiority of Nikon over Canon. But Nikon fans, like you and me, should better press Nikon to get back the throne in every aspect from Canon and not overestimate the success of D300/D3.

[elliot1]

Anthony, during the short period of time I had the D3, I had time to explore its high ISO abilities. Research and my own hands-on results indicate to me that most of the image enhancements are a results of advanced in-camera image enhancement and noise reduction.

 

I did several side-by-side comparisons between the D3 and 5D. For this test, I shot the D3 at ISO 4000 and the 5D at ISO 3200 (RAW, of course). As the D3 applies noise reduction in camera by default, I turned this feature off. Images were opened in CS3 with default camera settings.

 

This is the full frame view of the original Image:

 

http://www.photo.net/photo/6770520

 

I cropped a small section of the original images, selecting an area with both bright and dark areas. This is a comparison of an extreme crop of the unprocessed images:

 

http://www.photo.net/photo/6770521

 

I processed the images with Dfine 2.0 software in CS3. I ran Dfine 2.0 in automatic mode and processed each image twice. This link compares the original to the Dfine processed crops:

 

http://www.photo.net/photo/6770525&size=lg

 

There does not appear to be any significant difference between the two images. It appears current RAW converters and noise reduction software really levels the playing field when it comes to getting high quality, low noise higher ISO images with virtually all recent DSLR cameras.

 

I am not sure of the settings on the comparison shots you analyzed but I suspect that NR was on on both the D3 and D300. It is my understanding that both the D3 and D300 apply two stages of NR, and you can only turn one off. I don't know if similar NR is applied in-camera in the same manner in the 5D but I doubt it.

 

If noise reduction were applied to the 5D and D300 images, they would likely look just as good as the D3 image.

 

Perhaps a more fair comparison would be to compare the final results of fully processed RAW images. After all, it is the end result that counts, isn't it? I suspect there would be virtually no difference between the 3 cameras. I suspect that a fully processed D200 image processed would not look all that different from the equivalent processed D300, 5D and D3 images either when printed unless you were printing posters or pixel peeping.

[tonybeach]

Well I can see that I am going to get a lot of objections, but nonetheless I stand by my findings, which are applicable to Capture One and I used all the same parameters (which are clearly displayed in the link on the right side). I have not jiggered or played with the settings in any way and everyone is free to do their own testing to confirm or discredit what I have found here. Certainly feel free to download the files for yourself and use your own converter of choice to do the comparisons. Please feel free to post your results here.

 

For TN:

 

> I have gone to the trouble to download a trial version of Lightroom and did the conversions there, I raised Luminance NR to 15 and Color NR to 30 for the D300 and 5D files and left the D3 file as is. The D300 file in particular suffers from that program's conversion algorithms, but it appears to be a half stop difference and you can see for yourself in the patches included at the end of this post.

 

Before anyone gets excited about the lower amount of noise in the ACR conversions, it should be noted that far less sharpening was applied to them. The standard deviation in the same patches from the Capture One conversions were as follows:

 

D300 is 6.47

 

5D is 5.99

 

D3 is 4.20

 

For Michael:

 

"...there is no difference at all between D300 and D200"

 

> I own both cameras and have done the testing using NX and Capture One, I consistently see a 1 stop advantage for the D300. I am not sure about ACR in this regard, but I find the color profiling for NEF files with that program to be problematic -- specifically they are flat (much flatter than how ACR handles the 5D file, or how NX or Capture One handle the files from all three cameras).

 

For Elliot:

 

"I am not sure of the settings on the comparison shots you analyzed but I suspect that NR was on on both the D3 and D300. It is my understanding that both the D3 and D300 apply two stages of NR, and you can only turn one off. I don't know if similar NR is applied in-camera in the same manner in the 5D but I doubt it."

 

> Elliot, I wrote at length about the specific parameters I used for the test. For Capture One I set NR and sharpening to identical settings for the 5D and the D300, and the D3 actually had less NR. In the sample patches (from ACR) below, I set all the sharpening and NR settings identically for all the files.

 

Your repetitious claim that Nikon is using some sort of in-camera NR is becoming very tiresome. All of the cameras are rendering similar detail with both comparable absolute and extinction resolutions. You may be confusing the effects of the AA filter for NR, and you clearly do not understand that all CMOS sensors apply subtractive NR at the level of the photosites; and that this applies equally to Nikon and Canon. I do not know how many times I have to tell you that any additional NR applied to the RAW data is a tag that instructs NX to apply NR and can be turned off, and that tag is ignored by third party converters.

 

"Perhaps a more fair comparison would be to compare the final results of fully processed RAW images. I suspect that a fully processed D200 image processed would not look all that different from the equivalent processed D300, 5D and D3 images either when printed unless you were printing posters or pixel peeping."

 

> That is precisely what I did; and yes, we are engaged in pixel peeping here and at that level there are differences between all four cameras. To be absolutely fair, I would rank them (from best to worse):

 

D3

 

5D

 

D300

 

D200

 

Depending on what converter is being used, the differences between the 5D and D300 vary by up to a half a stop with ACR and are only marginal using Capture One. The D200 is about a stop behind and the D3 is at least a stop ahead.<div></div>

[mmene]

I said "By the way if you download images of D300 and D200 at 1600 iso and manipulate them at PS CS3, that I use, there is no difference at all between D300 and D200". So I continue to insist that this is right but only till 1600 iso. Of course at 3200 there is one stop SUPERIORITY of D300 than D200. AND DO NOT FORGET THE D200 SUPERIORITY IN DYNAMIC RANGE AT 100 ISO AND PARTIALLY AT 200 ISO. http://kammagamma.com/articles/nikon-d300-how-much-of-an-improvement.php

 

So my final conclusion is that for studio images and photography of high clarity and details D200 is better than D300. For documentary and sport photography and under very dim conditions the superiority of D300 is obvious.

[elliot1]

Anthony, the settings I was referring to were the in-camera settings, not the settings you used for processing.

 

I have read about the in-camera NR of the new Nikon cameras. I did not make it up, although before the D3s release, I discussed with several people that the hish ISO images looked like they had NR applied to them. They agreed. I will try and find the reference for you. Sony openly advertises its A700 as having two stages of NR in the camera.

 

Overall, I agree with your results. My main difference with your conclusion is only that an expert at PP can make all the images look pretty much the same, which is a good thing, isn't it? Your examples clearly show that the differences between the various cameras is minimal and possibly insignificant for most typical photography applications.

 

If suspect that if you apply NR from Dfine 2.0 software or equivalent to your D300 and 5D images (the ones you linked) they would look pretty much as good at the D3s. Why not give it a try just for fun and then repost them?

 

The features of the D300 and D3 clearly give them a huge advantage over virtually all other cameras including Nikon or Canon. My point is simply that when it comes to image quality, the playing field is now somewhat level. Even high ISO D40 RAW images can be processed to look surprisingly impressive. Pretty amazing for a camera you can now buy for about $300. In my testing ISO 1600 and even ISO 3200 are quite usable if you shoot RAW and process with DXO, Bibble or equivalent software.

 

I believe it may be more important to compare detail or loss of detail rather than noise. My 5D adds small amounts of color noise starting at ISO 800 which is easily removed in CS3 or DXO without loss of image quality. And while there is more noise present and ISO 1600 and much more at ISO 3200, the loss of detail after applying NR is minimal and certainly less than many other cameras. The D3 seems to do this exceptionally well too. I assume the D300 is similar (I obviously don't have one). Testing with my D80 at ISO 3200 revealed some but minimal detail loss after processing with DXO version 5. This is quite amazing considering up until now, I felt very uncomfortable shooting above ISO 400 with it.

 

Your testing is well done and will surely help many better understand and get the most out of their cameras.

[pete_s.]

These tests are flawed. Nobody knows what kind of processing is done in the camera software/hardware prior to the RAW file being written to the card. The RAW isn't really raw so you are not comparing apples to apples.

 

For instance, a nice little blur filter in the camera firmware increasingly powerful at higher iso will show low noise (and less detail). (The D70 has some kind of smoothing algorithm).

 

The astro guys knows this, for example look here: http://www.astrosurf.org/buil/nikon_test/test.htm

 

Peter

[lwg]

I would disagree with Peter that the tests aren't valid. I really don't care what the camera does internally as long as the results are good. If Nikon or Canon comes up with an internal noise reduction that doesn't obscure details I would want that applied. Sounds like they have already done that with the way CMOS sensors work. It simply becomes part of the camera. My interest is in how the RAW files produce images. If it is noted that the D70 looses detail at high ISO then it is dinged it for that. I don't see a loss of resolution between the three cameras examined here at high ISO using RAW.

[ellis_vener_photography]

I don't like Capture One for raw processing for the simple reason that unless you set the sharpenign to something like -128 , you haven't turned off the built in sharpening algorithm, and I suspect they bias their sharpening algorithms for their backs.

[dcraton]

A comment from the Geek Techno Peanut Gallery.

 

As much as I love diving into the design architecture of a camera, lens, software, display monitor, I would like to add another perspective here.

 

Yes, it is great to understand the mechanics and design of a cam bod or whatever to make an intelligent purchase and get the best use day to day. However, people buy different cam bods for a variety of reasons. With that being said, for us, it is the end result that counts. What goes on the wall, website, presentation, or in the clients' hands matters most. What we shoot defines what cam bods we buy every 18 to 24 months.

 

All to say, for us (not pushing this on anyone else): A D3, 5D, D300, or D200 (and others) all work fine equally per the end result. The end result is our main objective. Yes you can argue per the noise and detail issues it takes more work for some cams that others, but really with a couple of mouse clicks and copied to the rest of the images shot with the same lens and ISO, it takes about an extra minute. And, the results are incredible and the client is happy and we make a dollar.

 

I do enjoy threads like this as they keep me abreast of field testing and appreciate Anthony's work and opposing replies as I can learn from different perspectives.

 

Yes, we are geeks. Hopefully profitable ones.

[tonybeach]

For Michael:

 

"I said 'By the way if you download images of D300 and D200 at 1600 iso and manipulate them at PS CS3, that I use, there is no difference at all between D300 and D200'. So I continue to insist that this is right but only till 1600 iso."

 

> It is not right, differences are apparent below ISO 400: Also, DR starts out even but immediately starts declining and is about a stop better on the D300 at ISO 1600.

 

"So my final conclusion is that for studio images and photography of high clarity and details D200 is better than D300."

 

> If that comforts you then fine, but it is also not accurate. At ISO 100-200 the camera are essentially equal with a slight resolution advantage for the D300.

 

For Elliot:

 

"I have read about the in-camera NR of the new Nikon cameras. I did not make it up, although before the D3s release, I discussed with several people that the hish ISO images looked like they had NR applied to them. They agreed."

 

> Before the camera's release, the only samples available were JPEGs. Since I don't shoot JPEGs and I was testing RAW files to satisfy myself about optimal workflow and expected results, I cannot comment on what happens to the JPEG files, nor am I interested as I have no plans to shoot JPEG.

 

For Peter:

 

"These tests are flawed. Nobody knows what kind of processing is done in the camera software/hardware prior to the RAW file being written to the card. The RAW isn't really raw so you are not comparing apples to apples."

 

> You are both confused about what is happening inside the camera and what effect that has on the resulting files. The D3 and D300 files have amazing amounts of detail preserved even at ISO 1600 for the D300 and to at least ISO 3200 for the D3. Read your source more carefully. It states in its conclusion that, "The behaviour of Nikon DSLR are radically different from the Canon ones for long exposure..." Long exposure NR is different from high ISO NR or the kind of subtractive NR performed on CMOS sensors. This is not a discussion of the applicability of Nikon DSLRs for astrophotography, it is fairly well understood by those that engage in that Nikons are not optimal.

 

For Ellis:

 

"I don't like Capture One for raw processing for the simple reason that unless you set the sharpening to something like -128 , you haven't turned off the built in sharpening algorithm, and I suspect they bias their sharpening algorithms for their backs."

 

> It may explain the higher levels of noise I am seeing in Capture One conversions than in the ACR files (although the older version of ACR had atrocious amounts of noise in my D200 files), but sharpening has to be applied eventually, so it is a matter of choosing when and not if sharpening will be applied. For me Capture One's UI and consistent colors are its biggest appeal, and files up to ISO 1600 look excellent. My biggest problem with ACR is that the colors just always seem undersaturated and "off" for NEF files. The newer version of ACR looks better than the older version as far as noise is concerned (but colors are still off), but I have not done head to head testing with D200 files to confirm this and I am not interested in spending hundreds of dollars to upgrade the CS3 or purchase Lightroom. The colors from the Canon 5D files look the same in ACR as they do in Capture One, but the Nikon files in Capture One also look like the 5D files -- so there is something wrong with the way ACR is handling NEF files.

[DB_Gallery]

Man, I am sure glad it is not me doing all this testing, I would never get any real shooting

done if I did do it.

[tonybeach]

Hi Daniel,

 

I've been home sick with a severe cold lately, and its been raining and cloudy here. I did get this flower shot today (I didn't have the energy to do much else).

 

I'm noticing the 16 bit pipeline on the D300 really seems to be devoid of the posterization I would sometimes observe in D200 images. It's not something I can easily demonstrate, more an intangible quality to the images, but if I ever see two images (12 bit versus 16 bit pipeline), I'll be sure to start a thread on that topic too. I'm sure if I ever do start a thread on that there will be those that will take issue and argue that I did something wrong and that my observations are not valid.

 

Lest you or anyone else think I never actually use my camera, here's one I got recently that I think you will like: http://photos.imageevent.com/tonybeach/mypicturesfolder/december2007/huge/_AB01704_2.jpg<div></div>