Methodological Validity of Comparisons of Canon 5D II and Nikon D700

<p>When DPREVIEW.COM does its comparisons of the resolutions of DSLRs, does it err in presenting comparisons of the resolving power of cameras with sensors with vastly differing numbers of megapixels?</p>

<p>Specifically, since (for example) the Canon 5D Mark II and the Nikon D700 give files of such vastly differing sizes, can one validly disregard or omit downsampled images in the comparisons?</p>

<p>The problem is not unique to DPREVIEW.COM, of course. To avoid copyright issues, here is the page that I am drawing from in the images that I will be attaching below:</p>

<p>http://www.dpreview.com/reviews/canoneos5Dmarkii/page38.asp</p>

<p>I am presuming here that in "journal articles" (and our forums are essentially brief online journal articles) one may use small images without violating fair use provisions. In any case, as I have said, above is the link to DPREVIEW.COM so that persons can look at their pictures (without doctoring such as I have done below) and try to answer the question for themselves. In any case, I have in the comparison below used one sample image directly from that site, and one sample image which has been downsized to approximate files of the same size. The downsizing of the Canon image is approximate, but the trees are close enough in size to get a pretty good approximation. The native noiselessness of the D700 is not being disputed here, and the effects of noise reduction "smearing" on the Canon image are likewise obvious.</p>

<p>Still, the practical questions remain: (1) Are such comparisons without downsized images valid? (2) Are they all that useful?</p>

<p>The core of the problem from a practical perspective is that, without downsampling or downsizing images of differing sizes it is very difficult to tell whether or not claims about superior resolution make very much sense.</p>

<p>I have chosen two images, both shot at quite high ISO: 12,800. The reason for this particular comparison is that ISO 25,600 produces images in both cameras that are essentially worthless, whereas images shot at ISO 6400 are reasonably clean for both cameras. The question of usability (and thus of making valid comparisons) is most obvious to me in that problematic area of marginally useful ISO, in this case 12,800 for these two particular cameras.</p>

<p>I will post the Canon image first, downsized so that the tree is about the same size as the tree in the Nikon image.</p>

<p>--Lannie</p><div></div>

<p>Here is the corresponding Nikon D700 image, cropped but without downsizing.</p>

<p>--Lannie</p><div></div>

<p>CORRECTION: Both images have been cropped, with only the Canon image also being downsized.</p>

<p>Please see the DPREVIEW site for the actual comparisons of the completely unmanipulated images with attendant discussion:</p>

<p>http://www.dpreview.com/reviews/canoneos5Dmarkii/page38.asp</p>

<p>--Lannie</p>

<p>I checked this site but I think scans from a magazine page, as apparently the published photos are, cannot be used to evaluate image quality, particularly when expected differences are subtle.</p>

<p>I understand the limitiations of doing this, Vasilis, which is why I direct persons back to the DPREVIEW.COM site more than once:</p>

<p>http://www.dpreview.com/reviews/canoneos5Dmarkii/page38.asp</p>

<p>I do find the issue compelling, however, even if I don't have the images to support my position that we could make better comparisons in at least some cases with downsized images. </p>

<p>Anybody out there own both the Canon 5DII and the Nikon D700? I would like to see comparisons. I confess that the final proof would be in the prints, but I hope that the practical point of raising these questions is obvious enough.</p>

<p>--Lannie</p>

<p>One more point, Vasilis, is this: this issue is not finally a purely methodological or other theoretical issue.</p>

<p>That is, the results of such comparisons do bear upon the very practical real world issue as to whether the cleaner image of the Nikon translates into anything meaningful in reality--especially the decision as to whether one should buy the Nikon if better noise control is the driving concern (as opposed to, say, faster shooting or faster or more accurate auto-focus).</p>

<p>It seems to me that, once downsizing is taken into consideration, the advantage of shooting the D700 because of more noise-free shots at high ISO virtually (if not completely) disappears.</p>

<p>--Lannie</p>

<p>Keep in mind also that at ISO 12,800 we talking about a SEVEN-STOP ADVANTAGE in terms of light-gathering ability over ISO 100.</p>

<p>That is, if we can downsample huge files and get virtually noiselessness at 12,800, the implications for shooting in very low light without a flash are enormous.</p>

<p>I am seriously beginning to want the 5D II. I told myself for the longest time that the 5D and the 1DsII that I have could give me results that are just about as good. Well, maybe so, but "just about" is not quite there when it comes to low-light shooting.</p>

<p>I would like to see more shots made at ISO 12,800 made with the 5D II. Does anyone have any?</p>

<p>--Lannie</p>

<p>I tried ISO12800 and even <em>ISO</em> 25600 for a few night shots and I've got reasonably good results.<br>

To get the best results, I slightly overexpose (1/2 stop), and later, doing post-processing I correct the exposure,<br>

apply some noise reduction to the RAW file and finally rescale/downsize the image.</p>

<p>Hi<br>

I am not trying to advertise my site here.<br>

If you follow the link below you will see two images I snapped and downloaded from the card at 25600 ISO. No modifications.<br>

<a href="http://roy-palmer-photography.smugmug.com/gallery/8153286_dvnPv#532036052_NQiiU">http://roy-palmer-photography.smugmug.com/gallery/8153286_dvnPv#532036052_NQiiU</a><br>

If you go to the upper right hand side of the image you have an option to see the image full size straight out of the camera.<br>

I know they have no merit as images so no critique required.<br>

I will leave them there for 7 days.<br>

I hope this helps the discussion.<br>

Roy</p>

<p>I too have done a lot of comparisons using dpreview.com. These are not scans of magazines. They are studio images, and in this case we are looking at the printed labels on a beverage bottle in that scene. While not a great subject to compare, oddly enough these bottle labels are the main ones that I have used in the past as well. </p>

<p>I have no idea why dpreview insists on displaying images of different physical size when they go to the necessary trouble of using equivalent focal length lenses for their tests. An 85mm shot from two full frame bodies taken at exactly the same distance with exactly the same scene should be presented at exactly the same cropping and presentation size.</p>

<p>When I choose to make serious, image on monitor comparisons (as poor as that may be) I download their full size images, save them to my hardrive, and then align them in photoshop at the same physical size without altering resolution and file sizes etc. I crop each window to the same size and then pan around each image comparing the same details from each. I compare many of the printed labels as well as some of the objects in the scene.</p>

<p>When comparing I use both their studio scenes and their outdoor scenes to make my own reasonable conclusions. I often don't agree with their evaluation of their own images. Sometimes I wonder if they lose detail when they create their internet pages. There have been times when I wondered if they actually got the focus correct in some of their shots. </p>

<p>I also wish they would set up one studio scene and leave it that way permanently! Just take an air hose to it every once and awhile to remove the dust.</p>

<p>Sample 5D2 ISO12800 image after downsizing.</p><div></div>

<p>Well, according to some people on this forum, the 5DmkII has awful noise and apparently is inferior to 1DmkIII and 5D. Not that I agree with them. I expect that you'll be hearing from them soon, since they're so fond of trashing the 5DmkII.</p>

<p>I am very impressed with the 5D2s ISO performance. I really don't see much point in going higher then 3200 in almost any situation. </p>

<p>I haven't seen a comprehensive High ISO noise review yet. Including this one.</p>

<p>Virtually every generation of bodies has increased high ISO noise performance over it's predessor. It's rapidly becoming the single most important factor to consider when upgrading. High ISO capabilities in recent DSLRs have changed the way every one of us can and do shoot.</p>

<p>I feel that proper review of this subject should be equal in length and breadth to the review of pro lenses, including shooting in a variety of real world situations, with a subjective evaluation of the results, laboratory testing with more objective evaluation of the results, and complete comparative testing with not only competitors, but also predecessors.</p>

 

<p>I'd like to make two points. Firstly, they do compare cameras based on price, and in some markets the Canon and Nikon are close. Secondly, looking at what a camera can do at 12800 ISO is pointless, don't you think? The images are crap. How many people do that and expect good results? 3200 is a practical limit in my book, even though 12800 is only two stops faster. One can gain* one stop by using a faster or better lens, and one more stop by going to a slower speed using VR or a resting on a monopod.<br>

One of the celebrity papparazzi was asked about this in one of our magazines and he said he stays with Canon 40/50D because they are fine to 3200, relatively cheap and DX gives him extra reach. He uses the *method above if its dark.</p>

<p>Roy and Fernando, thanks for the great shots. One thing that came through for me while downloading (and then resizing) your shots is that all are potentially usable, depending on the amount of resizing one intends to do. (It would seem that some images that are not good enough for large printing might be good enough for the web--if downsized and post-processed correctly.)</p>

<p>Another thing that came through from viewing original files is just how much image degradation really does occur between ISO 12,800 and 25,600--but, in spite of that, just how potentially usable 25,600 could be. Roy, your shots are proof of that. I hope you don't mind that I downloaded them and then resized them to varying degrees to see what I could get (for my private viewing, of course). Those are very good test shots, in my opinion.</p>

<p>John, I share some of your same frustrations about the site in question, although it is surely one of the more reliable sites around, and I go to it frequently. (I knew that the photos were not scans from a magazine, but I let it go. I'm glad you didn't.)</p>

<p>Peter, I'm sure you're right about the naysayers. If this thread survives long enough, they will surely get here.</p>

<p>--Lannie</p>

<p> </p>

<blockquote>

<p>looking at what a camera can do at 12800 ISO is pointless, don't you think?</p>

</blockquote>

<p>Stephen, I generally agree with you, although I have to repeat that, at least for web posting, shooting at such high ISO could be useful--not least of all for snaps to be sent to be relatives, when a flash is such a bother and no one is going to print them at any size anyway. Even so, with Fernando's results at 12,800, I am inclined to think that 12,800 just might be useful, even when 25,600 might not be.</p>

<p>Thanks to all who have responded. I will be interested in seeing what else people have to say.</p>

<p>--Lannie</p>

<p>Images can be manipulated in so many ways using so many different programs that it makes it quite difficults to validate many of test results. While shooting at ultra high ISOs should be a last resort option, it is a viable option when necessary and both cameras can give good results with a bit of creative post processing.<br>

<em><br /> </em><br>

<em>"(1) Are such comparisons without downsized images valid? (2) Are they all that useful?" </em> Perhaps and perhaps. I have seen numerous references to images such as the ones you point to and others. Often the differences are so small that simple PP would equalize the visible differences. If, when pixel peeping, the images are very close in IQ, you can bet that you won't see any difference in normal sized prints.</p>

<p>I am a DXO user and have processed ultra high ISO images with DXO with impressive results that are far different (superior) to unprocessed images.</p>

<p>My comparison would have been better, I think, if I had brightened the downsized shot borrowed from DPREVIEW.COM:</p>

<p>http://www.dpreview.com/reviews/canoneos5dmarkii/page38.asp</p>

<p>Marcus, I also would like to see "a comprehensive High ISO noise review." No, this thread is not that, either, but perhaps someone will be inspired to do it if they think that shooting at high ISO is as potentially promising as I think that it is going to be--and increasingly already is.</p>

<p>--Lannie</p><div></div>

<p>ISO 25600 shows a lot more noise than 12800.<br>

I'm a Linux user and thus I cannot use all of the fancy software available for Windows/Mac.<br>

Fortunately I've found the UFRAW freeware program to process RAW files, including a wavelet noise filter.<br>

I shot this picture just to test ISO25600 (handheld).<br>

These pictures are far from perfect, but in some extreme cases the images can be used for some low-res applications.</p>

<div></div>

<p>Fernando, ISO 25,600, handheld at night! Wow! Sure, it has been downsized, but it is still equivalent to shooting more than five megapixels. That surely is not trashy for me--even for prints, since I have gotten some pretty good prints from five-megapixel cameras.</p>

<p>--Lannie</p>

<p>I have no expertise on these things, but I think the best way to compare cameras with different resolution is by making same sized prints. dxomark.com and imaging-resource.com do this type of comparison.</p>

<p>Lalon, if someone has done a side-by-side comparison of the 5D II and the D700 at high ISO, I have not been able to find it.</p>

<p>As a Canon user, it is nonetheless sobering for me to see what Nikon has been able to accomplish. On the one hand, one may say correctly that the D700 has less noise [in large part] because it is "only" trying to pack 12 megapixels onto the same size sensor that the 5D II has packed 21 megapixels onto. On the other hand, it is quite a shock to see that the D700 gives its admittedly awesome results without any downsizing, as these shots from a Japanese website show:</p>

<p>http://dc.watch.impress.co.jp/cda/review/2008/07/03/8782.html</p>

<p>About the only thing that I can say so far is that, even if one were to control for downsizing, the Nikon would surely still win on the high ISO/low noise tests--and shoot more frames per second and have superior autofocus performance to boot.</p>

<p>On the other side of the coin, the 5D II gives results at low light that are still rather awe-inspiring, considering that one also has this capability in a landscape/studio camera that can still blow the D700 away in deliberate, unrushed shots with plenty of light.</p>

<p>Nothing that I have said in this thread is meant to suggest that there is not a trade-off to be made, and persons are sometimes going to disagree vehemently on which attributes are most important--not necessarily because of brand loyalty, but because of the type of shooting that they do. Elliott Bernstein's surfing shots on his main page are a case in point. He shoots the D3 on those and gets the results he wants--he needs the extra frames per second and the quick AF to get those shots. Another Nikon user, however, might want the extra megapixels of the D3X--and have the bucks to make it happen.</p>

<p>Perhaps we should be calling for a side-by-side comparison of the D3X and the 5D II, or--even better--the D3X and the (surely, sometime real soon now) forthcoming Canon 1Ds Mark IV.</p>

<p>Yes, I said Mark IV. If and when it comes out, I would hope that dpreview.com, our own Bob Atkins, and others will give substantial attention to low light performance comparisons. If Sony comes out with a good high-ISO camera to follow the A900, watch out!</p>

<p>All that I can say at this point is that, for my purposes, I think that the Canon 5D would be the best mix of the available choices for me--and I am frankly in awe of the shots at high ISO that some of you are getting with it, without claiming for an instant that it can even begin to outdo the D700/D3 at high ISO. It is pretty clear that it cannot. I am still impressed, given what else it can also do.</p>

<p>I yet suspect that Nikon's in-camera processing must still be superior. That is, not all of the high ISO advantages are traceable to the respective number of megapixels on each brand's full-frame sensor. There is something else going on here that Canon needs to catch up on, in my opinion. The 5D II was a great leap forward, and it is an excellent candidate for "Best All Around DSLR"--for some of us. It will yet soon be history--and one wonders what Nikon already has in the pipeline.</p>

<p>--Lannie</p>

<p>I did a few comparison shots with the 5D Mark II. The top shot is at ISO 800, the 2nd shot is an unprocessed shot at H2 and the bottom shot is the H2 shot processed with DXO 5.3 (standard settings).</p>

<p> </p><div></div>

<p>Here is an extreme, extreme crop:</p><div></div>