Jump to content

Response on my post comparing sensors


brent_bennett

Recommended Posts

<p>To all who responded on my question about sensor quality of Canon vs. Pentax, Sony and Nikon:<br>

Thank you for your responses. I didn't expect to have so many make negative comments about DxOMark, especially when I didn't mention it by name. I think the best response from my point of view is the one from<br>

Laurentiu. Those who bash the DxOMark testing have not, to my knowledge, suggested an alternative. Does anyone know of any other testing organization that attempts to make similar tests, but does so with more general approval? <br>

I have never appreciated comments like "tack sharp". I am more analytical by nature, so the DxOMark service appeals to me. And I look at lens tests to make choices on lenses.<br>

I am fully aware that there are many other feature issues that must be considered in making a decision on which camera to buy. I started with the sensor issue, because as a film photographer for several decades, it is the most obvious feature that I look at. I have never owned or used a digital camera! But I plan on buying one this year. I will be using it for pictures of people and some use in astrophotography. Perhaps eventually using it for landscape as well. Of course I should buy a full-frame or larger, but I don't wish to spend the required higher cost at this time.<br>

I do landscape photography in prints as large as 24x36. With film cameras from 35mm to 4x5, I have never had to choose a camera based on sensor limits. So this is a new experience for me. It is from that perspective that I asked the question.<br>

Maybe I should start with a cell phone camera! (just joking)<br>

Again, thank you all for your thoughts. </p>

 

Link to comment
Share on other sites

<blockquote>

<p>I do landscape photography in prints as large as 24x36. With film cameras from 35mm to 4x5, I have never had to choose a camera based on sensor limits. So this is a new experience for me. It is from that perspective that I asked the question.</p>

</blockquote>

<p>It's good that you've told us that landscape photography is your primary application, Brent. But, since that's the case, why are you considering a crop body at all?</p>

<p>Just pick up a 5DII, and stop worrying about whether the Pentax sensor may or may not have lower noise at high ISO's than the latest Canon crop sensor. There are DSLR's that have (marginally) better high ISO performance than the 5DII, and there are bodies with (marginally) more resolution, but none combines low light capability with high resolution as well as the 5DII does. Just ask experienced landscape photographers (such as photo.net member G Dan Mitchell) which body they use.</p>

Link to comment
Share on other sites

<p>As mark says - for landscape go full frame. This gives better quality, great high ISO performance (although I rarely find I need it for landscapes) and the best range of wide angle lenses. Going full frame gives you basically two choices - Canon or Nikon and either will serve you well. Sony makes full frame bodies but they have a limited lens range. Thus the best option is probably a 5DII or D700<br>

A camera like the 5DII with some good glass (you should consider the Canon T/S lenses like the 17F4 or 24 F3.5 II) will serve you well. You should also consider using your MF lenses with an adaptor (especially a tilt / shift one). This is a low cost way to get great results and a good lens selection.<br>

You suggest that you cannot afford to go full frame but the difference is not that big and you should consider waiting until you can. I have a 7D body and it is a fine camera but I rarely shoot landscapes with it as I find it a less satisfying experience. <br>

I read your DXO mark post on the K5 but wonder about their tests. I will post a chart to show my issue - it compares the S/N ratio of the K5 with the D3X and 5DII. Up to 1600 ISO the K5 clearly trails but beyond this it improves to be as good or better than the other two bodies. However, coincident with this step increase in performance the data on the K5 changes from actual to smoothed. I am not sure of their methodology but a change in data interpretation coupled with a change in performance does look suspect. You similar with Tonal Range and colour sensitivity. While I have no doubt that DXO Mark is objective the K5 does seem to benefit from some unusual testing approach compared to the other bodies. I also find that their tests do not tell you anything about the image quality - just the performance of the sensor.<br>

http://dxomark.com/index.php/Camera-Sensor/Compare/Compare-sensors/(appareil1)/676|0/(appareil2)/485|0/(appareil3)/483|0/(onglet)/0/(brand)/Pentax/(brand2)/Nikon/(brand3)/Canon</p>

Link to comment
Share on other sites

<p>DXO isn't exactly wrong about stuff, but you have to understand the methodology and how they come up with their rankings. Just because a number is assigned to something and the number is higher than the number assigned to some other thing does not automatically make it better. A problem is that quite a few people understandably simply want to have a value that ranks things in order of good, better, best - but it isn't at all that simple.</p>

<p>In this case, especially if the OP wants to print landscape photographs at 24" x 36" on a regular basis, the "best" choice probably comes from among one of the high MP full frame bodies from Canon, Nikon, or Sony. (Here I presume that the OP is not in the market for digital MF or digital mini MF systems.) With any of these systems excellent print quality is available and things like noise are basically not an issue for this type of work.</p>

<p>Dan</p>

Link to comment
Share on other sites

<p>DxOMark is pretty upfront about the limitations of what they gather and how to use their data (and not use their data). They also are pretty clear that just looking at sensor data is not "the" factor in making a decision on a camera:</p>

<p>http://dxomark.com/index.php/Learn-more/DxOMark-scores/Sensor-scores</p>

<p>Unfortunately, they set themselves up because these stated limitations of their data are buried in their note in this link and the their online comparison tools allow comparisons way outside the realms of what even DxOMark recommends for a good comparison.</p>

Link to comment
Share on other sites

<p>At the end of the day it's all about the image you want. For most users, the differences between any of the DSLR's, regardless of whether its from Canon, Nikon, Pentax. Sony,etc. is negligible. For the pros, getting the right shot is less about which sensor to use but more about having the right gear and using light effectively. If you sell your photos, no one is going to ask what camera you used.</p>
Link to comment
Share on other sites

<blockquote>

<p>If you sell your photos, no one is going to ask what camera you used.</p>

</blockquote>

<p>Good point by Alex<br>

-<br>

-</p>

<blockquote>

<p>What is their methodology for determining the scores? Is it actually published anywhere?</p>

</blockquote>

<p>They don't publish much directly. Here is an example: http://dxomark.com/index.php/en/Our-publications/DxOMark-Insights/Detailed-computation-of-DxOMark-Sensor-normalization</p>

<p>They do sell their evaluation system and offer to provide details about it. Here is a link to their product offering:<br>

http://www.dxo.com/intl/image_quality/dxo_analyzer/analyzer_overview</p>

Link to comment
Share on other sites

<p><em>"If you sell your photos, no one is going to ask what camera you used."</em></p>

<p>A nice idea, but not accurate. Many image buyers have equipment specific criteria. I have run into sensor size, format and MP count rejections.</p>

<p>For the vast majority of users there is virtually no difference between equal generation equipment, and only small differences that are easily adjusted for in post between subsequent generations.</p>

<p>For somebody wanting to print landscapes regularly to 24" x 36", the very smallest format to be considering is a 135 format digital. Crop cameras, despite the earnest comments some might make, just doesn't cut it.</p>

Link to comment
Share on other sites

<p>Scott, there are a very small number of buyers/licensors who will specify a MP dimension (e.g. "12MP file") and an even smaller number who will specify a specific format, but these are a very small percentage of the sales opportunities. I can't say that I have encountered a buyer/licensor who wanted to know what brand of camera I used.</p>

<p>The real question turns out to be "will this work for the use I have planned?"</p>

<p>Regarding the DXO methodology, I haven't checked since their first round of rankings, what, a couple years back. I recall a post at Luminous Landscape that went into some detail about what the rankings actually meant, though I have to say I have not looked since then. </p>

<p>Dan</p>

Link to comment
Share on other sites

<p>Brent: The best "testing organization" is you... High(er) end equipment is easy to rent and that's the route a thinking person should take. Take the cost of equipment as a rough guide (i.e. more expensive = better, however you define the word "better"), shell out a few clams for rental and test a variety of rigs in your price range. You may need to learn a bit of post processing (easier, less involving and less time consuming than with film but still...) so you can draw meaningful conclusions. And perhaps 35 mm digital is not for you and you'll choose MF digital but you'll never know unless you test.</p>

<p>As an aside, among many cameras I shoot with, and I can use any camera in existence, Canon 1Ds Mk. III is a standout. Not something that DxO agrees with...</p>

Link to comment
Share on other sites

I find DxO to be somewhat useless for a lot of the reasons stated above. I find sites that post consistent photo test

subjects to be a much better guide, I can compare somewhat similar photos and see a difference or not. Photography

is a complex subject when you get down to the pixel level. Any quantitive number you assign is at best a single point

along a continuum, some times in several dimensions. IMO any averaing/summation/analysis introduces subjectivism

that may not relate to my needs or values. So, yes, a number is great to have, but after using them for a while you

see how useless they are.

 

Full frame 35mm or MF digital. IMO MF digital is the better choice because they usually do not have AA filters over

the sensor. One day when the digital Genereations slowdown some I may purchase a MF digital. That is pretty far off

into the future give the price of MF digital.

Link to comment
Share on other sites

<blockquote>

<p>As an aside, among many cameras I shoot with, and I can use any camera in existence, Canon 1Ds Mk. III is a standout. Not something that DxO agrees with...</p>

</blockquote>

<p>It's actually in the top 10 of their ranking. I would say they agree it has a top sensor.</p>

<p>And here is the disclaimer that accompanies their sensor reviews:</p>

<blockquote>

<p>This dxomark review evaluates only the selected camera’s RAW sensor performance metrics (i.e., Color Depth, Dynamic Range, and Low-Light ISO), and should not be construed as a review of the camera’s overall performance, as it does not address such other important criteria as image signal processing, mechanical robustness, ease of use, flexibility, optics, value for money, etc. While RAW sensor performance is critically important, it is not the only factor that should be taken into consideration when choosing a digital camera.</p>

</blockquote>

Link to comment
Share on other sites

<blockquote>

<p>I will post a chart to show my issue - it compares the S/N ratio of the K5 with the D3X and 5DII. Up to 1600 ISO the K5 clearly trails but beyond this it improves to be as good or better than the other two bodies. However, coincident with this step increase in performance the data on the K5 changes from actual to smoothed. I am not sure of their methodology but a change in data interpretation coupled with a change in performance does look suspect. You similar with Tonal Range and colour sensitivity. While I have no doubt that DXO Mark is objective the K5 does seem to benefit from some unusual testing approach compared to the other bodies.<br /><a rel="nofollow" href="http://dxomark.com/index.php/Camera-Sensor/Compare/Compare-sensors/(appareil1)/676%7C0/(appareil2)/485%7C0/(appareil3)/483%7C0/(onglet)/0/(brand)/Pentax/(brand2)/Nikon/(brand3)/Canon" target="_blank">(link)</a></p>

</blockquote>

<p>Note that the main differences in the performance of those sensors are noticeable at low ISO, not in the areas where the K-5 curve is marked as smoothed:</p>

<ul>

<li>K-5 High ISO score is lower than that of the other two cameras - SNR graph is below the others</li>

<li>D3x has highest color depth - its graph is over that of the other two cameras</li>

<li>5DII dynamic range is lower than that of the other two cameras - its DR graph is below others</li>

</ul>

<p>Everything is consistent. Notice how the graphs get closer at higher ISO - the large differences are at base ISO.</p>

<p>As for what is the deal with the smoothed portions of the Pentax graphs - they are explained here: <a href="http://www.dxomark.com/index.php/Our-publications/DxOMark-reviews/DxOMark-review-for-Pentax-cameras">http://www.dxomark.com/index.php/Our-publications/DxOMark-reviews/DxOMark-review-for-Pentax-cameras</a>. So, the smoothed curve just indicates those areas in which dxomark has detected a level of noise reduction present even in RAW files (normally, NR occurs during RAW processing). So instead of measuring IQ with that NR effect, they used some interpolation to estimate the performance if that NR wouldn't be present. But it doesn't really matter, because like I said, the score of the sensor is predominantly determined by performance at low ISO. You're not going to see a camera with a high score in a dimension where its graph is trailing at base ISO.</p>

<blockquote>

<p>I also find that their tests do not tell you anything about the image quality - just the performance of the sensor.</p>

</blockquote>

<p>I'm not sure what you mean by IQ, because dynamic range, color depth, and SNR are big parts of IQ and DxO makes it clear in their disclaimer that they only test the sensor and that's only one aspect of getting a camera. As for sharpness, chromatic aberrations, distortion, vignetting, etc - those are covered in their lens reviews. Really, it's hard to be more thorough than these guys.</p>

<p>As for the original question on what camera to get, I would check sensors (dynamic range score for landscape photography), lens lineups, camera features - the whole deal. For example, K-5 has great DR, but probably you can get a wider choice of wide angle lenses on a FF camera, so now you should guide your search within the realm of the FF cameras and the brands that offer you the focal lengths that you want. But the scores can help guide your search and narrow down options, even if they don't outright pinpoint the most suitable camera.</p>

<p>One important thing that should be obvious, but maybe it's not: dxomark scores are only useful if you intend to shoot RAW. For JPEG shooters, they are truly irrelevant.</p>

Link to comment
Share on other sites

<blockquote>

<p>It's actually in the top 10 of their ranking. I would say they agree it has a top sensor.</p>

</blockquote>

<p>Another way of looking a it is that 1Ds3 is behind all three Nikon D3 models, its only competition. (Which to me, having used all 4 cameras, seems ludicrous.)</p>

Link to comment
Share on other sites

Laurentiu. You are obviously a big believer in DXO mark so perhaps you can explain the following:

 

DXO mark rates the K5 above the 5D II at almost every ISO yet they claim the 5DI can shoot sports at ISO 1815 while

the K7 can only shoot at 1162. Leaving aside the fact that this is obviously a calculation (unless you use ISO 162) I

am baffled how the lower noise sensor is over 1/2 a stop worse.

 

Most reviews rate the 5D II above the K5 even the lab tests, for example:

 

Popular photography tested the 5D II with higher resolution, better colour accuracy, and superior noise performance at

the tested ISOs

 

Dpreview staters that the K5 and 7D noise performance is similar and if you compare the coin tests between the K5

and 5DII (e.g. ISO100, RAW and no noise reduction the 5D II crop is clearly better)

 

I have not seen any landscape shots published from the K5 but I have seen hundreds from the inferior 5D II and 1Ds

III

Link to comment
Share on other sites

<blockquote>

<p>DXO mark rates the K5 above the 5D II at almost every ISO yet they claim the 5DI can shoot sports at ISO 1815 while the K7 can only shoot at 1162. Leaving aside the fact that this is obviously a calculation (unless you use ISO 162)</p>

</blockquote>

<p>K-5 high ISO score is 1162. 5DII score is 1815. But a higher score indicates a better performance, not the other way around. If you look at the SNR graph, the K-5 graph is below the graph of the FF cameras (which means it's worse - low SNR means there's less signal noise ratio, i.e. there is more noise). So DxO measures the 5DII as being superior in high ISO performance, just like the other reviewers you mentioned - I see no discrepancy here - their conclusions support each other.</p>

 

<blockquote>

<p>I am baffled how the lower noise sensor is over 1/2 a stop worse.</p>

</blockquote>

<p>1162 doesn't indicate lower noise - in this test higher numbers indicate better high ISO performance, so the 5DII number is better. Read the description of the scores here:<br>

<a href="http://dxomark.com/index.php/Learn-more/DxOMark-scores/Sensor-scores/Use-Case-Scores#Sports">http://dxomark.com/index.php/Learn-more/DxOMark-scores/Sensor-scores/Use-Case-Scores#Sports</a></p>

<p>I posted this explanation of the high iso score in the other thread - it's the highest ISO value at which SNR, color depth, and dynamic range are still above a bar picked by DxO as indicating decent IQ. The bar is the same for every camera - the 5DII can be pushed to higher ISO while the K-5 drops below that bar earlier at 1162 ISO. Does it make sense now?</p>

<p>Also note in the SNR graph that the part where the graphs are similar happens after the SNR dropped under 30db - below the bar, where it doesn't influence this score anymore. I also just noticed now a neat feature in the DxO graph - hover the cursor over the right side, over the green-red bar and you will see an image that simulates the respective SNR, to give you an idea of what noise an SNR number corresponds to. It's not an actual test shot, just a helper to put the SNR numbers into context. They have this for the other graphs too - you may need to hover the cursor over a graph first, to bring the green-red bar.</p>

<p>On the other hand, the K-5 scores higher in dynamic range and this is the reason it ends up getting a higher aggregated score too - you can argue that the aggregation is meaningless, but they had to make one to come out with a single number - this is why you need to look at individual scores carefully. Incidentally, the higher dynamic range of the K-5 is also supported by the dpreview reviews:<br>

<a href="http://www.dpreview.com/reviews/canoneos5dmarkii/page25.asp">http://www.dpreview.com/reviews/canoneos5dmarkii/page25.asp</a><br>

<a href="http://www.dpreview.com/reviews/pentaxk5/page14.asp">http://www.dpreview.com/reviews/pentaxk5/page14.asp</a><br>

Except the results are not that easy to read - but look at those scales where they compare the DR. The 5DII is in between approximatively 32.5 and 6.5; the K-5 is between something like 33.5 and 5.3, indicating a wider range. So again, no discrepancy. </p>

<p>Now, take a look again at the graphs for the DR and color sensitivity and hover the mouse over that bar to see how different an image would look for a difference of 1EV, or 1bit of color depth - notice how subtle the changes are there. No wonder you can't see them simply by looking at test shots.</p>

Link to comment
Share on other sites

<p>Let me just say that it is possible to over-think this stuff, perhaps even possible to <em>way</em> over-think it.</p>

<p>I recall reading a wonderful post a Luminous Landscape a year or so ago - perhaps even related to one of the earlier DXO test postings - in which the author pointed out that every one of the cameras in the category under consideration (full frame 20+ MP DSLRs) was capable of producing truly outstanding results. Once you "get" that idea, the whole concern about which body rates .1 higher than which other body on whatever scale you happen to come across somewhere becomes much less significant, if not completely insignificant.</p>

<p>This is kind of like thinking about going on a very expensive vacation to a very wonderful place and finding three that you think would really be outstanding five-star experiences. You know that any one of the three would produce something truly wonderful. Yet... you have heard that the comforters on the beds at one of them tend to be 1mm thicker than at the others... and you sometimes do worry about getting chilled... and you are looking for something, anything to help you decide which of the three to select. So you end up selecting on the basis of some "difference" that is so small as to be completely inconsequential, meanwhile congratulating yourself on your ability to make logical, data-driven decisions.</p>

<p>There are other far more significant differences among cameras in a given class that have the potential to affect your photography in more significant ways than what the DXO test number is.</p>

<p>Dan</p>

Link to comment
Share on other sites

<p>While reading this thread it becomes obvious many posters come at this from their own relatively narrow viewpoint in terms of arguments pro and con regarding the 5D2 and the K-5, and DxOMark testing.</p>

<p>I'm no exception but I would offer the following:</p>

<p>To say that any serious landscape photographer should only be shooting FF is pretty much nonsense or to claim that buyers will reject a print because of the hardware used to capture it. I've sold hundreds of images that I've captured on everything from 4x5s to an Olympus C-5050. You go with what works the best for a given situation or you do the best job of image capture with what you have to work with.</p>

<p>FWIW, I have personally decided that I wanted more than my 5D2 could deliver and I've moved on to a Pentax 645D. At that point I also decided my Rebel backup system should go as well and I purchased a K-5 so that I could use my 645/67 MF lenses with my K-5. Does that mean the 5D2 and Rebel are 'crap' all of a sudden? Of course not but the K-5 does offer a couple of capabilities for some of my shooting needs that are clearly superior to the 5D2.</p>

<p>The threads on Photo.net seem full of posts that seem full of angst and lack of civility. We should all be out taking pictures.</p>

Link to comment
Share on other sites

<p>Charles,</p>

<p>I hope you are not referring to me when you say <em>"To say that any serious landscape photographer should only be shooting FF is pretty much nonsense"</em>. My comment specifically related to printing landscape images at 24" x 36". I am sorry but whilst you can print any sensor size to any output size, most serious large print sellers would agree that the 135 format is an absolute minimum for regular 24" x 36" sale quality prints, I am very rarely happy with such large prints from my FF camera.</p>

<p>Superb landscape images can be shot with much smaller sensors than a FF one, but to pretend you can do it, and print to that size regularly, is, for a MF digital user, disingenuous.</p>

<p> </p>

Link to comment
Share on other sites

<blockquote>

<p>Once you "get" that idea, the whole concern about which body rates .1 higher than which other body on whatever scale you happen to come across somewhere becomes much less significant, if not completely insignificant.</p>

</blockquote>

<p>Except when one rates twice better. Some differences you cannot afford to ignore.</p>

Link to comment
Share on other sites

<blockquote>

<p><em>Except when one rates twice better. Some differences you cannot afford to ignore.</em></p>

</blockquote>

<p>No full frame DSLR rates remotely close to "twice better" than the competition. If you think that data support that, you are misreading the data. In addition, you are likely misapplying the data - unless we can also see that photographs are "twice better" when made with the higher rating camera.</p>

<p>And, Charles Wood, I like your post. It reflects the real world, not <em>Imaginary Forum World</em>. And, to Scott Ferris, my view regarding the use of cropped sensor versus full-frame bodies for 24x36 prints of typical landscape subjects is in line with yours - here I think that full frame is likely the better choice.</p>

<p>Dan</p>

Link to comment
Share on other sites

<p>Scott,</p>

<p>I can't agree with you that my prior post was disingenuous. Over the years I have always attempted to maximize the quality of my images based on my skills and equipment at hand and it has been an ongoing learning process. I have developed my own style largely centered around large mural size southwestern US landscapes. Solutions that work well for me may not be particularly well suited for others.</p>

<p>I did fail to point out, as others have many times in the past, that it is possible to create critically sharp large prints with small sensor cameras if one has a shooting situation that lends itself to stitching multiple frames. I don't disagree in general with your comments regarding single frame print sizes but the actual size limitation is more often than not based on actual viewing distance. Customers generally are not pixel peepers. I display a number of single frame images taken with 8-10 meg APS-C cameras from several years ago and when a customer asks if it is possible to print larger, the first thing I ask and qualify is their typical viewing distance. I then go on to explain perceived sharpness vs viewing distance. Across a typical room it is usually very difficult to judge whether a 20"x30" print was shot with an APS or FF sensor.</p>

<p>I do display and sell large prints, some 3'x9' that I originally captured with 617 format film. To create images of those dimensions with digital can be a bit daunting whether shooting multiple frames with APS or FF 135mm sensors. Digital MF makes it a bit easier.</p>

<p> </p>

Link to comment
Share on other sites

<blockquote>

<p>Those who bash the DxOMark testing have not, to my knowledge, suggested an alternative.</p>

 

</blockquote>

<p>I know a man who tests the vocabulary and literary skills of fence posts by simply throwing a bag of flour at a wall to see what shapes appear. But hey, his findings must be correct because nobody has suggested an alternative.</p>

Link to comment
Share on other sites

<p>I wonder often about people who remark in these threads. I know some of you print large, but it you don't, it doesn't matter! I doubt half of you responding print large.</p>

<p>Last weekend I printed a dozen Super-B prints (13x19 inch, full page) at home with my now old Epson R2400. WONDERFUL output even a couple at ISO 1600 and 3200! The satisfaction you get yourself by DIY is pretty high. It's even better with a good glass or two of wine while you work.<br>

Equipment: EOS 7D with **good** lenses. I wish I had a newer printer that could tackle a 24x36 inch print! </p>

<p>Ilford Galerie paper: smooth HW matte at 200 gsm (double sided, but I use an ink pen to record details on the reverse side).</p>

<p>I cannot think of one print in that bunch that would've improved even 1% with a "better camera or lens". They are Super Sharp and pleasing. Don't worry about the labs like DxO. DIY and examine your technique.</p>

<p><strong>GUESS WHAT?</strong> It's 2011 now and Canon, Nikon, Epson and similar have got it down. You can't lose. You simply can't.</p>

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...