How real and reliable are DXOmark scores?

[uday_maripalli]

<p>Hello all,<br>

This might have been asked before multiple times but I haven't found a solid answer in any of the previous forums. I started my research and read a lot about different Canon models and almost decided to buy a Canon 70D. As a final step, I did a quick Canon 70D vs "competition" search on youtube to see how it compares to other cameras. Many of the videos were mainly comparing it to Nikon D7100. Both of them came neck to neck in terms of features but I still liked 70D for its amazing auto focus system, other bells and whistles such as swiveling screen, wifi etc etc. But many people started comparing the "sensor quality" --> "photo quality" and Nikon came out on top.</p>

<p>I mainly use my camera for taking landscapes (just an enthusiast) and travel pictures. Nikon D7100 seemed to be better than Canon for landscapes (according to reviewers). Many youtubers quoted DXOMark scores to back it up. I went on their website to do a comparison of sensors to find out that Nikon D7100 scored more than 70D in every category. Ofcourse, these scores do not take into consideration the features like the new auto focus system etc., but they looked only at sensor quality.</p>

<p>My question is how real are these scores? So purely from image quality perspective, Nikon D7100 is better than 70D? Here is the link comparing these two cameras:<br>

<a href="http://www.dxomark.com/Cameras/Compare/Side-by-side/Canon-EOS-70D-versus-Nikon-D7100___895_865" target="_blank">http://www.dxomark.com/Cameras/Compare/Side-by-side/Canon-EOS-70D-versus-Nikon-D7100___895_865</a><br>

Also, the ratings are higher for 3rd party lenses. For example Sigma 18-35 has higher ratings for D7100 compared to 70D (although the difference is very less).</p>

<p>Is it recommended to go by these values to make a final decision?<br>

Thanks.</p>

[rick_m.]

<p>So here is the deal. I have a D7100. It is a great camera. That said....</p>

<p>You should buy the camera you want. You will not be able to see any difference in the image quality at all. Why?</p>

<p>Because you will shoot 90% of your pictures under nearly average conditions. You will not use a tripod. You will not be fanatical about your steady hold techniques. You will probably not exclusively use the very finest lenses carefully adjusted for your camera. In other words, you will live in the real world and not a laboratory.</p>

<p>The question to ask yourself is this: What will one of these cameras do well that the other won't? Nothing. If you want to know which one will take better pictures, the answer is the one with the best lens hanging on it at the moment. <br>

If you want to improve any one thing on any camera that will make the most difference in the outcomes, fix the photographer. </p>

<p>Try not to become a gearhead or pixel-peeper. Try to become a photographer....;)</p>

<p> </p>

[uday_maripalli]

<p>Ha Ha! Thank you Rick! I understand that these ratings are not end of all. But given the same lens attached to the camera, if one camera has better ratings than the other then does it not mean that camera is better than the other? <br>

I understand that you can get excellent pictures with a low end camera depending on the photographer. But my question was, all other things being equal (photographer, subject, lens attached, lighting etc), is D7100 better than 70D? DXOmark seems to conclude so. Wanted to know if it was right. </p>

[bobatkins]

<p>It's better if you define better as having a higher DXOmark rating.</p>

<p>Whether it's "better" in any practical sense is debatable.</p>

[uday_maripalli]

<p>Thanks Bob. That is what I was looking for. If Higher DXOmark ratings would mean anything in practical sense. </p>

[Matt Laur]

<p>Whatever nit-picking one might do between the two cameras in question (from a DXO-ish point of view), those difference pale compared to the difference in ergonomics and user interface. <br /><br />I'm a Nikon guy because I've been one for decades. I pick up one of their bodies, and it just fits my hands and everything is pretty much where it's supposed to be, doing what I expect it to do. Hand me Canon's finest body, and I'll be a clumsy fool missing focus, bumbling shots, and altogether failing to take advantage of that Canon's fine capabilities because it just doesn't feel right in my hands, and the menu system seems wrong to me. <br /><br />You need to physically hold both bodies, or at least any two similarly-classed bodies from those manufacturers so that you can get a sense of how each company thinks you should hold, use, and look through an SLR style body. The impact of those details will have more to do with your end results and the pleasure you get out of achieving them than will any ten of the variables about which DXO frets their lab-coated selves when they run their somewhat capricious tests.</p>

[Wouter Willemse]

<p>What Matt said - handling is a bigger deal than "200 DxOMarks" more or less. Those lab values are not the real world.<br>

There are some aspects, though. First: how important video is; you quote the amazing AF of the 70D - the D7100 has Nikon's top AF, which is no slouch in any way. Except that the 70D has a pretty clear advantage for AF during video; on the D7100 this is a bit more an afterthought. For still photography, I wouldn't be surprised to see the D7100 come out on top (but it's very hard to compare/measure).<br>

Second: which lenses do you think you need, and which brand has the best options in that respect? It's quite easy to say: "how do bodies compare, assuming the same lens", reality is most of the time the lenses are not identical. And lenses tend to have more visible impact on an image, or on your bank account, than miniscule differences in the sensor.</p>

<p>Otherwise, these bodies are so close to one another that the choice really should come down how things feel in your hands.</p>

[rick_m.]

<blockquote>

<p>I understand that you can get excellent pictures with a low end camera <a id="itxthook5" href="/canon-eos-digital-camera-forum/00cvQA?unified_p=1" rel="nofollow">depending on<img id="itxthook5icon" src="http://images.intellitxt.com/ast/adTypes/icon1.png" alt="" /></a> the photographer. But my question was, all other things being equal (photographer, subject, lens attached, lighting etc), is D7100 better than 70D? DXOmark seems to conclude so. Wanted to know if it was right.</p>

</blockquote>

<p>Neither the D7100 or 70D are "low end" cameras. The problem with these nit-picking details is that "all things are NEVER equal".</p>

<p>But you insist on pole-vaulting over mouse droppings. Fine. The D7100 is, in most measurable criteria, a "better" camera than the 70D. So even if you have a room full of Canon lenses and have become used to Canon's interface, you should instantly sell all of that stuff for whatever you can get and go for the D7100. What if people found out you were using s substandard camera like the 70D? Horrors!</p>

<p>What ever you do, don't look at the 7DMkII. Just don't look.</p>

[Spearhead]

<p>The idea that a single number, or even a bunch of numbers, has any bearing on what kind of photos someone will take is ridiculous. Unfortunately, it's part of a whole culture that has grown up around numbers in place of any serious thought.<br>

<br />As an example with the two cameras referenced here...I bought a 70D a while back. I wasn't considering a Nikon camera, but I was considering another Canon model. The reason I chose it was simple - it has an articulated screen. I shoot in situations where I have to shoot over my head. Sometimes I am being paid and I really need to get the shot. Before the 70D, I would go as wide as possible, hold the camera over my head, and shoot with no way to tell if I was getting what I needed except to look after the shot. A little late... So what I got with the 70D is something that no number can fix.</p>

<p>There's a good analogy with wine. Wine seems to be sold in a lot of places on a single number from one of a couple sources. People will buy blindly based on the number. Then the $50 wine with the high rating is drunk with ribs dipped in sweet barbecue sauce. The right $10 wine would taste better in that situation and save $40.</p>

<p>So stop looking at numbers and ratings and look for the features and quality that <em>you</em> need. The information is out there in detailed reviews and in places like photo.net. Sure it takes more time and effort, but in the end, you will make an informed choice.</p>

[DickArnold]

I think everyone is making sense here. Go over to that other site where they constantly fight about how many

angels can dance on that pin or how much better is 2db dynamic range one over the other in deep shadow when pushed four stops. I just did the best NE fall series I've ever done with a "lowly" 70D. I will take that

same camera to a five hundred person, three day swim meet soon and it will be more than adequate. I like the touch screen and the full axis tilting screen as well. To tell the truth I could and have done all this with several earlier Canon bodies

over the years. The new cameras have more flexibility and better focusing but DSLRs have been able to make really good pictures for a long time. DXO, IMO is about numbers, not pictures.<div></div>

[uday_maripalli]

<p>Thank you all for your responses. Now I am convinced that its not intelligent to make a decision purely based on DXOmark scores. Other factors considered, both these cameras might end up with same image quality eventually.</p>

<p>Matt and Wouter: I think you guys have a very valid point which I ignored. I am used to using Canon cameras but never handled Nikon before. I assumed that I would eventually get used if I get a new system. Its probably a good idea to go to Best Buy to check them out.</p>

<p>Rick: You might be rich enough to consider these things as mouse droppings. For me it is a BIG deal investing $1500 to $2000 for a "hobby". New camera and lenses are not something I would buy/change every year. I wanted to do my due diligence so I would not have any regrets in the future..... and I would like to have the best product for the money I spend. For a noob like me the only sources of seeing practical information is via reviewers (websites and youtube, DXOmark being one of them) and awesome photo communities like this one. I can get all the "technical" numbers from online sources but only a community like this can give me some practical advice. And I never remember mentioning that I was heavily invested in any one of the systems.</p>

<p>Jeff and Dick: Thanks for confirming that DXO scores are just one part of the equation. I did like the swiveling screen and the auto focus system in Canon 70D. Although I never do videos or sports photography, these are still pretty useful during travel and parties.</p>

<p>Thank you all. I got so much engrossed into reviews and articles that I was up all night reading. Now I am interested even in Sony A6000 mirror less camera as it seems to have a really good sensor with some awesome features. So I am back to comparing DSLR vs compact mirrorless cameras. That might be a discussion for a different thread though.</p>

<p>Again, I would like to thank all of you for your valuable time and advice. I now have a much better frame of mind compared to when I started this thread.</p>

[bobatkins]

<p>You can think of numerical ratings like those published by DXOmark as being analogous to topspeed, acceleration, max cornering Gs, engine power and torque etc. when it comes to cars. Small differences in these numbers are in no way a predictor of which car you will like best. They do give you some information, but it's a small part of the overall package. Large differences in these numbers probably would be significant, assuming whatever was being measured was something you really needed. The difference between a car with a top speed of 183mph and 197mph would be of little significance unless you were racing it, even though 197mph is "better". On the other hand if the top speed was 40mph, then that would be a factor even if you only drove to the grocery store.</p>

[Wouter Willemse]

<p>Uday, just to reiterate one point: do not focus only on the body; without decent lenses, any discussion on sensors becomes a matter of tackling the wrong bottleneck. Also budget-wise, consider lenses as being at least as important (and at least as costly). I'd rather have a 700D or Nikon D5200 with good lenses than a 70D or D7100 with only one kitlens (which will be slow enough to autofocus to render the advantages the 70D/D7100 have a completely moot point).<br>

Think about the system you're getting into; the camera is just part of the equation.</p>

[rick_m.]

<blockquote>

<p>Rick: You might be rich enough to consider these things as mouse droppings. For me it is a BIG deal investing $1500 to $2000 for a "hobby". New camera and lenses are not something I would buy/change every year.</p>

 

</blockquote>

<p>And you think investing that kind of money is something professional photographers do at the drop of a hat? You came here looking for advice. We all unambiguously told you that there was no practical difference between the two cameras image quality wise. Every one of us. You persisted in making it an issue.</p>

<p>Bob Adkins summed it up. The differences between good cameras lie at the extremes. As I said in my first post, you are not shooting at the extremes. Either camera would do well. </p>

<p>It appears that you have not actually held either of these cameras. You said that you have been "using Canon". Let me ask you...Have you had a DSLR before? If so which one? What lenses do you already have? I ask because it very well may be that if you are on your first DSLR, neither of these cameras may be your best choice. Especially if, as you seem to assert, money is an important consideration. So why not let us help you better. Answer those questions and see where everyone goes. </p>

[uday_maripalli]

<p>Rick,<br>

I never made an issue. It is clear AFTER every one replied that DXO scores make very little difference. That was my question to begin with. So now I agree with everyone who said so.<br>

You are entitled to your opinion but you need not be condescending. It would have been better if you replied straight to my question rather than writing useless stuff (like "mouse dropings"). It would have been more helpful if you had contributed something substantial rather than rambling. What would I get by making an issue? <br>

I simply wanted to know if one was clearly better than the other as mentioned by a professional website. If I knew everything I would never have started this thread. <br>

Thank you for your time.</p>

[mike_halliwell]

<p>About the only variable <strong><em>not</em> </strong>open to debate is fps. Does it or does it not take more pictures per second.</p>

<p>The 7D MK II goes way, way faster than a D7100. If you need speed, Canon's your body of choice.</p>

<p>I'm starting to waver...:-)</p>

[pics]

<p>DXO mark is just a means to evaluate one particular component/aspect of a camera. For that reason it is a valuable tool for someone looking to make an informed purchasing decision without relying strictly on the manufacturers spec sheet. Of course it is all relative to what is important to you when buying a camera. One person may want the best low-light performance, another individual prioritizes resolution, another FPS, and someone else a wants particular ergonomic design. I find the site useful for what it is designed to do, evaluate sensors in as neutral a way as possible given the endless amount of variables. Given that I use an ancient camera by today's standards, it isn't the be all/end all benchmark for me, but it's a nice database to have access to.</p>

[dcstep]

<p>I've found direct correlation between the DxO DR ratings and actual performance of my Canon 7D, 5D II and 5DIII. That said, I just bought a 7D MkII and could care less how its performance compares in the DxO comparison, because I'm already happy with what it does in my hands and with my PP.</p>

[kerry_grim]

It would be interesting to take both cameras and then test them the way you intend to use the camera and then try to guess which camera took which photos. I would be willing to bet it could not be done. The fell of the camera is more important.

 

Each person has different needs. I could give a rats ass about whether the screen swivels on the 70D, but to the next person that could be an important feature. I would be more concerned about the viewfinder and wonder how it compares to my ancient 40D.

[hjoseph7]

<p>In general DXO usually scores Carl Zeiss lenses on top, followed by Nikon, then Canon. They are pretty brutal when it comes to Pentax lenses, but I have not found this in real life. Most people judge pictures by their content, aesthetics, technique, emotion, etc., not on a bunch of numbers. </p>

[ilkka_nissila]

<p>I think DXOMark's <em>sensor</em> measurements reasonably accurately portray the quality of the raw data, but in some cases the results can be biased if the raw data has been "doctored" by the camera's signal processor, or in some medium format cameras there can be patterns of noise which smaller format cameras subtract before writing the data in the file, but these cameras expect the subtraction to occur in raw conversion. However, overall I think their sensor measurement graphs give a good indication of what to expect. The overall score can be misleading as it seems to be heavily weighted to reflect base ISO performance, which may or may not be your primary interest, depending on subject matter and shooting style. But the graphs seem to be reflective of reality for the most part.</p>

<p>I find however that their <em>lens</em> testing and especially scoring have problems and the scores certainly do not consistently reflect image quality as perceived by a real world observer. I disregard them except in case no other tests are available on a specific lens. I generally prefer reviews where there is both formal testing, but also real world subjects photographed in the field of expertise of the photographer who carries out the testing. </p>

[Marcus Ian]

<p>You did mention that you will use the camera primarily for landscapes (amateur) and travel pictures.<br>

<em>If</em> you were shooting landscapes <em>professionally</em>, a difference in DR of 13.6 vs 11.6 could <em>easily</em> be <em>critically</em> important. </p>

<p>Bob Atkins analogy was by far the best. Only you can decide if you will ever need that extra 2 stops of DR, just like a race car driver has to decide whether or not a 197mph top speed is important given he drives tight courses. For him, better handling may be vastly more important than a higher top speed. If, OTOH, all he does is speed runs on oval tracks, that top speed is critically important.</p>

[Landrum Kelly]

<blockquote>

<p>How real and reliable are DXOMark Scores?</p>

</blockquote>

<p>My personal belief is that they are fairly irrelevant.</p>

<p>SHOW ME THE SAMPLE PHOTOS, NOT THE NUMBERS.</p>

<p><a href="http://www.dpreview.com/reviews/canon-eos-70d/17">http://www.dpreview.com/reviews/canon-eos-70d/17</a></p>

<p>Move the box to the woman's face on the right. Look for shadow noise under the chin at very high ISO, etc. </p>

<p>The DXO numbers do not square with what I am seeing.</p>

<p>Are you going to trust DXOMark or your lying eyes?</p>

<p>--Lannie</p>

[Landrum Kelly]

<blockquote>

<p>I am used to using Canon cameras but never handled Nikon before.</p>

</blockquote>

<p>I switched to the Nikon D800E two years ago just six months after selling my Canon gear to avoid bankruptcy after losing my full-time teaching job. (Yeah, I'm a resolution freak, and so the 36 megapixels lured me in.)</p>

<p>For the life of me, I can't figure out why my photography has not improved.</p>

<p>--Lannie</p>

[Landrum Kelly]

<p>Dick, that is a great series!</p>

<p><a href="/photodb/folder?folder_id=1065937">http://www.photo.net/photodb/folder?folder_id=1065937</a></p>

<p>--Lannie</p>