Opinion on TechRadar?

[fotolopithecus]

<p>I've noticed recently that TechRadar while seemingly giving very complete reviews of DSLR's is also very much at odds with the results dxomark gets. This is odd because they use dxo equipment to get their results. I asked them why the difference, and they said it's because they test using the software that comes with the camera, whilst dxomark uses their own unavailable to you or me software.<br>

If this is true it makes one wonder of what use is dxomark in giving you useful metrics to help you decide on what camera you want to get. I was surprised to see Pentax faring best of a group of cameras that included the Nikon D5500 in everything from noise to dynamic range. In fact the D5500 seemed to do worst of the group in everything except resolution, while at the same time doing best at dxomark.<br>

Any of you more technical among us have some helpful perspective on all this?</p>

[Wouter Willemse]

<p>With the pre-amble that I never found DxO Mark a useful metric for most discussions, nor to select one camera over the other, according to DxO they analyse the raw data that comes from the camera - no interpretation of software at all, but the actual real read-out of sensor data. In that sense, I expect their software to not actually produce a picture, but really data and graphs. It'll probably share code with DxO Optics, but be optimised to measure rather than bring out something esthetically pleasing.<br>

The path chosen by TechRadar, to use the manufacterer provided software, is debatable. It makes a bit apples-to-oranges comparisons. The counterargument could be that the manufacterer software should be most capable of extracting the maximum, as they know most. But they're camera makers, not software developers. After having switching from manufacterer software (Nikon CaptureNX2) to 3rd party software (CaptureOne), I found that the manufacterer software not necessarily gets the best from its raw files.<br>

The approach of DPReview is overall for me the most reasonable: settle on a common available 3rd party converter, document clearly which settings are used to approach apples-to-apples (it isn't fully, but closer). </p>

<p>In the end, all these metrics however are too much just a very small part of the picture. Apart from obvious important characteristics as lens availability and prices, handling and features related to handling, AF systems and so on, there is also the point of how noticeable the differences are in real life - what is the treshhold in "DXo Marks" where people really observe a visual difference? How well do the raw files of a given camera work in your prefered raw editor and workflow, and how easy do you get to the results you like? Camera reviews seems to focus on measuring the sensor, but it really is just a part of a chain; in the end, getting the chain to work well end-to-end makes a truckload more difference than 1% more shadow noise at high ISOs between brand A and B.</p>

[SCL]

<p>Relying solely on test results of anything to make decisions, without knowing the variables and how they were treated in the testing, is caving in to consumerism at best and foolhardy at worst. The testing one often sees rarely defines how well an item will actually perform in real world usage and expectations. I'm not trashing the concept, as I think most tests provide fundamentally useful information to guide consumers to begin to evaluate a product in terms of their own usage, expectations and workflow. That is, the tests provide a starting point for personal evaluation - all information, particularly if it is industry wide standardized, can help an individual in the decision making process. But most decisions don't boil down to simplistic cases of A is better than B.</p>

[fotolopithecus]

<p>Presumably dxomark isn't just providing metrics, graphs, and charts for the academic interest of engineers. I don't see "any" practical value they have given that there's no way to get their results yourself. Now I would think that quite a few cheapskates use the software that comes with the camera, so at least TechRadar is providing some sort of useful information.<br>

I wonder where one could go to find what kind of dynamic range, and noise performance to expect if you use some other software's. Lol, all these years of thinking dxomark means something. If your interested in what I'm talking about check out TechRadar's comparison of the Nikon D7200, Canon 70D, Olympus EM5II, and the Fuji XT1. If this is the case throw in the towel dxomark.</p>

[Tim_Lookingbill]

<p>Of all the white papers and technical research documents on digital imaging/photography conducted by a number of highly educated color scientists, engineers, programmers and mathematicians I've attempted to read through, not one of them produced a decent or even accurate looking image included as sample images to support their findings. </p>

<p>Pick a camera with the features that suit your working habits and go with it.</p>

[bobatkins]

<p>I think it illustrates the fact that technical testing on cameras (and lenses) is full of pitfalls and putting much faith in any one test can be a foolish thing to do.</p>

<p>There's no defined methodology for camera, lens or sensor testing on the web. Everyone uses different techniques and if those different techniques yield different results it only amplifies that point that depending on any one of them is foolish.</p>

<p>I'd say pick the camera you like, and unless it has some obvious and significant weakness, go with it. For example, detailed technical testings shows that the Nikon/Sony sensors have more low ISO dynamic range than the Canon sensors. In practice, most people will never see a difference. I've continued to buy Canon bodies despite this and I've been perfectly happy with them.</p>

[andylynn]

DXOMark does specify that their test is of the sensor - not the camera, but the sensor. That's not going to give you a

complete review of a camera, just one data point.

 

I was under the impression that the software they use for raw files shares a large chunk of the code base with DXO

Optics Pro. Only instead of the Optics Pro UI and all the front end stuff that makes images look good it would output some

boring flat image data pulled from some early point in the postprocessor's workflow that Optics Pro won't show you on the

screen but that's suitable for testing. But the DXOMark results would reflect some metrics of image quality you could

expect to see in Optics Pro.

[fotolopithecus]

<p>Interestingly dpreview's way of looking at raw dynamic range seems to agree more or less with dxomark in most camera comparisons, only less dramatically. I guess what they're doing is testing more than just the sensor though so this throws more confusion on the issue.</p>

[shutterbud]

<p>Digital imaging is incredibly complex. This is something we all know but when trying to evaluate a potential purchase most people, specialists & nonspecialists alike naturally tend to want hard facts to work with. There is little doubt that the factors routinely discussed [DR, CD, ISO noise etc] are important when considering a new body and other factors [CA, Distortion etc] regarding a lens are equally important but of course they are not the only data which need to be known. Some people will reject one product for reasons others find irrelevant or even ridiculous. However, given how much time, effort and money many of us devote to photography, we do require a way of deciding whether this or that camera is right for us. Unfortunately, most of the time, this only comes with experience, which is why many of us, having found a company we trust, stick to it. Our ability to overcome the known shortcomings of a product can sometimes be more important than the shortcomings themselves. I sometimes feel rather sorry for review sites. The amount of effort involved in providing a comprehensive review these days is high. And then someone inevitably comes along to point out a 1% difference in measurements between this or that site. It is bewildering.</p>

[Brad_]

In the end it makes little difference.

 

A photograph's power comes from the photographer's eye, imagination, life experiences, ability to see and

compose what's before him/her, skill, post processing, etc. Outside of obvious needs/requirements for

edge case situations (such as shooting sports, as an example), lens and camera minutia in the end adds

mice nuts to the power the photographer contributed from the above.

[andylynn]

<p>True, and you gain very little by nitpicking sensor stats between otherwise reasonably similar current models - if a D750 file does have more dynamic range than a 5Diii file, is that actually a big deal in the field? But nevertheless, there are images I can make with my current gear that I would not be able to make with my old D60 kit if I still had it.</p>

[ellis_vener_photography]

If you want to decide which camera you should buy, I have one tiny piece of advice: if at all possible put the cameras you

are looking at in your hands and work with each of them for about 10-15 minutes. How does the view through the

viewfinder feel to you? How do the cameras (with roughly equivalent lenses) feel n your hands? Are you comfortable with

how the camera balances? With the control layout?

 

In defense of dxO's methodology, they are running the output from the camera through the same process for all camera.

In other words they have an objective set of standards and criteria. Nikon, Canon, Sony, etc., and Adobe and Phase

One are designed to make pleasing looking photos.

[Spearhead]

<blockquote>

<p>DXOMark does specify that their test is of the sensor - not the camera, but the sensor.</p>

</blockquote>

<p><br />How do they do that? Do they open up the camera and bypass the electronics?</p>

[fotolopithecus]

<p>Ellis, the problem with dxo's methodology is that if what TechRadar told me is true it has absolutely no practical value. Of what use is it to know that the sensor of camera A has more potential dynamic range than camera B's, but actually doesn't because the rest of the pipeline, or post software doesn't take advantage of it. I mean the tests that TechRadar does measures noise, dynamic range, etc, with the cameras supplied software. Now it may be that some other softwares can extract more of dxo's figures than the supplied software but who knows it could be worse too.<br>

If what I'm told is true then dxomark is worse than useless, it's misleading because they must know that people are basing their purchase's at least in part on their scores. It's just amazing to me after all these years.</p>

[andylynn]

<blockquote>

<p>How do they do that? Do they open up the camera and bypass the electronics?</p>

</blockquote>

<p>No, they just run tests on raw files for specific image quality metrics while not evaluating everything else about the camera that helps take a good photo, like handling, speed, AF etc.</p>

[Spearhead]

<p>Well then, that's not evaluating the sensor. It's evaluating the entire image production chain in the camera. It includes more than the sensor.</p>

[andylynn]

Is there anything it evaluates that any reasonable person would have reason to consider as separate from the sensor?