Objective Lens Testing Sites

I have a few question regarding the some of the objective lens testing sites.

Is DXO still around? I have not seen any new lens or camera tests on that site since February and was concerned that they are going the way of LenScore/SenScore. I am concerned that they shut down all lens testing in favor of the latest smart phone testing.

I have completely given up looking for DPR lens reviews because they are so few and far between,

The active sources that I have looked at recently with some form of objective testing include Optical Limits, Lens Tip, Photography Life and Camera Labs and what I could find at Lens Rentals. Optical Limits and Lens Tip seem to be the largest data bases with the former being the most exhaustive though generally speaking none of the sites seem to be keeping up.

With Nikon glass their MTF charts that are published on their web site Nikon USA and seem to reflect the lens real world resolution weaknesses and strengths. Are there other sites that would be better to look at if I m comparing or shopping for glass? Thanks

Proper lens testing is a lot of work and it must be difficult to find sustained funding for such projects especially when camera and lens sales are on a decline. DxO Labs reportedly split into DxOMark and DxO Labs. I suspect Adobe's domination in the software field may have had something to do with it; when just to keep the existing version of Adobe software requires a montly payment, there is no opting out of paying for upgrades that you may not need. This means photographers may have less money to pay for other software; previously one could simply decide on a case-by-case basis, which software upgrades to get and from which company. If the money is not coming from software, other possible sources are private funds (buy the product, test it and report the results), public funds, or advertising. There is also the possibility of volunteering the work and testing lenses that others have purchased and lent for testing.

I was a little confused by the DxO split - I'd kind of assumed that DxO effectively had to calibrate lenses anyway in order to produce the lens profiles used by OptixPro/PhotoLab, and that DxOMark was just the result of processing their database. Actually doing the work twice seems odd.

 

For what it's worth, I don't hugely mind paying (admittedly quite a bit) each month for Creative Cloud, because buying or upgrading Creative Suite every few editions was preposterously expensive. Worse in the UK - you may recall that at one point it was cheaper for someone in the UK to buy a return plane ticket to New York and buy Creative Suite there than it was to buy the UK version. At least I'm getting current versions of things and not deliberately putting off an upgrade to save money. On the downside, if I'm happy with the "if it ain't broke don't fix it" philosophy, I'm paying Adobe for a load of busywork. (And I still haven't worked out why the docked tabs have decided to position themselves blocking half the screen in the latest Photoshop update.) I also pay DxO, who provide regular updates, and they're part of my standard import workflow.

 

Reviews... I read what Thom Hogan has to say when he gets to things, but they're generally expressed as (informed) opinions. I see what Rockwell has to say about stuff, mostly in case I missed anything in the specs or handling, although I treat statements such as "this lens is tack sharp wide open" with a pinch of salt. On older lenses, Bjorn's reviews are much appreciated, and the Photography in Malaysia content for technical descriptions. LensRentals too for recent lenses, though it's clearly not their primary business (again, in their case "reviewing" a lens is a way to check whether their rentals are coming back without being knocked out of alignment). Mostly I look at DxOMark or google around for other reviews, but I really miss when dpreview would let you mouseover the sharpness field map and give a pop-up of what the test chart looked like - that told you not just whether the image was soft, but what it looked like. I happen not to like what the 85mm f/1.4 AF-D looks like wide open (off-centre), for example, but others do like it.

I don't know if the DxO test procedure for their software profiles is the same as for testing lenses, I would guess that it might not be. DXO software has the support for 200-500 for example, but they never published test results for that lens.

Are there other sites that would be better to look at if I m comparing or shopping for glass?

 

Frankly, I never saw the "objective" lens test sites as the only or even main go-to sites. I still check OpticalLimits (former Photozone) on some interval; they seem to still test though things certainly slowed down. Their technical testing seems fine; though I have little means to validate. PhotographyLife also does a fair amount of tests, and overall I find their reviews fair and level-headed. Both sites are testing a limited amount of samples - so there is always some level of risk there. No idea how other sites compare in that respect.

 

 

But personally, I feel these objective tests really just paint part of the picture. There are a fair number of speciality lenses that test mediocre or bad, while the resulting images of those lenses can be very pleasing. And some lenses that test extremely well seem to prioritize sharpness and resolution to the point where the rendering becomes a bit harsh. All of this is highly personal and basically impossible to test objectively - still, I rather have a lens that renders images in a way I love than I'd like to have a lens with perfect scores that renders in a way I don't like.

So I've always found having multiple subjective reviews is far more valuable than objective test results. Of course it needs level-headed writers, but writing styles are usually revealing enough. There are plenty of sites, forums, blogs where people will discuss their pros and cons, and most important show example photos to underline their point. Pictures do speak a thousand words in this case.

With quantitative, ie measurable variables, like resolution, i find Lenstip reliable enough.

 

Despite there weight and limited zoom range the Sigma 18-35mm 1.8 DX and the 24-35mm 2 FX really are, resolution wise, 3 primes in a single body.

 

Sharpness readings on the site bear this out. But as Wouter said, that is only part of the story.

 

Somethings like copystand work requires hi-res, low distortion, zero vignetting, zero coma and negligible lat/long chromatic aberration. But OOF rendering is not relevant.

 

Portraiture has different needs and priorities. Hi-res might not be very nice, optical distortion of a few % is not noticable, OOF rendering and vignetting may well be hi priority.

Analyses by Larry Cicala, founder of LensRentals, are the most interesting and thorough with regard to technical details. DXOMark rates a lot of lenses, but their methods and weighting are proprietary. Any company which consistently dissed MF digital compared to small format stuff doesn't have much to offer, IMO.

 

In the end, your best bet is to read as many reviews as possible. Competent reviewers give better insight into handling and mechanical quality than mere resolution tests (and guesses). It is particularly revealing when the tester opts to buy the gear he has the opportunity to test and evaluate. When the choice is too expensive for trust in strangers, see if you can rent it for a week or two. My go-to place is LensRentals, where I can be assured gear will be in tip-top shape.

Analyses by Larry Cicala, founder of LensRentals, are the most interesting and thorough with regard to technical details.

I assume Ed meant Roger Cicala:

Roger Cicala

 

Cicala has the unique position to deal with multiple samples of just about every lens, sometimes maybe 10, 20 samples, such that his opinion is not affected by one sample that is way out of the norm. But of course part of his objective is to drum up business for his rental shop.

I am kind of curious about how DxO tested camera and lenses especially lenses. I think they just use camera and lenses then take pictures of some target then use software to analyze the results. Back in the old days when testing lenses they often not use camera but test the lenses on optical workbench with instruments to analyze the image form by the lenses.

Yes-- LensRentals -- So far as I know they are the only ones who have large enough samples of current lenses to have their results be scientifically acceptable.

 

Almost everybody else tests just one (1, count it) lens and there is enough sample variation to make this a very uncertain basis for decisions.

 

I am not at all convinced that DxO, useful as some of their apps are, is a completely level playing field for lens testing.