How many defective lenses?

[bobatkins]

There seems to be a lot of concern about getting defective lenses and we see

quite a lot of "defective lens" reports here, so I wondered just how frequent

defective lenses are.

<p>

Since I can't access the photo.net database for surveys, I posted one on my

Lensplay.com website and it's now gathered quite a bit of data. There are

currently reports from other 1000 users, with reports on over 3500 Canon

lenses, as well as smaller data sets on Nikon, Minolta/Sony, Pentax and 3rd

party lenses.

<p>

Most of the Data is on Canon EOS (and Canon EOS compatible)lenses because my websites deal mostly with Canon EOS systems and so that's what most people who have followed links to the survey use.

<p>

The conclusion so far, with all the caveats about surveys not being truly

representative, the possibility of people supplying bad data, etc. etc,. are

that for the the major manufacturers (Canon, Nikon, Minolta, Pentax), the

chance of getting a lens which you think is defective varies from about 1 in

10 to 1 in 17. For 3rd party lenses the odds a a bit <strike>lower</strike> higher, ranging from

around 1 in 5 to 1 in 8.

<p>

Are these accurate numbers? There's absolutely no way to tell. You can't tell

if they are high (because people are more inclined to particiapte if they are

angry about getting a defective lens) or low (because most people don't test a

lens thoroughly enough to find every small defect).

<p>

Still the results may be of some value. It does look like the major

manufacturers do a slighly better overall job with quality control, at least

as perceived by those who chose to provide input and with the data obtained so

far.

<p>

You can participate in the survey at

http://www.lensplay.com/lenses/lens_defects2.php

[pico]

Bob, I'll bet that many here would appreciate some of the caveats concerning self-reporting

(surveys), especially some of the nonintuitive twists of probability theory.

 

I'd visit your survey, but I haven't had a lens failure due to manufacturing mistakes or design

for thirty years; it's always been from hard use or operator error... and early on, unrealistic

expectations.

[bobatkins]

And that's a classic example of self selected surveys. You say that you would participate but you've had no defective lenses. So you've just self selected yourself out! It's important that a random cross section of users particiapte. Those reporting lenses with no defects are just as important as those who have received defective lenses. If only those who have received defective lenses particiapte, then the whole survey becomes meaningless!

 

Surveys are just surveys. Garbage in, Garbage out. You can twist statistics into all sorts of knots if you try hard enough.

 

These results are mostly unfiltered, though the data is run though some sanity checks and data on 50 lenses within a few minutes from different (but close) addresses in the same IP block will likely be flagged as bogus. There are a couple of other checks and balances too. However less than 1% of the input data suffers that fate.

 

A statistical analysis of the incoming data shows that it really needs at least 300 reports, and perferably 600, before fluctuations die down and the data smooths out. There's enough Canon data for that for sure, but data on some other brands is still a bit thin.

 

I'd be wary of putting too much faith in the numbers, but overall it does seem like maybe 10% of lenses are seen by users as being defective.

 

I'd agree that this number seems high in my experience. I've never bought a new lens that was defective (at least not in the last 10 years). I've had a few used lenses with problems, but they don't count as there's no way to tell if they were dropped or otherwise mishandled. User error and unreasonable expectations may well account for some of the "defect" reports, but there's really no way to tell.

[peter_hughes6]

I bought 2 Sony XBR CRT HD TVs. I was a part of a very active forum avforum. On that site there was an open thread about problems with the TVs. Threre were negative comment from a sizable percentage of posters.

Both my TVs are perfect, I don't have any reason to post to that thread.

I'm not interested because I have no issues. Could that be an indication of the validity of a forum?

[bobatkins]

Exactly. In a froum, people generally only complain when they have a problem. If they are happy, they tend to keep quiet. Nobody posts "I just got a 65/2.8 and it has no defects", but you will often see "I just bought a 65/2.8 and it front focuses".

[pico]

<i>And that's a classic example of self selected surveys. You say that you would participate

but you've had no defective lenses. So you've just self selected yourself out! </i><p>

I'm glad you took the opportunity to note the irony then comment. It worked.<p>

Then there is the issue of a reward for one to participate, and it is not always tangible.

[gdanmitchell]

<p>As others have pointed out, this type of survey tends to attract participants who fall

into two categories, neither of which are necessarily representative:

 

<blockquote>Those who had or think they had a problem with their lens and are upset

about it.</blockquote>

 

<blockquote>Those who are fans of a particular brand or product and will do anything to

build it up.</blockquote>

 

To find more useful information you would have to randomly select a group of users of the

product you wished to consider and then ask questions that were not leading. For

example, you wouldn't ask questions like "what was wrong with your lens?" but you might

ask them to rate the condition of the lens on a scale of some sort.

[trothwell]

This would add complication, but I'd be curious to take a a step further. Of the respondents, how many ?...

 

- EF-S lenses were bad?

- L-series lenses were bad?

- Standard (non-"L") EF lenses were bad?

 

Might give some hint as to more precisely where quality control might be deficient, if there is such a statistic to be had. It's obvious that less of SOMETHING goes into the 24-85 zoom compared to the 24-70 L zoom, but does that SOMETHING include QC?

[mendel_leisk]

How about a survey of lens retailers?

[gdanmitchell]

"For 3rd party lenses the odds a a bit lower, ranging from around 1 in 5 to 1 in 8."

 

I think you meant "odds were a bit HIGHER" here, right?

[damian_tinsley]

But in general, you will also get some skewed results based on the lenses people buy. I have a 35-135 mm which is so soft at the long end it's almost useful as s SF lens. I wouldn't consider complaining about it as I bought in on 'bay for a song and knew what it was likely to be like!

[bobatkins]

I don't believe there is such a thing as a perfect survey. There's always something you could do to improve it.

 

The REAL ststistics are known only to the lens makers and they will never release them. They know exactly how many lenses are returned, what was wrong with them and which lenses they were.

 

I'm sure some lenses have a higher defect probability than others. However I'm also sure that purchasers of a $1500 50/1.2L are going to be a lot more critical of image quality than those buying the EF-S 18-55 kit lens, so more (on a percentage basis) may complain about getting a defective lens!

 

I could run a survey that took in every single detail, such as when the lens was bought, what the serial number was, where it was bought from, what the defect was, how complaints were handled etc., etc., but I'm not really sure how useful that would be. It would be like looking at an image through a microscope. You could see details but it might make the overall picture harder to see. It's also to harder to get people to participate if yuo ask for too much information.

 

The present survey collects quite a bit of information which I may analyze at some point (such as where the survey participants came from). Maybe photo.net users are more likely (on average) to complain than survey participants who found the survey via a different website! That would be an interesting observation in intself...

 

And yes, I meant that the odds of getting a defective lens from a 3rd party manufacturer were higher, not lower, according to the data collected so far (and including the page full of caveats already noted).

[Marvin]

For along while the herbal supplement Black Cohosh was being touted as providing relief for the symptoms of menopause (http://ods.od.nih.gov/factsheets/BlackCohosh.asp). People and their physicians were recommending it.

Recently, an actual scientific study using the herb vs the placebo demonstrated that the herb had no effects greater than a placebo (http://www.eurekalert.org/pub_releases/2006-12/acop-hsb121706.php).

 

The point of course is that surveys, anecdotes, etc often represent preconceived notions or, in this case, wishful thinking and are often just wrong.

[james_alexander]

And maybe there is another problem. For many users it may be difficult to tell if their particular copy of a lens is substandard. Mechanical problems might be fairly easy to spot, but degraded optical performance may not be as obvious especially to someone who has less experience with other lenses.

[beauh44]

I got a chuckle out of this thread over at dpreview:

 

http://forums.dpreview.com/forums/read.asp?forum=1029&message=21365715

 

There are some people who, no matter what they get, are absolutely convinced that there's a good chance they'll get a better one if they just return it enough times. I bet B&H and Adorama love these guys.

 

I could be wrong but I'd swear none of this was an issue a few years ago - and not just because we can pixel peep so easily now. I recall reading a few articles (I dunno, circa '02-'03) in some of the more well-published photo magazines about how "the pros" will go to a camera store (say B&H) and go through 20 copies of the same lens until they find the very best one.

 

I think some of those articles started this "fever" regarding alledgedly "bad" lenses. I'm not saying some aren't bad - I'm sure some are.

 

I also think there's something that a lot of people miss. I recently had my 1Ds II in for service to replace the shutter. In addition to doing that, I saw listed on the repair sheet "Adjusted autofocus".

 

When I got it home - and I realize this is completely anectdotal - I swear all my lenses suddenly became "sharper". I rely heavily on AF due to poor eyesight so manual focus just isn't an option for me. Just thought I'd share!

[25asa]

They say you get what you pay for, but even in Canon L series lenses you hear of bad copies with those. It seems you have to pay Leica dollars to get something that wont have as many problems. So far my lenses have been fine (maybe Im not that picky), but it makes you wonder when you read so many postings of bad defective $2000 lenses. Im concerned as Ive got a 50L on order and the chances of getting a good copy at this time are 1 in 10, meaning most are bad copies. Thats pretty bad considering the lens costs $1600.

[bobatkins]

How do you know your chances are 1 in 10? The survey doesn't indicate that. It does indicate that maybe 1 in 10 people are unhappy with their lenses - if you average over all lenses - but it says nothing about the probability of getting a defective copy of the 50/1.2L

[PuppyDigs]

Bob you need to increase the max amount of lenses. Many of us have owned more than 10

Canon lenses during the past 5 years...

[trothwell]

Sorry, Bob, no complaints intended.

[pico]

<i>about how "the pros" will go to a camera store (say B&H) and go through 20 copies of the

same lens until they find the very best one.</i><p>

What shop would put up with that?<p>

Many years ago in a far away land called Chicago at the Worlds Largest Camera Store there

was a customer who asked to see the lot of serial numbers to find "a cool one". Man, how

sorry was that!

[bobatkins]

Anyone who has bought more than 10 new Canon lenses in the last 5 years probably isn't a typical user!

 

Limiting user input to 10 lenses is deliberate for a couple of reasons. One is to avoid the abuse which could occur if an unlimited number could be entered. I won't go into details on the other reasons, but I think limiting input to 10 new lenses per manufacturer should be fine for 99.9% of users.

 

If you really want to enter data on more than 10 lenses, just leave a note in the comments field with supporting data.

[25asa]

I say that 1 in 10 stat because all that have posted stating they've bought the 50L lens, most of them have had problems with the lens (typically backfocussing). Hardly anyone so far has been too impressed with this new lens. Ive only read one so far thats had no issues with it what so ever. And this is not for the lack of postings on this lens.

[robyn_s]

Bob,

 

I very much appreciate that you have devoted bandwidth on your personal site to put some type of statistical face on the ever popular spector of defective lenses. It is obvious that there is no perfect way to conduct such a survey, but you are to be commended for the effort, and I really appreciate it.

 

Having said that, I'll have to admit to not participating in the survey. Talk about "talk is cheap!" Here I say I appreciate it, yet I don't participate. With that shameful admission, you at least deserve a brief explanation as to why:

 

It's simple. I don't feel qualifed to determine whether or not any of my lenses are "defective" to the standard that matches what I have read from so many posters on so many photography forums who concern themeselves so studiously with this question.

 

I have yet to photograph a brick wall... or a newspaper on a wall for the sole purpose of comparing the sharpness of a lens. I don't buy lenses very often, but when I do finally get around to buying a lens, I already needed it yesterday, and the job I need it for next usually starts the day after (or sometimes the day of) the lens arriving overnight. The lens gets used immediately for the job I bought it for, seems to do the job ok, usually better than whatever options I had before, so I make room for it in my bag and forget about it.

 

It's only later (after it's too late to return it) when I read these detailed lens sharpness complaints on fora everywhere, with comparison pictures posted at every aperture, of modeled mannequins and made up mummies... that I start to feel guilty of not performing this due diligence on my own equipment.

 

Deep down, I know I should be putting every lens I buy through some set of methodological paces. Deep down, I think I'm afraid to test. Afraid that I'll misinterpret my own analysis. Afraid that I'll get really, really, REALLY, bored, really quick, about half way through staring at the first brick wall pic, and then abandon the test. Afraid that I don't really know what I'm doing anyway. Afraid that even if I sent the lens back, the next lens I'll get might be worse. Afraid that I'm just wasting time... since no clients have complained. Afraid that I'm just plain lazy. Afraid that I'll prove that the lens really is defective, and it will be to late to return it, and too wrong to foist on someone else. Afraid that I'll simply prove that I'm the one that is defective.

 

So, I conveniently avoid the problem by just using the lens for the jobs I bought them for. Having not applied any type of scientific test to the standards that I have read, I can't really say with certainty that any one or more lenses I own are defective or not. Therefore, I feel unqualified to respond to the survey.

 

And just now I feel even more guilty... first for having not tested my lenses in a timely and methodical manner, and second, for not completing a survey sharing my findings.

 

I wonder how many others don't have the time to devote to such tests, and just plod along with whatever lens is first delivered?

 

I've also shopped inside B&H's store, and there is no way that they will bring 20 copies of a lens for you to pick through. You don't even get to see the first copy, as the gear you select is routed directly from the back to a pick up counter. They don't let you see, handle, or carry it yourself to the cashier... you only pick it up at the counter after you present the receipt.

 

Other camera stores where you can pick and paw usually don't even have 20 copies of any particular lens in stock to poke through... maybe 2 or 3, but not 10 or 20 of the exact same lens. So, I'm quite skeptical about that article mentioned above.

 

Or maybe I'm just feeling more guilty, because now that I think about it, I never even ASKED the guys at B&H to "fetch me five copies of this lens from your warehouse pronto." I just told them the lenses I needed, and they handed me a piece of paper with an order number, and I walked over to the cashier.

 

After that, I walked to the pick up counter to finally see the lenses I just bought. I did at least open the boxes, made sure that actual lenses were inside and not pieces of brick, and removed the caps to look through the glass to make sure it wasn't cracked or dirty. Seemed clean and clear enough, so off to work I went. That kind of assessment hardly rises to the rigours of testing that I read of here and there. So I don't think I'm qualified to participate.

 

 

[twmeyer]

Hey. I have had various lenses throughout the "career", notably LF ones, that had what I recognized as "sweet spots", for instance the 90mm F4 Rodenstock is waaay better at 16 1/2, acceptible at f22, but I never use it at 32 where it is soft across the field. The 135 Sironar N is best at f22. My first LF lens was the 135 Scheider Xenar, which was awful when focused beyond about 10 feet, but razor sharp at 6 feet and less, or with a 6x9 back. My 24mm F2.8 Nikon AF-D is much the same, and with a Nikon diopter it makes a killer macro lens. My $75 Nikon 50mm 1.8 AF has a terrible red flare when the subject is heavily back lit, but it's a favorite portrait lens in any other scenario. Are these lenses defective? I think not. I prefer to regard them as "specialized". <p>I suck at quantum physics, but I'm a damn good cook. Defective? many (I hear you) might say yes, but at least my wife has yet to return me to the manufacturer (who, as you may expect, thinks I am flawless)... t

[kelly_flanigan1]

My take is there are really fewer "defective lenses" than folks think. The greater problem is basic operator error, faulty rangefinders, faulty ground glass backs non LF cameras, faulty eyes of folks doing the focusing, faulty reasoning and understanding of depth of field, faulty TLR's with mixed up viewing and taking lenses, faulty useage of autofocus. In most returns of consumer items there really is no problems at all, just returns by weekend warriors wanting a free rental, or folks with buyer remorse/sticker shock.<BR><BR>In the last decade there has been a radical increase in folks concerns with looks over function the dogma. In most folks minds a spec of dust or spot of fungus will drop a lenses performance radically. They probably think super glueing a BB to a dump truck will radically drop the trucks gas mileage too.:)<BR><BR>In LF lenses and my Phase One 4x5 scan back I have found that I get great performance, even with an enlarging lens used at infinity focus. Here I can back out all the focus errors by doing several scans and get the focused points spot on, and cheat the usual GG to film plane errors. <BR><BR>With the ancient T4 lenses such as Tamaron, Soligor, and Vivitar on would often get a "dancing stopdown diaphram" when one used an autodiaphram lens on a motordrive camera. Here the diaphram on say a 135mm F2.8 set to F11 might be F11, F8, F11, F9, F11, F11, F9 during a burst. The dancing might happen at some frame per second rates, and not others, due to like a resonance. Here the true camera makers lenses rarely had such weirdness.