MTF tests a shock to my system

[jonathan brewer]

I have just gone

here..............http://www.photodo.com/prod/lens/index.shtml.........where

they have conducted MTF tests of just about every major lens produced

in 35mm, and MF, and it has been a shock to my system.

 

They've rated every lens I've known and lusted after, a few which

are about the cost of a small care, according to these tests, ain't so

hot, some to my surprise, are hotter than a 'two dollar pistol'.

 

Maybe some of the optical experts can peruse this site for

possible flaws in their methodology, I'd be curious as to what they

think. There are some highly touted 'APO lenses of various

manufacturers graded here, that despite their stratospheric price,

ain't so hot.

 

According to this site and its tests and grading system, Mamiya,

particular a few of the 67 RB KL lenses come off like champs, giving

performance that stretches Rollei and Hasselblad all they handle.

I've really just begun to over this site a second time, and if the

tests are valid, then it is surely a testament to the notion that hard

date cuts through a lot of manufacturer hype.

 

One lens that has been tested was one I've always lusted after,

but according to these tests, while not being a dog, it's bark is

pretty loud, I lust no more.

[steve_teale]

Jonathan,

 

You've no idea what you've just started ;)

 

In my opinion these tests are a good way of identifying lenses with INDIVIDUAL examples which when tested shows obvious defect (i.e. true canines). It takes no account of factors such as colour fidelity, bokeh and other sometimes almost intangible characteristics that make a lens 'appealling'.

 

When you want a lens that can take good pictures at infinity of test targets under controlled conditions it's a good place to find the one you want. Otherwise rent or borrow one and use it in the real world.

 

One good thing about the tests I think which can't be underestimated is the fact that they remove a lot of variables from the equation i.e. lab processing, lighting conditions or bad technique impacting on perceived lens performance.

[jonathan brewer]

Looking over the grading, there are some Mamiya RB KL lenses that grade higher than any of the RZ, Rollei, and Hasselblad lenses, and this was a surprise to me, albeit a pleasant one since I have the RB 180 lens that grades out to 4.3.

 

Again I would be curious as to what the optical experts think of the methodoly and it validity as it is used on this site. Is the data and the way it is being used, giving a true picture of the performance of the various lenses tested.

[jonathan brewer]

Steve.........Yes, I agree with you about all the other variables tested, I don't need to rent some of these lenses, I have them, I looked at these tests and was surprised not by some of the lenses I knew/expected would do well, but by some lenses(according to these tests) that haven't been hyped as much, and that's my mindset, these tests versus hype.

 

I have no expectation that these tests replace the other variables to be considered when thinking about a lens and its particular pallete, but I did want to ask if these were valid tests using the appropriate methodology for MTF tests.

 

Many a good lens has been damned because of someone handholding for a shot @ 1/60 of second, who obviously couldn't blame the resultant 'mush' on their technique, ditto vibration, bad eyesight/old eyeballs like mine et al. It seems to be that what this site really says, at least to me is that the major camera/lens manufacterers are all in the same ballpark performance wise, and you need to shop around for a particular pallette rather than being led by the nose of reputation/hype.

 

I don't beleive anything written down as being Gospel, which is why I posted this, I thought it would be interesting to those who hadn't checked it out, as interesting as it was to me.

[steve_teale]

Jonathan,

 

I find the site very interesting but haven't been back there recently - I don't think they've published any tests for a while.

 

You're right about the major players being so similar.

 

The Canon L series don't dissapoint in their ratings and just about all the modern true macro lenses get a good grade as you'd expect.

 

Check out the rating for the discontinued Tokina 90mm macro - I've heard good things about this lens but I'm not sure if that's driven by the rating or just backed up by the ratings...

[frank_bunnik]

Steve basically said it all.

 

Photodo tests under laboratory conditions the linepairs per milimeter a lens resolves. Unless you make the same pictures or have to make a brochure for a Colombion cocaine cartel, these tests are useless.

 

Nothing is said about bokeh, flare resistance, sharpness at close focussing distances, light fall off at the corners, distortion and built quality.

 

In 2 different Dutch magazines a couple of years ago, the Nikon 28-200 was compared with the Tamron 28-200. One magazine tested only by using mtf graphics and decided the Nikon was the best of the two. No word was said about the enormous distortion of the lens, nothing was said about that it could not be used on the F801s without an expensive change of chips in this camera.

 

The other magazine refuses to use mtf graphics for the simpe reasons stated before. By taking a picture of the front of a building, the enormous distortion of the Nikon lens was shown. This was nothing like the little distortion normally found in most zoomlenses. The Tamron did not have this amount of distortion. Also, the tamron could be used without a problem on the 801s and was consderably cheaper. This magazine advised that Nikon should take it lenses back to the factory and make a completely new design. Sure the Tamron was not as good as the Nikon on the optical bench but in reality, the Tamron was the better of the two.

[chris_waller]

This is interesting. It's like all the arguments that go on in the world of hi-fi. The best loudspeakers I have ever heard for vitality and life-like reproduction return objective test figures which suggest they should sound very ordinary. I have an RB67 and the 180mm lens delivers, to my eye, superb results so I'm pleased to see they also tested well. And I've always favoured Mamiya since their cameras are so reliable. My old C3 gets dragged around the fields, up hill and down dale, through rain and wind and still it soldiers on. And given its vintage the sharpness of the lens amazes me.

[marbing]

MTF is an important and accurate measure of the resolving power of a lens. However, as others have noted, there are other lens characteristics that are equally important but which are not addressed by MTF testing. MTF will tell you the resolving power of a lens across the optical field and, as expected, show that most lenses are better in the center than at the edges. The better lenses retain more resolving power ("Sharpness") as you move further from the center.

 

However, a good lens also needs to be "contrasty" as well as "sharp". The out-of-focus highlights must be pleasantly "creamy" and distortion must be minimal...etc.

 

It is possible to have a lens that is sharp but with low contrast or a lens that has good contrast but poor sharpness. The 'great' lenses incorporate all of the best qualities together. MTF is just a single characteristic out of many.

[photomark]

What a weird site. I say we get a hold of a couple of Ansel

Adams', Edward Weston's, and Walker Evans' lenses, send

them to photodo in order to find out how good these guys really

were.

[godfrey]

Rating lens quality by a single value metric is foolish. Accepting someone else's numbers as definitive is more foolish.

 

By photodo's rating, the '51 Rolleiflex 3.5MX Tessar I sold recently probably has a Coke bottle bottom for a lens. Yet it takes beautiful photographs.

 

Don't believe everything you read, or rather, try to understand what value photodo's metrics might have as just a piece of the information you need to evaluate a lens.

 

Godfrey

[negro y blanco fotógrafo]

Well, you can't believe everything that you read but it should come as no surprise that many of the 35mm lenses are 'sharper' than many MF lenses-they have to be. They have a smaller neg to work with. I also agree that there are other characteristics that make a good lens that have not been discussed here. It�s simple, MF lenses are sharp, there�s not doubt but you make up so much by having a larger neg that maybe the MF lens manufacturers feel that they can get �lazy�. Who knows�go take some pictures�

[gene crumpler]

Photodo makes for some interesting reading and can help make decisions. I was lusting after a Leica outfit until I looked at the ratings of comparable Leitz and CZ lenses for the G system. I picked up a G1 and three lenses for about 1/2 of a M body alone. I've been very impressed by the quality of the results from the G1 lenses. My limited resolution tests of the G lenses and several nikon lenses rank the lenses about the same as the photodo results. The testing is done on Hasselblad's optical benches, so I suspect the results are reliable.

 

Resolution is not every thing in a lens, but if you are sharpness nut like myself, testing is useful in sorting out the coke bottles from the gems.

[steve_rasmussen]

"There are some highly touted 'APO lenses of various manufacturers graded here, that despite their stratospheric price, ain't so hot."

 

The problem with APO hype is that it only addresses one aberration out of the seven common ones. Advertising a lens as APO gives the buyer no idea about the other aberrations, only longitudinal chromatic. So we're supposed to think that just because it says APO, that we're going to trust them on the remainig aberrations? Leitz now has a 90mm APO. My old 90mm f/2.8 Elmarit-R was a great lens without being APO. Take all the marketing with a grain of salt. Real world results are better than MTF testing.

[roger_urban3]

I own several Zeiss lenses for my Contax. Photodo trashed the 28-70, yet this site with MTFs for Zeiss lenses rated it much better:

 

http://www.geocities.com/ilprode/TestZ.htm

 

So, did Photodo botch the MTF test? Did Photodo drop the lens first or get a dog lens?

 

Since I own the lens and compared it with the 35-70, the 28-85 etc. It definitely holds it's own and has remarkable color. But, after reading Photodo I guess I'm supposed to throw the lens out. I've had people absolutely praise pictures taken with the 28-70 lens. Guess they didn't read PHotodo and know in advance that they are not supposed to think for themselves, but are supposed to follow what the self-elected authorities say.

 

As has been said before, take Photodo with a spoonful of salt.

[jonathan brewer]

Roger Urban...........They didn't trash your lens, they trashed their lens. It would be ridiculous to give up a lens you're happy with because of the published results of a test, or get rid of your wife(if you're completely happy with her) because someone approached you and said she doesn't rate very high, I don't think anybody's suggesting that, certainly not me, although I understand the bemused reaction to results that don't jive with the performance of your lens.

 

I take these tests as a probable indicator of one thing, that while while CZ and schneider may have been ahead of the pack many years ago, everybody else has pretty much now pulled even.

 

Also they give pause to the issue of hype, that because a lens is advertised an 'Apo' w/asheric elesments by one of the top outfits and costs a fortune, doesn't mean it is superior.

 

I'm not going to lie, I was please with the fact that one of my lenses were rated highly by these tests, but I knew that before I surfed onto the site, just as you know your lens performs to your satisfaction irregardless of the tests.

 

As an hopefully humorous aside to you, I was never going to pay $6000.00 for a leica zoom even though I've read a few times how it will cook you breakfast and then wash the dishes, before it blows away every other lens in existence, I didn't beleive that.

[huw_evans2]

I'm seriously confused here. I've visited the Photodo site many times, and have just checked it again, and not a single RB lens of any description has been MTF tested. Not one. Am I missing something?

[dermot_conlan3]

Jonathan your RB 180mm lens was NOT among the lenses tested on Photodo. There were 39 MF lenses tested. Go to the MF lenses tested section, not the complete list of lenses available. You see a bunch of M7 and RZ lenses tested, only the rangefinder lenses of the M7 score over 4.

[jonathan brewer]

Huw, Dermot.......Yes, thank you, I stand corrected on that, I misread the list. I'm still happy with the 180mm KL, and feel it is at least the equal of anything from the other MF manufacturers.

 

I rechecked the other grades regarding the lenses I comented on and the Mamiya 180mm KL notwithstanding, those comments still apply.

[ptucci]

In defense of the MF lens manufacturers: perhaps it is not because they are lazy that resolution is lower, but technically more difficult to light an area 2.7 to 5.4 times the size of 35mm (645 thru 6x9).

 

To make an analogy, have you looked at a cheap 19" color TV next to an expensive 32"? The smaller TV will be brighter and sharper every time.

[andrew_schank]

Most of my Pentax 645 lenses got only average ratings there, but are capable of super crisp 24 X 32 inch prints. By the way, the thing that people grab on to the most from photodo site, that single number they assign, is not that relevant. If you believe the lens they test is identical in performance to the one you own (which is definately not a given), the MTF graphs themselves have more useful information in them to people who understand that sort of stuff than one silly "2.8" or "3.6" number.

[Spearhead]

If people took the energy they put into reading (and testing) lenses and put that same energy (and a lot more) into processing and printing, their photographs might improve. 99% of the photographs I see (and I see a lot) suffer not from any lens problems but from poor processing and printing, especially printing. Look at what makes photographs great, and it sure isn't the lens. Look at what affects the final quality of a print and it sure isn't the lens.

 

A good image from a "poor" lens, properly processed and printed, will always look better than a lousy image from a "good" lens that is carelessly processed and printed.

[thomas_diekwisch1]

Agree with Jeff.

 

The photodo site is great. They sat down and did uniform tests of a large number of lenses. With the obvious flaws that their and any other test system has. Think about Kornelius Fleischer's tests where he got his film developed in a nearby grocery market's automatic processor. Nothing against grocery markets, by the way...

 

Anyway, the result is that there is a lot of manufacturer hype out there, that we all know of (see the recent H1 Zeiss/Fuji debates).

 

In my eyes, and please forgive my interpretation, these and many other tests (Mamiya recently published a comparison between their 645 and the 645 Zeiss lenses) show that the lenses of the big manufacturers are pretty much on the same level. The big lesson is that Zoom lenses in general are not so good as fixed lenses, but beyond, there are very few general statements that hold up.

 

Back to Jeff. Once you know they are all very similar take the one you like most and shoot...

[kelly_flanigan1]

I love this stuff!<BR><BR>Its like rating women; wines; hifi's & speakers; coffee's', pocketknives, religions, politics, fishing lures, movies, best bible passages, best books, best places to live, best places to vist, hammers, shovels, best way to raise children, best pickup truck type, best music, best actor or actress, best motor oil to use, balh blah blah!<BR><BR>THE ABSOLUTE BEST TOASTER WE EVER USED WAS THROWN AWAY IN 1965; BECAUSE IT HAD ONE ELEMENT BROKEN.......now after several decades; I got a similar one at a thrift store....It needs a slight rebuild; but I am a happy camper again!<BR><BR>Several slides shot with my 3 element Rodenstock lens in my Graflex Graphic 35 exceed the resolution of my 2700dpi scanner. I will rescan these again with my 4000dpi scanner. The Germans in 1956 knew a wee bit about optics; and did a fine job.

[cliff_henry]

This subject has been beat to death on this and many other threads. I have read most of the post and a couple are correct about what MTF is acturally testing, i.e, measuring ability of lens to resolve line pairs of different widths from its center to its edge. There were even a couple of post, on another thread, by some engineers that used words with more syllables that I can count.

 

The one thing I have never read in any of these post is that the results of a MTF test is for ONLY the lens that was tested. Yes, I know lenses go through quality control before leaving the factory and the results of a MTF test might be representative of the basic design of the lens tested. However, all manufacturing has acceptable tolerances for all the components that go into the finial lens. Simple said, the design of a particular lens may be good - some are better than others - but every lens of the same design will give DIFFERENCE MTF test results.

 

I have worked for - represented in sales - Leica, Fujinon, Nikon, and most of the major manufactures of scopes and binoculars. I will never forget a factory tour of a major scope maufacturer's lens factory. They made their own lenses as well as most of the lenses for the other scope factories. Interesting to watch lens glass being ground. The quality control manager showed how each lens was tested after grinding. They had a tolarance for each factor measured, i.e., resolving power, distortion, etc. End result - some lenses (glass elements) are better than others. The QC manager also showed me the special place he kept the lenses that were of minimum tolerance - the best glass - which he used in assembling "special units". I don't know what he did with those "special units", he wouldn't say, but my guess is they were the units that were shipped to the press and product reviewers and well, you get the idea. As a manufacturer, you are always sure that the people that are going to hype your product to the buying public get the best - "special units" - you have to offer.

 

So, while a MTF test MAY be helpful in discerning the general quality of lenses from brand to brand or within a brand, from consumer to pro, IMHO it will not tell the resolving power of your lens unless you have IT tested. If you have a good lens keep it, if not sell it and test another.

 

Cliff

[thomas_diekwisch1]

Cliff has a very good point. I remember that I had bought in the mid-80ies two Rollei HFT lenses, one a 80mm Planar, another a 250mm Sonnar. Both were not very good and made me wonder why I had moved up to medium format. A 120mm f5.6 Zeiss S-Planar however made up for it, and a Schneider Variogon as well. Needless to say, I sold the Planar and the Sonnar. Actually I even sold the Variogon, but not because of sharpness, but because it was such a brute. Later I bought a 50mm Rollei Distagon and a 150mm Sonnar for the SL66E which were both top quality. The moral: it wasn't Rollei or Zeiss or Schneider or the Japanese. In my case, I had just been unfortunate to have bought two bad-day Rollei lenses in a row.