Interesting Test?

[glenn nk]

This has been on the net for a few hours:

 

http://clublexus.com/forums/showthread.php?t=326811

 

And it's causing some considerable angst.

[colinsouthern]

I think that that test will break a few records for the number of arguments it starts!

[bueh]

Amzing results! Only, if I could afford an 1D III....

 

And that lens looks tasty, too.

[denny_wells]

This is playing with equipment miles out of my price range - but it was interesting reading and pixel peeping anyway.

 

One of the most thoughtful responses I saw was this:

http://forums.dpreview.com/forums/read.asp?forum=1021&message=26241433

, pointing out the relative pixel density of the cameras involved (simple math on sensor size and number of pixels).

 

This morning there was a thread about the fine quality of a large print from a 6 MP camera - an example of a real world application demonstrating that the in practice, MP's are not always king (though they are undoubtedly important). And then later in the day we get this pixel-peeping thread that demonstrates . . . well, that with more pixels you can do better clinical pixel peeping.

 

Fun reading!

[frankie_frank1]

Glenn, that is a 100% crop comparison. 1DIII has twice the resolution of D3. With 100% crop, that is the difference.

[leopoldstotch]

<p><i>"1DIII has twice the resolution of D3. With 100% crop, that is the difference.</i></p>

<p>The 1d Mk III is 10mp, which is not twice that of the d3.</p>

[John Crowe]

Interesting reading. Fascinating lens that I can add to my wishlist along with a Nikon 300/2. Of course I'll be sure to mount them on a Canon!

[zigzag]

Interesting - and about as likely for me to try as the satellite test. Like the moon landing - as long as the technology flows down to the proles I'll be happy.

[kevin conville]

Frankie Frank, Dec 31, 2007; 10:13 p.m.

:Glenn, that is a 100% crop comparison. 1DIII has twice the resolution of D3. With 100% crop,

that is the difference."

 

Whaaa? Anyone with a keyboard....

[robin_sibson1]

It is indeed an interesting test, but what its actually shows isn't obvious. If, just to make a wild conjecture, Canon were to introduce a FF camera with a 16Mpixel sensor using the same photosite technology as the 1DIII and achieving the same performance, then this would be a like-for-like comparison of camera performance. That's distinct from a comparison of photosite performance, for which you would need to scale the crops to matched pixel density and, to make comparison easy, ensure that the test text was the same size on each of the comparison crops, either by changing the subject distance (we're assuming that the lens is 'perfect' regardless of subject distance) or using a different font size. For most practical photographic applications, it is camera performance not photosite performance that matters.

 

When comparisons are made across different formats, matching the physical scale of the sensors, which is what has been done in this test, is surely the least interesting of the three options from a photographic point of view, although I am full of admiration for the basic approach of the test, of eliminating variability due to lens performance or limitations in order to concentrate on the performance of the sensor. To me, the most interesting version would be to scale the comparison crops so that the whole frame would be the same size in each case, and then to match the text size (again, either by changing camera-to-subject distance or by using a different font size) to make comparison easy. I very much hope that the person who did the test might explore this possibility.

 

Incidentally, just for interest, how much do these lenses/telescopes cost, and will I be sorry that I asked?

[brucecyr]

"Incidentally, just for interest, how much do these lenses/telescopes cost, and will I be sorry that I asked?"

 

$10,000 +/- a few thou -- not sure how that strikes you, but it makes me feel fiscally-challenged :-)

 

BTW, I enjoy your commentary for throwing some perspective on the test that didn't occur to me.

[robin_sibson1]

Thanks, Bruce, I was expecting the price to be higher, but then I realised that it is only a 4-element lens and doesn't have IS ...! It's still well into the range where (for me) utility for money exceeds utility for toys, but it's certainly interesting to know that such lenses exist and apparently leave current-generation camera lenses way behind.

[eric merrill]

Seems to be pretty cut and dry. If you want to take photos of newspaper taped to a wall using an exotic telescope in low light conditions, the Canon wins this round.

 

 

Eric

[robin_sibson1]

Eric, you're not entering into the spirit of the thing at all. You have to remember that this is a gearhead forum, nothing to do with taking pictures.

[John Crowe]

If you read through all the original threads you will find that they had no intention of worrying about the sensor size difference since they were only going to pixel peep. It made sense to me at 1:00 AM but now I am not so sure, although the 1DIII was considerably better not just a bit better. I'd still like to know how the 400/2.8 with 1.4x faired against the Takahashi.

[kevin conville]

"Bruce C, Jan 01, 2008; 08:54 a.m.

"Incidentally, just for interest, how much do these lenses/telescopes cost, and will I be

sorry that I asked?"

 

$10,000 +/- a few thou -- not sure how that strikes you, but it makes me feel fiscally-

challenged :-)"

 

Again, a wealth of misinformation on these forums :( Why make stuff up?

 

The OTA (optical tube assembly) which is what was used for this test and DOES NOT

include the equally expensive (at least) mount for extraterrestrial viewing is $4K.

 

Here's where you can buy one:

http://www.buytelescopes.com/product.asp?t=1&pid=12521&m=

[kevin conville]

This talk of pixel density and photosite design is kind of missing the point isn't it?

 

The Canon has better image quality, period.

[brucecyr]

"Again, a wealth of misinformation on these forums :( Why make stuff up?"

 

To assert an error represents willful misrepresentation is... indicative of your frame of mind, I suppose. My info came from here:

 

http://www.takahashiamerica.com/catalog/index.php?cPath=21_26_31_36&osCsid=c13e33353a01d12777da9a3f6ffaa40c

 

which apparently represents "Set Complete" pricing. Sorry for the mistake.

[kevin conville]

Bruce,

Whether you want to admit it or not, it was an error.

You didn't know what you were looking at and passed along bad info. Granted this isn't

the crime of the century but it is just the thing that weakens these threads and causes

them to breakdown into tangental discussions.

 

The implication that you were making it up was inappropriate. I could have used more

tact.

I'm sensitive to bad info being parroted on these forums I guess because I've had more

time than usual during the holidays to actually read some of these threads and I'm

disheartened by much of the ignorance.

 

Sorry if was accusatorial.

[brucecyr]

"The Canon has better image quality, period."

 

Kevin,

 

First, I suppose I should thank you for pointing out a tangential error -- grump grump grump and smattering of formalistic applause.

 

Second, that has nothing to do with my next point: Basically your "period" is misspoken, as I suggest below :-)

 

In this test by an ostensibly well-heeled photographer with a decent sense of experimental procedure, the Canon appears to do better than the Nikon. But anyone with engineering or scientific training knows that experimental claims have to be independently validated. Experiments can have set-up flaws, and results are not necessarily generalizable beyond the immediate context in the absence of a randomizing strategy that is notably absent in this experiment.

 

As someone has noted, all the Canon has been shown to do so far is take better images of text at long distances. Actually, I would guess that finding might generalize to other subject matter, but lets wait to see.

 

More importantly, this trial is anything but definitive in the marketplace. The mere appearance of a full-frame Nikon body cum Sony sensor is enough to enhearten Nikon diehards -- they can always look forward to future developments, especially since Sony is a large, diverse corporation which could devote huge resources to developing better full-frame sensors. Or it could decide there's more money to be made with less effort in movies and game boxes. But my guess is they're prepared to slog it out for the long-haul, especially given their presence in the sensor market, their acquisition of Minolta's SLR expertise, and their corporate pride in their technological prowess.

 

Its way too soon to jump to categoric conclusions on the basis of a single trial -- period :-)

[frankie_frank1]

The D3 when cropped down to a 1DmkIII size is the equivalent of a 7.4MP camera versus a 10.1MP, respectively. 1DmkIII wins.

 

The D3 when cropped down to a D300 size is the equivalent of a 5.2MP versus a 12.2MP, respectively. D300 wins.

 

The D300 when cropped down to a 40D size is the equivalent of a 11.1MP versus a 10.1MP, respectively. D300 wins.

 

The D3 when cropped down to a 40D size is the equivalent of a 4.7MP versus a 10.1MP, respectively. 40D wins.

[kevin conville]

Bruce, et al-

The guy's test wasn't scientific, I'll concede. But it wasn't bad either. And, one has to trust

his integrity to buy into the conclusion. I bit. Maybe "period" was a little narcissistic.

 

I understand that strictly speaking his test "only" shows that the Canon bested the others

with a Takahashi APO scope attached, photographing printed words indoors, but I don't

think it's too hard to extrapolate from this that Canon "probably" excels elsewhere.

 

The part I find a little funny, and maybe I'm not understanding what people's actual

intentions are, is the pixel comparisons. So what is it? Pixel density is good now? Are

Nikon and Canon pixels more or less equivalent now that noise is being so well

managed? So the pixel race is back on?

 

My point was that it seems somewhat moot as long as the image quality is there. I don't

buy lenses based on MTF graphs either.

[robin_sibson1]

Kevin, let me respond to your last two paragraphs in a slightly different way from my original posting, to try to clarify the points I was making.

 

To me, what matters is achievable image quality. So I agree with you on that. Understanding what contribution the sensor makes to any shortcomings, independently of the optics in front of it, is something I find helpful.

 

Except when I am working at the limits of the telephoto end of my longest lens, I expect to make pretty much full use of what the sensor can capture in the majority of cases. So I am interested in comparing the whole of what one sensor can capture with the whole of what another sensor can capture, even if they are of different physical sizes. That means I want to look at images (or test crops from them) where the whole of what the sensor records has been scaled to a common size. To make such comparisons easy, I want the test images at the end of that process to match as well, and that can be achieved by either of the methods I suggested. That comparison process is the one I would regard as photographically the most significant, and it makes no reference to pixel count, or pixel density, or even sensor type - it is equally relevant for a Foveon as for a Bayer sensor.

 

Whilst black-and-white test images don't tell the whole story - and images of proper test charts should tell more of it than images of printed text - I do also agree with you that there's a worthwhile amount to be learned from them. You could say that you don't need test charts to rubbish a lens when you've got bird plumage to do it for you. More generally, there are plenty of real photographic situations where defects visible under test conditions will re-surface.

[brucecyr]

"I don't buy lenses based on MTF graphs either."

 

Kevin,

 

How ironic! You've just showed how natural it is in discourse to corrupt a train of thought with tangential issues!

 

While you may not buy lenses based on MTF graphs, I'll bet your desire for various lenses is closely correlated with MTF graphs, or with functions that map an MTF graph to a much smaller set of numbers.

 

For example, I have used nearly a dozen non-Canon lenses on EF-mount adapters based on old.photodo.com MTF graph ratings in excess of 4.0, and everyone of them happens to be an absolute gem in terms of image quality, both in my own experience and that of the photographic community at large -- Contax, Leica, and Nikon lenses.

[davidlong]

The thread has been updated (see the posts at the end of it). The difference was just an artifact of the RAW converter used.