Camera review standards for photo.net -- opinions solicited

[charles_miller]

Let me give specific instances of typical camera body limitations which are seldom, if ever, mentioned in reviews:

 

1) The built in flash only works with (physically) short lenses because longer ones block part of the flash output.

 

2) The built in flash will cause red eye, only a little less severe than a point and shoot camera.

 

3) The built in flash requires several seconds to recover, but if you shoot in continuous mode, the camera doesn't wait for the flash to recover, thereby wasting film.

 

4) When autofocus assist illumination comes from an external flash, taking the flash off camera can direct the pattern away from where the autofocus sensor is pointed, i.e. no autofocus.

 

5) If the SLR sync speed is 1/125, almost any point and shoot camera can do flash sync faster than it.

 

6) SLR lenses have to let in enough light for human vision, and must be designed to clear the mirror. Accordingly, inexpensive SLR zoom lenses are not optically much better than point and shoot lenses.

 

7) The focusing screen in some SLR models is a poorly designed for fine manual focusing.

 

8) The goals of a 100 percent view, a short shutter release lag time, and low mirror vibration cannot simultaneously be provided in a lightweight inexpensive SLR body.

 

9) One of the stronget advantages of an SLR over a point and shoot camera is depth of field preview. However, several SLR model don't provide a DOF function.

 

10) The number of people who think that a microchip in a camera body is capable of reading their mind is amazing. That function isn't available yet.

 

11) LCD panels that are not backlighted are useless in the dark.

 

12) The mere listing of a feature in a camera brochure in no way gaurantees that feature has been implemented in a practical way. As Phil points out several places on photo.net, having to remember and perform an obtuse programming sequence using "leetle buttons" is a bummer.

 

13) Many SLR cameras provide an exposure lock function which uses an older analog electronic circuit known as "track and hold". This is accurate for only a few seconds before the charge on a capacitor bleeds down. These are the SLR models that have a tiny, difficult to push, impossible to hold, spring loaded buttons for exposure lock.

 

14) If you like taking pictures of people, a quiet camera is a big advantage. Children in particular are distracted by any noises made while zooming or focusing just before the shot. This rules out several SLR bodies as well as most point and shoot comeras.

 

15) With any camera that uses removable lenses, there are many extra opportunities for dust to enter and accumulate inside the lens and inside the camera.

 

Well, I've got other thing on my plate now, but I'm certain the above is only a short list of the deficiencies in typical camera reviews. Anyone else care to add?

[bill_tuthill]

Great discussion, but instead of arguing, I'd just like to say that photo.net's Minolta section is badly out of date.

[charles_miller]

I have a few moments now, so I will continue my ode to under-reported camera inadequacies....

 

16) In almost all SLR bodies, metering doesn't function while the mirror is locked up, so you have to use manual settings and hope the light doesn't change. And in almost all SLR bodies, the camera will not automatically bracket exposures in the manual mode. So the photographer has to change the exposure settings manually, and with most cameras this risks bumping the camera out of position during macro shots.

 

17) If you use an eyepiece shade on an SLR, you have to take it off to get the film back open.

 

18) If the camera only has a central spot meter, and the camera is on a tripod, the meter will probably be reading off something in the scene which gives unreliable results.

 

19) In most SLR models, results from the evaluative metering mode will sometimes defy any analysis.

 

 

Come on guys. Help me with this.

[andrew l. booth]

Wow Charles! None of my cameras have even half the features you mention. Makes me wonder how I manage to take any pictures at all...

[charles_miller]

Andrew, I use an old and simple Nikon N2000 as well as other cameras. I guess one point I was trying to make about SLRs is that in the rush to incorporate gads of "features", some models have been rendered less than desirable. I could name a couple of such models, as could most folks at photo.net. I would simply be more impressed with camera reviews if some of these short comings were revealed more thoroughly and clearly.

[bobatkins]

Charles - most of your points apply to every SLR. I suppose you could add:

<p>

(20) If you drop it it may break<br>

(21) If the batteries go dead it won't work<br>

(22) You can't get sharp images with a hand-held 300mm lens at 1/15s<br>

(23) It won't take a 3/8-16 tripod threaded screw<br>

(24) It won't do TTL metering with a $10 Vivitar flash<br>

<p>

And so on. My guess is that the list is almost endless.

[colm boran]

One minor correction to Charles' list:

 

13) The circuit is called "sample and hold", not "track and hold". If it "tracked", it wouldn't be much of a exposure lock, would it?

[charles_miller]

"My guess is that the list is almost endless"

 

That sounds like it may become the basis of an excuse not to consider the short-comings problem at all. Boo, hiss! Why not just put the top thirty possible short comings in a standard check list to be considered by the reviewer?

[yuriy y. vilin]

Phil is looking for input, right? I think the short one is best one, so here is mine;

<p>

1. Review must be written by a person who used reviewed gear extensively.<br>

2. Review mustn't be a mere reproduction of manufacturer's specs<br>

3. Review must be accompanied by images taken with it. Ultimately, every major statement should have a "Figure" or "Figures". Sorry, that's where my scientific background kicks in.<br>

4. Review must be thorough, cover all issues related to actual picture taking (AF/AE accuracy), ergonomics and built quality.<br>

5. Review mustn't focus on direct comparison with competing product.<br>

6. Review must be well organized, divided in sections and have "Conclusions" part.

 

<p>Other than that, suggested guidelines cover all essentials.

[yuriy y. vilin]

I guess there is one more thing;

<p>

Submitted draft must be reviewed by a Photo.net editor prior to posting.

[patrick_ingram]

Mr. Miller... I think Mr. Atkins has rather a different point. Some of these shortcomings seem not to be problems with specific camera models, but rather problems with the universe as we know it (i.e., 6, 8, 15, 18 to name the most obvious). In place of 18 one might as well simply say that if the camera is pointed randomly, it's probably not looking at something you want to take a picture of. Some of the other comments are more about cameras, but I still wouldn't want every review to mention that the camera in question cannot meter with the mirror locked up, just as I don't want to be reading that a certain SLR body won't iron my shirts for me. It would be a nice feature, but so far no bodies have it so it's not much of a review issue (more of a "send a letter to Nikon/Canon R&D" issue).

 

I think it's great if reviews mention a few obvious oversights on the part of the designers, but those should probably be restricted to features often available on other units (i.e., mirror lockup).

[ole_tjugen]

I'll comment on Large Format only:

 

What's important to me is:

 

Type of camera (bench, field?)

Format (4x5, 5x7, 8x10 ?)

Range of swing/tilt

Lens board type ('standard'? easy to find?)

Ease of use

Construction

 

 

What I mean is - why do some people use Gandolfi, if not for the beauty of the construction :)????

[bobatkins]

The issue with displaying images taken with the camera is that they are essentially useless. They make nice decorations but there's no way you can get any sense of image qulity from a small web image. If you post a 20MB tif file you might get some idea, but even then the quality probably depends more on the scanner than the lens.

 

You could post photomigrographs of resolution test chart images and they would, in fact, be useful, but you'd need to specify a standard film, standard processing and a standard chart - and you'd need the ability to take photographs through a high quality microscope. The number of reviewers able to do this properly would be very small.<div></div>

[yuriy y. vilin]

Bob, if you are talking about sharpness/microcontrast, then yes, online scan tells little about this. However for things like vignetting, flare/ghosting and distortions web image is a valid way of demonstration.

[bobatkins]

True to some extent, but things like flare and ghosting are very difficult to quantify. I can take pictures with a Canon "L" lens which show flare and ghosting and I can take pictures with Samyang 28-300 zoom that don't. Distortion is easier to show. Vignetting is tricky since it depends on what contrast (gamma) level you use for the scanned image. You can reduce or increase the visibility of vignetting quite easily depending on your exposure/scanning parameters - and the scene you shoot.

 

Attempting to do "scientific" testing without a standard is a route to confusion.

 

I'm not saying images hurt, just that their actual (as opposed to perceived) utility is small.

[yuriy y. vilin]

Well, you brought that up, Bob. Next step would be a development of lens testing guidelines. Assuming that only a few of us have opportunity to use optical bench, the set methods should be easy to perform and available to anyone (camera and lens is still required).

<p>

1. Test in contra-light conditions to test flare/ghosting (sun in a viewfinder)<br>

2. Distortion test (brick wall, architecture shot, anything with straight lines within peripheral areas of image where distortions are most prominent).<br>

3. Vignetting test (open sky shot, wide open and stopped down)

4. Rendering of out-of-focus areas at maximum apertures (background blur, bokeh, whichever term is preferred)

 

<p>These are criteria and techniques that I've used to test my lenses for a few online reviews.

 

Yuriy.

[charles_miller]

Patrick said, "

 

I think it's great if reviews mention a few obvious oversights on the part of the designers, but those should probably be restricted to features often available on other units (i.e., mirror lockup). "

 

Yes, thanks for your comment. Any idea must used with discretion, perhaps more so than I indicated with my lists. But the advertising and promotional campaigns for many cameras are deceptive, and I simply would like photo.net reviews to provide a tactful, yet EFFECTIVE, counterpoint whenever it is deserved.

[bobatkins]

Can you cite a specific example of a deceptive ad, i.e. one that makes you think a camera has a feature it doesn't really have or can do something it can't do?

[charles_miller]

OK Bob, here goes,

 

Nearly all camera brochures for entry-level cameras show pictures taken of a professional model at an exotic location, not mentioning the use of thousands of dollars of lighting or light modifiers, the use of several professional quality lenses, or that the whole shoot was aided by an assistant or two who used hand-held meters. No starter SLR with a cheap zoom lens will do that. No big SLR with the best lens and an attached flash will do that either.

 

Quotes like "We take the best pictures". This is in the same class of silliness as a feature once listed on the box of a cheap vacuum sweeper, i.e. "Dirt Seeking Headlamp".

 

Specifications like "multiple exposures are possible". "Possible" if you know in advance how many exposures you want. "Possible" if you can remember how to program the camera. So every time you see the word "possible" in a camera specification, subtitute the phrase, "possible but difficult".

 

Pictures on a Contax TVS carton I saw were taken using MF, but the brochure inside the box showed a smaller image taken by a TVS, implying the whole image was taken by a TVS. Tsk, tsk.

 

The N6006 I had only took sharp pictures when it was bolted down to a large tripod because the mirror vibration was ferocious. It was said to be popular with photojournalists because of its relatively low shutter lag. Was I supposed to guess from that "information" that the N6006 mirror-up goes KA-WHOP?

 

The N80 has all sorts of complicated autofoucusing features to track moving subjects, but only a 1/125 flash sync, which is an obvious mismatch of capabilities. How long until the average N80 buyer will realize he really needs something more for action photography?

 

I never saw even the slightest notice from Canon that a camera with a pelicle mirror has special problems. Any light entering through the eyepiece not only throws the exposure off, but it reflects onto the film off the pelicle and affects the final image too. And any dirt on a pelicle mirror degrades the final image, which is a problem not experienced with a regular return mirror. Come on Canon, puleeeze give us some sort of little hint about things like that.

 

Every manufacturer produces a super sharp and inexpensive 50mm lens, and the photo rags really reach their stride saying a $100 lens is as good as a $500 lens. And of course, bokeh rhimes with hokey-pokey, right?

 

Beginners are encouraged to get an autofocus prime lens or two right away, but many autofocus prime lenses are formulated for extra sharpness in the center at the expense of edge sharpness. The overall quality of autofocus lenses may be about the same, or even a little better, than manual focus lenses. But if someone is going to do landscapes, high uniform sharpnes is more important than fantastic center sharpness. It is amazing, but some very recent cameras don't even work with manual focus lenses. (expletive deleted).

 

A friend was complaining about inconsistent focus and exposure on her Rebel 2000, and when I tried out a Rebel 2000, I found that it often would not refocus when pointed at something closer. But the Rebel G I tried worked flawlessly. Many of the same features of the Rebel 2000 are in the EOS-7, where they are said to work well. So think of the Rebel 2000 as more of a "training bra" than a real camera, for instance.

 

Bob, we are in the Fraud Neighborhood. Can YOU say "APS"? Try it now. And how many of those amazingly superior APS SLR outfits have you bought so far? Exactly how many...hold up your fingers.

 

I still hope that photo.net will bring useful objectivitiy to camera evalutions, but if photo.net is going to have a Unique Selling Position (USP), it can't play dumb about camera shortcomings forever. Over a dozen commercial rags already do that very well. On the other hand, nothing succeeds like success, so maybe photo.net should just emulate them. It seems that you are debating it, and I don't want photo.net to fail from a debilitating case of honesty either. Good luck following the "Luminous Path".

 

[patrick_ingram]

I'm not sure that I agree with everything Mr. Miller says, but there is a very good point about camera ads, and in particular camera ads that we don't know are camera ads. No-one expects camera ads to point out obvious flaws. It would be nice if they did, just like it would be nice if instead of "We love to see you smile" McDonald's ads said "We spit in your burgers but don't worry, they're made out of rat ass anyways". But that's not gonna happen since the small increase in sales from people thinking "Honesty? Wow!" would be overwhelmed by the large decline in sales from people thinking "They can't even make their product SEEM good." My question is: Are reviews in magazines really more objective than ads when the magazines are largely funded by advertising dollars from the companies that make the products that the magazine reviews? I'm not saying there's some grand conspiracy, but Pop Photog writing a negative review of the F80 seems a lot like biting the hand that feeds it. That is the advantage that PDN has, and that is why the reviews here are not redundant. Of course, if we're asking for loaners we might be in a similar situation, but there are ways around that.

[bobatkins]

Charles

 

All that can be summed up by "Advertsing exagerates", pretty much a self evident observation that everyone is aware of.

 

Most of your points equate to "if you stick your fingers through the shutter it won't work", or "A $250 body doesn't focus or expose as well as a $1200 body". Yes, correct, what else do you expect. If reviews point out the obvious they're going to be very dull and boring.

 

I suppose if you're writing reviews for the terminally clueless, you'd have to include all that stuff.

 

Obviously we disagree, so if you'd like to take a camera body and review it, submit the review to photo.net and see what sort of feedback you get.

[charles_miller]

Bob,

 

Well, I'm better at hyperbole than I am at eloquence, so please accept my sincere apology for any unwarranted directness. I simply wanted to express my view very clearly, but of course it is indeed only one view.

 

Best regards,

 

Charlie

[bobatkins]

Directness is fine. Different users want different things from reviews. Personally I don't find camera body reviews very useful. I can read the manufacturer's spec sheet and brochure and learn pretty much everything I want to know. Lens reviews are more useful since the spec sheets don't tell you about performance, but the problem there is objectivity. What one users may regard as good performance, a more critical user might not. Getting objective data is tricky and comparing "objective" data from a random set of reviewers is even trickier.

 

Some users may want to know what I think is obvious. Clearly you can't meter when the mirror is locked up because the metering sensors are in the viewfinder (actually on the prism) and that light path is blocked by a locked up mirror (and the sub mirror used for spot metering in some cameras is retracted). It's not a design defect and all cameras suffer from the same "problem". You can't autofocus with the mirror locked up either, but again that's obvious, all cameras suffer from the "defect" and I really don't need to be told that in every review.

 

If I had no idea of how a camera worked, I suppose I might need to be told that, but by the same token I'd need to be told just about everything. A reviewer has to assume some level of basic knowledge on the part of the reader or he/she would have to explain everything from the meaning of "f-stop" to the ISO rating of film.

[patrick_ingram]

C'mon, Bob... camera manufacturers could just put a second metering system on a small retractable arm behind the mirror... then when the mirror locks up you can still meter, and when you want to shoot the second metering system would swing out of the way and... oh. :)

[charles_miller]

Thanks Patrick and Bob.

 

I think part of the problem I have with most consumer gear, including cameras, is that some great features have been offered only in the past. With some Minolta models, you could plug a hand-held spot meter directly into the camera metering system. With some Olympus models, the OTF sensor was also accurate for normal exposures with the mirror up. Also with some Olympus models, several spot meter readings could be averaged.

 

With one of the Contax models (or is it a Leica model), you can trigger studio strobes with the camera, and the camera will set the lens aperture correctly for the subsequent shot. The Nikon F5 allows exposure bracketing in the manual mode, which is a super convenience for macrophotography on slide film. Canon models compute DOF, which is really nice.

 

So why can't a top Nikon, Canon, or Minolta body have several optional backs, each designed for a specific working requirement? Presently, top models usually have only one optional back, but it enables a motley assortment of features. Most buyers only use one or two of these features, and often only the features that are built into other camera bodies, often from the same manufacturer.

 

In the PC world, if I want to change from Word Perfect to MS Word, I don't have to swap computers. In the audio realm, if I want to upgrade to a twenty-bit player, I don't have to sell off the speakers and amplifier. Ho hum.

 

Sorry guys, but I'm still not impressed with the design decisions which have been made for most SLR bodies. I want the camera manufacturers to make enough money off casual shooters that they can continue to design and sell the really neat, limited quantity, pro stuff at reasonable prices. The alternative is that an F5 would be priced about the same as an equivalent Hassy or Rollei. But few consumers are attracted to SLRs, and a useless or inappropriate feature set simply compounds the problem, in my opinion.