Camera review standards for photo.net -- opinions solicited

[michael_moon1]

I agree that the review should follow a format. However, it shouldn't just recite the camera specs. The review should state the specs, possibly even in tabular form. I also think it makes sense to have commentary restricted to that paticular family of camera - as a Maxxum user, I really don't care about what things a F5 or a 1V do better or not as well as a 9. I have far too much money into lenses to care whether I should have held off and saved and bought Leica. When it coes to handling, we do need a prose report of how the camera functions at its user interface - the downloaded photos will show the the camera's ability to produce images (and possibly flaws or shortcomings too) but the reviewer's report has to tell us whether it's easy to swich to spot and lock AE in a backlight or other uneven lighting situation, how easy it is to call up the onboard flash to get fill at -1.5 FC. Also, and especially because the reviews will be here for decades, we need to know something about how the camera fits into the brand's evolution, which would (along with all the other info & specs) help people decide on used bodies. There has to be flexibility in the reporting format; how else is a reviewer going to give a worthwhile assessment of the panoramic option in the XPan, the film door screen in the Maxxum 7, or the AF bracketing of the N1? But I welcome this development, and although the comments from the community at large are great they should supplement, not supplant, a thoroughgoung review done to a rubric.

[sidney_cammeresi]

It's a pretty lame request, I know, but I'd like to see a picture of the camera along with the review. Perhaps it is not important for modern 35mm bodies, which all look fairly similar, but if the camera is something much different from that (such as some spiffy new Happy Magic Family C5 medium format camera which no one has ever heard of), then it's nice to see how the controls are laid out on the body.

[mark_sirota1]

Philip, I think this is great. As someone who has only recently gotten back into photography (and therefore had to buy in), I've read a lot of reviews in the last year or two, and I agree that standards would be immensely helpful. Your proposed standards reflect the general approach of photo.net well.

 

I would suggest that this camera-review-standards document could be rewritten for a different target audience -- the reader of the reviews -- and a link to it provided at the top of every review page. That way people won't wonder why this article isn't comparing a Nikon body to a Canon body.

 

Keep up the good work!

[daniel flather]

Sidney has a good point.<p>

Pictures of the camera in the review. With 35mm, not so important but with MF & LF it

would help the reader in understanding the review better. Also most of us live in an area

that limits us to seeing these items in person. Where I live a LF camera is a rare beast, so

it's Ebay or nothing.

[john_bald]

<i>The tool focus means that we're interested in how a camera performs when asked to do a task.</i>

 

<p>I suppose only a technical writer like myself would get excited when someone writes a good set of writer's guidelines, but I say Bravo Phil! Your directions for writers will make for good and useful reviews -- the sort I'd expect to find on photo.net.</p>

 

<p>I'm looking forward to strong opinions backed up by intelligent observations. That's what I find when I go searching through the photo.net forums, but it will be fun to see them focused upon a given piece of equipment. (I mean <i>tool</i>.)</p>

 

<p>Thanks, Phil. And thanks in advance to the writers who will take the time to do the work!</p>

[robert_roaldi1]

Just one point to add: I second the suggestion of submitting a pic of the camera being reviewed but I would go further and ask that the photo be taken with the camera in someone hands. This gives an idea of size, which is difficult to otherwise determine. Depending on where some readers live they may not have easy access to a store with that model in stock for real world comparison.

 

Or, If possible a photo of the camera alongside another camera that's better known or more commonplace.

 

Such a photo comparison would easily answer, "Is the Oly E-10 the size of an F5 or an MX?"

[john_bald]

Sidney, Daniel and Robert have an important suggestion -- including pictures of the item under review, and shown with a familiar object for scale. Indeed, many of us live in the hinterlands and don't have access to stores with a wide variety of bodies, lenses, etc. (Thank goodness we have access to the Internet!)

 

I remember the first time I saw a display of digital cameras. Are you kidding? I thought. In magazine articles, their photos seem to show them as about the size of a standard SLR, but that's deceiving. In "real life," they seemed ridiculously small.

[plattro]

camera-review-standards looks good. How about some guidelines for lenses and other accessories? Do we also need some guidelines for 'Member opinions' in ezShop? I wrote a couple comments there, and I would be happy to write some more. I'll say right away there are certainly many people much more qualified than I to review some particular 'tool', just perhaps I had more time at the moment to write something.

 

Maybe I missed it, but is there a planned submission or posting process?

[patrick_ingram]

One more thought: the anti-spec people have me on their side, but it might be nice to also compile some spec charts comparing the features of all of the major slrs out there today. I know that when I went shopping for my first slr a while ago (needing something to replace the older slr borrowed from my mother), I spent a lot of time flipping between spec sheets to see which bodies had which features. Not everyone finds this important, but it might be useful to have a basic "features at a glace" chart.

[jay_scott]

I find it easy to locate information about the features and performance of cameras. I would like to see reviews that concentrate on the feel and usability. Feel: is it satisfying/solid/cheesy/finicky/whatever. Usability: is it logical/confusing at first, and once you get used to it; are things that should be easy actually easy; what kinds of photographic tasks are natural or awkward?

[steve_singleton2]

Please request the "warts" feature you mention, and the reverse, of every review. Learning what someone likes best and least is often very revealing, both about the product and the reviewer's viewpoint. It also helps the reviewer to get off the fence and take a position. Also, please ban use of the word "cool."

[kodus]

I think I rating system will help as well. The best ones I have found are the consumer reports ones - eventhought I disagree with their SLR camera reviews (last one was in 1993). Please let me explain by what I meant about the ratings. <p> We could use ratings on: 1) handling - how easy is it to hold the camera steady? How does a big lens affect the usage? How does it feel light despite having a fairly heavy weight? 2) easy of use - how are the controls laid out? 3) Features 4) exposure consistency - how accurate the metering system is? 5) Weight. Note that I have omitted a final score. I think this is very personal, so one has to judge by oneself. <p> The downside of a rating system is that it implies consistancy in judgment. Someone needs to have a number of cameras and use them enough.

[joachim_gerstl]

What a great idea! Reviews from real users. Most of the time you can find reviews of guys which don´t own the camera but own just the leaflet. Endless lists of features and technical data together with comments like well built, very good image quality, ...

Only if you really know something you are able to write a useful review. I plan something like that on my own website ( www.joachimgerstl.com ) but think I will concentrate on things like tripods, ballheads and other useful accessories.

Reviews like this and www.photo.net together with www.dpreview.com ( for all technical data and pictures of the product ) will be the perfect sources to preselect equipment.

[bobatkins]

Personally I wouldn't trust a review from anyone who hadn't lived with a piece of equipment (and used it!) for at least a month, preferably 3 months. Initial delight with a new camera/lens can turn into dislike/frustration when you really get to know its bad points as well as its good points.

 

Of course this means you don't get the first reviews out there, but what you do get are worth a lot more in the long run.

 

"Instant reviews" such as those you find in Petersen's Photographic magazine are a prime example of what not to do. They reprint the spec sheet, show a few images and rave over how great it is. I'm sure manufacturers love them and I'll bet they NEVER have problems getting equipment on loan. The trouble is the reviews are useless other than as marketing tools. They could probably be written (uncluding doing all the "testing") within an hour of opening the box, in fact I'm not sure some of them aren't.

[patrick_ingram]

Mr. Atkins, do you think it would be possible to borrow something for that long from Nikon, Canon, or any of the big producers (I'm actually asking, not rhetorizing). The problem is that, if it isn't, you're only going to get reviews by owners of the equipment. And my psyc. major girlfriend tells me that it is well established that people will give favourable feedback on products that they have spent a lot of money on (to avoid feeling like chumps who threw money away). Just look at the Nikon/Canon debate here on pdn. Is one really so much better than the other in ANY area that someone would get so worked up about it? I think most of the flame wars stem from the fact that people have invested a great deal in these systems.

[bobatkins]

Well, it's possible, though perhaps not as easy as a 1 week loaner. Or photo.net could do what the better consumer magazines do and purchase the equipment, test it and then sell it (by auction). That way you also get around problems of manufacturers sending out "hand picked" samples for review (not that they'd even think of doing that of course). It costs more of course, but if photo.net is running as a business there will be operating costs. Then again, if you buy at a really good discount price and sell on Ebay you might even make a profit. People do!

 

As for psychological factors, they do exist of course. Since Canon users should test Canon equipment, Nikon user test Nikon etc. there's already some built in bias. Personally, if I have a piece of equipment for over three months, I like it. Otherwise I would have sold it. Either way I can give an unbiased opinion.

[GerrySiegel]

If someone could write a page even based on a three month+ use of a camera/equipment and that person is a Community member of some standing I would likely buy on that basis. I did with the Hexar two years ago. (Your writing guidelines make good sense and as guidelines I see no holes)I think the point maybe is-'Go and Start Writing, Others will Help to Edit as Needed.' Or that could be the message.There are volunteers who can do that sort of thing,I trust, even I could collaborate on broad editorial review,looking for things that need fleshing out and getting ready for publication part, if that would help,even at this distance. Note: I got an emailed comment I solicited from a person I queried about the Minolta Scan Elite II, a hot new item in digital land,for film scanning(that is my next buy). This sort of networking goes on and is a real asset to the community.I think the identification of interesting items for review and soliciting of writeups is a worthy step. Good project. I hope to be able to participate. I use old stuff and my new stuff like the little Leica Mini is out of production. Need a writeup on the Stereo Realist,no sweat.

[charles_miller]

Phil,

 

Thanks, it looks good so far to me.

 

photo.net attracts the attention of so many photographers and wannabes that it will be impossible to serve them all equally well.

 

It will be a sufficient service if photo.net injects a degree of rationality into the camera market. The general corporate trend to keep consumers on a price escallator cannot be totally overcome by the easy availability of rational information, but it will help. Most manufacturers change models and features continually....so it becomes difficult to evalute the ratio of "manure to mollasses" on any given model. And I don't want to see photo.net reviews simply becoming recounts of slack-jawed amazement over every new transistor or photocell added to a camera, and enforcing that seems to be the trickiest part. Maintain the focus on "usable" and "practical", of course. And do it without getting way too serious or formal. That's the ticket.

 

Cheers,

 

Charlie

[bobatkins]

In my opinion, reviews need to be written by people who are not in love with cameras. Basically photographers, not collectors or equipment junkies. The trouble is that it's the equipment junkies who want to get their hands on the latest stuff and write reviews. Photographers are probably too busy with photography to bother about such things!

<p>

Photographers go through 3 phases:

<ul>

<li> Phase 1 - Equipment is everything!

<li> Phase 2 - Equipment is nothing!

<li> Phase 3 - Equipment is equipment

<p>

Some photographers get stuck in Phase 1 or Phase 2. Reviewers should be in phase 3.

[jeff_drew4]

I agree with Bob A. I have lusted, purchased, used, and then discarded several systems and formats over the decades. I rarely get excited about any gear unless I've laid hands on it and put it through my paces. I would trust some reviews from some writers, especially if I got to know them to a degree and learned how they arrived at their conclusions. I still get excited when I buy a new toy, but usually I've already shopped a solution prior to finding the good reviews. How many of you have used reviews to justify your purchases or choices after the fact?

[philg]

I'm surprised by how many people in this thread accept the idea that someone who owns something will be biased in favor of it. Maybe that is true of a 13-year-old kid who is proud of his new toy. But how many adults do you know who say "My house is the greatest house in the world" versus who say "My house is falling down and the electrician won't return my calls and I wish the kitchen and dining room weren't separated by such a long hallway..." And a house is a lot more expensive than a camera. In the world of aviation, which is renowned for attracting opinionated blowhards, you don't find Cessna 172 owners saying "My plane is so much better than that overrated Beech Bonanza"; they all say "Gee, if I had $600,000 to spend, I'd be in that Bonanza right now."

 

And really why would any bias matter? The reviews are supposed to be authored by someone carrying out a project with a camera or lens. If the reviewer is diligent about noting what went well and not so well in the project and the reader is critical in looking at the same images, people get the information that they need. If the author says the Holga is the world's best camera and her pictures are 100X better than mine, I really can't argue with her results, can I? If her pictures suck, why would I want her unbiased opinions since she obviously doesn't have any practical photographic ability.

[bobatkins]

Philip - We all know that people cannot be trusted to be unbiased, no matter how scientific the test and how much they try not to be.

<p>

For example, look at medical research. The only true tests are double blind where neither the doctor nor the patient knows if a particular drug was taken. In science in general, if a result is expected, it is often obtained. If it isn't, the experiment is repeated until it is!

<p>

A photo.net equivalent might be for one person to do the testing (take the pictures) while a second person - who does not know anything about the equipment involved - evaluates the results.

<p>

About bias you say "<em> Maybe

that is true of a 13-year-old kid who is proud of his new toy</em>". Well, have you read through some of the postings lately? It's amazing how much passion people feel about their equipment. It almost amounts to a religion with some people. They will defend it to the death. Photographers (of almost any age) are generally pround of their new toys. Bias is alive and well and living right here. I wouldn't expect anything else.

[pete_su]

I don't think bias should be an issue in camera reviews.

 

At some level, camera reviews are sort of meaningless and

boring anyway. No one can really tell you if you will like a camera.

 

All they can do is tell you what a camera is like to use and then

tell you if they like the camera. The first part of this can be

relatively objective (i.e. the camera has no auto modes and a

100% viewfinder vs. the camera has only an auto mode and an

optical viewfinder with the rangefinder in it).

 

The second part is bound to be somewhat subjective (the

camera is too heavy, the camera is too small, etc).

 

I think most people reading reviews want information of the first

kind. I think when they get information of the second kind the

flame wars start.

[patrick_ingram]

Now, I don't want to prattle on about bias, because I don't think it's a MAJOR problem in reviewing (though something to think about, always), but I must respectfully disagree with Philip. Yes, people complain about their houses and cars... and I am the first to admit that, if I had the money, I would likely buy a Leica system and leave my Nikon stuff at home (yes, equipment is equipment... but the simplicity and quality are so appealing). That said, I still think you'd get a more objective review out of me by dropping some kit in my lap and asking me to wander around shooting with it for a couple of weeks than you would by asking me to review my own stuff. For starters... I picked my stuff over the other stuff out there (of course, this could be a useful bias in a review, since readers might be able to determine if they have the same shooting interests as you). And I stick to my story that many people are hesitant to admit to themselves that they may have wasted money... especially on stuff purchased recently (it's easier to say "I was so dumb" than "I am so dumb"). And Bob A. has a point: some people on pdn are, with respect to equipment, exactly like the 13 year old boy (and some are stuck in phase 2).

 

Is bias a critical factor in a review? I guess if people write reviews well it isn't. The guidelines posted are certainly a leap in the right direction. So, Mr. Greenspun... how might I get involved?

[bobatkins]

Unless you chose reviewers carefully, all you'll end up with is a clone of "epinions.com". Random reviews from random users. If you want to know where all the controls are, what the specs are etc. it's usually all on the manufacturer's web site. Might as well just copy that. No reviewer is going to weigh the camera, calculate viewfinder coverage, verify exposure range etc. And who is going to risk testing weather sealing on a camera they don't (or do!) own?

<p>

There's not a chance that any photo.net reviewer without access to an electronics and optics lab is going to come anywhere near a Popular Photography SLR or lens review for example. You might not BELIEVE the Pop Photog tests, but they have the equipment and the time to do it right (at least in principle).

<p>

The question then is what can photo.net do better than the rest? I'd say carefully considered long term reviews of cameras. Perhaps something like the long term tests that automobile magazines do. I know of no photography magazines that do anything similar, nor web sites for that matter.

<p>

The only problem is that equipment may become obsolete (in the sense that it's replaced by a new version) before the long term test is up unless you start the test right after an item is released!

<p>

There's also certainly a place for "epinions" too - and photo.net really already has that.