Photo Rating Suggestions

Just a vote to ditch the ratings, but keep the comments. Mark Meyer's idea of a "photos this member thinks are interesting" link from members' pages is good. I'd like to see that.

Get rid of the rating system, it's <b>pointless</b>. How does one assign marks out of ten to a piece of art? Do you see people in art galleries comparing Turner's works and saying, "well, that's a 5", or, "he could have put less cows in that one, I'll give it a 7"?

 

<br><br>One idea is to have <b>two</b> text boxes, one for technical criticism and one for comments on the artistry of the image, respectively, "you under-exposed, so the shadow details in the doorway are missing", and, "if you'd cropped more radically and placed the door off-centre it would have made the image more dynamic". Whatever, you get the idea. I'm no critic. ;o)

 

<br><br>Another issue is that criticism <i>itself</i> should be recognized as part of the photographic process, and people should be given credit for being good at it. I, for one, would like to practice and improve, because how is my photography ever going to improve if I can't look at another photographer's images and learn from them? One worry is that an ivory tower mentality could be creeping in, and the really good photographers out there may feel it beneath them to discuss the work of us lesser mortals. It isn't, it's something you <i>should</i> be doing, because how are the rest of us even going to improve if you don't, therefore how can you call yourself a good photographer if you can't hand over your skills to the next generation? Unbiased, well crafted criticism is a vital part of that, and should be as much a part of a photographers repertoire as compositional ability or skilled use of equipment. Recently someone posted a letter written by Ansel Adams in which he critiqued an image that had been sent to him - fantastic stuff.

 

<br><br>How about if the good critics out there, they can be selected by reading back through the critique pages, were asked to make a page on how to write a good critique? Not everyone on the forum is a native English speaker, and even those who are can be at a loss as to appropriate descriptive terms, so why not explain some of the words and terminology that <i>are</i> appropriate? People who write, "yes, I like it, the blue bits are nice", are wasting their time and that of everyone else.

 

<br><br>PS/ Looking at the critique page at 7.45am (Japan time, whatever the hell that is in the real world), all of the images presented are, IMHO, worthy of comment! There are no 'holiday snaps' in there.

No ratings without a critique, no critique without a rating. It's ok to rate a photo poorly, and a rating is a good reflection on a person's opinion and whomever doesn't like it can always ignore it but all ratings should be justified with a comment. (I know this could provoke "asdf or ..." comments, but it's a start).

- Ditch numerical ratings<BR>

- limit uploads to 3 a week (nothing worse than getting 20pics of the same subject in a row)<BR>

- limit folders to 5 pics (so when you click on a folder it doesn't take forever display)<BR>

- overall total of 20 pics per person (might help people edit their work to good shots)

I think your rating system is a good idea. However, how useful it is really depends on the quality of the community members. I've seen some good and helpful ratings + comments and _many_ bad ones. If only you could screen out those "immature" ratings via a report-warning-privillage suspension system. But then again, we all hate censorship...

<u>Principles/my operating assumptions:</u>

<p>

1) Quality filter is valuable for those wishing to learn by viewing examples of good work.<p>

2) Feedback is more important than ratings.<p>

3) Ratings still have value for feedback (at least to me).<p>

4) No ratings is better than abused ratings.<p>

<p>

<u>Suggestions (original or other)</u>

<p>

a) Eliminate self-rating (direct or spoof)<p>

b) Verify email address or other anonymity eliminator<p>

c) Require membership in order to post anything (ratings/comments/msg/pics/etc)<p>

d) Make rating system simpler - I like J. Watmough's idea or alternatively - Novice, Amateur, Pro<p>

e) Do not separate rating and comment process - if one rates, one must comment also.<p>

f) More limits on photo critique postings - maybe once per week?<p>

g) Require 2 (or n) comments on other's photos before a user is allowed to submit for critique.<p>

h) Have a process (no good ideas in my noggin right now) to delete comments without content (eg, "asdf").<p>

i) Establish cross-rating algorithm to auto-identify abusers (I'm sure there are some smart people at MIT that would love to put their collective noodles on this project)<p>

Lastly, I agree with just about all Tom has to say above with regards to improvement ideas.

<p>

Ta ta for now...

The numerical rating system is causing people to focus on the wrong things. It should be eliminated, and if people want to engage in constructive dialogue about their work, they should exchange comments or email.

 

I've said it before, and I will say it again. I am a college professor. If I could stop giving my students grades and could give them only narrative comments, I would do so immediately. I don't see why we should employ such a system here.

Get rid of the numerical ratings, keep the comments.

Why doesn't the "Services Photonet" people take a second to share their ideas to be critiqued by the Photo.net community?

After much thought, I'm also for getting rid of ratings but keeping comments (even though seeing who rated you and what they thought is kind of fun). Certainly get rid of the "top rated photographers" list.

 

I realize this will make the random non-POW photo on the front page difficult to choose, so maybe get rid of it, too, or make it random from previous POWs or from Phil's photos.

I knew you services people were really working down in the bunker below Harvard Square subway. You know where I stand vis a vis numerical rating. Rating system= 1/10. If something's worth fixing,do it. I predict the value/discontent ratio is going to be minus, whatever the "fix."... A critique is what? Statement of informed opinion by someone who cares to help- via words- of how one sees an image,accompanied with ideas (See some of the comments on Mr Hald's recent figure work for example.) If you get words, you reflect, go back and say, well I tried that but it failed, how would you eliminate that distraction. How do I get this kind of mood,impact,etc by use of color, composition, digitization and the like. What was the emotion I got from you vs what I sought? How would you out there do it to achieve what I think I was hoping to achieve.Where did I succeed. Where did I really go ploof?... The thinking person's critique. And even more, others can participate by proxy in the dialogue! Without, pray, the hostility which is creeping into other parts of the site. As you get more complex, then it becomes a challenge to defeat the system for- you know-er, those lovable rascals. It syphons off energy that could be spent in doing good stuff (for those who are eating their fiber daily and want to enjoy the company of others in the online experience).Consider just keeping written comments and no ratings. If you must have ratings to sort wheat from chaff, use a PANEL of raters. Enlist a panel of the week from members who have the wit, temperament, and experience to judge other's with interest and affection. Ladies and germs, idea is to keep photonet civilized and serious. Has a two factor,10 scale rating system helped or hindered?.Rhetorical yes. It seems painfully obvious with unacustomed sniping and fudging going on. Someone has to keep the dialogue high class.Good luck for trying. Rajeev and Lisa and Dr Strangelove and whoever else is down there with you :-))

I thought someone had better kick-off a strong "pro-ratings" case, as most of it so far has been pretty "anti".

 

Call me "Wet" about this but the Ratings System has done a lot for my own photographic self-confidence since I started posting pictures in January after 25 years wondering if I was any good (or, "Did I make the right choice of a day job in 1980?"). It probably has done the same or similar for others. It's also made me a few good friends too (amazing the number of people you meet on the road in Europe who subscribe to photo.net... and when two people with cameras get talking in a taverna....). These are all positive things.

 

The "Drys", on the other hand, who would abolish the system, characterise anyone who participates in the ratings (and worse, succeeds in rating well) as ego driven, impure beings who do not subscribe to the lofty goals of "sharing" and "learning" and other such nice New Age concepts. Some of these above have even proposed that introducing pictures to the site ruined it! So let's get rid of JPEG posts and then we would have a photographic site (some say the best in the world) with NO PHOTOGRAPHS! Wouldn't that be a great idea?

 

Don't throw the baby out with the bathwater. Reform the system.

 

One of the main ideas behind photography is to get others to actually see and appreciate real photographs ("Drys": please keep that foremost in your minds). We aren't all mired in the finer points of Dektol, Rodinal, T-grain emulsions, the such-and-such f2-point-whatsername lens as opposed to the OTHER f2-point-thingmybob lens, ENDLESS discussions as to why film is dead/alive and incident/reflected metering. We like others to see our pictures and we enjoy some (however inaccurate and prone to abuse) feedback as to how we're going. In my own case a lot of this feedback somes directly to me via email and, if it also involves matters technical, I always answer as accurately as possible. I also post a lot of technical stuff next to the photos. But I enjoy the appreciation too. What's wrong with that?

 

Where does all the "Dry" purist nonsense come from? Usually people who have not posted any of their own pictures, or if they have done so, rate averagely in The Big List. Does this mean they're not "sharing" their best with us, not "teaching" us how to achieve their lofty levels of expertise? Live and let live, "Drys".

 

As to a "comments only" forum: Comments are fine, but there are too many "wows" and "just superbs" etc. Don't really tell ya much, do they?... fairly inarticulate. Also, comments about photographs (and indeed about any visual art) remind me too much of Art School weekly assignments. Sometimes you can't say "why" you like a picture... you just like it and wish to give it a high numeric rating... or the reverse, of course. That method of appreciation should be allowed in parallel to commentary: photography's a visual medium. Writing about a photograph never substituted for looking at the real thing. If it did, we wouldn't take photographs, would we?

 

Some suggestions as to how to improve the ratings sytem:

 

1. Delete and prevent patterned ratings (all consistently "1/1" and "10/10" ratings would be a good start, especially from obvious bogus members). This does not include self-ratings as this is one way for the photographer to say what he/she thinks of their own work. If the system is abused, then only the photographer looks a fool.

 

2. Prevent more than one rating per set time period (say once per 30 seconds - then the blitzers would need to stay on line for a long time to wipe everybody out like "DA BombeR" did yesterday).

 

3. Up the limit of photographs for "Top Member" from 3 to 10, or maybe 20. It's too easy to score high with three flukes. The proper place for these photographs (they can hardly be called "portfolios") is in the traditional "highest rating photographs" filtered gallery.

 

And I agree with Vuk about Jo Voets - except for the "he" bit. She's a girl, Vuk.

Ditch the numerical ratings. If the mission of photo.net is to provide a forum to interact with others wishing to become better photographers, I don't see how a numerical score helps anyone. The number only acts to boost or shatter ones ego.

Make it so you have to leave a comment with the rating. Months ago a few of us caught a "professional" using kids and fake email addresses to raise his ratings. What I think is funny is when 2 people have had a difference of opinion on any subject, you look now and see 1's across every picture by that person you sparred with.

Here's a listing of ideas I'd think are good. Some are from others and summarized here, others are from me.

 

<ul>

 

<li>Reducing the spread to something like 1-5 or 1-3 (aka poor/average/good).

 

<li>Increasing the number of rating dimensions (categories like "exposure" for example).

 

<li>Make the rating system go away. I'm not too in favor of this one, but should ratings prove (reasonably) unfixable, this would probably be preferable to keeping a stupid system.

 

<li>Make the "do not submit this photo for rating" checkbox actually do something. What the heck does this checkbox do?!? When I check it on a picture, people can still rate them, they are still listed among my choices when I want to submit a photo for critique. I didn't notice whether they go through the "rate yesterday's picture", too many pictures to find out conclusively.

 

<li>Make the "do not submit this photo for rating" checkbox checked by default.

 

<li>Eliminate self-rating.

 

<li>Require a working e-mail address to create an account on photo.net.

 

<li>Do not allow 1/1s and 10/10s, as they <b>clearly</b> are wrong (ever seen a <em>perfect</em> photo? or a <em>perfectly bad</em> one?).

 

<li>Allow rating only by photo.net subscribers. This would kinda defeat the idea of leveraging a lot of people in the community though.

 

<li>Add a "mark as interesting" feature to photos that will list it on the public home page of the user, so that the world can see what I find interesting (and for me, so that I can go back and check out details or whatever). Maybe this could be used as an alternative/replacement rating system ("N people find this picture interesting"). Note that this is only a positive system: you cannot say a picture is bad, only that its interesting.

 

<li>From the oh-my-god-that's-crazy department: add a meta-critique system similar to what Slashdot has. This would indicate failure of the photo.net community to me, as it previously seemed (to me) to be a community of serious photographers, and didn't waste too much of my (or others) time.

 

</ul>

<p> Since numeric ratings are highly subjective (beauty lies in the eye of the beholder) scrap them altogether. Keep the comments.

 

<p> Failing that, give the ratings numbers some meaning. Like </p>

 

<ol>

<li> Poor

<li> Average

<li> Good

<li> Excellent

</ol>

 

<p> As mentioned in comments above, 1 to 10 is too wide a spread for a meaningful judgment.

1) Mark Meyer's suggestion is excellent--as it should be given that I had similar thoughts recently ;-) To accompany it, perhaps a "random photographers" list alongside "random recent" photos, with a formula to enhance frequency of appearance as a function of web-depth (the intricacy of interconnection to other photographers--I'll be glad to donate free statistical services to accomplish this)

<br><br>

2) The people expressing sentiments along the lines of "it's not perfect, but it's still OK" have probably not seen the kind of damage the ratings system can inflict on an individual. Have a look at the bogus negative ratings on <a href="http://www.photo.net/photodb/ratings_breakdown?photo_id=224669">one of Tom's photos</a> or go through my folder and see what happened there this afternoon. Top member-rated aside, a <b>visible</b> average score is likely to bias the next rater in the same direction and provide ultimately flawed (skewed) feedback (I have carried out graduate level research on this sort of thing, so trust me). Similarly, making the entire list of raters and scores visible has all kinds of other potential biases associated with it--for example, just the other day I found myself spontaneously going over to the portfolio of someone who'd given me a nice score/comment and looking around for a picture that I could be generous with. This sort of thing invalidates all "psychometric" validity of the numeric ratings.

<br><br>

3) Competition brings out the worst in people, especially men (who seem to be the majority here) and any sort of "top list" will encourage it. If photonet wants to do that, then they must ensure an iron-clad system to protect people from the sort abuse described above.

<br><br>

4) I spoke to Rajeev earlier today (actually urged him to start this thread) and noticed he kept referring the site <i>photo dot net</i>, whereas I and everyone else I've ever talked to simply call it <i>photonet</i>.

<br><br>

Thanks Rajeev!

I'm a newbie and I can't say for certain that I know what the gallery area is intended to be for. For instance, last evening I uploaded 6 photos of not my best, but just some stuff that was scanned before my scanner bombed. My intention was primarily to give an example of 'here's what I'm interested in photographically' and share a few photos. But, perhaps that's not the intended purpose and some would prefer that the area be left for exceptional work. In some ways I think this topic is just as germaine to discussion as the ratings system itself. I get the impression that some may feel there is too much posted to the area and the sheer quantity of material affects the ability to rate, comment, and discuss.

 

So, (and perhaps this is implemented and I am not aware) is it feasable/desireable to have an area for 'hi, this is what I do' photos and a seperate area/mechanism for persons to display photography that they consider to be worthy of and desiring serious critique? Just a thought; if it sounds off topic or far fetched, please excuse. Best, Greg

Greg, I think you should save postings for your absolute best, especially if you do not feel you are very accomplished. The more expert/artistic people will pay little attention if there is no spark --- whether technical ability or artistic vision --- in the picture, and you will gain nothing.

 

When you will do something of which you are truly proud, then take a chance on other's opinions (I am still terrified to do so). Then you will learn something.

 

Best regards.

If the rating system stays... and I don't mind if it does if there are some changes that include a total removal or reform of the top photographer page... I would argue that we need a larger scale not a smaller scale. There are a lot of good photos on here and a lot of bad ones. When I first started rating photographs a nice photograph would be a 5-6, a great one 7-8. I felt I needed to leave room for a photograph that really blew me away to deserve a 9 or 10. And there were a few. That is why I didn't feel right rating a photograph that was nicely lit and composed and technically correct higher unless I could see that picture hanging on my wall or someone else's wall if it wasn't in my taste but still a good photo. After a while you realize that not everyone was using the same system. If there's a pretty girl in it it gets at least a 7, if there's nudity and it get's me stiff 8, I think it's pretty 10. There's no standard.

 

The only protocol for rating is outdated, insufficient and most people probably don't even know how to get to it let alone know that it exists.

 

I liked the idea of a panel. That sounds cool. Give them directions on how to rate. Some people have given some great examples of how to rate based on how photo clubs judge photos. It's in focus +1, it's properly exposed +1, etc. Maybe instead of aesthetics and originality there should be technical and aesthetics. Technical at least isn't that subjective.

 

Some people have made suggestions about passing the numbers through different algorithms but I personally don't think that would help. What I think might be useful is to judge the rater taking into acount his rating vs the average and standard devation for each photo s/he rated then averaging them all. Then maybe even replace the top member page with the most "fair" rater page then you would go in and see a listing of the top photos that the rater rated based on how fair he rated them and they couldn't be his photos.

 

Uhm. That sounds confusing but it seems to make sense to me. It needs to be hashed out more but I'd like people to think about that last paragraph and try to make sense of it and see if it could work.

Have only "photo of the day, week, or month." Do away with best rated photo ever conceived or delivered by mankind. And also- end self-ratings. That is terrible. Love your site. Love your show.

Substituting a ten-level rating system with a tri or quad-level system would simply substitute "3/3s" or "4/4s" for "10/10s" and "BADs" for "1/1s".

 

The Ratings system has its benefits for those who participate (either as viewers or photographers) and does no harm to those who don't. It just needs reforming and tightening up.

 

I never read such a bunch of negative comments as some (many) of the above. Would you abolish democracy because it is abused? Or because you couldn't get into office? Or because you personally didn't want to be President or dog-catcher or whatever? Would you get rid of money because some people are poor and others rich? And the Professor wants to censor grades...as long as they don't come to me looking for a job, Prof, armed only with a touchy-feeley "narrative comment" and a new suit.

 

If you don't like the ratings system or the Big List, don't participate. If the site is being corrupted, elves, you should (and can) fix it... don't trash it because of the above photo.net Puritans (we call them "wowsers" in Australia).

 

BTW: a previous thread (about 2 weeks ago) suggested charging a small membership fee. Why not?

>Whatever is done, with all the fraud that's been exposed, all

 

>current ratings will have to be deleted in order to ensure a level

 

>playing field, so why not take that necessary first step

 

>immediately?

 

>

 

>-- Vuk Vuksanovic

 

Seconded! Seems futile, counter-productive, and a waste of time to carry on with the existing system whilst we are debating it's flaws and value to this community.

 

Deleting all current ratings and freezing the rating system would provide an opportunity to observe the effectiveness of the "comment only" suggestion whilst carrying on an effective debate about what should be done in future.

 

On a purely technical level, for beginners who are still developing their skills I feel there is a need to retain something which can demonstrate a "measure of excellence" to which we can aspire. How about changing POW to "Technical POW" to highlight and critically appraise a particular photographic technique used to produce a particular image? I like to see a "nice" pic as much as the next person, but doing so doesn't assist my own photographic development unless there is discussion of precisely how it was achieved with specific details concerning use of aperture, metering, exposure, filtering, Photoshopping and such.

 

At the end of the day, aesthetic opinions are like assholes, everybody has one however useful or relevant it may be... at least when discussion gets technical, aesthetics are put to one side and we can usually get a gut feeling for whether or not the person who posted a reply is talking complete s*** or not...

<i>"And I agree with Vuk about Jo Voets - except for the "he" bit. She's a girl, Vuk." -- Tony Dummett (Australia)</i><br><br>

No way, dude! Now I'm really interested. Jo, are any of the shots self-portraits?

<br><br>

<i>"As mentioned in comments above, 1 to 10 is too wide a spread for a meaningful judgment."-- Anoop Iyer</i><br><br>

The empirical data from numerous published studies suggests very clearly that you are incorrect in your assumption. The reliabilty of a scale will improve all the way up to 9 or more anchors/points, although the gains beyond 7 are minimal and drop off very steeply. It is also best to have and odd (rather than even) number of choices.

<br><br>

<i>"is it feasable/desireable to have an area for 'hi, this is what I do' photos and a seperate area/mechanism for persons to display photography that they consider to be worthy of and desiring serious critique?"-- Greg Sindewald</i><br><br>

This is an excellent idea, very much along the lines of having a <b>useful</b> check box to determine whether or not one is seeking ratings/critique. Maybe a "newbie corner" would be a good thing to implement (and perhaps also a dog-with-frisbee-in-back-yard snapshot jail cell).

I think there is some value in the numeric ratings. In general, higher rated photos/albums are better than lower. So, how do we keep the bozos from polluting the well? I vote for:

 

* Requiring that critiquers (either numeric or textual) to have posted some of their own work first. I think this is essential so that you can rate the raters. If you respect their work, you will respect their comments or ratings. It also raises the bar to creating bogus accounts. I guess there is a danger of tit-for-tat rating but that will happen regardless. (I practice what I preach here. I did not dare to comment on other photos until I had uploaded some of my own.)

 

* Go back to 1-5 and add make the values more clear (e.g. 1 = trite, 3 = Canventional, 5 = Imaginative) Image the fun discussions we can have coming up with names for each value!

 

* I also agree that the default size for the "batch" ratings is too small. In addition, there is not enough context. I also want to see the photographer's comments to see what they are intending with the photo. But I *don't* want to see the current ratings -- that always biases me. In fact, remove the "batch" rating altogether. It encourages skipping comments. Also the juxtoposition of widely different techniques and subjects is jarring.

 

4. Restrict the number number of photos submitted for critique per person per week. If someone is posting that often, they are probably just talking and not listening or thinking. (But it's easier to upload in batch. Is there any easy way to submit an already upload photo for critique?)