Photo Rating Suggestions

There is nothing inherently wrong with having the Photo.net �community� rate images. I understand arguments regarding the arbitrary nature of assigning a subjective rating to someone�s �work of art�, but it has been my experience that healthy competition brings the collective quality of images to a higher level. When I was in music conservatory in Manhattan, we had to perform recitals that were adjudicated. You may say, �How on earth do you <I>grade</I> a performance of a Mozart concerto or an improvisation on a Charlie Parker composition?� Well, <I>nobody</I> came un- prepared or gave a sloppy performance. That stupid grade, that arbitrary number made people work harder, and pushed them to excel on a higher level.

<BR>

<BR>

 

I proposed to the Photo.net staff via email a system where the lowest grade is thrown out, deleted. This would halt the 1/1 hit and runs raters who either think it�s funny or some kind of perverse strategy to prevent someone from getting a decent score. Just because there is abuse within the framework of the rating system doesn�t make the <I>system</I> inherently flawed. People need to grow up and use the critique forum with intelligence and compassion. Many people enjoy the critique forum and for those who don�t have any images posted to gruffly say �Get rid of those damn ratings� is not entirely fair. If you don�t like the critique forum and posting, there is plenty to keep you both occupied and happy at Photo.net. You can sit for hours and talk about lens sharpness and ballheads. And the people who enjoy looking at and writing about images and technique can have their area of interest as well in the form of the Critique forum. Obviously it needs some tweaking but to do away with it would be completely unfair to those who enjoy it. Lastly, please do not put too much importance on the �comment� section. Comments from someone with experience who knows what they are talking about are invaluable. But many of the comments sometimes miss the mark � things like �Have you considered cropping 2/3�s of the image or all of the area around the subject?�

I don't think that people without photos online should be banned from rating/critiquing photos. Some people may not have the time or equipment but their input isn't necisarilly any less valuable. Also some people have admited their images suck but that doesn't mean they can't point out what doesn't work in other's photos. They obviously recognize the flaws in theirs :)

<i>"Seconded! Seems futile, counter-productive, and a waste of time to carry on with the existing system whilst we are debating it's flaws and value to this community.

Deleting all current ratings and freezing the rating system would provide an opportunity to observe the effectiveness of the "comment only" suggestion whilst carrying on an effective debate about what should be done in future --shiu wan</i><br><br>

A 10/10 for intelligence and comprehension!<br><br>

<i>"If you don't like the ratings system or the Big List, don't participate. -- Tony Dummett (Australia)</i><br><br>

Tony, although I love your photographic judgment, I think you need to think a little more carefully about the influence of "top-rated" status on your perceptions of the system. Being at or near the top of the pile ensures numerous ratings and, consequently, a lot less vulnerabilty to revenge raids, not to mention the bias involved with assuming the system works when it places one at or near the top; this, of course, is not to say that you don't deserve to be there (you know what I think about your work), but are you realy anywhere near 700 positions ahead of Jo, with all the "cuddly puppy" and "Velvia canyon" people in between? Most importantly, your suggestion to not participate in ratings isn't choice right now!!! It's all or nothing for posting viewable pictures.

(1)throw out the two low ratings and the two high ratings..makes it a little tougher to cheat

(2)change it to a 1 part 1-10 rating...do i like it or not.1= i wouldnt wipe my ...er,nose with it,10=i would hang it on my wall if i could afford it(even if it isnt tack sharp and it IS alittle dark/light)

(3)make it anonymous,i do not want anybody giving me a high or low rating because i gave them one

(4)get rid of the rate yesterdays uploads section..it takes absolutely no thought to zip thru and rate these(hoping to get to something you might find interesting)

(5)limit raters to those who have shots posted

(6)do not under any cicumstances take the rankings so seriously(after all we all shoot for ourselves..haha)

I have also had my work panned by "members" with dead rock star names and hotmail addresses. They have posted no pictures and display a rather limited vocabulary. This is not a simple problem to fix (just as I have noted some members rating their own work as 10/10 to inflate their ratings, I guess the new format which shows who rated your work will put a stop to that one).

 

My only suggestion is that members should have posted pictures before rating others (I know this will not solve the problem but it may wave off the casual "crank call" types). I don't think any photographer should be able to rate their own work EXCEPT in their own comments about the work, I see the rating system of a way to guage others reaction to my work, uninfluenced by my own opinions.

 

I realize these suggestions have all been made but I feel that the biggest problem with the system is getting your work noticed in the first place, and this is not a problem to solve, just an unfortunate situation which is not easily dealt with. Maybe if there were defined categories of work, even such broad ones as "black and white", "portraiture", or "manipulated digital" so your work would be seen by people interested in that particular area... oh well I guess that just opens another can of worms, if it ain't broke (too bad) don't fix it.!

I actually like the rate yesterdays uploads. It gives me a chance to view photographs and photographers I might have never seen. Although I hate it when I have to scroll through a whole roll of someone's birthday party pictures. There should be some sort of restriction on this. I think it has the possibility of being a great feature. As someone suggested earlier it should have more information.

 

It would be nice if there was a feature similiar to that for the photo critique forum that allowed you to go through all the current photos in the forum or subsection of the forum you haven't rated.

 

I remember when I wanted to rate all the concert photos. Click on the picture rate it, comment on it, go back 4 pages start over. After the second one I realized you can do Open in new window, rate it comemnt on it close it. But not nearly as convienient as the rate yesterday's uploads thing.

As to posting photos to be allowed to rate take a look at this person's average ratings then look in his portfolio. Do any of the photos look familiar? Can you guess who he's been giving his average ratings of 9.55 to?

<p>

This really cracked me up you have to <a href="http://www.photo.net/shared/community-member.tcl?user_id=357711">Check it out!</a>

<p>

Also the user's name should be recognizable to most.

Getting rid of grades or ratings is not "touchy feely" at all. The object is to get people who want to learn to focus on what is wrong with their work, what they can improve, and what they do well, so they can maximize their strengths. I teach at a pretty elite institution, and for better or worse the students get there by being ultra-grade conscious. I want them to think hard about difficult ideas, not about their grade point averages. Narrative comments potentially convey useful information. Ratings convey virtually no useful information. They harm the entire community, because they cause people to think about the wrong things. If you think the rating system has helped you, then it has already numbed your mind.

 

photo.net existed quite productively for a long time without a ratings system. It is still quite new. I say nip it in the bud before it gets worse.

I'm in 100% agreement with Tony from Oz. And...

 

>> Vuk - "The people expressing sentiments along the lines of "it's not perfect, but it's still OK" have probably not seen the kind of damage the ratings system can inflict on an individual." <<

 

If a photographer is that influenced by what a crowd of virtual strangers think, especially the 'thought-less' ones abusing the system, then its probably best that the photographer *not participate*. Very simple.

 

>> Vuk - "Top member-rated aside, a visible average score is likely to bias the next rater in the same direction and provide ultimately flawed (skewed) feedback (I have carried out graduate level research on this sort of thing, so trust me)." <<

 

I use the photocritique page the way I imagine it was designed - you do not know what others feel about an image until *after* you critique it. Seems fair enough to me. Why not use the system in the way it was designed? It solves exactly the problem you describe.

 

>> Vuk - "Similarly, making the entire list of raters and scores visible has all kinds of other potential biases associated with it--for example, just the other day I found myself spontaneously going over to the portfolio of someone who'd given me a nice score/comment and looking around for a picture that I could be generous with. This sort of thing invalidates all "psychometric" validity of the numeric ratings." <<

 

And are the ratings designed to be authoritative, I might ask?

 

I still see nothing wrong with the basic system as it now stands. Sure, make it more difficult to cheat, but do not change the fundamental principles behind the rating methodology today. The system allows for rapid and broad participation. The system was not designed to support serious competition.

 

Methinks people are taking this far more seriously than the issue warrants. Just my opinion.

A moderation system seems to work well enough for slashdot (http://www.slashdot.org). The idea of 'karma' - those users that post useful information earn you points that you can use towards rating others. Rating points are a resource that you have to earn.

 

Wouldn't your ratings mean more from people the community admires vs. joe blow shooter or mean shmuck (aka anonymous cowards).

 

I also propose that you can also gain karma points by posting highly rated photographs (kind of a self propegating system).

 

While we are on the topic of moderation, I also propose that we add a moderation system to the user comments, so that if someone gives obviously bad advice (or potentially dangerous advice as sometimes happens) it can be moderated down. Again, I point to you slashdot to see how this works (and works VERY well). FREELY available code exists (slashcode - as written in perl) exists that could be modified to fit in the photo.net framework.

 

See http://www.slashdot.org/faq/com-mod.shtml for more information on how the moderation system works at slashdot for ideas. We would of course have to tailor the system to photo.net, but I think it would result in a much more meaningful rating system!

 

Plus, it encourages people to invest time and effort in answering questions and gives people reasons to post only their most excellent photographs! (Charge people karma to post pictures, this would prevent people from posting TOO many photos).

 

Oh... one more thing. I propose that Tony Dummett and Ellis Vener start with +50 points, and the rest of us shmucks at zero. <grin>

 

Cheers!

"Tony, ..., I think you need to think a little more carefully about the influence of "top-rated" status on your perceptions of the system. Being at or near the top of the pile ensures numerous ratings and, consequently, a lot less vulnerabilty to revenge raids."

 

I copped it the other week Tom (as did a lot of others) and dropped off the Big List screen for a while. When I was able to look at the ratings (a few days later) I saw patterned "1/1s" and patterned "10/10s" on most of my images. I have (and others have too) "friends" and "enemies" (although sometimes I think it's a case of "the enemy of my enemy is my friend") - respectively annoying and embarassing to me. My suggestions above in this thread included detecting and deleting these patterned ratings (good and bad).

 

A server-based photo.net filter to disable or delete these irrational "friends" and "enemies" would solve a lot of the current ratings system problems. If it's easy for my addled brain to intuitively detect false high and low ratings, why can't the Math PhDs at MIT do the same? (In fact I bet they already have mapped this out). To simply just abolish ALL high and low ratings ("patterned" and "genuine") because they're there smacks to me of censorship, totalitarian style. Get rid of them and we could then all live in a Wonderful World where all people are perfectly equal and get to talk about lenses and emulsions all day.

 

As to your comment re. unavoidable ratings, "It's all or nothing for posting viewable pictures", what's the problem here? If you don't care about ratings you could just ignore them. Anyway, the elves could sure fix this bug, couldn't they? After all, we're talking about future solutions here, as well as current problems.

Michael.

 

You've been a member for about 3 weeks and you haven't uploaded any photos. Perhaps some newbie reserve is called for here.

TOM

My comment on bandwith was referring to the concept that sending a couple radio buttons clicks for numerical rating would take no more internet bandwith than a textual comment. That does NOT mean I know what I'm talking about, just a flippant opinion.

 

VUK

Indeed I don't have any idea of "the kind of damage the ratings system can inflict on an individual".

What kind of damage can be inflicted? Are you referring to (commercial) damage to a professional's reputation? Surely the photonet community is not a major determinant in that respect is it?

Or maybe it is. Not being a pro any more than a bandwidth expert I don't know.

Are you refering to ego bruises from lowered averages?

That is a self-inflicted wound from choosing to care about the numbers. Solution as I said before: don't read'm.

I suspect that you meant something else & I just missed the point, but thru in these 2 opinions anyway.

 

TONY

Thank you for providing your defense of the numerical ratings.

I did not have any real reason to form an opinion either way when I commented earlier, just wanted to lip off about the option to ignore the numbers rather than get upset, but you pointed out how it allows expression of wholistic reaction to a photo without reduction to textual analysis of a visual experience. Good point.

 

All take me with a grain of salt now, eh! I have no axe to grind on either side of the issue. Live & let live.

Leave Mike alone, you...you...brute!

 

(Note: this has nothing to do with his "100%" endorsement of my own comments).

 

Cheers.

<i>As to your comment re. unavoidable ratings, "It's all or nothing for posting viewable pictures", what's the problem here? If you don't care about ratings you could just ignore them.-- Tony Dummett (Australia)</i>

<br><br>

Tony,

<br><br>

If it were just me viewing, I could ignore it, but my mother looks at these photos too (doesn't rate!!!) and a Montenegrin mom is roughly equivalent to a Jewish one. How could I ever explain these bad scores? What a shame for the family! ;-)

>Just because there is abuse within the framework of the rating

 

>system doesn�t make the system inherently flawed. People need to

 

>grow up and use the critique forum with intelligence and compassion

 

>-- Markus Arike

 

 

I think the evidence shows that there is abuse within the framework because people are not using the system with intelligence and compassion, which renders the framework inherently flawed.

 

...

 

Think the elves need to evaluate whether the intrinsic value of the rating system is greater than the effort required to minimise it's abuses to acceptable levels. As it stands, the current system is being undermined by a small minority who seem intent on "damage"

 

The elves should also consider that nothing requiring an opinion is foolproof because fools are so ingenious when it comes to finding ways of cheesing off others. If you enforce the requirement of email addys prior to rating, fools will create additional aliases. If you track IP addresses to cut down additional email addys, fools will log in elsewhere. If you restrict the use of extreme values in calculating a rating, fools will adapt to it. If you introduce a complicated algorithm to limit the effect of foolish intervention, fools find a way to circumvent it. If you create a panel to rate pics you get Big Brother Fool and his elitist, foolish opinion. If Big Brother Fool says your carefully silhouetted pic is under exposed, his foolish opinion becomes gospel...

 

Fools are everywhere, but even a fool has to admit that 1+1=2 ... unless they're a moron which is something else entirely ;)

<p>Andy Graham: <i>throw out the two low ratings and the two high ratings..makes it a little tougher to cheat.</i></p>

 

<p>I'm somewhat puzzled by the suggestions to discard low ratings. Say I upload a photograph, and someone rates it 9/9 (hah!). The next person who comes along and rates it lower has their rating discarded, and the next, and the next. There's no way the photo will ever get a lower rating. Unless I'm missing something it makes the whole system pointless.</p>

 

<p>Aaron Helleman: <i>A moderation system seems to work well enough for slashdot (http://www.slashdot.org)</i></p>

 

<p>Aaron, moderation sounds like a good idea, except that the sheer <i>volume</i> of the images posted to photo.net and the number of members able to rate each one makes it impracticable. I can't even begin to imagine the amount of work involved in overseeing it.</p>

Vuk, just in case you feel I was singling you out, I wasn't, its just your comments caught my attention and I initiated a reply. Frankly the rest of the comments were coming into this thread far too quickly for me to bother scanning them for other relevant points. Nonetheless, your comment back to me deserves a quick reply.

 

> You've been a member for about 3 weeks and you haven't uploaded any photos. Perhaps some newbie reserve is called for here. <

 

I pointed out in this same thread earlier that I recognized that I'm a recent member without having contributed. However, those two points do not make me a 'newbie'. I've spent some time using Photo.net as a resource prior to signing up for posting priveledges, have read hundreds if not thousands of postings, and feel I have a decent sense for the community. Otherwise I'd not have commented at all.

 

The issues being discussed here are not unlike those found in other on-line communities, and I've been involved in a number of them, some quite actively, going back in time before most users on this system ever heard of the Internet.

 

As long as people are civil and being constructive, I see no reason why a 'newbie' should refrain from joining in on a discussion if a person has an honest and thoughtful comment or contribution to make. It is exactly this open access to contributors from around the world that Photo.net the terrific resource that it is.

 

PS to the person suggesting Slashdot, not a bad idea. At least the community has a hand in floating worthwhile commentary to the top. - cheers all.

I don't care for the rating system-all is has done is brought bad feelings to this site. I'd get rid of this whole competitive thing which seems to bring out the mean spirit in people. People rating there own stuff as a 10 to try and be at the top of some silly list-what is that? Other people rating every image they see as a 1-1, I think these must be the same folks that get off by keying other people's cars. I never even asked anyone to rate my images and still some of them in my portfolio have been attacked by the hidden critics (whose have now been named so I can know who these idiots are) I don't see any benefit whatsoever to the system where a bunch of strangers are rating images like this is some kind of Olympic event. I don't need it.

IF ratings are going to be kept, I might make a few suggestions:

 

A. I like the idea of a probation period where new members can't rate images. It would mean folks would have to be more industrious to beat the system.

 

Secondly. How 'bout a cumulative ratings floor and ceiling? I've noticed that most really low ratings come from folks with average ratings in the 3-4 area, and some really high ratings come from folks in with averages in the 8s. I've seen fewer of the latter, and I'm inclined not to see that as so much of a problem; one might be compelled to rate and comment only on outstanding images, but what kind of person is the other way 'round?

 

They (them - you know who they are) could institute such a floor, so that after, say 10 ratings, if the numbers were below the threshold, a warning could be displayed. Or better yet, the ratings by those people could just suddenly not be counted in the overall numbers.

 

Of course, then one could use the same bogus ID to bump up some images' ratings and murder others. But then THEY could implement a filter that finds such IDs, which only have extreme rankings, without middle-of-the-road ones. THEY could implement a series of criteria, without publicizing them, that could severely impair the non-sophisticated cheaters' ability to affect rankings. Of course, the clever cheaters will always be able to foil any such system, but hey, we're not all Tom M. (joke - don't flame me!)

 

(Aside: today I found a former Picture of the Week winner who has four 'fans,' all email addresses on his domain, that have ranked all 20 of his images 10/10, and have no other rankings. Such behavior surely merits the removal of his image from the past POW page, doesn't it?)

 

Cheers.

Tony in your "Response to Tom" comment above you quoted Vuk not me. I didn't say anything about you.

Rachel: The beauty of the moderation system is you make it user run. No work involved by the maintainers, it runs its self.

 

How you ask? When every someone posts something, others in the community that find the information valuable (that have moderation abilities) can mark the posting as +1, +2 up to +5. Or, if they feel it is bad advice, can mark it as -1.

 

How do you become a moderator? At first, people well respected in the community could be approached to be a moderator. As time goes on, those people that visit photo.net regularly are randomly asked to become moderators automatically. The system tracks users and their habits and those that qualify randomly are asked to moderate.

 

This works very well for the tens of thousands of visitors daily to slashdot.

 

The score of a posting is public, and users can choose to view all comments or set a threshold viewing level (ie I only want to see postings who's moderation level is higher than 2 to weed out junk). Or for those who are short on time, they could set their level to 5 to see only the GOLD postings, really juicy stuff.

 

If you get your postings moderated up, this earns you karma. Also, being a regular visitor earns you some karma. Also, posting highly regarded photographs earns you karma.

 

Then, when you go to post pictures OR rate photographs, you have to spend your karma.

 

Cool, hey? And the best part is once it is set up, it runs itself !

 

Cheers,

What's wrong with you lot? Rating photos and having a league table of "best" photographers makes perfect sense. You can see who's doing good work in the eyes of the majority - it is a learning forum afterall. Besides, it's fun.

 

Yes, some people will abuse the system - that's human nature. And of course a numerical system can lead to differences in interpretation - but this isn't the Olympics. There isn't fame and fortune waiting for those who rank in the top ten. It's an internet site for crying out loud!

 

Maybe it could be improved if there was a little random validation from the moderators from time to time to confirm scores - especially if things look suspicious. But to abandon it because of a few idiots or for fear of the nastiness of competition??! C'mon, lighten up. The fools will get sick of playing dumb games and we all can tell who the cheats are and who's being targeted.

 

My vote - keep the rating system and the highest rated photographers list and let the moderators do a little validation from time to time. I also say let a photographer rate there own pics (once) - isn't it interesting to see what a photographer thinks of there own work?

Can't read them all, but good thread. I'd vote to abolish the numbers (it's ART! everyone to their own thing!), keep verbal comments, and create categories for photos so that all the folks interested in architecture can go there, etc. I think most of us would sample several areas and occasionally all of them. I also like the idea of pro's being involved in the comments somehow. Perhaps also a moderator's group by invitation only? And, I would think the idea of neither posting or commenting for at least three months after joining, is a good idea to foster community. And anything else that's reasonable to keep down the B.S. and junk posts.

 

As far as clogging the message board, categories are really badly needed. How many can even scan all those headings usefully, and then, there are so many meaningless subject lines that people use. So, thanks for a great site!

I mostly agree with Michael Watkins.

 

I like the entire system as it is, some people are taking it too seriously. Ratings are obviously subjective, somewhat arbitrary, and can occasionally be rigged.

 

I like all the changes made in the last months- the once a day posting rule, the top rated photogs, the viewing of ratings and of those who have rated and what their average rating is.

 

I think that Photo dot Net HAS SHOWN A LOT OF WISDOM in how they have set things up.

 

Perhaps a rules and etiquitte page.

 

Scammers and anonymous flamers will always be with us, let's hope they don't pollute the site. They haven't for me, yet.

 

Quit trying to conrol the web, it's not yours. Many of your issues are free speech issues. The increased knowledge of your critics, their nature, and their e-mail addresses are full compensation for any disargreements you may have with them.