Changes to Rating System

[bobatkins]

Just an interesting observation. In the latest Pop Photog there's a review of some website or other (not this one) which has images and ratings. Pop Photog considered it a plus that you could get back at people who gave you low ratings by giving THEM low ratings in return.

 

Ah yes, the new sport of photography where nobody ever wins.

[mottershead]

Tim, I feel that you are the one who is not getting it. We are debating whether you should be allowed to give ratings in the range of 1-4 without a comment. The reform does not stop from you giving ratings in this range; it only requires you to make a comment.

 

The problem with the 1-10 scale is that it is too broad. With a nominal average of 5-6, that makes 1-2 two standard deviations below average. Only a tiny fraction of photos are that bad, and nothing is served by rating them. SOmething similar could be said of 9-10. When the average is 5-6, 1-2 and 9-10 are so far below and above the norm that they are effectively meaningless from the point of view of providing feedback, and they tend to defeat the accuracy of the system.

 

This is one of the reasons (non-anonymous ratings being the other main one) that the average rating has now risen to 7. With an average of 7, 9 and 10 become meaningful ratings, but 1 and 2 become even more meaningless and 3 and 4 start to be highly questionable as well. As it stands only 5% of the ratings are in the 1-4 range, with most of those being 4's.

 

A more logical reform would have been to make a 1-7 scale and recompute all the existing ratings. However, I am working on a system for the photo.net Gallery visitors to "select" photos not "rate" them, and I wanted the easiest-to-implement reform that would stop people from giving thoughtless and useless low ratings, which cause the admin volunteers no end of trouble.

[bobatkins]

Brian - following that logic, why not just make the scale run from 4 to 10. If ratings below 4 are meaningless, dropping them should not be an issue. So just delete all 1-3 ratings and make 4 the lowest score you can give. A scale running from 4 to 10 has 7 as the mid-range and since you say "7" is the average rating, all will be well and you don't have to recompute anything.

 

If the lowest score you can get is a "4", people won't get highly pissed off when they get a "4". After all, it's much nicer then getting a "1" right? With a 1-7 scale you still have the problem of sensitive users being offended with "1" scores.

 

Yes, it's totally artificial and anyone with a brain can see it's just a sham, but that's not necessarily a reason not to do it. We're not dealing with logic here.

 

I still think the whole idea of ratings is flawed, but if you must have one keeping people happy is much more important than attempting to make it "accurate" or "scientific". It doesn't have to be fair, it just has to look fair.

[tim_vanblaricom]

Brian, I do get it, we just have a fundamental difference in philosophy, you see the new system as 7+, I see it, at least conceptually (as it has not been implemented and I will give it a chance to see how it serves me) as a 2. Look at that, we can both approach the same subject with good intentions and come to completely different conclusions. As an administrator, I respect your right to implement whatever policy you see fit, but I must admit I don't feel this will enhance my photo.net experience. Possibly I am wrong, we will see. Interestingly enough I got a 3 originality, 8 aesthetics rating today, a comment was left, it actually enhanced the positive rating and said nothing about the 'negative' rating. I found, even without the comment, it to be a very informative rating, my mountain pic was very pretty, but not all to original. I wish more people would leave ratings, just superficial first impressions I'll take, some I will take with a grain of salt, some will open my eyes a little more. I emailed my 'negative' rater and thanked him for his honest opinion.

I find Pop Photog's position on retaliatory ratings rediculous.

[john_oliver]

There should not be a problem with giving or receiving low ratings. I have received and given my share. Each time it has made me think "why does someone not like this shot". It has helped make be a lot more critical of my own shots - I now see flaws that didn't occur to me before.

 

It should be quick and easy to rate a shot - right now we have to tab over one. Minimizing the number of mouse clicks to rate would be an improvement.

 

Here are some suggestions for the scale - that IMHO would be best:

Aesthetics:

1 - below average - or do not like

2 - average

3 - above average - like it

4 - outstanding shot - love it - one of the best of the site

 

Technical:

1 - below average -some noticable technical flaws

2 - average

3 - above average or outstanding technical example

 

Originality:

1 - common subject & composition

2 - average

3 - very original.

 

 

All this adds up to a score out of 10 and is more meaningful to me.

 

For the purpose of an average rating for the top-list, etc, just drop the highest and lowest scores to get a bit of balance.

 

Would be glad to discuss. Let me know.

 

Regards,

John.

[bob stewart jacksonville]

this truly has gone from the sublime to the ridiculous. the ratings always have been fairly meaningless. But now when you look at a photo that could have been taken by a monkey that's rated 7.9, it just is absurd. personally, it has discouraged me from even bothering with posting or critiquing.

I've looked at photo recently where the horizon wasn't square, the subject wasn't in focus, the exposure was off, and it get ratings like 9/9 and comments like I really like the effect here. I guess I'm becoming a curmudgeon.

[bobatkins]

Perhaps the people who used to like to "monkey" with the ratings by giving good images a low score have discovered a new game. After having ther low score efforts thwarted by the new software their new method of attack is to seek out the worst images and give them high ratings!

 

That, or perhaps we have an unusually large number of monkeys online at present?

 

Clearly more work on the software is needed to stop this, perhaps by requiring a 100 word essay on "why I like this image" for any score except a 5, and by requiring proof of species before registering users.

[bobatkins]

How do you prove your "professional" status? Send photo.net a copy of last years tax return? Some "amateures" (sic) are much better critics than "professionals" anyway.

 

If people don't like your images they have a perfect right to give them a low rating. They don't need to justify their ratings. If you like your work, why would you care anyway?

 

The more complex the system, the more ways there will be to screw with it. Everyone should be able to rate any image with any rating they want to, with or without a comment. Then drop the high and low 10% of all ratings and "problems" with "unique" opinions vanish. Make ratings and images anonymous and you remove the personal conflicts too.

[fabriziogiudici]

<i>How do you prove your "professional" status? Send photo.net a copy of last years tax return? Some "amateures" (sic) are much better critics than "professionals" anyway. </i><p>

 

This is indeed a problem of the scoring proposals.<p>

 

<i>If people don't like your images they have a perfect right to give them a low rating. They don't need to justify their ratings. </i><p>

 

They have that right, but we also have the right to understand how skilled and experienced is anybody who gives us a rating.<p>

 

<i>If you like your work, why would you care anyway? </i><p>

 

So what is the rating system for? I believe that most of us likes most of our photos; but we are interested in a more objective opinion!<p>

 

<i>Then drop the high and low 10% of all ratings and "problems" with "unique" opinions vanish.</i><p>

 

Yep, but what if you get zero ratings? Out of my latest three requests for critique, 2 ratings and one critique... I'm thinking again if the effort to upload is worth while.

[bobatkins]

<em>Yep, but what if you get zero ratings? Out of my latest three requests for critique, 2 ratings and one critique... I'm thinking again if the effort to upload

is worth while</em>

<p>

That will always be the problem unless you are prepared to PAY for critiques. There will aways be 100x more people wanting advice than people prepared (and qualified) to give it. If each of them uploads 100 images, you're screwed.

 

<p>

If people were limited to one image upload for critique per month, maybe there'd be a higher probability of actually getting one. However if you upload only 1 image per month and everyone else uploads 100, you're screwed even worse.

<p>

See <a href="http://dieoff.org/page95.htm">this</a> for a theoretical treatment of similar situations.

<p>

The critique system won't ever work unless it's torn down and rebuilt on a sound foundation, something I think it currently lacks. That's the result of legacy, like building Windows on top of DOS!

[mg]

I was wondering whether you received my e-mail about all this... And besides that, I just thought I would drop by to add my 2 cents here more directly about your original question.

<p>

Yes, I do think that the new system is an improvement compared to the old one. I also agree with Bob to say that a complete reform would be even better of course, but for the time being, it's certainly a good thing... now people have to justify their low ratings... How can that be bad ?

<p>

I suppose that a 2 / 3 without a word was just plain useless. If critiquers now want to piss people off, they can't really succeed... If someone leaves a 2 / 3 ratings with an empty word as critique, it will now be clear that he was just trying to piss the photographer off, and the photographer will know what to do with this info - just disregard the critique. So... Good job, Brian !

<p>

Another thought came to my mind today, as I saw a technically PERFECT image - which to me was also excellent in many ways -, by a photographer who already had his POW on photo.net, and who is obviously not a nobody...

<p>

I found this picture had an average of 7 / 7, which seemed a bit low to me, and I went to check the ratings it had received. Most were good - 7 and above... And there was a 1/1 - of course...

<p>

So, the ideas I had were:

1) Why not implement your reform retroactively...? Meaning: why not somehow put up the long list of all ratings below average which received no comment in the past ? I guess the computer would manage that easily, no ? Then, maybe, Photo.net could publish the resulting list, and ask all the low raters to return justify their ratings with a comment (within 30 days or so, failing which the rating will be deleted). If the list is too long, maybe it could be limited to the ratings that were way below the other ratings this particular picture received.

<p>

I'm not too sure how exactly you could do something like that in practice, but I think you get my point: there are a lot of old pissing contests - revenge ratings, etc - which have discouraged decent people to post critiques and pictures... The idea is to first get rid of all the old agendas...

<p>

2) Why not introduce the notion of deviation from the average as soon as possible...? Meaning that if a picture is rated, say, an average of 7 / 7 after the 11 first votes, ratings from 5 to 9 (deviation of 2 points) will be ok, but any rating deviating by 3 points or more will need a written justification - maybe even a justification of 3 lines minimum... Here again, I'm not too sure whether this would be possible or not... just an idea... The same principle would then apply all the way... An interesting sophistication to this could be that each picture uploaded would be presented anonymously till it reaches 11 ratings... That way, pissing contests would just be impossible...

<p>

Does this make some sort of sense ?

[suzanne_berry]

I've just sat and read a lot of what has been said here and

it's obvious that everyone will never agree, but will put my

thoughts about it on here ...

 

I'm still at the bottom of the 'learning' totem pole with my

photos and for me, both the ratings and the comments work well.

My photos aren't all that great and to expect someone to have

to "comment" on WHY they rated it good or bad, is a little much

I think. If it's an 8, they think it's good ... if it's a 5 they

think it's average etc. etc. It only takes a minute to go to that

persons portfolio and see what kind of work they have on here and

it pretty much tells you if you should take what they rate or comment

seriously or not.

 

It's always nice to have the comments with the ratings. I have learned a lot on here from those comments. I had a few that made me want to throw my camera away, but it wasn't long before I noticed that those harsh comments were sometimes the ones that I got the most out of ... but only when they were done by someone that I knew what

they were talking about, by the work that they had on here, which leads me to what I think IS wrong ...

 

If you don't have photos on here ... you should NOT be able to comment

OR rate!! It's nice that people can come on the site and look, but if

they have no photos, then why on earth should they be allowed to!

Whenever I see the 1's ,2's and 3's it's almost ALWAYS someone with

"0" photos uploaded. To me that seems to be the biggest problem.

[rich815]

It continues to puzzle me why so many put so much importance into whether a person has photos online and using that critieria to decide if that person is qualified or has "the right" to comment/rate photographs. I guess people want to have some kind of measurement tool to try and understand a person's background but in my opinion a person's ability as a photographer does not necessarily mean they know how to critique or discuss photography. Does a top-notch movie critic have to know how to make a great movie in order to know a good movie from a bad one? Does an expert art museum curator have to be a great artist himself to know what good art is or to discuss it intelligently? No! Of course not. <p>Having photos online at photo.net and whether they themselves are good or bad is not an indication that one is adept at critical discussion of photography. One of the best U.S. photographers in outdoor photography, in my opinion, must be one of the most boring persons to talk to about photography, and many other consider him an awful teacher as well even though he is a fantastic photographer himself (he shall remain nameless, but you get my point).<p>What good is this really? That you will see a critique that "hurts your feelings" and then you'll go and see if YOU think they are a good photographer or not, and then use your personal opinion of them as to whether you should discount or accept their criticism? What if their photography is a very different style than yours? What if you think it's bad (after all we ourselves are not experts, right?) but others think it's good? And I'd hazard a guess that there are many good photographer's and/or people who know photography very well who have not uploaded images to photo.net for a variey of reasons. In my opinion this is not a good way to "judge" your critics. When someone begins to judge the comments they themselves receive using this method they lose most if not all of the value of the whole idea in the first place, and it becomes more and more meaningless.

[suzanne_berry]

Can't say as I agree with you Richard, but that's OK ...

Like I said in the beginning, we'll never ALL agree on

this subject.

You think yours is right and I think mine is, etc. etc.

etc... ;)

[mg]

Just saw this critique on a picture: "Her best shot. I agree! That's why I rate it at 2/2."

<p>

That's the entire critique, not just part of it... and the person who wrote this hasn't uploaded anything...:-)

<p>

I thought that the person who wrote this left a written comment, after all, and therefore was entitled to leave a 2/2... and I thought, that this critique might raise a few questions...

<p>

The question that came to my mind was: "Is such a critique actually acceptable for Photo.net ?". And if so, then what's the point of changing the rating system at all ? And if not, what does Photo.net intend to do or what CAN Photo.net do at all about this kind of written critiques ?

<p>

Just thought, that this example would be interesting for further attempts to improve the system... because it looks as if what would need to improve would mostly be the attitude of certain members...

<p>

Suggestion: what about yellow cards and red cards ? Since it often looks like a football match, why not after all ? :-)

[peter_lawrence]

Like Richard, I�m also very puzzled by the suggestion made by

many who have uploaded photos onto photo.net that they only

want people who have also uploaded photos to be the the only

ones to rate their photos. Well fine, if they strongly feel that way,

why don�t they just state that request on the comment field of

their uploaded photo. I, and probably others, who choose not to

upload are photos to photo.net would gladly oblige. But then

don�t come back and start to whine why hardly no one rates your

photographs like someone on this thread already has.

 

There are variety of reasons why many photographers on

photo.net have not uploaded photos. Many probably don�t have

access to a film scanner. Others don�t have the time. Others, like

me, don�t particularly care for photo.net�s terms and conditions of

use in regards to uploaded photographs. Specifically, I don�t

care for the statement that by uploading a photograph that I

�grant photo.net a perpetual non-exclusive worldwide royalty-free

license to modify, publish and reproduce that material ...� In

particular, I object to the words �perpetual� and �modify� in that

statement. So I probably won�t upload any significant photos of

mine to photo.net. But does that make me unworthy to rate other

people photographs? I would like to think that my thirty plus

years as a serious amateur photographer, and for the past five

years, as a serious art collector of high quality fine art

photographs, that I could add some value to photo.net by giving

my honest appraisal of other people�s photographs. Silly me.

 

It�s pretty obvious that many people who upload photos to

photo.net just want a �pat on their back� complement about their

budding photographic skills -- not honest feedback. I and many

others have tried to give people honest feedback through

comments and ratings of their photographs. Yet anything less

than a 7/7 rating causes dismay among many of these

photographers.

 

I�ve greatly reduced the amount of feedback I give to

photographers in photo.net since the requirement of a comment

for �originality� ratings below 5. Come on now... Photo.net is full

of photos that I would have no problems at all giving ratings of

7,8,9, and 10 for the photograph�s aesthetics, but I would be

VERY hard pressed to rate these same photographs higher than

a 4 for originality. Why does anyone need to leave a comment

when they rate photograph low for originality? A low rating for

originality is self-descriptive -- the rater thought the photo was

unoriginal -- what more needs to be said?

 

Did some people abuse the old rating system? Of course they

did! But any open rating system will, sadly, have people who

choose to abuse it. I think my suggestion on my previous post to

this thread to have photo.net users rate the raters could help

weed out the abusers. Another simple way to weed out those

who abuse the rating system would be to have the photo.net

administrator simply suspend the rating capabilities of those

who choose to abuse the ratings feature of photo.net.

 

The current �reform� in the rating system is a big step in the

WRONG direction. It makes the ratings of the photographs even

MORE meaningless by discouraging honest critiques of the

uploaded photos.

[mg]

Just to make one thing clear... I have absolutly no objection, from the start, about members with no images uploaded on the site, and who rate other people's work - WHEN THEY SAY WHAT THEY THINK ABOUT THE IMAGE ITSELF. It seems to me that you are doing just that, Peter, so please don't believe that anything in this thread is directed against people like you.

<p>

The problem we are here addressing, and that Brian's reform was, I think, trying to address - among others - is different.

<p>

Suzanne wrote this : "If it's an 8, they think it's good ... if it's a 5 they think it's average etc." That's exactly what I call a wrong assumption, Suzanne. I can guarantee you that there are people here who don't rate images according to what they think of these images... but rather based on a) Whether this picture is taken by their friend or not b) Whether they have or not a personal agenda with the photographer c) Whether they like or dislike this or that photographic genre - I hate PS, or I hate flora photographs, so 1/1, and such attitudes.

<p>

Part of these people with no pictures uploaded were going around the site playing God and distributing 1/1 ratings (even to obviously good or very good pictures) with no justification whatsoever. Now they have to write something, and I believe it's a good point.

<p>

The sad part of the whole story is that decent critics with no uploads are affected as well, though they aren't the ones targetted by the new system. Fakes, manipulators, and critics with agendas are the targets.

<p>

Maybe, Peter, you should look at all this from the bright side: this will help to minimize the manipulations and to identify the mean spirited people. As a side effect, you might now, yes, need to write something if you want to leave a rating below 5. But don't you think, that doing so is also just fair to the photographer...? I mean, put yourself in a beginner's shoes, for example, when he receives a 2/2 rating with no comment... Is it really such a major problem for you or anybody else to be a bit kind and to justify your rating by a few words...? People who have no pictures uploaded should continue rating images as they wish, but should also, like all of us - no more & no less -, get used to explain their opinions - which is educational and courteous.

[peter_lawrence]

Marc,

 

While my little rant about the rating changes covered a number

topics, I think you missed one of my main points -- the

requirement to add a comment when one wants to rate a photo

lower than 5 for �Originality�.

 

I don�t mind having to add a comment when I rate a photo lower

than 5 for aesthetics (I usually have done that in the past). The

photographer ought to know why he or she received a low

aesthetics rating - how else can the photographer make sense

of that rating. But it�s totally pointless to have to add a comment

when rating a photograph low for originality. As I wrote before, a

low originality rating is self-descriptive -- no additional

comments are needed.

 

Maybe the Originality rating should be eliminated altogether.

People seem not to use it for its intended purpose. Instead it

seems most raters seem to use it as an addition vote to double

the score they gave the photo for aesthetics. As Richard

Sintchak memorably wrote in another thread a few months back

�It is NOT just a chance to vote twice.�

 

Still, I think there are better solutions to this ratings dilemma

than the one currently implemented. Why doesn�t photo.net

administrators simply disable the ratings capabilities of users

who have clearly abused their rating privileges? I think this

would be a quick and easy solution to implement and more

effective in improving the ratings system here at photo.net

[kcfox]

I see all this squabbling about good ratings bad ratings etc. I have been a photographer (amateur) for about the last 15 years. I was told about this site just a week ago and think its a great way to just get exposure of my work. Ratings or not I now have a means to display my work on a global scale to this I thank you photo.net.

[indra]

I think all we can do is to keep improving a system in light of our experience. Decent people will make use of the system. Bozos will continue abusing it. After all, isn't that life? One learns to go on in spite of the clueless people throwing spanners into the works. I think this new rating system is a good idea. The same should be applied to ratings above, maybe, 8. I am least interested in medians and averages. A few sensible opinions/advices are all I care about.

[robert_mcgee]

leave the originality rating alone.... I like working to make a neater

photograph. It helps me keep my photo clients happy with neat and

interesting photos. I am a firm believer of a great photo getting a low

originality rating. If all it comes down to was point and click.... whats so

original about that. That is one thing photosig does NOT have going for it. I am

a firm firm believer in the photo.net rating system. I like this new change. <P>

Now vote adding is another issue. However I believe this.... if the person

thinks they deserve a higher rating and decides to pad their vote, let them.

Come time to sell their photos or actually make some money they are going to

come to some harsh realities from people who will REALLY critique their pics.

<P> But please leave originality as it is....

[tonys]

Oh boy! I do not envy people like Brian who have to face this avalanche of heated opinions, but I envy them for their patience and resolution in trying to solve some of the problems. It took me more then an hour to read all this thread and I would really like to summarize it in order to put some order in this sea of ideas (for myself and possibly for others).

 

Let�s start with the facts.

 

- Photo.net is an international site for the lovers of photography.

 

- Photo.net is interested to have as many members as possible.

 

- Photo.net is supposed to be the home of �the art of photography�.

This doesn�t necessarily mean that all good photographs are

artistic photographs.

 

- Photo.net uses English as a means of communication with its users and between users.

 

- Not all members master the English language to the level of intelligent and articulate English speaking person.

 

- All the members have some interest in photography.

 

- The age, experience, photographic abilities, talent, tastes and favorite genre of its members varies wildly.

 

- Some members love the rating system to the point that without it would give up their membership.

 

- Some members think the rating system creates animosity among the members and should be abolished altogether.

 

 

- Most of the members agree on the drawbacks of the present rating system.

 

- All members disagree with each other regarding a specific rating system.

 

- The stated purpose of the rating system is to classify the photographs, in order to allow other members to filter the photographs that are worth the time of the viewer.

 

- In order to properly criticize (anything, not only a photograph) one needs, first of all, time.

 

- Time, �nowadays� is a commodity in short supply.

 

 

How can anyone solve this impossible puzzle?

 

I think, this goal can be attained, by eliminating the conflicting �facts�. I won�t get into details because this would initiate another thread of heated but fruitless debates.

 

I will finish with a question.

 

How can two numbers quantify the quality of a photograph? Books were written about a single work of art and you expect an average amateur to summarize and classify it by two simplistic numbers?

[mg]

I read above that there will always be people to abuse the rating system, and that this is life, or such. I disagree. It isn't life, it is a negative usage of positive tools. Like atomic energy isn't a bad thing in itself, but a bad thing if it becomes a bomb. Will there always be people to abuse any situation ? Yes, of course - IF WE LET THEM, IF THEY MANAGE... I see no reason to give up on something simply because some people abuse it. Could a rating system based on numbers ever be useful ? Yes, I think so. If well adjusted, it will help to find better pictures, and it will give photographers an indication of how much a viewer liked the shot.

<p>

I went through this thread much earler this month, and sent a proposal to Brian. Not sure whether he got it or not. I received a message saying it couldn't be delivered. So, I'll post it here today. I started from what I saw in this thread that was constructive and tried to build on it. Here's the result. It is very long and written, as you would expect, in a quite poor language - since I lack vocabulary in English...

<p>

This rating system seems to have polluted Photo.net for quite some time now. I found Brian's latest ammendment to the system very positive, but I thought the system was still in the need for improvements.

<p>

Answer to Brian Motterhead's initial question:

<p>

"I don't quite get the insistence on low rating when (a) it doesn't help the site classify the photos; and (b) it doesn't help the photographer. Nobody is asking for raters to surrender their integrity and rate everything higher than they think is warranted. Just don't rate them or leave comments." said Brian Mottershead.

<p>

Agreed 100% - which is why this change you made can only be for the better.

<p>

Another important post by Brian: "Ratings that are consistenly lower than the norms serve no useful purpose. In theory, neither do ratings that are much higher than the norms".

<p>

Agreed 100% - unless a good written critique is posted to justify the rating of course. Now if such "out-of-norm" and probably biaised ratings are useless, why keep them at all ? This leads me to part of the proposal, that you will find at the end of this post.

<p>

First of all, here are a few messages previously posted in this thread and which have caught my attention... and my reply to them.

<p>

Appeal system ? - If Mark Duke's idea of an appeal system can be made easy, then I like it... Worth thinking of easy ways to set it up... if there are any.

<p>

Critiquing critiques ? - Good idea, but I've just experienced such system lately, elsewhere, and it was abused as well... Who said that someone critiquing a critique would be more honest than critiquing a picture ? :-) And anyway, no longer necessary with the changes proposed here below.

<p>

Re-normalized ratings ? - "Quite clearly a 10 from someone rating every picture a 10 is worth much less than a 10 from someone who gives on average a rating of 5. The situation is clearly the same for low ratings, but due to basic human psychology, this has already been pointed out. So what I suggest is that besides the "raw" ratings, each picture gets a "renormalized" rating, calculated as follows: for each rating, substract the average rating given by the user who rated the picture (and possibly divide by the variance of its ratings). This will result in positive ratings (above average) and in negative ratings (below average). This measure is much more robust than the current one with respect to being biased by someone with a few accounts giving extremely high (or low) ratings. just my 2c." said Olivier Pelletier.

<p>

To me, clearly the best idea in this thread !! WHY NOT JUST DO THAT ? Or is it too complicated ? I have basically proposed here an amended version of Olivier's idea... I feel it would all need to be refined, but the solution is somewhere there imho...

<p>

A single overall rating ? - "How about one overall rating? Possibly doing away with the originality rating completely." said Ken Talheimer.

<p>

Makes sense to me IF the rating categories I propose below are impossible to implement. Why does it make sense ? Because the originality rating is being understood differently by everyone. To just be different doesn't mean one is being different in a smart way ! Many shots I see are different from the norm, but to me, have no concept at all, or a very poor one... So, is it an 8 in originality in such case, or a 2 ? My way to rate such shots was to give the picture the benefit of "my-doubt-about-the-originality-rating", and assign the shot a 6 or 7 or so...:-) I feel the originality rating I assign to images is always stupid. I can't manage to rate well on originality. Add to this that of course a standard picture of a flower isn't original, but it isn't a crime either, in itself... So give a wonderful flower shot a 2 in originality will deprive you from finding this wonderful shot easily with the high-rated pix filter. Lately, we also see an increase of the amount of digital work in the high-rated pages. It is normal. Because originality is easier to achieve with a photo-montage or an abstract, than with a pure photograph. That seems to be very unfair to pure photographs. If we could add categories about images like fauna, flora, fashion, portraits, digital art, etc, then, the originality rating would no longer be necessary anyway... Yet, I remain convinced that the 5 rating categories / criteria I propose are the most accurate way to rate a shot.

<p>

Should members with no uploads rate other members ? - "everyone who rates the photos should have at least one photo uploaded. For example, I get 20 ratings. 19 of them are eights , nines and tens. And then there is one user who rated my photo: aesthetics - 3, originality - 2. The first thing I would like to know is "is that man a good photographer?" Is he competent? Maybe the 19 other users were wrong rating my photo so highly. So I go to this guys page and see "no uploaded public photos".... wrote Dejan Kosanovic.

<p>

Of course, this is true... But every abuser can upload an uggly shot or 2, or 5 - what difference will it make ? And a bad photographer can well be an excellent critic. And please note that Olivier Pelletier's suggestion of a "renormalized rating", or my proposal above, or any similar system, would actually help people in that sense as well... If someone has no picture uploaded, and if PN doesn't want to deprive him from participating, this will help as it will minimize the importance of any faceless harsh rater - faceless raters with no comments being often (not always admittedly) abusers... Not taking into account ratings that deviate too much from the other ratings a picture received, would help.

<p>

"There is a real problem of irresponsible or malicious rating going on, which, if nothing else, upsets a lot of people." wrote Brian Southward. And then this: "I think many of these antisocial raters would disappear if they had to post images. If we lose a couple of world-class critics along the way, that's a price I'm willing to pay."

<p>

I agree with the fact (1st part). I also agree that it would discourage a few, but not many. After all, what's the problem with uploading 5 shots before being nasty...? And no, I AM NOT prepared to loose a few world-class critics along the way... So, why wouldn't PN accept people with no pix uploaded ONLY IN CERTAIN CASES... PN is entitled to decide that SOME will have the right to rate but with a comment, whereas the rest would just be banned from ratings. What's wrong with that?

<p>

"Three boxes. Good (I like it), Bad (Why even bother) and Mediocre ? -No numbers, no ranking list, ratings total only visible to the poster of the image. Images submitted for critique posted without attribution to the photographer. No egos to boost = no problems." said Bob Atkins.

<p>

I could agree with that, but in the end, good, bad, mediocre is not a useful information, not for the photographer, and not to make the best pictures more accessible to the public.

<p>

Then, why not 5 choices, rather than 3, and this available separately for each of these 5 rating categories I propose: 1) Concept 2) Light 3) Colors/tones 4) Composition 5) Overall technique. I think that would serve your purpose, Bob, and still carry a lot more information, which are both useful to the photographer, and to make the best shots easier to find... No ? For example... One search could be about the "Best technical shots", another one about "Best lighting", etc... And there could be (or not be, too) an "Overall best-rated images" - NOT photographers... Then this search could even, ideally, be available for each category of images: Fauna, Flora, fashion, Portraits, Still life, Digital manipulations, etc... How do you like that ?

<p>

"Say good words when it is good and simply shut up when it is not" said S Liu ? -

<p>

As Bob said, "absolutely not. When an image is poor, the photographer needs to be told..." I used to agree with S Liu, but no longer... Since I came back to Photo.net, I noticed, that an incredible number of truly average shots get sky high ratings... That, to me, is the end of learning. It also deprives us from finding some of the really great shots in the high-rated pages. Low ratings are necessary too if a rating system is in place.

<p>

Bob said "However it hurts much less when neither the critic nor the photographer know who each other is. Opinions will be more honest that way since there are no personality issues to deal with, no axes to grind and no egos to be delicately preserved. It would also tend to cut down on the hate mail."

<p>

I'm very prepared to agree, BUT... Don't we need to know, not WHO, but HOW GOOD the photographer who rated us a 4 / 2 is ?... So what about this: Every picture uploaded has no author name visible on it till it gets to, say, 25 ratings, or such...? That would abort biaised and mean ratings based on agendas all together, no ?... OR, a different idea all together would be that we could ONLY KNOW OUR RATER'S GENERAL RATING PROFILE, BUT NOT HIS NAME. A rating profile would tell us how many pictures our rater has uploaded, what's the total number of views his work has attracted, what's the average rating he received for the whole of his work. That's enough for us to know roughly who rated us. Knowing more is not only un-necessary but leaves the door widen open to abusers.

<p>

Thumbnail pages ? - "The majority of 339,188 photos are hidden in the database. Is it possible to modify the system to show the uploaded photos (or at least those for comments) as pages of thumbnails (you can decide the best size and navigation tools) so that people have an easy way to go through them? I don't know how the staff manage to view 700 photos everyday. Can you open this system to other users? I don't think this would add too much burden to system. The current system forces me to rate each photo in the list instead of showing thumbnail page and discourages me from viewing more photos efficiently." S LIU

<p>

Yes, yes, yes, and yes !!! :-)

<p>

Rate and comment on same page ? - "It would be helpful if we could rate, and comment on the photo in one process, and on the same screen." said Mubeen Mughal...

<p>

YESSSSSS !!

<p>

Rating scale 1 to 5 with worded definitions attached ? - "The problem with the 1-10 scale on photo.net, in addition to openness of misinterpretation, is that malicious "1s" can really throw off an average. Make the darn thing something like "1-5," with a simple statement along the lines of "1" is a complete failure, "3" is average, "4" is well-done and "5" is top-notch." wrote George Day

<p>

The "2" was forgotten - probably something like "Below average" - but this was another good idea imo. Might not solve problems, but would be clearer at least.

<p>

Deleting extreme ratings ? - "if you MUST have a scoring system and you don't like a few lowball scores, simply drop the bottom and top 10% of ratings and average the rest." said Bob Atkins.

<p>

YESSSS !! Almost perfect to me... But rather than top and bottom 10%, I'd suggest the 10% of ratings that are most deviant from the picture's standard average.

<p>

" Reduce the chaff " ? - "Make photo.net a place where only stringently limited examples of one�s very best work can be submitted for critique each year. Dramatically reducing the number of allowed submissions would similarly reduce the need for a sorting device to find quality work: Each photographer would pre-filter his own submissions. Anyone who is impressed by one of the, say, four submissions allowed a photographer in a twelve-month period might wish to peruse that photographer's folders of additional, un-rated work." Brian Walsh

<p>

I like the idea but... Let's rather say, since we learn from bad pictures as well, that each pototog would have a "Portfolio" limited to 30 or 40 of his best shots, and a "New uploads" folder - each could be divided in categories like Portraits, Scenics, etc. Up to the photog to keep and delete what he wants... but the maximum total would be 50 pictures (if 50 isn't enough to some, people could pay to post more). Here's my take on the rating issue, basically a rough proposal for a new system.

<p>

PROPOSAL BASED ON A SYNTHESIS OF THE OTHER POSTS IN THIS THREAD:

<p>

1) What are the problems ?

<p>

a) Some people leave low scores with no comment, and people are tired of such practices.

<p>

b) People can't identify whether the low scores they get from some users are sincere or just plain mean.

<p>

c) People are sometimes creating fake IDs, it seems, to carry on with personal agendas.

<p>

d) Some people seem to be getting "score support" from family & friends who rate their stuff sky high.

<p>

e) A picture that gets hit by low scores a few times leaves the "high-rated" pages and will receive far less critiques after that. Therefore, mean hits affect people's chances to receive real feedback, therefore piss people off.

<p>

f) There are more pictures on PNet than one can ever hope to see.

<p>

g) It is a bit difficult right now to find new photographers we'd like - though not impossible.

<p>

h) Imo, the manipulations by ego-driven mean raters have resulted in many pictures getting to the high-rated pages though they obviously (to me) didn't deserve the ratings they had. As a result, many good pictures proposed by people who don't manipulate the system disappear in the depth of the lost pages. And the quality of what's easily accessible is lowered.

<p>

i) People are scared of by manipulators. If they tell what they think, retaliation could occur, etc. Consequence: most ratings are not really sincere.

<p>

2) The fix

<p>

a) Limited number of pix on the site for each photog. "Portfolio" folder & "New uploads" folder. That way, many worthless pictures will disappear after a while, and good pictures will be easier to find.

<p>

b) Pictures separated in genres. This way, PN will help us to find what we like best, but will also minimize the competition that's going on here nowadays. Mean people will mostly do their manipulations in their category - and spear the rest of the site...

<p>

c) Ask each member to pick 3 genres which are his "specialties" and 3 genres he knows quite well - or any other number you like. He will then be able to upload in other categories, but his crits will count at a scale of x 3 in his specialties, x 2 for the critiques in his quite well known genres, and x 1 only in others. Or something like that.

<p>

d) PN could assign a special value to a certain number of critiquers and, for example, decide that Photographer X, Y and Z's critiques would count more than others - basically setting up a scale for ratings. Nobody would know who PN's selected critiquers are... Nothing's wrong with that imo. Best part would be, that the scale of each rating we receive would be visible to us, but the names of critics selected by PN with a scale of x3 would not appear. So, basically, those who would hate to receive a 3 / 3 by an unidentified critiquer at a scale of x3, just wouldn't know whom to hate...

<p>

e) Take the standard average of the image, then delete the ratings that are deviating far from this average (after 20 critiques or such). Example: A picture has 20 times 7 / 7, with 1 time 2 / 2 & 1 time 10 / 10 . Majority rules and the 2/2 and 10/10 ratings are deleted - or not counted. A threshold must be determined here, meaning that if the rating on a pix gets more than X points difference compared to the picture's standard average, it gets deleted, unless we find Y ratings similar to this one. When we are voting to elect a president of our country, and he gets 52% of the votes, he is not given 52% of the trone !! :-) Basically, not counting in ratings that are too far from the picture's average would be a great addition to discourage abusers and give less importance to the minority... [ Abusers would need to create more fake IDs to really continue to abuse the system effectively, and will be a lot easier to trace if they do so... ]

<p>

f) Force EVERYONE to write comments for each picture they rate - no matter what's the rating. Ratings with no comments could be allowed, but would have a scale of x0 - meaning they wouldn't be counted in any calculation...:-) Or maybe a scale of 0.5 or 0.25 - basically so low that it wouldn't really matter much...

<p>

g) Break the present rating system in 5 rating criteria instead of 2. I feel that the categories we need are: 1) Concept (includes originality) 2) Light 3) Colors / tones 4) Composition and angle 5) Overall technical quality. If that's too complicated to implement, I'd take Charles Barcellona's suggestion of 3 categories for ratings -Aesthetics, Technique, and originality as 2nd best.

<p>

h) Implement the re-normalized rating system proposed in this thread by Olivier Pelletier or something similar.

<p>

i) To finance the costs incurred by all these changes, open a pay-feature whereby any member can buy detailed critiques by photographers picked from a list that PN would propose based on their knowledge, keen eye and good will. If I want 5 pictures of mine critiqued by these 10 critiques, I'd have to pay 20 USD for the 50 critiques I would receive. Out of which PN would keep at least 10 USD, while the rest will be shared among critics, or such. (See Chris Prouty's request in this thread: obviously there is a demand...) I offer my contribution to write critiques if such a system is set up, and I expect nothing in return.

<p>

Conclusion: Let's have a stronger critique section on this site, with a few pay-features. Let's not abolish the rating system, which many are attached to, but let's improve on it by implementing a strong control over abusers as described above. Let's partition the site in categories of work... Let's set up 5 rating criteria (or at least 3) instead of our 2 "Aesthetics" / "Originality" system. Let's work on the best way to implement a certain anonimity for the ratings given - see my comment on rater's profile above.

<p>

I'd be glad to hear what you all think of this proposal. Don't you think this could actually work ? If it needs raffinements - and I'm sure it does -, please tell us which ones. Regards.

 

 

 

 

 

 

 

 

[root]

We need a system that promotes good images and constructive technical

criticism. I suggest we rate the raters by putting a little tag or

icon - whatever they're called - after your name if you have, say,

ten highly rated images. A highly rated image is simply one that has

been flagged by ten or more people as being deserving of placement on

the highly rated page for a period of time. Nobody benefits by low

ratings, so don't allow for them. If you want to offer constructive

criticism, offer it, knowing there is no way for the recipient to

retaliate.

 

You would encourage people to visit your gallery by responding to

their requests for critique and/or a positive rating. If your name

gets attached to too many lame images that clutter up the high rated

pages, people will stop rating you because of your lack of

credibility. Limit the number of images you can flag, show a list or

gallery of your selections on your page.

 

There should be a limit to the number of images in a personal gallery.

High rated photographers may upload a greater number. Those that

offer constructive criticism could also be tagged as a recognized

'judge' and might draw more interest and/or allow for more uploads.

 

miscellaneous ideas. . . . use pages of smaller thumbnails, not a

slow loading list, for selected daily uploads. . . . separate highly

rated images into three categories: B&W, color, and manipulated (see

foundview.com). These two ideas will allow for a far greater number

of easily viewable highly rated images. Thirty is no where near

enough. Discourage or prevent multiple similar images from appearing

on the highly rated page. . . . perhaps only two images per

photographer at any one time

[neil]

My suggestion:

 

Get rid of number ratings. Each person rates on a different scale anyway.

 

Ratings seem to have two intended purposes:

1) Provide a way to find the most liked photos on photo.net.

2) Provide feedback to the photographer.

 

So why not address these separately. For 1, here's a suggested system: let each user nominate a photo as an audience favorite. Someone can nominate a photo once per week. Also each week, all the photos with 2 nominations are displayed in the gallery for everyone to see. Of these, let everyone select 1 (maybe 2) favorites. The 3 with the most votes will receive commendation. There could also be 5 runner-up photos selected.

 

Now for the other part. Numbers don't help much anyway. Often, a photo has a critical flaw that lowers the rating. Other times, something wonderful about it raises the rating. So, why not expand the feedback system. Written comments are great, but perhaps more people would comment if there was the option of a short "questionnaire."

 

A couple of ideas for fields:

 

Strongest point: tone/composition/emotion/originality/color/interesting - have choices, and then a line to explain "other".

Weakest point: similar

 

Have radio buttons or maybe even check boxes for multiple selection. There could be other fields.

 

This could provide much more interesting feedback to people who really are interested in feedback.