Changes to Rating System

The photo ratings system at photo.net system serves two major

purposes. Together with the comments, the ratings provide the

photographers who submit photos with sought-after feedback on their

work. For other users, the overall rating averages provide a

mechanism for finding photos that may be of interest to them, and

ratings underpin many of the most popular features in the photo.net

Gallery.

 

However, low ratings that are unaccompanied by comments advance

neither of these goals. They provide no useful feedback to the

photographer who submitted the photo. On the contrary, they tend to

be frustrating and discouraging and frequently lead to fruitless

disputes. And low ratings do not play any useful role in helping

users to find good photos.

 

Accordingly, today we are introducing two reforms to the rating

system.

 

First, it will no longer be possible to give a rating lower than 5 in

the Photocritue User Interface. This user interface makes it too

easy for people to give large numbers of low ratings without due

consideration of the photo.

 

Second, it will still be possible to give a rating lower than 5 when

visiting the Photo page, but only if one has previously made a

comment on the photo, which presumably will explain the low rating.

Great. Now we'll get anonymous, quickly set-up hotmail accounts with their "detailed" comments. ("This sucks", "1") This solves nothing. Good try but you're still just pandering to people who actually think ratings mean anything. Just let those people find their way to photosig where they can cavort to their heart's content in meaningless ratings madness. Ratings may be "popular" but IMO right about the time the ratings system was instituted here is when photo.net started it's decline. This site used to be high in substance and integrity. Now it's full of too much fluff and meaningless banter. I think there's a direct correlation. I say remove the ratings system all together. Then those who think ratings mean anything will leave and maybe, just maybe, photo.net's magic will return.

I think the change is great. Now change the "originality" rating to something less ambiguous and everything will be all right :)

Richard, we aren't pandering to the people who think the ratings mean something. We believe the ratings mean something.

This may be a good idea but I have noticed a lot of very high ratings lately with no explanation. Perhaps any rating of 7 and above should also require explanation. That way high ratings would also be beneficial to the photographer. :-)

I think the changes are good, but suggest one further change - and that is to make it such that before a person can dole out a rating, they too must be able to be rated, or, as it would be, have images of their own posted in public forums.

 

In this manner, it may serve to weed out a few more of those meaning no good, and also serve to make those, who do mean the best, apply a rating of thoughtful consideration.

Why not just do away with the critique system altogether. Your never gonna please over half the people. People will still get around new rules. They will still get pissed when they get bad reviews. Do away with the critique forum and save lots of dasd space and bandwith. It will also reduce the number of "I got a bad rating on my picture, boo hoo hoo" post. Just my opinion.

Photo.net: Where all the images are above average...

No, Ethan. The consequence of this change is that only the above average images will get a lot of ratings. This will mean that some below average images won't get many ratings at all, but they might get some comments. Besides people will be still able to make low ratings; it will just require a comment. That does not seem like a problem.

I agree, the demand to leave a comment before posting low ratings is a good thing. It may not solve the problem entirely, but will certainly help. We can continually seek to improve the ratings system, but let us institute what new measures we can to help now.

A few suggestions are that we comment for any rating, not just low or high, and that those who do rate must also have public photos available... not for retribution but to show they truly have a desire to improve and grow photo.net, and not just upset people with unusual ratings.

Okay, I won't argue about limiting the "aestheticics" to 5 or less without trouble. However, why limit the "originality" rating in the same way? There's nothing insulting (or even the least bit troubling) about getting a 1 or 2 for originality if you're doing normal macro pictures of flowers, etc. It seems that you're cutting off the majority of review (after all, most of our photos are unoriginal) with this...

I think you are thinking. The fuss has been about low ratings that are seen as spiteful, arbitrary, and capricious, or retaliatory. Forcing a comment to justify a low rating seems a reasonable way to keep down the discontent. Who could argue about having to explain a rating of 3,or 2, when the mean is certainly higher.I think its the golden rule thing to do anyway. I am surprised PN Services chose number 5,but I expect it was done with some analysis of the stats. Never numerically scored a picture.And never got a really hot e mail either,yet. Although a few thank yous and some visits to my portfolio to see who this here bozo is I expect:-) I can't bring myself to assign a number to originality and aesthetics. Might as well ask 'Does It Work?,'from 1 to 10?Blecch. I like seeing some photos on photonet. So you are dealing with the distractive fighting one step at a time. Good work Brian.

The only thing this change does is to effectively constrict the scale from 5-10 instead of 1-10. It will make us all feel good, since we get such nice and high ratings on our photos, but ultimately the feedback only becomes even more useless. I for one have stopped submitting photos for critique because the only thing I get back is a random collection of numbers - nothing that will help me become a better photographer.

 

What I would suggest is that the ratings are expanded to convey something more meaningful. For example, if there were four ratings: subject, exposure, lighting and composition, the numbers would start to tell me something. Let's say I find a photo with a strong subject, which is underexposed, but with ok lighting and good composition. I could quickly rate it "8, 3, 5, 7". Those numbers would say a lot more than an aesthetics rating of 7 and an originality of 8. Furthermore, those four ratings invite the user to take a more critical look at the photo - a critical look that might invite the user to explain his ratings by adding a comment. And comments after all, are much more helpful than the ratings. Finally, it can be done without breaking the rest of photo.net, which has several features that depend on those ratings. Just average the ratings (5.75 in this example), and you can still have a top-rated page and a top photographer page.

As I recall, the original system was set up for rating 1 through 5. When it was expanded from 5 through 10, I criticized it as being "anal".

 

What's the difference between a rating system that is 1 through 5 or a system that is 5 through 10? why not just go back to 1 through 5 and require ALL images to have at least one comment, even if just a word.

This change is just a patch and is by no means ideal or intended to be be final. The current average is 7, and the ratings of 4 and below are too low compared to the averages to be meaningful to anyone; they only cause trouble, especially without comments.

I'm fairly new to Photo.net and I didn't read the comments above so I can't promise this hasn't been said but on one of my photos I got low rating and I sent out an email to them asking to tell me why. Only one out of three wrote back. I say, "get rid of the ratings." Correct me if I�m wrong but photography is an art. Art cannot be rated because art is subjective and not everybody appreciates the work. Comments are great if they are not the type of comment that I am sadly guilty of (ex: great photo, POW, etc). These offer no help in improving the photographer�s skill. Thanks for hearing my opinion.

I cannot really say I like this change. I understand some people don't like the low ratings given, but not all good photos are above average in both aspects. I don't like the 4 I got either but what if my photo deserved it?

 

I would appreciate if everyone read Phil G's primer on how to rate. He has no problems giving a nice but not creative photo a very mixed rating. In order to follow his wishes for how the rating is supposed to work we need to comment on every single photo just to give say a 4 for originality while it still deserves a 9 or 10 for aesthetics. Even if the reason for such a mixed rating might be so obvious no further explanation would be needed.

 

Sych ratings would not imply a bad photo, just because it's nice doesn't mean it has to be as origninal. Many times simplicity is best.

 

The ones that really love giving out 1/1 ratings will only create more fake accounts.

 

I personally only rate photos that I like, it's my way of giving the photographer a pat on the back saying: "Nice work!" -I feel this change will brin an inflation of positive ratings.

 

This change closes the community more, making it more troublesome to rate peoples photos with the effect of fewer ratings. If it ain't broke don't fix it.

On the page of every photo, add a check box "I like it". Once a user click on it, his/her member page is added to a list associated with this photo. The member has the option to remove his/her name from that list later. On the member page, there is also a list of the photos he/she likes. The cross links between photos and users indicates the popularity of the photos and the preference of the users.

 

It is something like what Amazon.com does in its book review pages.

I think it is more useful than rating from 1 to 5, 1 to 10 or 5 to 10.

I think this is a good move on the right direction, but it is not enough.

My two cents:<p>

 

<ol>

<li>Why the comments should be forced only for ratings above 5? Instead, make a comment mandatory for each vote. We will have much less votes but more meaningful.

<li>Let's split the originality/aesthetics in more values as Fabian suggested. I would suggest subject,

originality, technique, composition, but it just an idea. Probably there should not be more than four items.

<li>Let's give people the ability not to give some ratings: for instance I could vote subject and technique, but not originality and composition (I found that I often gave a pair of equal votes - for instance 7/7 - when I had ideas about only one of originality/aesthetics topics).

<li>Only at the first time a new guy puts his first vote, you could force the display of the page of explanation about the rating system - I think that most people doesn't read that page.

<li>Finally, reduce the range from 1-10 to 1-5; or let's define a mnemonic scale at least for some items. For instance originality could be: not original at all, somewhat original, etc (my english fails here in giving the proper terms of the scale). I found myself rather embarassed in giving 1 or 2 for originality, even if, as Jon said, this should be an usual thing. I prefer to rate a work "not original at all" rather than 1.

</ol>

 

Thanks to the Services Photonet staff for their work. I would add the suggestion to support the site with the 25 bucks subscription: we will give the staff more resources to improve their work. I did a few days ago. Bye.

From a statistical viewpoint, this is obviously meaningless, as it will simply remove most of the below five ratings (which should constitute about half of all ratings). It will simply move the threshold for a good average up by a couple of points.

 

Force comments on all ratings or none. Nothing else makes sense. Not that I care much. I expect to upload some pictures eventually, but I couldn't care less about the numerical ratings - the more numerical rates a picture gets, the less meaning it gives, much like the length of the emperor of Chinas nose.

 

The numerical ratings don't help the photographers, only the comments do, once in a while when there is some thought put into one.

 

As for other functions like galleries of the most popular images, that could in fact be solved without ratings. For instance the number of comments can serve as indication of the quality or "interest factor" in an image, or there could be a simple button for voting for images as worthy of extra exposure or not, with the number of pro votes determining the popularity (pro as in yes, not professional). That would leave only difficult ways for individuals to seriously alter the "rating" of a photo one way or another, and no really sensible way to affect it negatively.

 

Again, I don't care about the ratings, and so I don't care about changing them, but from a mathematical/statistical viewpoint, this patch is just plain silly. It will simply shift the weighted center of the scale - and probably result in a lot more fives being dealt out.

 

In fact, now I think about it, it will do more than that. For instance a very boring picture, that doesn't inspire anyone to say why they think it's bad will get nothing under five, but of course a few ratings above (there's always someone to like a picture), while a more interesting and generally better liked picture will inspire people to explain why they think it's only worth three after all, and will consequently get a lower average than the worse image.

 

One of the reasons I don't rate pictures is that I should rate all pictures or none, if the rating is to give a correct view of how good I think pictures are. If I only rate pictures I like, the data is worthless - particularly if everybody else does the same. But this method will cause people to primarily rate pictures they like, even if they previously tried to rate everything.

 

Of course, there is a fairly simple way of ensuring reliable ratings. It would be simple, and absolutely correct to discard ratings from anyone with rating averages far from five either way. If a person has a rating average of 9, it is fair to say, that his/her ratings are of little value to anyone, and the same goes for a rating average of 1. Where to put the limits is another thing, and not so easy to determine.

 

How good for me that I don't care about the ratings, that must be the reason I spend so little time writing about them...

Here are my pair of cents worth:

 

1. Have the ratings go from 1 to 16.

 

Anything 5 or less

 

or

 

14 through 16

 

requires a comment (a sentence or two?) before the rating is posted.

 

Along with the rater having several photos posted on photo net.

 

 

2. You all will require a 'spell-check' for the all the one-word comments that can adhere to a photo rated lower than the 5 score you are attempting to create as a 'mean.'

 

 

Good luck...

S LIU has hit upon more or less the system I am planning to implement. I'm glad at least one person is going to like it. Meanwhile, the requirement of a comment on a low rating will help to prevent the many thoughtless (or deliberately provocative) low ratings.

First of all, thanks to Brian and everyone else doing all the great work to keep this site up and running. To the point - I think <5 is too high to require a comment. A low cutoff of <3 would be much more appropriate - I have quite a few photos whose average rating is between 4 and 5 and I think most of them deserve it. Especially for originality - a photo can be nice yet still deserve no more than 2-3 for originality.

Brian, you had told me in a couple recent emails that you were planning on designing the option for users to disable <b>all</b> ratings for their pictures. Is this still part of the master plan for the silly number game?<br><br><b>Curious, J.A.S.</b>

John, yes I am still planning on doing that.