Should the Rating System be changed?

[guyscrivner]

I have been a PN member for a much shorter time than a lot of you but

I would like to still throw in my 2 cents worth on the issue of

ratings abuses and critiques. I feel that you do not have to go far

to see that (1) there is ?mate rating? going on in a lot of places,

that (2) there are a lot of ratings that are given low just to harass

someone or for revenge, that (4) most PN members do not feel there is

a non-biased method of determining top rated photos or photographers,

and (5) that because of the above mentioned issues PN members seem to

be divided into two camps, those that have given up on ratings and

those that haven?t. There could be a third group here and that would

be those who think it is all just a big joke anyway and why bother

with a fix.

 

My solution is to change the rating system. I think that the current

system invites and promotes most of the problems that have been

mentioned above. The new rating system would (1) make it hard to

mate rate because it would take more time to rate each picture and

the mates would soon realize that it wasn?t worth the effort, (2)

make the ratings more meaningful to photographers who really do want

to know how others feel about their images, and (3) provide less

thrill for those who like to give out low ratings for kicks or for

revenge.

 

The proposed system would work something like this. There would be

five categories to rate a picture.

 

COMPOSITION: Carefully composed and/or cropped to eliminate any

distraction and to maximize aesthetics. The rule of thirds used or

appears successful without the rule applied. Rate 1 if YES, leave

blank if NO.

 

FOCAL POINT: Image clearly establishes a focal point that captures

the viewer?s attention. Rate 1 if YES, leave blank if NO.

 

 

CREATIVE: Successfully uses pattern, line, color, texture, pose,

perspective, subject, etc. that results in something other than a

snapshot. Rate 1 if YES, leave blank if NO.

 

TECHNICAL: Camera settings and tools results in appropriate focus,

depth of field, color, contrast, and high lights and shadows. If

PhotoShop or other tools are used there is no evidence left of the

use of such tools. Rate 1 if YES, leave blank if NO.

 

IMPACT: The image has a level of emotional or aesthetic appeal that

causes it to stand out from other images. Rate 1 if YES, leave blank

if NO.

 

There would also be a optional comment section as part of the

rating. It would be worded something like this,

 

COMMENT: If you were the photographer and had a chance to redo this

image what would you do differently?

________________________________

 

This rating system would not replace the current critique method. I

personally like to be notified when photographers I like post a new

image. I do believe, however, that access to their pictures in this

manner should allow me to only comment or critique. If I do not have

access to rating there pictures from there folders, I can neither

mate or revenge rate.

 

If done in this manner, a member who chooses to just post an image an

who is not interested in ratings can do so and not worry about

ratings. His friends will still know that he has posted a new image

and they can comment or critique. What they can?t do is rate the

image.

 

So, that brings up the final point, if the photographer is interested

in having his image rated then it goes into que when he requests that

it be rated. It will be presented to members for rating in sequence

with all other photos submitted either as one large group or by

category. There is little chance for manipulation, all photos get an

equal opportunity to be rated and it discourages abuse. In either

case, the image presented for rating should not reveal what others

have already rated the image. This should be revealed after the

rating has been made.

 

 

Well, that is what I think, what do you think?

[mark lucas]

Guy,

You raise some interesting points, but, unfortunately, I think you'll find your ideas somewhat flawed.

 

Management do not want to introduce a whole new way of rating images as there are millions of archived images that have already been rated and have their place in the rankings, however you sort them.

 

Also, nothing you've suggested would stop mate rating. How would rating something a 1 or nothing leave much statistical data for sorting? This might sort a handful of photo's, but when millions are involved, too many would end up on an equal footing. That's not to say there aren't merits in your suggestions. I, too, have thought how it would work if there were a third or fourth rating criteria, based on the current system, that introduced something like a 1-7 rate for technical achievement i.e. is the picture in focus (amazing how many 7's get awarded though to out of focus pictures).

[pics]

You make many good points but the bottom line is that the ego-obsessed want a number next to their photograph because they believe there is a prize to be won by having the highest average rating. These are the same people who send in ratings complaints on a bi-weekly basis. The whole thing could be solved by abolishng numerical ratings and only allowing verbal suggestions and critiques, but then how would you prove you're "better" than someone else?

[nomade]

"abolishng numerical ratings"

 

Rates are a way to get feedback for many of us photographers, but also an instrument for the site to be able to classify photographs with different criterias. If the numerical rating system would be abolished, you wouldn?t have the TRP anymore.

 

All the years, all the months, even almost on a week or daily basis, people complains and wants to change the rating system. What for? You'd have another system that many others would disaprove.

 

So forget about it, keep shooting and posting and enjoy the way things are right now -not as bad as they seem.

 

Regards,

 

Nestor

[howard_foto]

I happen to agree with Nestor... that the ratings is a keen KISS - keep it simple stupid - system. so why complicate it? if many members who use it don't give a hoot about the TRP and find its feedback useful then why mess with it? chances are if your better photos score higher than your lesser ones, then it's working for YOU. so I prefer to read my tea leaves via the nuances of comments. diff strokes for diff folks.

[pics]

"Rates are a way to get feedback for many of us photographers"

Wouldn't actual sentences and thought out written critiques be considered a superior means of feedback than numerical rates? What critical feedback do you derive from a number? I only suggested getting rid of numerical rates since they seem to be an incessant source of confrontation due to the types who send in abuse complaints when they see that their average score dropped 1/10th of a point. Getting rid of the source of the problem would allow moderators to concentrate on more pressing issues than trying to settle the ratings scoreboard. People would still get feedback on their work and the constant squabbling over "mate rating", "revenge rating", "robot rating" or whatever the phrase of the day is, would be over.

[nomade]

"Wouldn't actual sentences and thought out written critiques be considered a superior means of feedback than numerical rates? "

 

You're right, but reality points out that most of the people dont write critiques, so if I have to choose between a rating system and nothing, I prefer the rating system.

 

For this same reason, I agree with Howard: the rating system shouldnt be replaced for an even more complicated one, it would serve no purpose. Just because some of us would receive the proposed one as an improvement, it wont work because thounsands of people out there dont think and act in the same way like us.

 

We all know and suffer from all the issues you're pointing (revenge and mate rating, bots, and so on), but maybe that's the small price we have to pay in exchange for... "something"...

 

On the other hand, having a system that classify the trillions of photos uploaded every day, to let us enjoy the better ones, is also something better than nothing.

 

And it works. Not perfectly, but it works.

[guyscrivner]

I am totally speechless! Thanks to everyone for your thoughts. I will go take some pictures now.

[keith turrill]

At the risk of sounding like a comedian making jokes about mathmatics and hypotenuses or hypotenae, I spent too many years in college studying math. The numerical ratings and averages are much easier for me to understand.

[WJT]

Yes Keith, and it is easier for the computers that run this site to understand as well. That is the problem with Daniel's suggestion above. We need the numbers to sort the photographs. They're a necessary evil, so to say. Regards.

[michaelhills]

Mate (said with an Aussie accent), what is the big deal with ratings? Like a few have already pointed out - it is the encouragement of constructive comments that should be pursued and applauded. A single constructive comment on a composition is worth countless ratings. Recently, I have started not to rate compositions, but to try and leave comments instead. For my own work, I want people to TELL me how I can improve my compositions. I get no useful feedback from a rating of 7 or even a 1. Each photographer has a unique style and that style will not appeal to everybody in the world. As for ?mate? rating? Why should it matter - unless the people complaining desperately want their own compositions in the top 10? Stop whinging, go out, take photos, post them up on PN and encourage others to actually comment on your work. Enjoy life (and capture it at the same time!) M

[mark lucas]

Michael, how are you going to get all these comments if noone sees your pictures because the mate raters have clogged up the TRP?

[michaelhills]

Mark, I go to the Critique Requests page (http://www.photo.net/bboard/forum?topic_id=1481) and browse the newest requests for critiques by category. Unless I am mistaken, these pages are not ranked by ratings etc. They are just the most recently submitted photos for critique. I would presume other members use this options? Am I wrong? M

[mattvardy]

i have found another site as diverse and helpful as photon.net, it's called photosig.com. like PN it has a rather simplistic rating system, using thumb-ups and full articulate sentences to judge photos rather than a number scale and one-word-blurbs, but ratings are only given with critiques. imo, it seems to be a much more effective, user-friendly, helpful, and meaningful set-up.

 

it has been suggested here many times by various philosophers that critiques should be considered most important, ie first, before ratings; that a comment should be mandatory before any rating can be given. despite this suggestion's popularity, admin have shot it down repeatedly saying that it would be too easy for users to write meaningless "i hate it" sort of comments.... but surprisingly, this does not seem to be an issue over at photosig - with the clever use of a points system. other users can 'rate' your critique based on its helpfulness, thereby giving you (the author of the critique) 'points' - making it advantageous AND rewarding to write thoughtful critiques with some sort of substance or lucidity. this among other things like a recent critiques section, displaying the most recent critiques written site-wide, promote proper critiques. its really a great site imo.

 

yet, even though this system might seem to be more favourable that found here at pn, there are those over at photosig who continually complain about ratings. i think no matter how hard admin try, there will always be people who find fault in any system.

 

i know photo.net wouldn't want to simply copy another site's format, but surely something similar can be conjured up by admin. the question is, how much of a re-vamp are we willing to endure; and, will there be an overall decrease in the number of complaints if a new set-up were implemented. i bet, there would be some people screaming for joy, and others cowering at the thought (and subsequent realisation) of change.

 

photo.net has obviously become accustomed to the whole numeric ratings game, and the complaints it brings. complaints and new suggestions have always surfaced throughout the years, no doubt. yet i have a feeling, things are going to stay the same.

 

if you don't like something about this site, explore others...because "you don't know what you have until its gone". leave for a while and you might find that there are far more +'s to PN than there are -'s, in fact, you might miss PN. or, you might find another site out there that better suits your personal preferences with a more forgiving rating sytem, or whatever.

 

personally i think comments and critiques should be what hold a photographic community together, not ratings. ratings have become the be-all of photo.net, everyone is concerned with them - despite their vague, non-descript, controversial, and mathematical nature. its true, ratings keep PN going... but barely imo. they're turning PN into a statistically-based analytical machine, with critiques being few and far between, laboured, and virtually unrecognised.

 

since it appears that admin have decided this is the ideal set-up for PN - whether the reason is hits-per-day, cash flow through TRP ads, laziness, the uncertainty of change etc - then who are we to argue?

[keith turrill]

As to a ratings comparison between Photo.net and PhotoSIG.com, the sight has some interesting features. However, before making a judgement on that site, it would be wise to study the online behavior of the moderators on controverial critiques.

[howard_foto]

Matt's points indicate what should be apparent to all. as long as this site rakes in the dough it barely changes its stripes in response to problems within it and developments outside it. for example: no useful system to categorize photos other than by ratings (or critique requests). an interactive tagging and companion search system would help bring it into the 21st century. unfortunately, the site has created an underlying monster, its competitive ratings system, so that no collaborative system dependent on cooperative sincerity would work.