Jump to content

A Ratings Reform


Recommended Posts

Here is some data on ratings over the last week. This shows the

number of "Aesthetics" ratings at each value.

 

<pre>

DATE K A A1 A2 A3 A4 A5 A6 A7

----- ------ ------ ----- ----- ----- ----- ----- ----- -----

05-01 3586 5.29 7 17 266 459 994 1586 257

05-02 3482 5.33 6 17 221 407 1020 1550 261

05-03 3094 5.34 2 4 168 423 884 1397 216

05-04 4091 5.15 13 9 409 669 1062 1695 234

05-05 3862 5.33 7 3 254 539 1027 1703 329

05-06 4139 5.22 5 23 244 723 1209 1658 277

05-07 3819 5.30 8 13 216 546 1127 1616 293

</pre>

 

<p>"K" is the number of ratings on that day, still in the database.

The Originality numbers are very similar, although the ratings are a

shade lower.

 

<p>One notices several things from this data:

<ul>

<li>The range of values that people use is essentially 3-7, with a

negligible number of 1's and 2's

<li>The ratings are not normally distributed (i.e. it is not a

bell-shaped curve). The most common rating is 6, accounting for 40% of

the values, followed by 5 and 4. 3 and 7 are relatively rare, less

than 10% each, and 1 and 2 are hardly used at all. The average

value, which is supposed to be 4, is actually over 5.

</ul>

 

<p>I don't think it is possible to get a normal distribution and to

get people to use the full range of values. The practice of only

"rating what I like", either out of friendliness, or fear of

retaliation, is just too prevalent. To get a normal distribution,

people would have to be equally likely to rate a photo they like as

one they did not like, and that is simply not the case.

 

<p>One thing that is good is that people are still, on the whole,

somewhat discreet in awarding 7's. However, there are some problems

with 7's.

 

<p>First, people vary enormously in how many 7's they give out. Some

people print 7's like the currency of a banana republic. These people

have a big impact on a photo's average, and if you are playing the

"ratings game" on photo.net, part of the game is to make friends with

the people who shower their friends with 7's. I can't prove it with

statistics, but I also have a hunch that the people who are a liberal

with 7's don't have as good taste as those who are more discreet.

 

<p>A second problem is that the required comment entered with a 7 is

almost always "Wow", "Excellent", or even "7". Such comments are no

more informative than the rating alone, and tend to fill up the

commentary of photos with flattery that is tedious to read.

 

<p>I am therefore considering the following reforms:

<ol>

<li>Remove the requirement for comments on 1-2 and 7.

<li>Require that photo comments be at least 40 characters long (about

7-8 words)

<li>For photos on the system less than 30 days, users would be limited

to not more than 5% of 7's. So, someone can go through older

pictures in the database and rate the ones he or she likes without

restriction, but on newer pictures would be limited to no more than 1

"7" per 20 photos rated.

</ol>

 

<p>I'm interested in feedback on this plan.

Link to comment
Share on other sites

  • Replies 81
  • Created
  • Last Reply

Top Posters In This Topic

It is obviously an improvement, Brian, especially p.3. I like such simple and reasonable reforms.Let me share another simple idea - making the self rating obligatory. You can not upload a photo without self ratings. They are not calculated, can not be changed, and are exposed in the "Details" field. I think more balanced ratings could be achieved this way. Blago
Link to comment
Share on other sites

<I>Require that photo comments be at least 40 characters long (about 7-8 words)

</i><P>Why not make that a little longer? <P><I> Wow, this is a great photo, way to go!

I wish I could do that! </i><P>That is just over forty characters. Make the minim 250

characters.

Link to comment
Share on other sites

Ya really wanna hear my thoughts? Ahh, what the heck. Even if I don't officially participate in the critiques anymore...

 

1. Comments should still be required for 1-2 and 7.

 

2. Comments should be in standard essay form, a minimum of a three sentences/statements. They don't need to be in a single paragraph since that's difficult for the eye to scan given photo.net's current display properties. But if someone choosing to critique a photograph can't construct three coherent statements addressing specifics about the photograph, they probably don't need to be leaving comments at all.

 

3. Good idea.

Link to comment
Share on other sites

Brian, interesting data...

 

I would love to see a "no comment, no rate" system here. I think the lack of commenting is horrible. For every rate I have given I have written something. I never point only... I doubt this will happen but I think it would be a good step...

 

I think it would also help the trolls that come through and just rate everthing low. They wouldn't want to waste time writing anything.

 

Just my 2 cents... Dave

Link to comment
Share on other sites

the rating values are too discrete to be able to form bell-shape, especially that, as we see, the most "popular" is 6, so it should be "centered" somewhere around 6...we have only the 7 for a higher rate -> psychological factor:most people don't like to give 7's easily, even if they easily give 6's coz 7 is really the max. While downwards, there's room enough so they give lots of 5's.

 

Why the whole stuff is shifted up to 6 instead of being around 4? i agree with your reasoning that people mostly tend to rate the photos they do like more than the average (except when there's one really bad) so their average is shifted towards the bigger values. It is also hard to realize that "average" rate means average photo on photo.net and not an average photo out in the world. Hopefully, PN average is better ...

 

See attachment:) :gaussian fit to (7-days averaged) statistical data is much better when we disregard all the 7's given as rate (black curve), and is within the deviation for each rate value. Deviation = coloured stars of the 7 days scattered around the averaged (black cross)value. The red curve (which includes the 7's) fits within the deviation only for 3 points out of the 7.

 

my conclusion: it's only the rate 7 as the abrupt end of scale destroys the bell-shape.<div>0054kr-12658684.jpg.d39ebb8c1f5baac8dc4eda644fda6610.jpg</div>

Link to comment
Share on other sites

I think the first propose is better all, and self rating is a VERY good idea.<br>

I don't think it would require comment for 1,2 & 7 critique, and 40-100 character minimum for comments is superb idea.<br>

I hope the changes come fast.

Link to comment
Share on other sites

Brian, you are to be saluted for your efforts to make the rating system more meaningful. I've pretty much given up on it. But, you've asked for comments, so . . .

 

<p>I think the steps you propose would probably have <i>some</i> effect in spreading the curve and dealing with its skew towards the high end -- but I have to think that the effect would be pretty modest. Seems to me that the "only rating what I like" tendency that you mentioned is what is really causing the problem you're trying to fix.

 

<p>A question, though: if the requirement to include a comment with ratings of 1, 2 or 7 is dropped, then comments would not be required at all, since those are the only ratings for which they have been required. But then in what sense could you add a "requirement" that comments be of a certain length? Would it just mean that if you added a comment that was too short, your rating would go through but your short comment simply wouldn't show up? Or would the system bounce you back and tell you your comment was too short, and not take your rating until you either lengthened your comment or deleted it?

Link to comment
Share on other sites

<i>1.Remove the requirement for comments on 1-2 and 7.</i><br>

<br>

The least that will happen is that the number of 7s will increase dramatically<br>

<br>

<br>

<i>2. Require that photo comments be at least 40 characters long (about 7-8

words)</i><br>

<br>

Nice idea, but people playing the ratings game will learn how to paste an

enthusiastic but meaningless standard text of what ever size you mandate

and you might end up with <a

href="http://www.photo.net/shared/community-member-all?proc_name=ad_general_comments&user_id=455896#ph_photos">comments

like these</a>, only a little longer ... <br>

<br>

<br>

<i>3. For photos on the system less than 30 days, users would be limited

to not more than 5% of 7's. So, someone can go through older pictures in

the database and rate the ones he or she likes without restriction, but on

newer pictures would be limited to no more than 1 "7" per 20 photos rated.

</i><br>

<br>

Excellent idea. Why not apply the same idea to the raters. Everyone has to

maintain a balanced (bell-shaped) distribution of their ratings and if the

distribution is to skewed, they have to give out a couple of 3s or 4s before

they can give  more 6s and 7s.<br>

Link to comment
Share on other sites

I agree with 2 & 3 but not with 1. I suspect that 1 was brought in partly because there was a rather strange character who rated every photo that offended his religious beliefs with a '1', there still seem to be plenty of people about who take pleasure in 'punishing' people who dare to different.<br><i>But if someone choosing to critique a photograph can't construct three coherent statements addressing specifics about the photograph, they probably don't need to be leaving comments at all.</i> But what about people who can't communicate well in English? Shouldn't they be able to provide ratings? Some sites seem to encourage comments in a range of languages, and when looking at comments on usefilm I can press the 'translate' button if I don't understand the language (mind you, my knowledge of a particular language needs to be poor for the computer translation to be better)!
Link to comment
Share on other sites

Hi Brian. I don't usually post on anything related to ratings and critiques for various reasons.

 

But since you're asking, here are my opinions.

 

1 - I agree with you; I have never thought that comments should be required. I realize that this requirement was made in an attempt to counter "spite ratings" among other things. However, if someone wants to give '1' ratings, all they need to do is comment "Bad!" or "Horrible", or even something like "abcdefg�".

 

2 - "�comments be at least 40 characters�"; I don't think there should be any such requirement. I see a certain number of posts already complaining about the lack of comments; I think this would reduce the number of comments even further. Those who like to say things like "excellent" can simply cut and paste a longer standard response; ie, "Excellent; I really love this photograph! Keep up the good work!", etc.

 

3 - I don't think there should be a rating restriction based on date of posting. If someone wants to give a '7', but the quota has already been reached, they'll just give the highest they can. On the flip side, they could go and hurt "competing" photos with '1' ratings, knowing that the supporters of that photo can only counter their bad ratings at the ratio of 1:20 (5%).

 

 

My personal opinion on the best way to combat rating games would be to force comparative ranking between images. I have not thought through how this could be done, but it might be along this line: if I wanted to rate three photos, I have to put them in order of preference; ie, 1, 2 and 3. I could not rate them all as "best".

 

Granted, one could not be allowed to select all (possibly not any?) of the photos to sort. If they could, they might select one of their images and two of yours, then rank theirs first.

 

Possibly, there could be an option to sort random sets of three photos (within a date range?). If enough people did this, it seems like there would be a way to ultimately calculate a rating for each. And it seems to me that this would normalize the ratings as well as make it very difficult to high rate your friends and low rate your non-friends. Again, I've not thought this through, but would try to do so if you'd like.

Link to comment
Share on other sites

If a commentary standard were enforced - whether by a minimum number of characters or words or sentences - and such a standard reduced the volume of comments, that's probably a good thing.

 

Anyone who can't be bothered to write more than a bunch of gratuitous generalizations or mere exclamations *should* be discouraged from commenting.

 

The POWs are strong evidence that there are plenty of photo.netters who do enjoy exercising their minds and engaging in spirited debates about the merits of a photo, or lack thereof, in a constructive manner.

 

I'd also suggest that owners of photos be given the option of deleting comments at their discretion.

 

Sure, some folks will declare intellectual bankruptcy and delete any comment that displeases them. Fine, they don't want constructive criticism, that's their loss. Allow the delicate flowers to indulge themselves in faint praise.

 

But it will at least keep noise, trolls and flames to a minimum. If someone's feelings are hurt by a deliberately hurtful comment, or even a well-intended critique, just hit the delete button and move on - no need to clutter up the commentary section below one's photos with retorts or the photographic equivalent to "Oh, yeah" and "Yo' mama!"

 

The rest of us who are mature enough to handle genuine constructive criticism, pro or con, will be able to confine our excercising of the delete button to tossing drivel such as "Wow!" or "Love it!" or "I've seen better" or "Nothing here for me" (whatever the hell that means) into the appropriate ether.

 

It'd also be nice if deleting such dross automatically e-mailed to that person a note stating: "Your comment, 'Wow!,' is not a critique. Please engage brain and try again."

 

Of course I'd like it if folks had the option to choose whether to make their photos subject to the numerical ratings system at all. I think Brian has said before that this may be infeasible, tho'.

Link to comment
Share on other sites

Since 1 to 10 and 1 to 7 have not worked, has anyone thought about going to 7 through 13? There would be no conversion possible, of course, so that all of the 7s would become the lowest scores and anything else would drop out of sight. We would have a gallery of unrated or lowest-rated photos. Alternatively, everytime someone gives a 7, perhaps one of their photos (randomly selected) just disappears. We need some way to randomize the system and get that bell curve back. Here's to inductive proofs. . . .
Link to comment
Share on other sites

<em>1- Remove the requirement for comments on 1-2 and 7. <br>

2 - Require that photo comments be at least 40 characters long (about 7-8 words)<br>

3 - For photos on the system less than 30 days, users would be limited to not more than 5% of 7's. So, someone can go through older pictures in the database and rate the ones he or she likes without restriction, but on newer pictures would be limited to no more than 1 "7" per 20 photos rated.

</em>

<p>

I think we have to ask what the goals are and what the effect of these changes might be.

<p>

Clearly the goal is a rating system which by some metric measures the aesthetic quality of an image. I think this is an impossible goal.

<p>

A secondary goal might be to stop ratings manipulation. Again I think it's an impossible goal.

<p>

A third goal might be to stop rating inflation, and again I just don't think this is possible.

<p>

So we have an impossible task. That doesn't mean we should give up, just that we shouldn't expect to succeed. Any improvement is better than no improvement!

<p>

Restricting the number of times you can give a particular score (e.g. "7") will impact people who only rate photos which they find to be of exceptionally high quality. I think it would be a good idea to have a requirement that anyone who gives a 7 should back it up with at least a 25 word comment. If you can't find 25 words to describe why you think an image deserves a "7", then you clearly haven't thought things through enough to give it such a high rating. If we do this I don't think we need to limit the number of 7 scores you can give. If you limit the number of 7s, people will just give 6s and the whole system will evolve into one where everybody gives their friends 6s and very few 7s get handed out so in the end all you'll get is a system where 6s "win" instead of 7s "winning".

<p>

1 and 2 ratings are pretty meaningless, so get rid of them. Nobody uses them anyway except to punish other users.

<p>

So partly for the sake of contraryness and partly because I think this might work better, I'd suggest the following:

<p><b>

1 - Eliminate 1 and 2 ratings. Nobody uses them, just about no image deserves them and their only function seems to be as a punishment for people who have offended the rater in some way. This violates the "7 steps" rule and 4 is no longer "average", but 4 is no longer "average" anyway. Average seems to be about 5 these days and that's the mid point of a 3-7 range. Images really fall into 5 categories anyway - bad, below-average, average, above-average and good.

<p>

2 - Don't limit the number of 7s anyone can give out, but require that any 7 is accompanied by a real comment. I'd suggest a minimum length of 25 words. Anyone obviously abusing this (by using the same word 25 times or by posting the same 25 word comment on every image) simply gets banned from giving ANY 7s to anyone.</b>

Link to comment
Share on other sites

There are some of us who do not rate the photo's we don't like. I guess that doesn't give the rating system the 'bottom' you are looking for. I personally have a problem with giving 1's and 2's because, it seems to be a matter of taste as to what is good or bad. I have a whole spectrum of ratings on some of my single photos, ranging from 2- 6 (for the same photo), with no real idea as to WHY. I have had one 7, which was immediatly followed by a 3 I think, so who can figure these thingums out?

 

I think if you feel strongly enough (good OR bad) to give someone a 1 or a 7, you owe the photographer at least an explaination so they can either do that 'gOOD' thing again, or try to avoid that BAD thing.

 

I REALLY LIKE that idea about using MORE WORDS in the comments, that would be good :)

 

just my two cents worth

 

holly:)

Link to comment
Share on other sites

See, people *will* write complete thoughts if they find something interesting.

 

Even tho' I'd give this thread only a 3 for originality because it's been beat to death. But 6 for aesthetics because of stylish presentation and well considered arguments.

Link to comment
Share on other sites

If you require a certain number of words for comments, then the comments

will be filled with paragraphs like:

<p>

<cite>I like it a lot. Yes, I like it a whole lot. This is a very nice picture and I like it. I can't think of much more to say, except that I like it and the software wants me to write some more words. Here are some more words that say I like it. Oops, they just increased the quota of words I have to write, so here are some more words to say that I like it. It's a very nice picture and I like it. Now is the time for all good men to come to the aid of their country. The quick brown fox jumps over the lazy dog. I like this picture. It's nice.</cite>

<p>

I don't do numerical ratings, because of the problems that have already been mentioned numerous times. I think

Bill C. is onto something very clever. It would be great if,

instead of numerical ratings, those who want to rate photos

are shown a randomly selected pair of images, and asked to

say whether "A" is better, "B" is better, or both images are

the same quality. Obviously, it would take quite a bit of

software work and a some thought about an algorithm to

turn a large collection of those pairwise ratings into

an overall ranking and/or a "letter grade" for each image,

but the result would allow the ratings competition to

continue, free from many of the problems inherent in asking

each rater for a numerical score on images he wants to rate.

Link to comment
Share on other sites

Okay. I'm new as of about MARCH 15, 2003. I'm a pretty good photographer.

Well, good enough to have earned a living at it for 35 years. Good enough to have

been a member of the White House Press Corps. Good enough to have been

published in most major magazines in the world. These are my only credentials.

I've never entered a photo contest of any sort (yet). Just never got around to it

i guess. I don't consider myself to be extremely egomaniacal. But I certainly do

like for my work to be appreciated. And I am always trying to grow and learn

more about photography.

 

Don't we ALL want praise and validation sometimes? Be honest! And aren't we

ALL more willing to accept critique from those we respect? (I studied the Zone

System with Ansel Adams in 1971 - Mr. Adams could rip up my prints all day long

I would simply come back for more, but if a fellow student did it I'd ignore him/

her.)

 

When I discovered Photo.net a few weeks ago, my first thought was, "Wow, this

is really great! Tons of amazing pictures here! By all sorts of photogs I've never

heard of, this will be fun!" - I was truly excited and impressed! At first.

 

I happened to have a few scans of some of my pictures, so I began uploading

images BEFORE STUDYING THE SITE. I mean, I'm a pretty smart guy - and being 50

years old, I'm fairly mature. My first instinct was to take it slow and rate only

those pictures which I liked. Not because I was consciously afraid of retaliation (i

was to learn how that worked later) but because it TAKES SO LONG to rate a

photo, and there ARE SO MANY GREAT ONES HERE, AND because it's only natural

for a photog who's been around a bit to be put-off by terrible photos and drawn

to the really good ones.

 

So I started rating my favorites first. I do tend to give EACH PICTURE, BASED ON

ITS MERIT, THE RATING I BELIEVE IT DESERVES, ALONG WITH A WELL INFORMED

CRITIQUE / WRITTEN OPINION. Since I hadn't even thought about the HUMAN

NATURE FACTOR yet, I pretty much stuck to these principles.

 

THEN my own ego started kicking in. We ALL HAVE AN EGO!!! And so did the egos

of those I'd rated!

 

I began noticing that some people who rate others harshly don't even have any

pictures up of their own. Then I began noticing LOTS OF OTHER PHOTOGS mention

this point in various forums. My bet would be that they too have been put-off by

the human nature factor, to put it nicely.

 

Then I realized, while the majority of top photos are by paying members, many

top photos are not necessarily by paying members. This made me wonder, why

would ANYONE, including myself, NOTPAY??? Then I started wondering, WHAT IF a

high percentage of those non-paying photogs paid?? Wouldn't that liven the

spirits of the elves and administrators??? So why aren't these people paying? I

had only to look at my self.

 

Then I realized that the SYSTEM is extremely complicated. You comment, with

some effort, on a photo and then want to rate it ( as most of the more

accomplished photogs here believe we should). This PROCESS takes a lot of TIME.

If you go the critique request page, it's much faster to rate lots of pictures -

but unless you have a photographic memory, you'll have to spend HOURS slogging

through members pictures first, and even those SEEM to be quite different than

the natural order of the top pages images. I'll bet this is a major factor with the

Brief Comments Problem.

 

It just isn't a logical system if fairness and intelligence are the principles to

which you expect everyone to adhere.

 

See what I mean, though? IT's slow and complicated. But that's not meant to be a

criticism of the site!!! If anything, it's more of a wake up call as to the REAL

PROBLEM: EGO EGO EGO!!!

 

WHAT IF:

1. EVERYONE COULD JUST PUT UP THEIR PHOTOS AND EACH WAS IDENTIFIED AS

TO WHETHER OR NOT IT BELONGED TO A PAYING MEMBER, BUT NOT BY HIS/HER

INDIVIDUAL NAME? (Think about it: IF what you're doing is

A. Offering our work up to be critiqued with a willingness to learn and grow as

photographers.... and

B. Interested in seeing other really great photographer's work - regardless of

their professional experience and / or "knowledge" so we can learn and grow as

photogs.... and

C. Receiving valued praise and / or criticism from both our peers and those who

simply "like" (or not) our work....

 

THEN WOULD IT NOT BEHOOVE EVERYONE to make this whole rating thing a bit

simpler as well as more FUN???

 

NOW I realize that if you were to take this as a criticism of this site and the

millions of hours and tens of thousands of dollars (or more) and blood and sweat

and tears that have gone into it, you're going to think I'm just another moron and

go give all my pictures double 1's in retaliation or something.

 

BUT - if you look at the big picture - that this site has thousands and thousands

of incredible images on it. You have hundreds if not thousands of good

photographers, a few great ones and lots of beginners too. EACH AND EVERY ONE

OF WHICH HAS AN EGO. And everyone wants to be treated with respect. Some

people deserve it and other's don't.

 

It's always been my opinion that PHOTOGRAPHY is about the IMAGE! You can dig

my picture or not. You may even be able to justify why you think it sucks, but

someone else may love it. IN THE END - it's the number of HONEST OPINIONS that

win when it comes to ANY PIECE OF ART being labeled "good" or "bad" or

"average" etc... No wonder most artist's works are not seen as valuable until

they die. If people don't "like you" no matter how good your work is, you're

screwed. So it is here on PHOTO.NET - no matter how good the image, if you

don't have lot's of friends, you're not going to get high ratings. If you never

make it to a top page and you KNOW it's because people don't like you, and your

NOTSURE it's because your pictures suck, well, you're going to feel bad and

people who feel bad usually get mad and retaliate, etc...

 

Okay - back to the big picture:

 

So WHAT IF I could upload a picture, it goes into a constantly rotating slot along

with all the other "newly uploaded images" for like a WEEK? After that week, the

X number of images with the highest ratings go into a more conventional

photo.net top page labled "BEST OF THE LAST BATCH".

 

THEN after that, the last batch (which is new every week) goes into a top page

labeled appropriately, "LAST WEEKS BEST" where it stays for a week (or 24

hours or 3 days or whatever) while the new "batch" is being judged.

 

So effectively, what you would get is a top page of the last weeks best which

could be a total of however many number of photos you want it to be.

Then you get a ROTATING GROUP of all newly uploaded images from the previous

week and updated just as you do now with "24 HOURS", except that as new

images go in, the images are #1. constantly rotated so as that there is no

priority given to more highly rated photos yet. #2. The photos are not labled by

photogs name yet so as to prevent friends rating friends highly until each photo

has been in the rotation a week. #3. The ratings may or may not show up when

giving a rating and comment (if they don't show, no one can be manipulating

according to whatever unscrupulous ways they have been. #4. And to rate, you

simply MUST make an attempt at an intelligent comment or critique.

 

IF YOU SIMPLY CHANGE YOUR RULES according to what you've proposed above,

you will not have overcome the problem inherent in us all - the HUMAN EGO

FACTOR.

 

At least TRY to do something about making the first few days of a picture's life

on Photo.net completely annonymous! DO that, and require comments with ALL

ratings, and don't priss around trying to put limits on the high or low side of

whatever SCALE you decide to use for ratings. EACH PHOTO should be able to be

rated and critiqued STRICTLY ON ITS OWN MERIT! AT least for a few days or

week. Wouldn't that be a lot more FUN!!!??? EVeryone would be able to anticipate

what their ratings are under a much more FAIR SYSTEM.

 

Plus, you'd be better able to assess more accurately later on who has been

attempting to play the friend rate game by looking at the names of the raters

AFTER that picture has been rated by everyone for a week. Plus you'd be able to

see more pure photography as a lurker or beginner before trying to "compete".

 

Make the ratings and comments annonymous during the first week, not to be

revealed until after that first week. Require a comment for ALL ratings.

 

Rotate that batch frequently at certain times so that every single picture has

had at least one chance to be on top during a certain cycle. eg... every picture is

on top 21 of "Latest Batch" twice a day or so. And don't allow "request critique"

until AFTER all the images in that first "Batch" has been rotated through to

wherever you end up putting it

 

Leave all your other rules the same until the new system has been implemented.

Leave all the other "top" pages the same as well. Don't change the rating system

otherwise yet.

 

NOW you've got a much more fair system, it'd be a lot more fun seeing each new

batch each week, and then there would probably be fewer people willing to risk

looking like sycophants when the truth comes down at the end of each "batch".

And because it would be more fair, more people would likely give you money to be

a part of it.

 

I'm repeating myself but since Brian is clearly a genius, my guess is he can glean

something worthwhile out of this - even if it is only that I am just another

newby?

 

But please do try to get this one point: Every photographer / artist has an ego,

few are true adults. It's a lot easier to take criticism when you respect the

person doing the critique AND you know others either like your work OR they

mostly agree that your work isn't as good as you thought. And as long as you

believe everyone is judged by the same basic standards, it's all cool. If you can

figure out a way to make this site offer that, you'll be swamped with new

members.

Link to comment
Share on other sites

Point taken, Richard. Which is why I advocate allowing the photographer to delete such inconsequential fluff, at whim, with cause or without.

 

At least then no one can accuse the poor elves of being heavyhanded on the delete button. We'll all be equally guilty.

Link to comment
Share on other sites

James - thanks for a thoughtful evaluation. As a new recruit to photo.net and as an experienced photographer your view are valuable.

However it's trickier than you might think. I beieve we have tried anonymous ratings - didn't we start out that way? The trouble there is that if you don't know who gave what rating people can "attack" you by giving you low ratings and you've no idea they are doing it and no way to detect their "games"!

 

I don't think we've tried anonymous images (i.e. not identifying the photographer). It's a bit of a programing problem I think since images are linked to the photographer's portfolio so you'd have to get real clever and hide URLs in some way. It would also not stop friends giving each other ratings since they could just tell each other which images were theirs. You can also just go to someone's portfolio and rate their images. I suppose you could "hide" images for some period so they didn't show up in the portfolio but what about people who dont' care about ratings but actually want their images in their portfolio. I suppose they could "opt out" of the rating system, but again it makes the programming tricky - and I belive we've tried that one too!

 

You are correct that EGO is the problem, but it's a universal human problem. The question is really whether it's practical and/or even desirable to design a system so bulletproof that it can't be abused, involving cloaking or URLs and user IDs. There's a school of thought which says let the idiots who want to play ego games play their games, but let's have an open system.

 

Is the huge amount of programming effort and mental anguish that goes into trying to stop people playing silly games worth it? Could that effort be better used elsewhere?

 

Richard - yes, people could abuse the comment system by posting text such as you suggest. However as I said earlier, it would not be too difficult to institute a system where anyone found doing that would have their rating privalidges suspended. If they're going to do that, nobody's going to miss their comments or ratings anyway.

 

I think any workable system has to be simple. If it's full of complex rules about who can do what, how often they can do it, who they can do it to etc. etc. it gets really difficult to make changes and people get confused.

Link to comment
Share on other sites

I guess it just seems that so much time and effort is wasted NOW on this

problem. Seems like you would want new people all the time and you'd want them

to pay something in exchange for using photo.net.

 

If a person put up pictures and the image itself remained annonymous for a

period of time MAYBE it would get MOST of it's ratings and comments during that

time based primarily on ITS OWN MERIT?

 

Once the picture has been through its "annonymous" period, it becomes like all

the others. At least, during that one time period, you'd MINIMIZE a large part of

the problem without changing the entire dynamics of the rest of the site? That's

gotta be worth something, doesn't it?

 

What if you remove the names from ALL pictures on the site? Keep the LINK to

the photog's portfolio page, but this makes it more likely most pictures will get

more honest ratings over the long haul.

 

All I know is, I'd rate a lot more photos every time I log on if I could whip through

them just going by what I see is the merit of the photo, and I wouldn't hesitate

out of fear, as happens now.

Link to comment
Share on other sites

Brian,

<p>

I've thought about this issue for some time now, although I haven't publicly commented. To me it seems the easiest solution is to make ratings and critiques separate and distinct.

<p>

When a image is posted it can be flagged as: Rate - Yes/No <b>or</b> Critique - Yes/No, <i><b>but not both</i></b>. If the submitter wants to stroke their ego and have friends and family give them 7/7, 10/10, 100/100, or whatever the scale is, who cares!!! Let them win the meaningless #1 ranking.

<p>

On the other hand those people who want an honest evaluation of their work can indicate that when submitting and those photos can be critiqued but not numerically rated.

<p>

Thus those who really want to improve can have a (separate) forum where their work can be critically evaluated by people who want to help. Those who play the 'rating game' for ego gratification won't participate in critiques because (1) there would be no reason to, (2) it would require effort and (3) there would be no benefit to be gained from a 'bad' critique.

<p>

Just a thought...

Link to comment
Share on other sites

Bob Atkins wrote:

<p><i>I don't think we've tried anonymous images (i.e. not identifying the photographer). It's a bit of a programing problem I think since images are linked to the photographer's portfolio so you'd have to get real clever and hide URLs in some way.</i>

<p>I think that actually each photo is identified by a unique photo_id number in the photos database (/photodb/). Photographer's portfolios are identified by user_id numbers. From the URL of an individual photo, I don't think you can determine the user_id of the photographer or otherwise figure out how to get to their portfolio.

<p>Bob also wrote:

<p><i>It would also not stop friends giving each other ratings since they could just tell each other which images were theirs.</i>

<p>The answer to this, is a system that "feeds" photos randomly to the rater, and requires a rating to be given before another photo is displayed, so that raters <i>can't</i> choose what photos they'll rate.

<p>Bob also wrote:

<p><i>You can also just go to someone's portfolio and rate their images.</i>

<p>That's true -- at least, it is the way things are set up now. So, eliminate that capability -- set up a system where the only place photos could be rated was in a rating interface that, as mentioned, feeds photos randomly to the rater.

Link to comment
Share on other sites

I don't think a system that prevented portfolio images being rated would fly.

 

I don't want to rate random imges. Most of them aren't very interesting. If I find an interesting photographer I'm inclined to look through the associated portfolio and perhaps rate or comment on images.

 

You could certainly do some sort of anonymous system, but it would require a lot of effort to program and it would have drawbacks as well as advantages.

 

I'm still not sure that the vast majority of gallery users aren't perfectly content with the way things are. The usual "silent majority". There a a few people who complain here all the time about unfair ratings, but they tend to be a small subset of the small subset of photographers who have a chance of actually getting their images on the "top" pages and who care about scores.. What most users seem to want are more constructive comments and they don't care about "rating cliques" and the like. However I don't see any way to get more constructive comments. You can't force people to comment, and even if you could you couldn't force them to make useful and constructive suggestions. There are just too many images and too few people capable of and willing to do that.

 

 

Brian's suggestions are fine (though of course mine are better!), but unfortunately I doubt that either result in what the "vocal minority" of users will accept as "fair". Life's not fair, why should we expect gallery ratings be fair?

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...