Jump to content

A Ratings Reform


Recommended Posts

Life's not fair. That's true. But photography, even if it IS your vocation (and let's

face it, that's not the case with the majority of users here) EVERYONE DOES

ENJOY knowing how they stand amongst their peers as well as their superiors and

inferiors. THAT'S a big part of life too!

 

You all know how it works here now. Don't you think that trying to figure out a

way to make the ratings thing work better is worthwhile? IF SO... wouldn't it be

nice to be able to simply log on and begin looking at images with NO NAMES on

them? Pages and pages of newly uploaded images which are constantly rotated

as far as positioning goes on those "annonymous" pages so that the ones on top

are not necessarily the highest rated? Where you can simply comment and/or

rate them without any outside influence? And wouldn't it at least be interesting

to find out where those images you rated /commented on fell at the end of this

annonymous cycle?

 

After these images are moved out of this "annonymous" section, everyone would

then get to see the ratings and the names of the photographers whose work

they had rated and commented on. Frankly, I think that alone would be worth the

price of admission!

 

Nothing else needs to change, necessarily. But at least there would be one way

for everyone to get their images rated and critiqued with the least amount of

gamesmanship and oneupmanship. It may not be possible to make it perfectly

uncorruptable, but it would certainly remove a lot of doubt. And most of all, it

would be a very positive experience for even the most cynical among us all. It

may even be humbling for some. But if it works at all, if people ENJOYED THE

EXPERIENCE, (even if their images didn't receive a high rating) everyone would

have the opportunity to improve the next time.

 

Yeah, I know life's not fair. But who wants to deal with retaliation ratings? Who

wants to bother with getting upset over sarcastic comments and snide

remarks? Who wants to show their creativity and hard work to people who don't

respect it, whether they like it or not? Who wants to pay for the privilege of

being told they're too thin skinned? Who wants to spend so much time playing

politics (and worse) just to get their pictures SEEN?? Let's face it - no matter

what your position in life, it takes a LOT OF TIME to become a good photographer

and it takes a LOT OF TIME to upload, rate, comment upon, figure out the

workings of this site, etc.. here at photo.net. IN fact, you guys and girls are

constantly having to explain this fact of life here to newbys.

 

The bottom line is, everyone wants as many people to see their pictures as is

possible. ( EGO EGO EGO -It's human nature, and darn sure the nature of any

artist! That doesn't necessarily mean it's BAD!) And the way things are set up

now, and you KNOW THIS IS TRUE, all you need are a few "freinds" to get your

images seen by more people everyday. So it seems that as soon as many people

figure this out, they start playing those games. Not everyone, no. Not all the

time, no. But so many as to make it quite rare for someone who isn't at least

fibbing sometimes, or hedging or pulling punches sometimes, to get their

pictures seen more, and thereby get more comments and thereby learn and grow

more. That being the ulitmate purpose of this site as stated elsewhere.

 

I've read it over and over here - ya gotta play the game if youse wants youse

pictures seen.

 

So what if: EVERYONE got to upload pictures annonymously and for that one

week (or however long) period of time, every comment and rating they got they

knew it was on the merit of that picture alone? And that their name would not

get them a comment or rating. And that they could tell all their friends, but all

their friends MIGHT BE the ONLY people who rate their picture. And if they keep

doing THAT, someone WILL NOTICE EVENTUALLY and bust them?

 

I'm sure you guys are a lot smarter than me, so I'm going to shut up now.

SHEESH!

Link to comment
Share on other sites

  • Replies 81
  • Created
  • Last Reply

Top Posters In This Topic

I don't know that everybody does want their images to be seen by as many people as possible and gather the best possible ratings. I suppose it's a good working hypothesis, but don't forget that in thsi forum we only hear from maybe 10 or 20 of the most active, ratings aware gallery users. I'm sure Brain has exact numbers but I'd guess that there are probably between 100 and 1000 times more gallery users that never make a comment than do. What do they think? Is the <1% who are vocal representative of the rest? Maybe they are, I just don't know.

 

I put images in my portfolio here. I don't care about the ratings, I never check to see if they make the "top" pages but I do occasionally check to see if anyone has made a comment.

 

Keeping everyone happy must be a nightmare of a job for Brian and deciding which, if any, of the dozens of often opposing suggestions to try to implement must be an even bigger headache. I certainly don't envy his job!

Link to comment
Share on other sites

An indepth tutorial on rating and critiquing with a link on the home page might help give us guidelines and instruct new members as to the value of honest evaluations and comments. I don't think just changing the numbers system or lenth of characters posted will do as much as educating attitudes and egos. I would also like to see some comments and ratings deleted if they didn't conform to accepted guidelines for a rating system. (Such as awarding a "1" just because a certain person appeared in the background of a photo)! Anyway, your concern on this issue is appreciated, and I'm looking forward to any reforms that will have a positive effect.
Link to comment
Share on other sites

How about this system:

 

1. One must allways leave a comment when rating. (Maybe some minimum length).

 

2. One can rate a photo by giving a approval (or thumb or whatever) or not giving it.

 

This means that there is no rating scale, only 2 values. And good photos have many of these points. The reason is that people usually either gives a '6' or don't rate at all. So why not restrict the scale to "Good photo" and "Outstanding photo".

Link to comment
Share on other sites

for hundreds of different persons, meaning of the same rate is totally different. solution to this might be: smaller scale, less divergency in oppinions. Reducing the scale to two or at most 3 rates, 1 2 and 3. Then 2 is average, 1 is bad, 3 is good. No more possibility to boost a photo's average from 3.3 to 6 just by making two "ratemates" who will "appreciate" it with two 7's. Since we do tend to rate mostly the photos we like, the (small?) differences between "good" and "very good" will come out simply in the number of ratings, and/or the sum. Finally, "top photos" page based on the sum of the ratings. Being able to choose what criteria you want to use for sorting, that should be kept of course, it's a good thing.

 

Two rates, bad and good, is even better. Noone will be offended by Mister Taylor(there are few hundreds of them, so don't worry, it's NOT you) giving a "1" for a photo, since there are lots of better photos always. Much easier to compare unanimously. Oh yes, you will be sure you get back your bell-shape with two rates. After 100 rates per photo:))

 

greetz

Link to comment
Share on other sites

Please do not put restrictions on comments. Even a short one could be better than none. People get more from comments than ratings no matter what. Maybe we could devise a bunch of verbal comments chosen by check box to go with the rating. Things like "I like the compositions", "The colors are off", "The lighting is good" and so on. The text from those choices would then be entered below the photo. Of course the rater could still enter their original comment. At least having a bunch of canned critisms might even make people think more about the rating.
Link to comment
Share on other sites

I think that if you want a bell shaped curve for the site, that each user should be forced to have a bell shaped curve of their own. There are plenty of photographs out there that could use a few 2's and 3's plunked down on them.

 

Unlimited 4's for everybody. Average is average and most of the images here are average.

 

For every 5 you give out, you must give out a 3 to balance it.

 

For every 6 you give out, you must give out a 2.

 

For every 7, you distribute a 1.

 

Rather than seeing an upsurge in the lower ratings, I think this will result in a downward trend in the higher numbers because the halloween distributors of 7's won't want to upset anyone by giving out a 1. This should result in a much more tightly packed center of the ratings scale.

 

I have no problem with removing the comment requirement on 1,2, and 7, because the bulk of the comments you get with those ratings is typically useless dribble.

 

A requirement of longer comments on any rating is fine by me as well as long as abusers are sanctioned.

 

I don't quite understand #3. If I am reading it correctly, it has nothing to do with the photo itself, but you would be limiting the distribution of 7's to new images by the raters. Each individual rater would only be allowed to distribute 5% of his total rates of new images with a 7. Is that correct? If so, anything that lowers the number of 6's and 7's distributed is good with me.

 

I happened to notice yesterday one members rating stats.

 

He/She had been a member for over one year with an average rating level of 6.22/6.22 with nearly 500 photos rated.

 

Over 20% of the ratings were 7/7's and over 25% of them to one photographer.

 

That same photographer recieved over 20% of the 7/6's and 30% of the visible 6/6's as well.

 

This is the type of behaviour that needs to be curbed.

Link to comment
Share on other sites

I have only been a PN patron for 9 months and have been away for the last two. So, I would like to acknowledge some recent added major improvements, i.e.: the "highest rated photos by this member" feature, "Number of Photos with more than 10 Ratings", "Critics Circle", "Featured Portfolio" on the front page and the new-look individual "Portfolio" page...<br>Sadly, I see that the ratings issue is continuing to be an on-going problem and the fact that Brian has posted this thread with his suggested reforms, has reminded me that some things never change (human behaviour).<br>I noticed that some PN members have been banned, some have become "0" uploads and some have simply stopped participating - this is very unfortunate.<br>

The proposed changes are a good idea, however, I agree that comments should stay for awarding 1,2 and 7's.<br>What I would like to see at PN is the inability of members to change their user identity! During the last week, I used the random image ratings Gallery and came across a fairly decent recent upload... I left a 6/6 rating, only to find out a couple of days later that the person who posted the image had changed identity! (from a matter of principle, I would normally <b>not</b> leave any ratings on this person's work). So, I felt cheated... This example illustrates that JV Knowles has made some valid points on this thread about the anonymity of new uploads. Some (many) of us <b>do</b> rate differently if we know the identity of the photographer...<br> Although I am not in favour of anonymous ratings, what I <b>do</b> like, is the feeling of picking out a newly uploaded image with 'no ratings', recognizing it as a genuine "top" image and then seeing one's judgement being vindicated during subsequent days, when others leave their honest opions.

Link to comment
Share on other sites

I like your proposal, Brian, all 3 points. I'm plain fine with them.

<p>

I would also agree with strictly everything Scott Bulger posted just before me. I also agree with James Vincent Knowles's philosophy, as well as to the possibility, worth considering, of having a week anonymity and mostly a rotation of all images on the front page - at least till each picture gets 2 ratings for example.

<p>

I basically feel that a few other issues would also need to be addressed, perhaps in a slightly more radical way, but I'm still considering how would be best.

<p>

Then perhaps a bit more than 7 or 8 words would be a bit better - maybe 15 to 20 words. But it seems that another site is having some trouble with the implementation of a similar idea, so I would suggest to really make sure most members approve or at least can accept your point #2 before implementing it.

<p>

Finally they are a few other ideas on my mind that could complement this reform, but I need time to formulate them and refine them a bit, so I'll probably be back for another post during the week-end or next week.

<p>

Scott's idea of a 1 for a 7, a 2 for a 6 etc seems interesting as well. Perhaps it needs to be slightly more permissive (a 3 for 2 6s or such), in order not to force people to spend too much time on images that they have no interest in), but I like the concept.

<p>

I also feel Scott is right to address the issue of some highly problematic raters. Some raters have more than 85% of 6s and 7s among the ratings they have distributed. To me, that makes no sense at all, and I would suggest to consider implementing retroactively as well this limitation of high scores submitted, or to somehow take care of raters for whom every crappy shot is a 7.

<p>

Meanwhile, congratulations for these 3 steps proposal, which would no doubt only help the rating system to work better.

Link to comment
Share on other sites

Am I understanding point 1 and 2 properly ? The way I understand it is that NOT ONLY ratings of 1, 2 or 7 should come with a comment, but that ALL RATINGS would now require a comment of a certain number of words ? Is that right or am I mistaken...?
Link to comment
Share on other sites

Great ideas Brian. A note to Bob... Eliminating 1 and 2 ratings will only cause 3 and 4 ratings to be the equivalant of a 1 or 2 and people will not want to insult the photographer by giving him/her such a low rate. As to comments... I think it would make for much more thoughtful ratings if we had to make a comment in order to rate.
Link to comment
Share on other sites

Mary, while removing the 1 and 2 ratings leaves 3 and 4 as the lowest scores, giving 3s and 4s won't bias the overall score as much.

 

Tell me, have you ever seen an image that deserved a "1" and if you did, would you bother to take the time to rate it? Ditto for a "2". The only purpose for giving a "1" or a "2" ratings seems to be retaliation. I've seen them given in bulk to a whole portfolio for just that purpose. If the ratings have no valid use except to prop up the rest of the scale AND they are a principle source of abuse, the answer is simple. GET RID OF THEM.

 

As to any scheme that dictates how many 3s, 4s, 5s, 6s and 7s you can give based on some mathematical formula, it's lunacy (sorry, I'm not trying to be nasty but I'm trying to make a strong point). You're crippling the whole system just to make it "fair"

 

There's a science fiction story, whose title and author I can't remember, but it involved the creation of a "fair" society. In oder to accomplish this various things were done, such as to tie sandbags to ballet dancers so that their performance wouldn't be so far from the "norm" when it came to dancing. Dictating what score you can give to an image based on some mathematical modes which limit syour options amounts to the same thing.

 

You can't (and shouldn't attempt to) stop people only rating images that they think are really good, and if you could, you shouldn't. Just because it's possible for some idiot to abuse the scheme by giving out all 7's you shouldn't restrict me if I only want to take the time to rate and comment on images that I think deserve a 7. Why do I get restricted for the actions of some other socially irresponsible imature idiot who's abusing the system. Restrict him (her) not me. You don't stop car theft by banning the ownership of cars. It would work, but clearly that's not the sole criterion on which a solution to the problem should be based.

Link to comment
Share on other sites

Bob, it�s Vonnegut�s "Harrison Bergeron". As for the proposal:<br/>

<br/>

1. Good idea. The forced comments tend to be uninteresting.<br/>

<br/>

2. This might help a little. Some people might be encouraged to think a little more about their comments, and that's a step in the right direction! Sure, some crappy comments will merely be longer. Win some, lose some.<br/>

<br/>

3. This sounds like trouble. Will users get to downgrade 7s to reclaim a few? Or are a few sacrificial 6s expected to "earn" another 7? Clearly the normal curve advocates have different goals than those who might use it as a <a href="http://www.photo.net/bboard/q-and-a-fetch-msg?msg_id=0054PA&unified_p=1">"favorites" feature</a>.

Link to comment
Share on other sites

Society already restricts all peoples behaviour in some ways because a few abuse things.

 

Liability insurance is so expensive because some idiot sues McDonalds for a million dollars because she spilt hot coffee on herself. We all pay.

 

They have speed limits on roads. Sure, some people are qaulified to drive 100 on the freeway safely, but not everybody, so everybody has to stay at 65.

 

I've never hijacked a plane, but I'm still not allowed to carry a pocket knife on one.

 

There are always idiots that spoil things for everybody.

 

Have I seen images on photo.net that deserved 1's and 2's? Sure, lots of them. By our current standards 1=Poor, correct? Lot and lots of poor images out there, but no one likes to give out the 1's and 2's. I don't want to encourage 1's and 2's, but if you are going to have to give out a 1 every time you dish out a 7, people are going to think harder about the 7's. You have never given out a 7 Bob, so you wouldn't ahve to worry.

 

As far as taking the time to comment on a 1 or a 2 rated photo, who needs the comment more, the 2 or the 7? If a photo deserves a 7, how much helpful critique can you give them? it's already excellent. On the otherhand, you should be able to write volumes about an image that deserves a 2, don't you think? That person certainly needs more help.

 

Take the stigma off of the lower numbers by making more of them issued.

 

90% of the photos here should should get 3's, 4's, and 5's. Currently, many folks are insulted by you telling them that their image is "Good" by issuing a "5". I thought that "Good" was a good thing. When did things become so skewed that "Good" was a bad thing?

Link to comment
Share on other sites

Brian, what an interesting thread. I've been a member for over a year, and haven't rated a photograph in months. No matter how you fiddle with it the ratings system will always be susceptible to manipulation, retaliation, and as James points out, ego. You've tried the 1-10 scale, now the (essentially) 3-7 scale, always to end up with the exact same problem. As I recall, your rationale for the 1-7 scale had to do with some distribution patterns- good idea that failed. Here is a suggestion: go the other way, make the scale like wew used in school, 1-100. The Wine Spectator has used this scale for years- an 85 is a good but not great wine, a 90 is terrific, a 98 sublime. At the same time 60 is just terrible, 75 mediocre, etc. Make the subjective descriptions integral to the scale; if you give someone less than 75 you are point blank telling them their work is less than mediocre. If you're a stroker, giving all 90+ scores will make you stand out as well. Then, every once in a while do a bell curve review and redistribute the scores. THIS TIME DON'T LISTEN TO THE WHINING AND DON'T REDISTRIBUTE THEM BACK. Fiddling with length of comments will do little. Take the new version of pSIG. Please.
Link to comment
Share on other sites

What I proposed was that a comment would no longer be required for any particular rating, since it seems that people intent on circumventing that requirement will do so. I think this change alone will get rid of a lot of the "Superbissimo!", etc, comments.

 

However, I would still add the requirement that if you do make a comment it should be more than a word or two. The "Wow", "Excellent", etc, comments may help with all the bonding that takes place and the photographer may like them, but it is kind of drag to read through the little love-fests that the commentaries on some photos become. I think I will change the wording so that the comments are described as "critiques", and people will be reminded in the "Add Critique" (currently known as the "Add Comment" form) that if they want to say "Wonderful, you did it again!" they can send that to the photographer via email, and that it doesn't need to be shared with the world. At the same time, it seems good to let photographers opt out of critiques and/or ratings when they submit a photo.

 

I have been thinking about the idea of making ratings more anonymous. Here are some thoughts:

 

1. If you are a subscriber, have rated 100 different photographers (note, not photographs) or more, and meet certain other tests (like not too many 7's, not too many ratings concentrated on one person, etc) you become eligible to make anonymous ratings. Anonymous ratings count double.

 

2. When he/she submits a photograph, a photographer can say how long he or she wishes to remain anonymous. The advantages of doing so are that (1) there will be special views of Top Photos for anonymously-submitted photos; (2) ratings on anonymous photos will count double; and (3) there will be a special anonymous-mode of the Photocritique UI which will present only anonymous photos.

 

3. An anonymous rating on an anonymous photo will count three times.

 

4. A photo submitted for anonymous rating must not have anything to identify the photographer (other than whatever distinctive style the photographer may have): no copyright statement, or special frame or border, etc. The photo submitted for anonymous rating will not be visible in the person's portfolio (folders, etc) during the time that it is anonymous. It will be presented only in TOp Photos and in the Photocritique UI, and the photographer will not be identified for the anonymous period A person who tries to cheat by identifying himself in the Technical Details or in some other way will become ineligible to submit anonymous photos.

 

3. A person won't be allowed to rate an anonymous photo, or to rate anonymously, if he has already rated more than 10 photos by that photographer.

 

These are just ideas, but I would be interested in comments. (By the way, I want to give credit to Dave Nance: he suggested double-blind rating a long time ago, and others have made suggestions like this periodically.)

Link to comment
Share on other sites

Renormalization works great in quantum physics (though it's not quite clear why!) but it's maybe not the best method to apply to the quality of images. The ratings don't lie on a bell curve and have no reason to. You can't assume people randomly rate images. They don't. People more often rate images they like than they don't like. Why waste your time rating crap? I suppose you can make an argument that if you have enough people ratings enough images over enough time - and that the general standard of photography is stagnant (people don't get any better at it) - then the overall distribution of the rating curve should have much the same shape and you could attempt to renormalize to a given (non-gaussion, non "bell shaped") curve.

 

Numerical rating of images is the problem. You can't do it accurately on a scale of 1-10, nor 1-7, nor 3-7, nor 1-100. Give people numbers and they think "precision". This image gets a 92.345 average, that image gets a 92.344 average, therefore this is a better image.

 

For subjective ratings you need to use subjective terms. You really need "Poor", "Mediocre", "Average", "Above Average", "Good" and "Excellent". For the purposes of internal ranking, assuming you feel the need to rank, you can assign numbers for the purposes of calculation, but the user interface which calls for a subjective judgement should be subjective. Images could be presented in terms of "Top 1%", without numbers.

 

 

People just don't stand in front of images in art galleries and say "Hmm, that's a 7.2". They say "I like that, it's good", or "Wow, that's excellent" or "Uck, that's aweful"

 

BTW when I say "when I give a 7", I mean when "insert name of reasonable person" gives a 7. I don't do much image rating because I don't personally put much stock in ratings. However I do accept that some people do care about it and if we have a system I care that it's a reasonable system that allows maximum freedom of choice with minimum openings for abuse.

 

However all of this would be a HUGE amount of work and I'm pretty sure is beyond the capacity of the photo.net programing staff (who's name is Brian!). Thus small tweaks to a flawed system are all we can do. Like I said before, Brian's tweaks may work, but dropping 1 and 2 ratings and requiring 25 word comments on 7s would work better.

 

Another "simple" change would be to adopt the olympic figure skating ratings system where you drop some percentage of high and low scores to try to eliminate bias.

Link to comment
Share on other sites

Here's an interesting quotation which I think is on-topic:

<p>

<em>

"When inequality is the general rule in society, the greatest inequalities attract no attention. When everything is more or less level, the slightest variation is noticed. Hence the more equal men are, the more insatiable will be their longing for equality" </em>

<p>

Alexis de Toqueville - Democracy in America

Link to comment
Share on other sites

I go into an art gallery. See a piece I dig. I buy it. See something that truly sucks

(in my opinion) and don't buy it. Maybe I comment to my girlfriend (or perhaps to

the gallery owner if I'm not thinking of one day having a show in this particular

gallery), "gee sweetie, this painting/ sculpture / photo really sucks because....".

 

I go into an art museum. A good one. Amon Carter. Getty. MOMA. This month it's

W. Eugene Smith. I dig it. Next month it's Bresson. Dig that too. Month after that

it's Picasso - groovin' there. The next exhibit is a bunch of tv's hanging from the

ceiling playing soap operas all day - don't dig it at all. BUT - lot's of people DO!

Whaddaya gonna do?

 

What if all these artworks had never been seen before? What if they were

annonymous? WOuld you dig a BRESSON or PICASSO if you had never seen a

PICASO or BRESSON and didn't know who they were? And even if YOU did NOT,

someone else may. In fact, maybe LOTS OF SOMEONE ELSE'S would like what you

do not.

 

Here's the thing, though... Each "exhibit" i see i'm going to like or not like or sorta

like according to what moves me, what turns me on when i see it, according what

I DIG THE MOST! The REASON famous artists of all persuasions are famous is

because there is a tremendous amount of AGREEMENT that their art is great.

Sometimes it takes until well after they die for this agreement to take place,

nevertheless - it is most definitely about agreement by those who know the

difference!

 

If you had never heard of Smith or Besson or Atget or Adams would you be able

to appreciate their work? Would you give them a 7 or just a 6? Maybe you'd give

Adam's MOONRISE a 7 but his BRIDAL VEIL FALLS a 6? Who knows? You might

even give some of his stuff a 2 for all I know. Just because you don't like

someone's work doesn't mean others will agree with your assesment / critique.

 

HERE at p.net, I see an image I think deserves a 7 I give a 7. At least that's how I

believe I should rate a picture that I recognize as being truly awesome from MY

OWN PERSPECTIVE, EXPERIENCE, KNOWLEDGE, UNDERSTANDING, PERSONAL

GROOVE INDICATOR, ETC... But see - that's just it - my experience and

understanding may be quite different than the next guy /girl. How ya gonna

FORCE people to understand stuff they haven't learned yet? Or even don't care

about at all?

 

If I see a picture that seems to be someone trying to be a good photographer,

but the image really kinda sucks, I'm not going to SLAM him/her. I'll critique it,

maybe even rate it a 4 or 3 or 5 depending on so many factors it's impossible to

explain in generalities. Gotta watch those generalities, don't we?

 

If I see 50 pictures worth a 7 I'll rate them a 7 because to each photographer

who made those images, my positive comments and ratings MEAN A LOT!

Whether they respect my opinion and or my work is irrelevant to me, though I

would hope that IF they look at my work, they will weigh what I've said about their

work with a little more sincerity. Scott might give the same pictures a 4 or 5.

Maybe he thinks there can only be 12 pictures a year on p.net worth a 7 from

him? Maybe he is waiting for the reincarnation of Stieglitz or Weston? I dunno.

Maybe Scott and others have been here so long they've run out of good pictures

to rate and comment on? Maybe some people feel it is their calling to TELL

OTHERPHOTOGS what they are doing WRONG? I dunno. But I'm pretty sure most

photogs who upload pics (not the obvious people selling junk or sharing class

photos or baby pics or selling watches etc...) want to know what others THINK

ABOUT THEIR WORK?

 

If the system is set up ( and most arguments here seem to say it is) to

COMPETE to some degree. TOP this and TOP that. HIGHEST this and HIGHEST

that. Then some people GIVE OUTpositive and constructive, well thought out,

educated opinions and comments and RATE accordingly, and because they've

chosen good pictures deserving of praise they are seen as kissing but and / or

playing games? Who's afraid of whom here? Others give out extremely critical

and negative comments EVERY SINGLE TIME they rate /comment. Their way of

being superior? EGO man, it's a big responsibity. It's tough growing up.

 

If you at least see that some people here are going to be competeing for highest

this and that, then why fight it? Like others have said above, no matter how

many ratings you allow, the lowest will always be viewed as the lowest. For those

who are so self diciplined and ego-LESS, "just comments" will satisfy them

forever, right?

 

Man, let face it, everyone who is even TRYING to be a good photographer wants

to know what others think of their work. And the beginner (who could be an 8

year old for all you know) might want his/ her picture of the family dog seen and

rated. Who knows what motivates some people to upload some of the goofy

stuff here? But if those images are mixed up with good stuff, and they are all

alowed their day in the sun on a TOP PAGE for some amount of time everyday

for like a week, when the ANNONYMOUS time is up, we all get to see both how our

own pictures have done as well as how others rated those images which we

chose.

 

The point is, there really are A LOT OF IMAGES ON PHOTO.NET worth the

absolutely highest rating available! Gems get through the system now though.

Games are played, reeking all sorts of havoc it seems. This can probably never

be stopped completely. So what?

 

Just give everyone a chance to have their pictures seen by everyone else for a

week with no I.D. - rate these pictures on MERIT. Doesn't matter who gives out

what ratings at this point because in the end, the general consensus will be

revealed. Ahh, the thrill of it! The sheer excitement! Just like in life - you never

know what's next. EVeryone loves a mystery.

 

.... And THEN - the pictures go into your normal system exactly however you

want them to. Maybe it would lead to more appreciation and respect by all for

the rest of p.net's services and efforts as well?

Link to comment
Share on other sites

Bob in answer to your question: <em>Tell me, have you ever seen an image that deserved a "1" and if you did, would you bother to take the time to rate it? Ditto for a "2". The only purpose for giving a "1" or a "2" ratings seems to be retaliation.</em> <p>I've actually given out about 34 2's and 21 1's. Not retaliation.
Link to comment
Share on other sites

About te origional proposal from Brian:

 

1) Remove comment requirement for 1,2 and 7? Maybe, the system is abused anyway, I have seen lots of photo that have "." as comment for a 7, and no comments for 1 or 2 (comment removed after the rate... I even got a comment that was "zzz").

 

2) That would probably just reduce the number of comments. True we can easily live without 1 word comments, but if the comment requirement for 1,2 and 7 rates is removed, these short comments will probably go away with it. If not, they are harmless, but I don't think that a lower limit in the number of words will make people think harder (they will just avoid to leave a comment at all, which is fine).

 

3) I see this difficult to implement with success. If somebody wants to cheat and give a 7 to his/her friends, would just have to go through the rating interface and give some random (or not well-thought rates) to the first 20 photos to get the right for another 7. Without a double blind system we will have always somebody to do some effort to cheat (people even creates multiple identities...). To implement this system will not prevent cheats, but will harm other people getting meaningless rates by somebody trying to reach the quorum for another 7.

 

The problem for me is that we are too much obsessed in the bell rate. Nature not always has a gaussian statistics. I am a physicist: if I try to apply gaussian statistics to all physical problems I have to solve, I would be bound to a professional disaster. Clearly the PN society shows a statistics that is not characterized by a bell curve, having a skewed average value. We shouldn't try to change the statistics of the physical phenomenon to please our wishes: a better approach is to understand what kind of statistical law the phenomenon is following, and leave with it. So I would say: no point in trying to force people to give 1 and 2 if they don't want to.

 

If I remember correctly, the main reason for reforming the rating system is that the average rates have changed with time, and old photos have disappeared from the top pages. If this is the main concern, just add some options in the top pages to show the top photos in given periods of time ("top 1999 photos", "top 2000 photos" etc...). These pages will have to be computed only once, after the year has ended, so can be served as static pages and will not be a load on the servers. Is it difficult to implement this feature (that I think was already proposed several times)?

 

Bob's idea of removing the numeric labels to the ratings is also good. You still use the numbers to do the average, but in the rating forms just write the bad, good, etc word. Then, instead of writing the numeric average just write the word associated to the numeric average. So 5.12 will be "good", 5.75 and 5.86 will be both "very good" and maybe we will be less obsessed with the ratings, if we don't see too many decimals. This should also be easy to do, as it only requires to change the templates for the rating interfaces (removing the number labels), and then to write a little script that converts back the numbers in words.

 

Finally, about the double-blind system. I like it a lot, even though it will mean quite a lot of work from Brian. It will also give more reasons for people to subscribe, which is good. I will be happy to submit photo anonymously if that gives me the chance to have them posted into a place where I have more chances of feedback, without having to complete (for visibility space) with the snapshots of the users that are not interested in critiques, but that only use PN as a web hosting place to show family photos to friends. If this system is implemented, though, an initial period will be necessary to allow people to adapt their rating habits to conform to the rule to become anonymous raters (I bet a lot of the people writing in this thread have given each other much more that 10 ratings ;-) I guess that what we want to avoid is not the maximum number of rates to allow from any photographer to each peer, but their distributions (e.g. no more that 20% of the rates to a single friend). If somebody has given 5,000 rates, who cares if 20 of them go to the same good and very productive photographer? But you cannot be an anonymous rater if you gave 20 out of 50 rates to the same friend. You get the idea, it's not the absolute number, but the percentage.

 

That said, I appreciate the effort that Brian is doing to improve the system, and I think we should all thank him for this. Maybe it would also be a good idea to post a link to this thread from the main page in the left column, so that more people will notice it and participate.

Link to comment
Share on other sites

Although I've been a contributor to the Forums for some years now, I only started uploading images to my portfolio a few weeks ago, my sole purpose being to have a small body of work that could be easily viewed by other contributors without them having to access my website. By doing so my images are now part of the ratings game and open to comments which is fair enough and I admit could even be fun.

 

However, if ratings are ever to be more than a bit of fun or a game, I feel they really need to be linked to meaningful comment or critique. Personally I would like to see any rating, no matter what the score, linked to at least a 25 word meaningful comment/critique. This would certainly make people think twice before rattling off ratings willy nilly and could only encourage them to examine the image and their feelings towards it more closely.

 

Personally I am against any anonymous ratings or comments on principle as this could only add to the problem of potential abuse, although I can see the possible advantages of the images being anonymous.

 

As I said before, I am new to the gallery and ratings, know nothing of the subtle nuances involved and apologise if the above is uninformed, naive or just plain boll*cks.

Link to comment
Share on other sites

Ratings are a game. You can't rate art in an objective way (unless you rate it based on what someone is prepared to pay for it!). You can comment on art of course in fact you can get a degree in commenting on art. There's no degree which prepares you to rate art objects on a scale of 1-10 (or 1-7). The idea of doing that is ludicrous.

 

Any meaningful system would require at least 25 word comments.

 

However people like numbers, Americans in particular like to rate things and see everything as a contest, whether it's how much money you make or who gets the highest photo.net rating.

 

If you junked the ratings and left only 25 word comments, the whole system would probably crumble and you'd go deaf from the volume of complaints.

 

The other very important thing to note is that the system we have now actually works. The "top rated images" pages actually seem to contain the "better" images posted to photo.net. Maybe they aren't in quite the order you would pick or I would pick, but most of what's there is worth looking at and there are very, very, few "dogs". The good stuff floats towards the "top" and the junk sinks towards the "bottom". I'm not quite sure what more you can expect then to generally move the better images "up the ladder".

Link to comment
Share on other sites

Perhaps if all ratings were linked to 25+ word comments it would deter those who merely enjoy the rating by numbers process and encourage only those with strong opinions on a particular image to rate. Trouble is it probably wouldn't do the hits to the photo.net website any favours.
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now



×
×
  • Create New...