A modest suggestion regarding anonymous ratings

[gdanmitchell]

<p>I think I understand the desire to have allow anonymous ratings at PN. I think I understand the

frustration about bizarre ratings from anonymous raters.

 

<p>There are ways to alter the process so that anonymous raters could keep their actual identities

secret

while giving those rated some indication of the meaning of those ratings.

 

<p>Those who prefer to not link their identities to personal pages and information about themselves

could be automatically assigned anonymous "names" along the lines of <i>anonymous00111</i>, and

these "anonymous identities" would be indicated in the photo ratings.

 

<p>No one would know who <i>anonymous 00111</i> actually is, but those who care would be able

to

determine information about anonymous ratings including:

 

<ul>

<li>how many ratings <i>anonymous00111</i> has made.</li>

<li>whether <i>anonymous 00111</i> had previously rated ones work and, if so, how.</li>

<li><i>anonymous00111's</i> rating history: average rating, etc.

</ul>

 

<p>This would go a long way toward answering the questions folks have from time to time about those

add low-ball ratings. (And those who don't care what ratings mean could simply ignore this feature.)

 

<p>A reply to this proposal that says, "Get a life and don't worry about ratings" won't be very helpful.

This isn't a proposal for those who feel that way. Ratings exist and many PN community members try to

understand what they do and don't mean. If anything instituting an approach like the one I propose

could help people determine what value ratings do or don't have. (It is a case of "Mothing to lose, but

possibly something to gain.") In addition, it isn't likely to require major changes to the site since the

data a likely already being collected.

 

<p>Thanks,

 

<p>Dan

[brucecyr]

Providing a rater's average rating would essentially "weight" the value of his/her rating. Some folks tend to give high ratings, some low, so if both rated the same image it would tend to be a rather average rating all other factors being equal. But if both raters give high or low marks relative to their average rating tendency, then the "ratee" could get a better grasp of the true nature of the ratings.

 

OTOH, it sounds like a good bit of extra work and cost -- would it really be worth it for all parties? After all, what proportion of the site's users use the rating system -- either as raters or ratees? If a small minority, then the extra costs would benefit a few at the expense of the majority. Assuming the resources would have to come out of existing operating revenue, maybe the parties who want these statistics should be willing to pay a bit to get them.

[gdanmitchell]

Bruce:

 

I suspect that the data needed to do this is already captured by whatever system runs PN.

There certainly would need to be some customization work to format and present it. Given

that the ratings issues continue to pop up it might be worth it though.

 

Your observation that this would help weight the ratings is right on. For those who are

interested in ratings, knowing the rater's history (but not identity) would help clarify what

a rating might mean. It would be more information for those who find it useful, and those

who aren't interested could still ignore it.

 

Dan

[john_h.1]

The reference to the 'odd' high or low rating itself reveals there is no need for adjustments. Time and time again people object to certain low ratings with inferences that these mysterious low ratings are invalid or require extra examination. Of course this is not required for 'normal' ratings as it is the 'odd' rating that was cited to suggest additional programming. But for the reference to the ratings that do not comply with expectations, there would be no suggestions to adjust anything.

 

Everyone who has raised this issue knows which ratings they consider to be devient. Merely ignoring these non-conforming ratings and considering the remaining ones solves the supposed problem. Its that simple.

 

Don't worry about the ratings. You already know which ones you consider to be odd, invalid, retaliatory, mate based, ect, ect.

[gdanmitchell]

<p>Hi John:

 

<p>There is no "request for adjustments," just a proposal to share a bit more useful

information about the ratings that would clarify the meaning of said ratings. While there

are those like you who "dont worry about the ratings," we also know that quite a few

others are interested in them and what they mean. (If nothing else, they <i>do</i>

determine the placement of ones work at various places at the PN site

 

<p>Having a number rating by itself is only so useful. Knowing a bit more about what the

number represents increases its value. For example, if I see a "4" rating it makes a

difference if it came from someone who on average awards "3" ratings or someone who on

average awards "5" ratings. It also helps me determine what value to place on a rating if I

know that it came from someone who has rated only 10 images vs. someone who has

rated 1000.

 

<p>The inclusion of additional rating info for anonymous raters (a randomly generated

but consistent ID, average rating, number of ratings, etc.) would be regarded as useful by

many. I concede that some would not find it useful, but they would be free to disregard

these data. Nothing lost, and possibly something gained.

 

<p>(In a sense, having these data available could even provide some evidence - or not -

to those who variously believe that a) ratings mean nothing, b) there are some anonymous

raters who assign punitive low ratings (or not), c) you name it... )

 

<p>Take care,

 

<p>Dan

[Dieter Schaefer]

Dan, your proposal is a good one IMHO. Problem to me lies simply in trying to make sense of the numerical ratings to begin with. Time and again we read responses like the one you correctly label as unhelpful ("Get a life and don't worry about ratings"); rating is a major feature on this site. Equally often, when questionable low ratings are brought up or a request is made to accompany them with a comment, responses inevitably shift focus onto the questionable high ratings. PN has so many restrictions on the high rates in place that the rating system is already too skewed to make sense at all. The best you can make out of a 3/3 is that the person didn't like the photo, for whatever (unknown) reason. A 7/7 to me doesn't require a comment as much as a 3/3 does although in the spirit of "education" it would be good if both had one to let you know what is bad with one and excellent with the other. And let's not even go there that there are as many 3/3 given for all the wrong reasons as there are 7/7 distributed (point (b) in your response above). <br><br> It took me a while to realize that having the critique of an image reduced to two rather useless numbers by complete strangers isn't very helpful at all (comments are, and I learned a lot through them) and trying to make sense of them can drive you crazy. Unfortunately, on this site playing "rating roulette" is almost the only way to get exposure for your images (critique only is a joke, the images vanish from view way too fast and the "critique only" forum doesn't even display thumbnails, if you are able to find it at all). When I realized this I stopped submitting photos for rating and stopped rating myself (except for the occasional 7/7 I leave for what I consider exceptional images). <br><br> In summary, I agree that your suggestion would make it easier to evaluate a rating and to convince yourself that a low one doesn't mean much because it comes from a habitual 3/3 speed rater; and I agree with your assessment of "nothing lost, possibly something gained".

[bobatkins]

There are no truely anonymous ratings. System administrators can see exactly who gives what rating to who and there are periodic checks to look for and correct anomalous rating patterns (e.g. "rating bots").

[gdanmitchell]

I'm glad to hear the the admins check for anomalous ratings and bots. I have always

assumed that the "new members" indication in the ratings explanations were related to

this.

 

My proposal, however, is not really about the bots and related "ratings spam" issues that

are the concern of the admins. It is more about providing photo.net users with different

and useful information about the ratings while protecting the identity of those who post

so-called anonymous ratings.

 

Rather than trying to prevent outlier 7 or 3 ratings, this would simply be one way to help

PN members make more sense of them. And that is a good thing. (Or a neutral thing for

those who aren't concerned with what ratings mean.)

 

Take care,

 

Dan

[vancouverphotographer]

Doesn't seem like the ratings system can ever really satisfy everyone. Does the 3/3 rating for some mean that they actually mean a 1/1 or 2/2 but they give it a 3/3 because the lower ratings don't count? Was the 3/3 arbitrary because they trying to decide between a 3/3 or a 4/4.

[gdanmitchell]

I certainly agree that the rating system will never be perfect, but that isn't an argument

against trying to make it better - and that's all that my proposal would do. It would tend to

lessen some of the uncertainty, though it cannot completely eliminate it.

 

(Sort of like good glass can lessen distortion, but it can never eliminate it. :-)

 

Dan