Can a simple change clean up the rating system?

It is unfortunate that this has become so complicated. I would like to repeat my previous suggestion; no anonymous members, no anonymous ratings, and no rates/comments from anyone who doesn't enter their pictures for rates/comments. If you only submit for critique, you only get to critique. The system can track the number of 7's a member gives and it prevents giving out a 7 to someone who has left that member a 7 within a certain time frame (24 hours?) so it can certainly track the number of 3's someone gives out; perhaps there could be a maximum number of 3's a member could give within 24 hours.

Last comment; I really would not like to see a panel of 'qualifiers' no matter how good they are; it is not needed and -to me anyway-insulting. My 2 cents (Canadian) worth-that's about 1.8 cents U.S. cb :-)

Oh for crying out loud!

 

He asked questions that suggest that he hasn't been following all the nuances of the ratings system daily for the last five years (like some of us have.) Read the whole proposal again and take him at his word.

 

SP, your request for a personal favorites gallery has been in effect for quite some time. Where have you been?

"Carl Root , sep 15, 2006; 04:47 p.m.

Some of these posts seem a bit paranoid, but much as it may look that way, Cyrus K's is not." <BR>

Thank you Carl for understanding. It was the first ever acknowledgement of this problem by anyone! But, no one has to believe me, the photo is still there and so are the records. The photo is under my previous account.<BR>

Yes, Ben, they go even further than that. Just look at the "top photographers" list and look at who SOME of those guys are and look at their rating habit and you WILL soon figure out how they got to be on that list and so close to the top. Just as that list is not a good representation of great photographers on this site, so is not some of the photos we see pop up top on the TRP. I have a clear understanding of how that happens too and you may ask me how!<BR>

Now, please understand. I am not anxious as why I am not on that list. As a matter of fact, I don't deserve to be anywhere near that list. I do however, care about this site and I do want it to work for everyone. I am a paid member and I have also paid for several others on this site to become paid members, the records should be there. But, it is discouraging to see that a good site like this can be hijacked by a group that is motivated by their own personal agenda.

<I>SP, your request for a personal favorites gallery has been in effect for quite some time. Where have you been?</I>

<P>

OK, help me out, where is it?

if you rate photos, you'll get a link on your community member page to 'photo srated highest by this member' which includes photos you've given 6/6 or above.

Cyrus K,

 

"It was the first ever acknowledgement of this problem by anyone!"

 

Well, exactly the same thing happened to me. Makes it 2. :-)) I believe a lot more people probably experienced the same thing. Over a couple of years I've seen so many "funny things" anyway, that I hardly take note anymore, and I surely wasn't used to having anyone listening either: you'll have to accept that too, I guess...:-) This thread seems like the first time since 2002 or such, that someone at the top actually finds it worthwhile to look down there what's happening in the gallery...

The anonymous ratings system is pnet's heart of darkness. It is the obscure, inarticulate man

in the crowd expressing his resentment at the literate class he feels has failed him. Give him

his voice. If you do not, he will nonetheless think his thoughts, and

you

will have merely deprived yourself of the only means of knowing what they are.

The anonymous rates are allowed (and preferred in the trp) in the current system because it doesn't allow a return of the favour to the rater. If I rate someone high on a photo, it is likely that the person receiving the rate will rate a photo of mine high. Thus leading to an escalation of the average rate. Notice how the nominal average 4 is far from being the true average of the ratings, although it is intended to be.

 

What I have proposed before is a normalization of the ratings by each rater. This way the mean and standard deviation of the rates by each person rating (this assumes the rater must have logged on) is reset to 4 (mean) and say 1 or 0.7 or something like that (for std) after each ratings change. This would enforce the validity of the attached meaning of the ratings (ie. 4 is truly the average). Some people seem to object to this because they will only rate pictures that are good. However, I think on average this would give a more meaningful picture of how the pictures are valued by the raters than letting them rate only 7s and 6s.

<I>if you rate photos, you'll get a link on your community member page to 'photo srated highest by this member' which includes photos you've given 6/6 or above.</I>

<P>

Yes, you are correct, but the problem I have with that it that under the current system I'm prohibited from only providing high ratings (which is exactly the images I would want to show other people). So in order to show you my favorites, I have to also rate some images low, which is not what I am interested in doing.

<P>

Think about it, the ability to show you my favorites comes at the expense of someone else? How is that fair?

That's not true. You can rate anything you want any way you want.

 

That's one reason rates never have and never should be "normalized."

 

A case can be made that the only "rates" that the site needs are the ones you use to put in your favorites pages. . . except for the problem of reciprocity which is almost impossible to avoid when you first sign up and start rating. Then you wake up one morning and realize that some people are going to be upset if you don't rate their next upload a 6/6 no matter what they offer. It doesn't matter if both photographers upload mostly "good" images. The resulting data amd image order is unavoidably based on site politics and ethics.

 

This stuff is far from intuitive, which is why people who haven't rated and uploaded plausible images extensively should sit down, be quiet, and read posts thoroughly,

 

. . . and above all, consider the source.

When you allow people to opperate with out having a true Identity its just a lot of nonsense and a few other words that I will not use . Its a very simple solution . Real names , real rates , all attached to the real owner . If some one wants to be an _____ _____ , at least we will all know . They will do them selves in . Yes its really that simple , it may not work perfectly , but why make it easy for the cowards !

Another thought , if anyone is concerned about other people getting 7/7 and you are not getting yours , well may I suggest taking some photos that will get those 7/7 s . You are in the right place to learn ! Robert :)

<I>That's not true. You can rate anything you want any way you want.</I>

<P>

Well if that is truly the case then I wasn't aware of it. I believe that I read comments from people wondering why they couldn't post a 7/7 when they wanted to. I'll spend some time searching to see if I can dig it up.

<P>

<I>This stuff is far from intuitive, which is why people who haven't rated and uploaded plausible images extensively should sit down, be quiet, and read posts thoroughly</I>

<P>

Was this comment directed towards me? If so I find your paternal tone a bit condescending, after all I've been here few a few years, following right along with you.

The current ratings system should be abolished, not modified or fixed. The whole idea of assigning a numerical score to a photograph is silly, even insulting. The system encourages the ignorant and unconstructive attitude that photography is a kind of competitive sport.

 

Still, the site needs a way of picking strong images for display to members and visitors. Some sort of rating system would seem an efficient way to do that.

 

So consider this: All ratings should be INVISIBLE to everyone except the site editors and managers. Moreover, only those who have passed some type of qualifying exam should be permitted to rate. Possibly these judges should be assigned to rate only certain genres of pictures -- specialists in nature photography probably shouldn't be rating street photography and vice versa.

 

It'd probably also be easier to have the ratings range from 1 to 5 instead of 1 to 7.

 

To select a set of good pictures for prominent display, it's not necessary that the ratings be shown to the photographer or to viewers. Ratings are meaningless and probably detrimental to photographers attempting to improve their skills, so the site shouldn't display them at all.

 

The (invisible) ratings should be explicitly treated as only a preliminary criterion for selecting photos for the TRP. The highest rated photos should be taken as a pool from which a panel of editors select which images to feature. The editors should be free to move an outstanding image from far back in the list to a more prominent position or to move a overrated photo to a less prominent position. This may sound unfair, but in the system I'm suggesting, the TRP is not a competition and fairness simply isn't an issue.

 

The name of the page on which the featured images are shown should be changed from "TRP" to something else, perhaps "Images of the week." The page should be set up to randomly mix the top-rated photos in different genres of photography, so that it is not dominated by landscapes, nudes, children, or any other genre. All members should understand that there is an element of randomness involved in landing in a prominent position, and thus they should have nothing to complain about when they land there sometimes but not other times.

 

If there were enough volunteer editors, they could take turns writing brief explanations of why particular images were selected. These comments would be helpful to learners.

 

The system I describe would be slower and more labor-intensive than the present system. But the present system satisfies no one, and there really is no need for the featured photos or TRP to be updated daily. Once a week is enough.

 

The key point is to get members to stop obsessing over ratings, thinking of them as a competition, or treating them as a source of meaningful feedback. Making the ratings invisible would instantly solve all of these problems.

An alternative to preventing reciprocal 7/7 rates could be to apply a punitive TRP ban for 7 days instead.

 

Perhaps people should not be prevented from rating any way they see fit, rather they should be prepared to suffer the consequences.

I agree with some of the proposals you are making. How can someone who is UNFAMILIAR with let's say Macro photography and the challenges it presents , get to put a number on a close Macro and complain about "not enough DOF"? <BR>

I am not so excited by numbers but I know a lot of members are. Even most of the ones who complain about not receiving comments, they really look for numbers<BR>

I'm just thinking out loud and everyone can help formulate this:

If you WERE to go with an appointed panel, What if:<BR>

1-Everyone can comment and rate<BR>

2-the panel will see some good critiques and fair ratings and they approve and check the names of those members.and the member get's a qualifying star next to his/her name. If you accumulate enough stars, you join the ever growing panel and your ratings will count towards photos going to the TRP, photos of the day/week or whatever you may want to call it.<BR>

This method may not be perfect but would give members an incentive to become better critique and join the panel. I think this may work better than a permanent group of people deciding all the time.<BR>

3- The name of the photographer should be anonymous at the time of rating and so should be the ratings of the panel but not necessarily the names of the members of the panel.

What if Ansel Adams were still alive, was a recent registrant and just starting handing out 3/3 ratings left and right?

Ray: with all due respect. The advice and suggestions given above are typical of a slew of folks who always pop up after a management change with their ideas of "how to set things straight" Any advice to the opposite is often regarded as negative and unconstructive, when in reality it is only checks and balances to the other remarks which have already been made. You have a tough task as a moderator. Participants in feedback forums who make remarks counter to the thread have an even tougher time. What might sound negative actually might be the very thing that you need to pay heed to. Don't be a Rumsfeld this early in the game.

My basic viewpoint on the rating system in contribution to the stuff above. The ratings right now appear to be a statistical bell curve. Once dominant images start appearing in the peak of the bell curve, other images and genres are less dominant. When folks see what's dominant, its only too easy to start posting similar images to enhance your chances of appearing in, or close to the top of the bell curve. That, essentially defeats the whole system, and is certainly NOT what I assume both Phil G and Brian M had in Mind. Yet, that is the state of the system right now. Instead of diversity and items of interest, we see monotone repetitiveness and little discussion outside of sycophantic praise. Some of (the)convoluted practices proposed above are only schemes to enhance the viewpoints/biases of those parties, and would only make things even worse than they are now. I don't see how that will do photo.net any good, to replace one system that is fixable with some kind of convoluted idea similar to Medicare. What is basically needed is a good tuneup of the sorting engine to prevent manipulation of the end result. Every party who posts an image SHOULD have the same basic opportunity for critique and review. It should NOT be the sole domain of any one specific category. It should NOT be the domain of a couple of photographers who can magically pop themselves into the TRP regardless of what they submit. The TRP has to be tolerant of diversity, which it is NOT right now. Too many of the parties above have expressed their intolerance for certain genres because of their personal inhibitions and biases. That for me, totally disqualifies them as having the right to input to the changes in the rating system.

I think that a lot of folks who are new are here to enjoy photography. They don't want to hear about the latest 3/3 paranoia attack, and how to go after the dreadful revenge raters. Enjoy photography. I still do after decades of it, as if its my first day with my preWWII Voigtlander. Let Photo.net worry about those things that they can control and fix. And let's make it more attractive to newer photographers young or old, by fixing the system so we don't have to look it images in reverse order to find something interesting to look at. All of us should realize that our personal photographic preferences play no role once we submit an image to photo.net. So If my images fall flat, c'est la vie. I can always take them off.

Many valid points have been raised and discussed above. There have also been a slew of off topic unconstructive posts which I have grown tired of deleting (sometimes from the same individuals over and over again). I set this thread to expire in 90 days to allow time for a complete analysis by site programmers. Again please refrain from any additional posts - unless you have carefully read all of the comments above and strongly feel you have a constructive idea that wasn't mentioned.

The rating thing is highly emotive but we shouldn't take it too seriously.

I rarely rate now and rarely bother with the TRP, prefering to browse through

the Critique Forum. Just how silly is the rating system? Well, <a href="http://www.photo.net/photodb/photo?topic_id=1481&msg_id=00I5AO&photo_id=4950317&photo_sel_index=0">this

photo</a>, posted today, is revealed as appreciably inferior to <a href="http://www.photo.net/photo/3861170">this

one</a>, posted in November. Occasionally I have re-posted images and the discrepancy

between rates received has been pronounced. <a href="http://www.photo.net/photo/4153012">This

particular photo</a> in my folio gained massively 2nd time round. Two serial

raters who first time round thought it worthy of 5/5 gave exactly the same image

6/6 on its 2nd appearance. A recent nude scraped together a dozen votes averaging

4.68. A re-post 2 weeks later sees it with 72 rates with an average just above

6. So which value should I take as the real? The 1st? The 2nd? The existence

of such a discrepancy dictates that I should accept neither.Probably the best

photo I've taken recently received one of the lowest marks I have in my folio.

Where do I go with this rating? I don't. In the end I go instead with my instinct.</p>

<p>I do a lot of work in one of the less popular genres (child) and although I

hope my work might cross genre boundaries and offer more than just a pic of

a cute kid, I appreciate that others might not think so. Carl is totally correct

to point out that rating in genres we have no interest and knowledge in is pointless.

I know nothing about insect, automobile and macro photos. Well, I know they

don't interest me. Probably if I took more time I could come to appreciate the

nuances involved but frankly I'm not going to. Equally I am not going to rate

these photos either. How could I? What do I know about it? I wouldn't ask a

plumber what he thinks of the carpenter's table, even though their toolkits

look similar. </p>

<p>Until the interface was changed and there was a lot of commotion about the

updating, I wasn't even aware of the existence of the RR system. I think I looked

at it once, decided it was a poor way to go about appraising a fellow artists

work, and forgot about it. I only ever rated through the critique forum. A radical

solution to the RR problems would be to simply abolish it. If a TRP and anonymity

are required make the rates from the Critique Forum anonymous and use that venue

to determine the TRP. Dwell on the idea for a minute of two. Think it through.

It makes sense. I could expound if invited.</p>

<p>Meanwhile, when the RR interface was changed the facility to easy locate photographs

submitted as Critique Only was removed. See attached picture for how it used

to be. I read elsewhere that Philip believes 'ratings are not core to the mission

of the site, which is peer-to-peer education.' A restoration of the Critique

Only facility would go part way to achieving that aim of P2P education. Photos

submitted for 'critique only' are now invisible. The newbie or the shy photographer

who would rather not undergo trial by rating has no other option if they wish

others to see their work. P2P education through useful critiques is worth a

1000 rates. If Philip is serious in his intent, an encouragment to critique

could quite easily and quickly be added to system. Provide a link to the 'Critique

Only' photos from the drop down 'gallery' menu.</p>

<p>Ray, I hope I have stayed on-topic and offered something which hadn't been

said before. Consider leaving my contribution. It took a while to write and

is meant to be of help.<div></div>

Instant P2P education<div></div>

This discussion has many great comments and some less so. First off, ratings are okay if you take them for what they are. A popularity contest. One fellow suggested hanging your pictures up and if they sold, then you were good (or something to that effect). Wrong! Rating by the masses is what gives us the photographic equivalent of a Thomas Kinkade painting. Perhaps there is no way to avoid that, but at the very least there must be a way to see who is handing you the rating, and a way to see what work they have produced. I often go to the site of a rater (or a commenter, for that matter) to see how I feel about their work. I do not make an effort to retaliate. It will cause far more problems than it will solve to attempt that.

 

Under no circumstances should a system be initiated in which raters are limited to a certain number of votes or inhibited in any way from speaking or voting freely. I have gravitated toward not rating stuff that bores my silly, even though I still try to be fair within the framework that PN sets out in the guidelines (how many of you have read them by the way?)

 

Most raters seem to be unable to differentiate between the two scores. I don?t have an idea for solving that.

 

You cannot draw the same conclusion from raters? averages. Suppose I only rate photos that I can tell I am very drawn to and which I can easily see are excellent photos and I pass on the ones that I would rate low. Then you would put me down as a high rater and unworthy of credence. Idiotic! Suppose I choose to rate a lot of really bad photos thinking I could help the poor blokes? Then I will be accused of rating too low, when, in fact, it was just a good reflection of what I was rating.

 

I may be the Lone Ranger here, but I never go to TRP. It?s just a cheerleader?s dream. I have no doubt there have been some excellent images, but the real gems are the guys that hang out on the fringes, and they don?t get TRP. I follow one good thread to another, and IMHO there is far more originality (which I value over popularity) in many, many of the lesser known PN members than one will ever find in TRP.

The corporate line used to be... "Comments left were for the photographer and Ratings were

for the site." In that the rates given helped the site distinguish between the everyday

snapshot and more polished work. Whamo the site can publish a TRP.

 

Make all rates anonymous and only visible as an average.