Can a simple change clean up the rating system?

Folks:

 

A lot of the threads in this forum are from people unhappy with the ratings

system. How much of that could be fixed with a simple change, I'm wondering...

 

Currently, as I understand it, a person can register as "Captain Rato" (i.e.,

anonymously) and, starting immediately, rate 1000 photos with low or high

ratings as he or she sees fit.

 

Why do we want ratings from people who just registered? From anonymous users?

From people whose own quality of work we've never had a chance to judge?

 

Why not say that to be able to rate photos, a user must be

 

1) registered with a verified RealName (once we get the system up and running;

should work if we can send email to Joe.Smith@hp.com or some other well-known

company or if an existing RealName user can vouch for the person or if the

person has a credit card and can do a $1 transaction, etc.)

 

2) nominated by the existing group of qualified raters, presumably based on the

fact that someone among that group has seen the new user's photos and liked them

 

Forgive me if this idea has already been put forth and shouted down for one

reason or another.

 

Thanks in advance,

 

Philip

Phillip -

 

This probably has been suggested before. I stopped paying attention to the debate about the rating system a long time ago because no matter what people suggested it was thrown back as unworkable or somehow against the policy of the site. I think Brian went a long way to reducing mate-rating but the system itself encourages gaming. Some old-timers told me it was corrupt and corrupting. I think he was right. People are very concerned with the score their photograph gets, much more than if their photograph is good or not. Those two ideas get confused. I think your suggestion would improve the rating system. Maybe not perfect it but eliminate some of the abuse.

sorry Philip not Phillip

I had an idea similar to that a while back, a PNetter had to have say at least 5 images (an arbitrary number) posted and have ratings on these imgages before being able to rate others photographs.

 

I am neither here nor there on the first idea. I joined because I was a trial member and got to use all the functions of the site and liked it. I dont' know if many people would like it if they had to give away cc numbers online.

1) I like this idea, and I believe there is much support from current users.

2) How would the initial group of "qualified raters" be established? Assuming this system would not be anonamous, how will the "revenge" ratings be controlled?

<i>Can a simple change clean up the rating system?</i><br>

Yep, just bring the 1s and 2s back and only show average of ratings received (no more ratings breakdown page).<br>

Do not see the connection between ratings and real name.

I don't why one has to be nominated to have an opinion. That's what a rating is, ins't it; an opinion?

 

Here are some ideas.

 

Design the critique forum so the critic can choose what genres of photography appear. For instance, I want to start rating photos. I go into the recently submitted photos for ratings and a list appears with all the categories and I am able to check the ones that I would like to see and rate and the ones I do not check won't appear. For those who want to rate all photos, there should be a "check all" option.

 

Make the comments field more visible. This might encourage more comments and written critiques. Also allow Critique Only photos to appear in the queue. Just disable the ratings function on that particular photo.

 

The ratings system isn't perfect, but the issue isn't with the ratings system. The issue lies with a hand full of people who abuse the ratings system by mate rating, low balling, and cheating. You will never create a perfect ratings system. People will always find loop holes to cheat it.

 

I propose that you focus on ideas that promote more dialogue than just raw numbers.

Philip, it is quite refreshing to hear admin being pro-active with this subject. This is quite a different approach from the previous regime where Kelvin Ball, (anonymous raters & rate recent,) with all due respect to Brian, was created to minimize abuse@photo.net work load on Jeremy as well as eliminating mate rating activities.

 

Although both of your points had been raised numerous times in the past in site feedback forum without further action, I would certainly prefer having real members rate per your point #1. As much as I like the idea of point #2, there definitely will be questions about what constitutes a qualified rater and the process involved.

 

I don't have a real answer here, but again just want to say that I appeciate the fact that you're asking feedback from members. Simple change? Sometimes the simpler, the better.

Phillip, Will King hit the nail on the head for many of us who are fed up with the ratings when he wrote, "The issue lies with a hand full of people who abuse the ratings system by mate rating, low balling, and cheating. You will never create a perfect ratings system. People will always find loop holes to cheat it."

 

The fact that there are folks who would corrupt the site in the past with bots and folks emailing each other to purposely down rate others is simply amazing. I base this on a previous post where this had been disclosed. (No I don't have the link to this.)

 

The common request I've seen when discussing the rating system is a requirement to have the rater be identified and an explaination with why he/she rates high or low. Some will argue why it won't work and rating battles would occure.

 

Sorry if my frustration is showing. Previously when such discussions were made posters were dismissed as whining malcontents. So I've tried to stay away from this topic.

 

Gmww

'I think Brian went a long way to reducing mate-rating but the system itself encourages gaming'... Well put, Kent.

 

Philip, I think your ideas are great and will definitely partially clean up the current rating system. I would like to also suggest that a registered member can only give limited number of ratings 'per category' 'daily' to minimize, not resolve though, the impact by raters 'who abuse the ratings system by mate rating, low balling, and cheating'.

 

My another suggestion is about the TRP. Since the deafult TRP (3 days Rate Recent Average) is based on the rating system so I think it's somehow related to this thread. I would like to suggest the TRP shows only the top rating image for each category and people need to drill down to see all top rated photos in that category. This will give all genres the equal opportunity to be shown on the TRP without having the page dominiated by certain category.

When submitting a photograph to be rated it is a forgone conclusion that there weill be some rates too high and some too low ... with that in mind I can still get a general feel of how others felt by looking at the overall average. I don't need Pnet to average it for me. Really the point is for me do I like the photograph and maybe someone else will like it too. Philip leave it as is.

I wouldn't be living up to my reputation if I wasn't an early responder to this thread. I appreciate your addressing this issue by asking for input.

 

1) I expect you would reduce the number of raters per image by a substantial margin - maybe half. That's not necessarily bad if we assume that most people who reigster properly might also take the task of rating somewhat more seriously, but that's just conjecture. (At this point, consider why anyone would want to rate images at all, even if the system was flawless.)

 

2) Having had some experience "judging" photographs similar to many of the kinds of images uploaded here daily, I can tell you that being qualified to rate them in any useful quantity is difficult because you can't properly assess the value of an image unless you appreciate the genre and have some idea of the technical skills involved in its capture and processing. This assumes that you're comparing an image to other similars. (I think the POW discussions support this belief.)

 

It has been postulated that in order for a rating average to have any statistical validity, you need a critical number of raters - fifteen was suggested as statistically significant in this context. My view is that one or two rates from just a couple people who actually know what you're trying to do and have done it themselves successfully is much more valuable than a half dozen numbers from perfectly nice people who just bought a camera.

 

But this assumes you need numbers. What are you going to do with them, and isn't there a better way to accomplish that goal without all the acrimony the has been part and parcel of the system from day one.

 

I'm sure by now you've noticed how many of the less than stellar images on the all/all TRP sort were posted by people who vote each other high and often. Many - but clearly not all - are the last people I'd ask to select or be selected as experts.

 

Judging by the comments I've read, there are a lot of people here who do know how to evaluate an image, but have been so turned off by the ratings system that they won't go near the gallery, much less the rating system. Setting up something that would attract them would go a long way towards improving the quality of both images and discussions. I would place a higher priority on attracting a broader range of images and critics than some sort of ordering of the images we see now.

 

There is a lot of evidence that supports the notion that giving and getting rates is a fun activity that has served as an introduction to this site and has attracted paying members. This subject can not be addressed seriously unless we recognize that the system has been used to do too many things and, as is usually the case, does none of them very well.

 

If you plan on fine tuning this sytem there are a lot more details to share, but I'll save them for now, hoping they won't apply to the substantial changes that I hope you're considering.

Open accounts at other websites and solve the technical and behavioral issues too.

"Do we want ratings from anonymous users?" No! In the same way we don't want anonymous users, we should not want anonymous ratings of any kind-let's stop these gutless wonders who love dishing out light speed 3/3's and who hide behind a mask of anonymity. If you want to leave a low rate and don't want to comment, fine but be identifiable.

I don't think it necessary to be 'screened' by members to join but at the same time, you should not be able to rate other photographs if you don't post your own-it should work both ways. cb

Lots of excellent comments here so I'll keep it really brief. I think the proposal is sound as

long as the approval process is quick for new raters. It should be as easy and inclusive as

possible.

 

And, as someone mentioned above, it's important to bring back the 1s and 2s or reorder the

scale from 1 - 5. The dead ratings make the scale meaningless in its current form. I also kind

of like the idea of showing only average rating, no the actual ratings themselves but I'm not

married to the notion.

Also, I think Charles point above is right on the money. If you don't post your own photos

you shouldn't be able to rate.

Philip, i have been a long time critiqic of the rating system, but i would welcome your idea. I think it is well worth trying. Thanks for listening to the members.

Sincerly Joe B.

A couple of comments. First off, the primary rating complaint is that ratings are too low- you even see it up above where people complain of the 3/3'ers. Attaching a real name to that 3/3 isn't somehow going to make anyone like it any more than they do now. It will however give the complainers a way to go low-rate the other persons work just to show them.

 

Someone up there said the rater should "have some idea of the technical skills involved in its capture and processing." I disagree with this. Suppose I take my 110 camera, shoot a shot, develop in my own home-brew developer, blow it up to 4' square, handcolor it with crayons, scan it, spend a few hours touching up digitally, etc. Meanwhile, you walk outside with your point-n-shoot digital, accidentally push the button, and get exactly the same photo. Is one of those shots to be rated higher because more work (and/ or skill) was put into it? I think not. The significance is when people are using 8x10 view cameras, doing exotic printing, etc., they expect the finished shot should be rated higher because of the process, even if the quality can't be seen in a 600-pixel wide jpeg.

 

It won't bother me personally how the ratings system works, but a lot of the changes made through the years were trying to resolve complaints of one kind or another, and those complaints are unlikely to go away.

"they expect the finished shot should be rated higher because of the process, even if the quality can't be seen in a 600-pixel wide jpeg."

 

I'm not talking about your ability to produce a web shot which will be seen on a thousand different monitors.

 

I'm saying that each genre requires a different skill set, which probably isn't obvious to people who haven't actually tried it.

 

I do agree with your comment about low rate whiners. Many posters here don't remember what the system was like before anonymous rating. If an average photographer gets a 3/3 from Joe Blow, the next time he sees Joe's image (or seeks it out intentionally!), we are meant to believe he'll be objective in his analysis. Not likely!

 

It's true that the above scenario rarely happened. Why? Because people learned about revenge rating very quickly and stopped rating inferior images, especially if they have a decent portfolio and could actually be of some help if they chose to include a comment.

 

Now do you all understand? You have to separate uploading and comments . . . and figure out a way to identify the better images without using numbers.

I don't care for the rating system, have only twice used it, and I'm sorry I did that. If I think someone is doing interesting work and I'd like them to know I'd just as soon write them and say it. My problem with the anonymous is that sounds like duplicitous. I have pulled my personal work from the site not because I care what people think of it but because I'm concerned about the changes that are afoot regarding copyright protection in the US. I don't mean to be ill mannered, but I'm really not very interested in what most people think about my work. Whether its the work for hire or my personal work. I guess I don't understand. I mean, what do you get out of being rated? There really aren't that many people out there who are actually qualified to even begin to discuss what makes one photograph better than another. Sure everyone has an opinion, however some are informed, others clearly aren't. What service does it do a young or beginning photographer to be rated by an idiot? I don't see it. Hayes

I suggest allowing all members to rate providing only an average after applying a simple (unspecified) algorithm to weight those in category 1 (real name) higher and those in category 2 (an experienced select group) even higher.

 

No detailed breakdowns - just an intelligently weighted average.

Setting aside your concerns about copyright for the moment, I would say that the goal here should be to infuse this photocritique forum with sufficient integrity so that given your credentials, you will consider participating in it in areas where you're interested and have some experience. There ARE loads of qualified people here; you just have to make it worth their while.

Don't you think the number of people in category #2 are so few as to have little effect on the input from category #1?

Carl,

not if weight for group 2 was higher than the ratio of group 1 to group 2 (just counting those actually rating an image).

What is the difference between the current members who use nicks or initials etc.

 

If they had just given a number what would you think. For example a comment from

"258746" etc. Totally stupid sounding isn't it. Using quasi names is NOT appropriate for

the type of community I believe you wish to have here.

 

Requiring a name plus postings plus time at the site plus a 'certain level' of ratings from

others are all ways of trying to prequalify who can rate.

 

Philip you are to be applauded if you will try out a few simple but BIG changes over a

period of time. It would be great to have the atitude of "LETS TRY IT" instead of listening to

all the people who do nothing because they 'think' it won'y work. In the end no dies from

getting a bad rate.

 

Feel free to change the system it currently has no real value to anyone here.