Can a simple change clean up the rating system?

Discussion in 'Photo.net Site Help' started by philg, Sep 14, 2006.

  1. Folks:

    A lot of the threads in this forum are from people unhappy with the ratings
    system. How much of that could be fixed with a simple change, I'm wondering...

    Currently, as I understand it, a person can register as "Captain Rato" (i.e.,
    anonymously) and, starting immediately, rate 1000 photos with low or high
    ratings as he or she sees fit.

    Why do we want ratings from people who just registered? From anonymous users?
    From people whose own quality of work we've never had a chance to judge?

    Why not say that to be able to rate photos, a user must be

    1) registered with a verified RealName (once we get the system up and running;
    should work if we can send email to Joe.Smith@hp.com or some other well-known
    company or if an existing RealName user can vouch for the person or if the
    person has a credit card and can do a $1 transaction, etc.)

    2) nominated by the existing group of qualified raters, presumably based on the
    fact that someone among that group has seen the new user's photos and liked them

    Forgive me if this idea has already been put forth and shouted down for one
    reason or another.

    Thanks in advance,

    Philip
     
  2. Phillip -

    This probably has been suggested before. I stopped paying attention to the debate about the rating system a long time ago because no matter what people suggested it was thrown back as unworkable or somehow against the policy of the site. I think Brian went a long way to reducing mate-rating but the system itself encourages gaming. Some old-timers told me it was corrupt and corrupting. I think he was right. People are very concerned with the score their photograph gets, much more than if their photograph is good or not. Those two ideas get confused. I think your suggestion would improve the rating system. Maybe not perfect it but eliminate some of the abuse.
     
  3. sorry Philip not Phillip
     
  4. I had an idea similar to that a while back, a PNetter had to have say at least 5 images (an arbitrary number) posted and have ratings on these imgages before being able to rate others photographs.

    I am neither here nor there on the first idea. I joined because I was a trial member and got to use all the functions of the site and liked it. I dont' know if many people would like it if they had to give away cc numbers online.
     
  5. 1) I like this idea, and I believe there is much support from current users.
    2) How would the initial group of "qualified raters" be established? Assuming this system would not be anonamous, how will the "revenge" ratings be controlled?
     
  6. Can a simple change clean up the rating system?
    Yep, just bring the 1s and 2s back and only show average of ratings received (no more ratings breakdown page).
    Do not see the connection between ratings and real name.
     
  7. I don't why one has to be nominated to have an opinion. That's what a rating is, ins't it; an opinion?

    Here are some ideas.

    Design the critique forum so the critic can choose what genres of photography appear. For instance, I want to start rating photos. I go into the recently submitted photos for ratings and a list appears with all the categories and I am able to check the ones that I would like to see and rate and the ones I do not check won't appear. For those who want to rate all photos, there should be a "check all" option.

    Make the comments field more visible. This might encourage more comments and written critiques. Also allow Critique Only photos to appear in the queue. Just disable the ratings function on that particular photo.

    The ratings system isn't perfect, but the issue isn't with the ratings system. The issue lies with a hand full of people who abuse the ratings system by mate rating, low balling, and cheating. You will never create a perfect ratings system. People will always find loop holes to cheat it.

    I propose that you focus on ideas that promote more dialogue than just raw numbers.
     
  8. Philip, it is quite refreshing to hear admin being pro-active with this subject. This is quite a different approach from the previous regime where Kelvin Ball, (anonymous raters & rate recent,) with all due respect to Brian, was created to minimize abuse@photo.net work load on Jeremy as well as eliminating mate rating activities.

    Although both of your points had been raised numerous times in the past in site feedback forum without further action, I would certainly prefer having real members rate per your point #1. As much as I like the idea of point #2, there definitely will be questions about what constitutes a qualified rater and the process involved.

    I don't have a real answer here, but again just want to say that I appeciate the fact that you're asking feedback from members. Simple change? Sometimes the simpler, the better.
     
  9. Phillip, Will King hit the nail on the head for many of us who are fed up with the ratings when he wrote, "The issue lies with a hand full of people who abuse the ratings system by mate rating, low balling, and cheating. You will never create a perfect ratings system. People will always find loop holes to cheat it."

    The fact that there are folks who would corrupt the site in the past with bots and folks emailing each other to purposely down rate others is simply amazing. I base this on a previous post where this had been disclosed. (No I don't have the link to this.)

    The common request I've seen when discussing the rating system is a requirement to have the rater be identified and an explaination with why he/she rates high or low. Some will argue why it won't work and rating battles would occure.

    Sorry if my frustration is showing. Previously when such discussions were made posters were dismissed as whining malcontents. So I've tried to stay away from this topic.

    Gmww
     
  10. 'I think Brian went a long way to reducing mate-rating but the system itself encourages gaming'... Well put, Kent.

    Philip, I think your ideas are great and will definitely partially clean up the current rating system. I would like to also suggest that a registered member can only give limited number of ratings 'per category' 'daily' to minimize, not resolve though, the impact by raters 'who abuse the ratings system by mate rating, low balling, and cheating'.

    My another suggestion is about the TRP. Since the deafult TRP (3 days Rate Recent Average) is based on the rating system so I think it's somehow related to this thread. I would like to suggest the TRP shows only the top rating image for each category and people need to drill down to see all top rated photos in that category. This will give all genres the equal opportunity to be shown on the TRP without having the page dominiated by certain category.
     
  11. When submitting a photograph to be rated it is a forgone conclusion that there weill be some rates too high and some too low ... with that in mind I can still get a general feel of how others felt by looking at the overall average. I don't need Pnet to average it for me. Really the point is for me do I like the photograph and maybe someone else will like it too. Philip leave it as is.
     
  12. I wouldn't be living up to my reputation if I wasn't an early responder to this thread. I appreciate your addressing this issue by asking for input.

    1) I expect you would reduce the number of raters per image by a substantial margin - maybe half. That's not necessarily bad if we assume that most people who reigster properly might also take the task of rating somewhat more seriously, but that's just conjecture. (At this point, consider why anyone would want to rate images at all, even if the system was flawless.)

    2) Having had some experience "judging" photographs similar to many of the kinds of images uploaded here daily, I can tell you that being qualified to rate them in any useful quantity is difficult because you can't properly assess the value of an image unless you appreciate the genre and have some idea of the technical skills involved in its capture and processing. This assumes that you're comparing an image to other similars. (I think the POW discussions support this belief.)

    It has been postulated that in order for a rating average to have any statistical validity, you need a critical number of raters - fifteen was suggested as statistically significant in this context. My view is that one or two rates from just a couple people who actually know what you're trying to do and have done it themselves successfully is much more valuable than a half dozen numbers from perfectly nice people who just bought a camera.

    But this assumes you need numbers. What are you going to do with them, and isn't there a better way to accomplish that goal without all the acrimony the has been part and parcel of the system from day one.

    I'm sure by now you've noticed how many of the less than stellar images on the all/all TRP sort were posted by people who vote each other high and often. Many - but clearly not all - are the last people I'd ask to select or be selected as experts.

    Judging by the comments I've read, there are a lot of people here who do know how to evaluate an image, but have been so turned off by the ratings system that they won't go near the gallery, much less the rating system. Setting up something that would attract them would go a long way towards improving the quality of both images and discussions. I would place a higher priority on attracting a broader range of images and critics than some sort of ordering of the images we see now.

    There is a lot of evidence that supports the notion that giving and getting rates is a fun activity that has served as an introduction to this site and has attracted paying members. This subject can not be addressed seriously unless we recognize that the system has been used to do too many things and, as is usually the case, does none of them very well.

    If you plan on fine tuning this sytem there are a lot more details to share, but I'll save them for now, hoping they won't apply to the substantial changes that I hope you're considering.
     
  13. Open accounts at other websites and solve the technical and behavioral issues too.
     
  14. "Do we want ratings from anonymous users?" No! In the same way we don't want anonymous users, we should not want anonymous ratings of any kind-let's stop these gutless wonders who love dishing out light speed 3/3's and who hide behind a mask of anonymity. If you want to leave a low rate and don't want to comment, fine but be identifiable.
    I don't think it necessary to be 'screened' by members to join but at the same time, you should not be able to rate other photographs if you don't post your own-it should work both ways. cb
     
  15. Lots of excellent comments here so I'll keep it really brief. I think the proposal is sound as
    long as the approval process is quick for new raters. It should be as easy and inclusive as
    possible.

    And, as someone mentioned above, it's important to bring back the 1s and 2s or reorder the
    scale from 1 - 5. The dead ratings make the scale meaningless in its current form. I also kind
    of like the idea of showing only average rating, no the actual ratings themselves but I'm not
    married to the notion.
     
  16. Also, I think Charles point above is right on the money. If you don't post your own photos
    you shouldn't be able to rate.
     
  17. Philip, i have been a long time critiqic of the rating system, but i would welcome your idea. I think it is well worth trying. Thanks for listening to the members.
    Sincerly Joe B.
     
  18. A couple of comments. First off, the primary rating complaint is that ratings are too low- you even see it up above where people complain of the 3/3'ers. Attaching a real name to that 3/3 isn't somehow going to make anyone like it any more than they do now. It will however give the complainers a way to go low-rate the other persons work just to show them.

    Someone up there said the rater should "have some idea of the technical skills involved in its capture and processing." I disagree with this. Suppose I take my 110 camera, shoot a shot, develop in my own home-brew developer, blow it up to 4' square, handcolor it with crayons, scan it, spend a few hours touching up digitally, etc. Meanwhile, you walk outside with your point-n-shoot digital, accidentally push the button, and get exactly the same photo. Is one of those shots to be rated higher because more work (and/ or skill) was put into it? I think not. The significance is when people are using 8x10 view cameras, doing exotic printing, etc., they expect the finished shot should be rated higher because of the process, even if the quality can't be seen in a 600-pixel wide jpeg.

    It won't bother me personally how the ratings system works, but a lot of the changes made through the years were trying to resolve complaints of one kind or another, and those complaints are unlikely to go away.
     
  19. "they expect the finished shot should be rated higher because of the process, even if the quality can't be seen in a 600-pixel wide jpeg."

    I'm not talking about your ability to produce a web shot which will be seen on a thousand different monitors.

    I'm saying that each genre requires a different skill set, which probably isn't obvious to people who haven't actually tried it.

    I do agree with your comment about low rate whiners. Many posters here don't remember what the system was like before anonymous rating. If an average photographer gets a 3/3 from Joe Blow, the next time he sees Joe's image (or seeks it out intentionally!), we are meant to believe he'll be objective in his analysis. Not likely!

    It's true that the above scenario rarely happened. Why? Because people learned about revenge rating very quickly and stopped rating inferior images, especially if they have a decent portfolio and could actually be of some help if they chose to include a comment.

    Now do you all understand? You have to separate uploading and comments . . . and figure out a way to identify the better images without using numbers.
     
  20. I don't care for the rating system, have only twice used it, and I'm sorry I did that. If I think someone is doing interesting work and I'd like them to know I'd just as soon write them and say it. My problem with the anonymous is that sounds like duplicitous. I have pulled my personal work from the site not because I care what people think of it but because I'm concerned about the changes that are afoot regarding copyright protection in the US. I don't mean to be ill mannered, but I'm really not very interested in what most people think about my work. Whether its the work for hire or my personal work. I guess I don't understand. I mean, what do you get out of being rated? There really aren't that many people out there who are actually qualified to even begin to discuss what makes one photograph better than another. Sure everyone has an opinion, however some are informed, others clearly aren't. What service does it do a young or beginning photographer to be rated by an idiot? I don't see it. Hayes
     
  21. I suggest allowing all members to rate providing only an average after applying a simple (unspecified) algorithm to weight those in category 1 (real name) higher and those in category 2 (an experienced select group) even higher.

    No detailed breakdowns - just an intelligently weighted average.
     
  22. Setting aside your concerns about copyright for the moment, I would say that the goal here should be to infuse this photocritique forum with sufficient integrity so that given your credentials, you will consider participating in it in areas where you're interested and have some experience. There ARE loads of qualified people here; you just have to make it worth their while.
     
  23. Don't you think the number of people in category #2 are so few as to have little effect on the input from category #1?
     
  24. Carl,
    not if weight for group 2 was higher than the ratio of group 1 to group 2 (just counting those actually rating an image).
     
  25. What is the difference between the current members who use nicks or initials etc.

    If they had just given a number what would you think. For example a comment from
    "258746" etc. Totally stupid sounding isn't it. Using quasi names is NOT appropriate for
    the type of community I believe you wish to have here.

    Requiring a name plus postings plus time at the site plus a 'certain level' of ratings from
    others are all ways of trying to prequalify who can rate.

    Philip you are to be applauded if you will try out a few simple but BIG changes over a
    period of time. It would be great to have the atitude of "LETS TRY IT" instead of listening to
    all the people who do nothing because they 'think' it won'y work. In the end no dies from
    getting a bad rate.

    Feel free to change the system it currently has no real value to anyone here.
     
  26. Philip,

    I believe you are putting the cart before the horse here. You are suggesting a change to the rating system to fix it, based on the assessment that since there are complaints, it must be broken.

    But are you sure that the system is broken? What is the purpose of the ratings? Does the site need them at all? Perhaps these are the questions you should be asking first. It is difficult to determine that the system is broken if you are unsure of what the purpose is in the first place.
     
  27. Philip,

    Another site had an interesting way of doing it after they re-formatted. Their program made it necessary to leave a critique or the act of simply rating wouldn't be accepted. Moreover, if the critique was too short it wouldn't be accepted. It certainly put a stop to the shotgun ratings and those who bothered to spend time critiquing were interested in the photo. Unfortunately the site's content basically went down the tubes, but their idea was an interesting one
     
  28. I know this has been suggested before, but it seems appropriate to bring it up again here.

    <p>

    What about making the images (submitted for rating) anonymous? This would help reduce the mate and revenge rating(?). One would only learn the identity of the photographer after the photo was rated.
     
  29. I generally like the idea, but I think there are a number of good critics out there with relatively few photos, and some people with good photos who make lousy critics, so I have some doubts about that element.

    Another way of rating that might be less prone to abuse: put two photos up on the screen when rating: the raters job is to choose the preferred (and, perhaps, to say one thing about the photo the rater preferred and give one way to improve the one the rater didn't prefer). Points could be given for each comparison (10 to the preferred photo), with increasing points as photos are compared to more highly rated photos.

    Because this is a direct comparision, there won't be a difference between high raters and low raters that skews ratings. Because someone's going to have to explain why that poorly lit nude is better than those artsy blue flowers, I'd think we'd get a truer comparison.
     
  30. The new Forum Listing page is, to coin a phrase, A Great Leap Forward. Some random thoughts on ratings follow.

    How about keeping the name of the photographer annonymous on the Rate Photos Annnonymously and the Critique Resquest Rating pages. I try to keep the name of the photographer below the edge of my screen because I found I was fully capable of adjusting a rating up or down based on such things as the photographer's gender, ethnicity and recollections of their forum postings or portfolio. Whether it is conscious or not, I believe all of us make a complex set of judgements about an artist the moment we see a name.

    I am also curious about the dude or dudedess who uses the moniker "Z". It has rated something like 60,000 photos but doesn't have a single photo of its own posted nor does it make its e-mail address available. The spread of Its ratings follows a relative normal curve. Likewise, I don't have many arguments with his highest rated photos. I simply find it interesting that Z chooses to use the site this way and wonder if there are others who take the same approach.

    Finally, I continue to find that about 90% of the 3/3's I get occur within five minutes after posting. About one quarter of these images rebound within the next 48 hours and end up with averages in the high 4's and low 5's. However, by then, the early 3/3 seems to doom the image from ever getting much exposure. BM presented various arguments on why an image will likely get a lion's share of its 3's in the early stages but I never totally bought into them. When all the dust settles from the mega rennovation of the site, I hope management we review this issue.

    Now, I really probably shouldn't go here but I rarely see the bawdier nudes on the Anonnymous Rate Recent Page. Perhaps I'm just looking at the wrong moments. But if there is some scheme which minimizes the appearance of nudes on the Rate Recent page, this, in effect, shields them from the phenomena of early 3's which, in turn, helps propel them to the top of the TRP. Of course this is all idle speculation from a guy that has too much time on his hands today. :)
     
  31. Philip, nothing like this good topic to stir up the users! A change sometime ago that Brian gave serious consideration was to allow new comers to rate (and use rate recent to compare their skills with the raters in general) ... but not to count the ratings for some period of time. This would generally elliminate newbies from newbie mistakes.<p>Also I strongly disagree with your idea on nomination ... you are making the assumption that the only critique is from an accomplished photographer. In many cases, we know what should be but don't for a myriad of reasons (like we just don't have the time to spend shooting for one ...).<p>Finally, I have always supported Brian in anonymous rating and the laws of averages. People frequently call out 3/3 as abuse ... a very tough call. On the whole the population IS average. On the old 1-7 scale that means a 4 is average. Somehow people get the idea that they are all above average. (Just ask my wife who is a 2nd grade teacher about how many parents think their kids are all gifted ... rather impossible.) Anonymous rating increased ratings dramatically and we should let the law of statistical interpretation apply rather than complaining about singular ratings at the extremes. In fact, we should all be our own toughest critic ... perhaps we really are ... but many just don't want to accept the truth!
     
  32. Phillip, can you post whatever the powers to be finally decide on this issue? As another poster has expressed, thanks for asking!
     
  33. All the issues listed above - Bailey Seals' rating habits, comparing images of different genres, anonymous uploads, images that seem to attract early 3s, and why you don't see popular images in the RFC queue have all been dealt with at length in the site feedback forum before.

    This is off topic, but given that you can't find the answers to these questions by searching the archives, who, if anyone, should be expected to address these issues?
     
  34. Anonymous rating must go , its just too easy to be abusive .
    Just look at some high quality shots with 3/3 and 4/4 om them , thats a discredit to the raters that put them on , and the system for allowing them to do it with out accountability . Its Just not right in any way for this to happen .
     
  35. Real names are for real people ! Great Idea ! Robert
     
  36. Real names - sounds good to me.

    One possible idea would be that each member gets a certain number of ratings they are allowed to give in any one day, based on a percentage of the members rating history.

    For example, I have rated 2900 photo's with an average of 4.75/4.91. So I'd get a certain number of ratings per 24hours as a percentage of that 2900. To get the math simple for example purposes, lets say 1%. So I get 29 ratings per 24hr. Of these, 35% would be 5's, 32% 4's, 20% 6's etc (these %ages are rough guesses based on frequency - I'm too lazy to search out a calculator and work them out properly).

    So now I have a fixed number of ratings I can give (and I wouldn't be allowed to "save" my ratings and rate 200 one day, then none for the next 6. It'd be 29 per 24hr).

    Naturally, comments/critiques would be unlimited.

    Philip, it's great to see you back, and great to see you asking the community here for their opinions.
     
  37. Philip,

    Before programming a solution there first must exist a well defined problem.

    Is the problem well defined?

    The only *consistent* complaints I heard are two:

    Low ratings with no comment attached. Also associated with this is the belief that ratings should not be able to be given anonymously.

    Disagreement with the Aesthetics and Originality categories. This is usually coupled with the belief that the average guy doesn't have the critical ability to parse an image into these categories anyway.

    While I don't pretend to know exactly what the true problem is, I think making a comment mandatory and non-anonymous before a rating can be issued would improve things enormously.
     
  38. I don't know what the answer is but I can tell you I agree entirely with Charles Becker above!
     
  39. Honestly i don't think there is a cure-all. Anonymous raters anger people, and non-anonymous raters attract abuse and 'revenge rates' or other admonishment and retribution. I think real names may eliminate some abuse by people who want to pad their ratings by creating dummy accounts, but it doesn't really address the larger issue at hand. It may also help deal with people who rate low en masse. It will also bring back the much maligned mate-rating and all the ensuing ranting.<BR><BR>Nearly everyone feels their images are at least above average, otherwise they wouldn't post them. In looking at ratings, people are looking for acceptance and reassurance that their photography is superior to other photographers' photography. Nobody really relishes being average. When they are brought down by below average rates, they get angry. I admit that I get peeved at 3/3 ratings when I feel my work has some merit- I have put time and effort into creating it, and it's a gut reaction to feel indignant when someone else doesn't appreciate it. i think it's a fairly common reaction, no matter how 'above' the system anyone claims to be. this is why I don't really rate any more.<BR><BR>There are many fine photographers who frequent this site. Their work deserves recognition, as they have worked hard honing their craft. however, there are a lot of photographers who come to this site to learn and develop their skills through observation and, hopefully, intellectual exchange with other photographers. I think it is here where your efforts need to be focused. Do you want this site to be a feature site for a small, elite group of photographers, or do you want this site to be attractive to the average photographer who wishes to improve their skills? A feature site is great, and we can all ooh and ahh about how superior the squeaky clean, polished peices the accomplished artists present really are. We can all ask them 'how'dja do it?' But that gets old pretty quick. An intellectual exchange forum, on the other hand, will be attractive to many many more folks. We can still ooh and ahhh over the flawless gems, but at the same time know that the less polished works won't just get tossed to the side. I think the 'fix' you're looking for is to connect the rating system to the critique forum. I feel that since I joined this site, it has moved progressively away from its focus on critique and idea exchange. The system has been streamlined to encourage gut-reaction number punching and mass processing as many images as possible. Simply: require a rating to be accompanied by a critique. No more pedestrian ratings from people without a real opinion. No more mass-rating forty shots in 15 minutes. Require the viewer to put some thought into why they feel like they do about a shot, and give an incentive to communicate that to the photographer. That is what the critique forum is for, right? Otherwise, we might as well just call it the ratings forum. It's been a while since I got a really good critique on this site. Been a while since I felt like giving one here either. This idea has its inherent difficulties, I know. Any system, though, is going to have kinks. All I know is that it is much more satisfying for me, as an artist, to hear someone's opinion than to see it simply represented by a 7, or a 3.<BR><BR>I appreciate that you're taking a proactive role in the direction of the site, and I'm interested to see where you go with it in the near future. Thanks for taking the time.<BR>-e-
     
  40. Why do we want ratings from people who just registered? From anonymous users? From people whose own quality of work we've never had a chance to judge?
    I agree. As I am using a screen name, and have not placed images in the gallery, I refrain from rating or offering critiques.
    But I think your "vouching" scheme is open to abuse.
     
  41. "Why do we want ratings from people who just registered? From anonymous users? From people whose own quality of work we've never had a chance to judge?"

    Because it reflects the real world, where if you put your work up for sale/appraisal/comment it would be judged in the same way.
     
  42. I would like to see critiques rather than ratings promoted. Often I put a photo up for rating because I want it critiqued but it becomes near invisible if I tick the critique only category.
    Jerry Ting's idea about having to drill down to view TRP in various categories is very good. As we all know there are certain genres that are always rated poorly and this would give those people a chance to have their work viewed.
     
  43. What is the purpose of the ratings? If it is meant to be a guide for improvement, then clearly the system at present is not working. I think there are two reasons for this. First, the prevalance of anonymous spoiler ratings which enrage many, and rightly so.

    But the second, unmentioned reason is that while people may initially seek feedback in the form of ratings in order to learn, all too often (i.e. with most who put up their pictures) it degenerates into an unhappy quest to try and get as much approval as possible from the photo.net community.

    So I really don't think ratings work well as a teaching/learning tool.

    Nevertheless, I think Philip's ideas may cure that. And I think Sylvie Leuders' idea to display the rater's average rating is a very good one.

    I don't have a dog in the hunt for ratings, but BTW this is an exemplary feedback discussion.
     
  44. I'm of those that think assigning a couple of digits on an abitrary scale to something as subjective and emotional as a personal reaction to a photograph is ridiculous. Whether the digits come from someone with a believable name or not seems irrelevent.

    Furthermore, photographers, on the web at least, seem peculiarly enthusiastic about rating their raters. I'm not a filmaker, fine artist or musician, but do such people fret so over their audience's qualifications to judge them, or are they more concerned with satisfying a wider audience? Why erect artifial goalposts by striving to be a photographer's photographer, when there's a world of potential appreciation?

    Are photographers even qualified to judge photography? Are the best picture editors the best photographers? Should we discount the opinions of all film, music, literature and fine art critics if they are not the best film-makers, musicians, novelists or fine artists, as judged by other film-makers, musicians, writers and artists?

    This is what perplexes me about any rating system. Other photographers think a photographer has made a good photograph. So what? Pre-qualifying the raters in the proposed ways seems only to maximise photographic incestuousness. Your opinion doesn't matter if you're not one of us.

    Please permit me to make an alternative suggestion. It amazes me that no online photographic community, amongst the proliferation now available, has earnestly pursued a collaborative approach to photography. In simple terms, "if you like this picture, you might also like..", or, "if you like photographs by this photographer, you might also like..."

    There's been solid research into collaborative filtering techniques and algorithms since the popularization of the internet. The tools exist. The success of collaborative filters is strongly related to the quantity of content and the size of the participating audience. Photo.net would appear to have no shortage of either. I might even suggest that by implementing a collaborative rating system, the popularity of the site might increase, the value to those posting photographs be enhanced and the loyalty of visitors rewarded - the more you rate, the more accurate the filter, the greater the satisfaction - a positive feedback loop.
     
  45. Is everyone missing the point? You can not rate a photograph unless it is submitted for critique. You can submit for critique ONLY but you cannot submit for RATING ONLY. When I post my images, I am looking for feedback. Any rating without comment doesn't help anyone. Getting a lot of ratings might give you some idea of the popularity of a photograph.

    I do not rate anything without comment and quite frankly I don't think anyone else should either.

    "We are now a group of more than 100,000 photographers working to help each other become better." If you are not interested in helping you shouldn't be welcomed here!
     
  46. I dont see how your proposal could make things worse, it might very well work. Here goes a wild idea: Why not give members an option to not allow ratings without comments? Lets say it is implemented: Anyone could rate 3\3 and write: fhgjygyutg on the comment box or give a 7\7 and write WOW, personaly I think either situation would demonstrate the value of the rate(r). Its also very likely that mate raters would enjoy this feature, so, speaking for myself, I would accept if the photos of those chosing this option would not go to the trp or had a seperate one. I supose it is a realy wild idea... I agree with yours anyway. Something does have to be done.
     
  47. Well, why not throw my pennies in? I don't think we need a numeric rating system at all. We can generate the Top Photos by critiques, and people can critique if they want to provide feedback. I really don't see a need for the ratings system. At least, if we go to all critque only, if someone wants to 3/3 an image, then they at least have to type it out in a comment. It will mean an end to instant feedback, but frankly, IMO, a person who needs instanteous feedback shouldn't be posting photos for critque.

    Just my thoughts

    Don
     
  48. Here are some ideas.

    Design the critique forum so the critic can choose what genres of photography appear. For instance, I want to start rating photos. I go into the recently submitted photos for ratings and a list appears with all the categories and I am able to check the ones that I would like to see and rate and the ones I do not check won't appear. For those who want to rate all photos, there should be a "check all" option.

    Make the comments field more visible. This might encourage more comments and written critiques. Also allow Critique Only photos to appear in the queue. Just disable the ratings function on that particular photo.

    The ratings system isn't perfect, but the issue isn't with the ratings system. The issue lies with a hand full of people who abuse the ratings system by mate rating, low balling, and cheating. You will never create a perfect ratings system. People will always find loop holes to cheat it.

    I propose that you focus on ideas that promote more dialogue than just raw numbers.
     
  49. I also have no problem with 3/3 as long as they go to to those that are clearly below average . Ive never had a problem with 7/7 either , they are the rarest to get for 90% of us here so who would complain about getting them ? Robert :)
     
  50. My suggestion is about the RealName issue.

    Some people have privacy concerns. Can you have a verification system that allows the site admin to verify the identity of a member, but still allows them to post with a "user name"? There could be a logo or icon next to their name saying "verified".

    Ways to do this include (as PG already suggested):

    . verifying by a credit card transaction (many websites have this, www.couchsurfing.com is one that I know of, ebay is similar too, right?)

    . e-mailing to a name.name@trustedcompany.com (those of us who use gmail etc can use the card method above)

    A member, thus verified, could still post under a user name, but with an "identify verified" icon.

    Other ways might be found by people with more knowledge of the Net than me.

    I don't want to re-start the discussion about why some people prefer not to have their full name on the web - I just think it is a fair point that some people prefer this.

    As long as the site admin knows who they are, and an icon tells other members that this is the case (ie that their identity as a real person has been verified), shouldn't this be sufficient?

    Of course, I offer this as someone who has posted no photos and only "rated" once, just to see how it works. Perhaps I would feel differently in other circumstances.

    richard
     
  51. I think rating photos is a good idea, When someone says "a good photo", we all want to know how good. Besides, it's what drives the system and creates excitement. But, we all know the problem with the rating system.<BR>
    Please, let's evaluate these suggestions and see why they would NOT work, before we dismiss them and they may very well be inapplicable . <BR>
    1- We all have a good idea of who mate/high raters are. Now, why not have a a way to anonymously identify a high/mate rater, let's say by hitting a hot button or sort next to the name. in a sense, everyone will be building up a record of sort of their rating behavior, judged by thousands and thousands members of the bigger community.<BR>
    We will set a threshold above which, the persons's name will appear, let's say in green and that should be considered as, let's say, not a serious rater. We will make this threshold high enough to get a better judgement. If the person gets more negative hits, up to a second threshold, then the name will appear, let's say in red and the rating of the person will no longer be averaged out for the TRP. This should discourage members from high/mate rating, it would discourage me! Or, maybe we can have more that two colors or no colors at all but a way to evaluate our ratings.<BR>
    2- some members don't like, let's say bird photos and I know they down- rate bird photos because they are fed up of so many birds making it to the TRP. Why not have a top photo in every category and that should alleviate some of the anger these member may feel.<BR> Please, think about it, wouldn't it help? I know this have been raised already and I DO agree with the idea. We are having genres compete and that doesn't seem to be fair. We may have a much bigger pool of members who find comfort and satisfaction in seeing a bird sitting on a branch than taking pleasure in looking at an abstract, no matter how skillfully it may have been done, the abstract will never be recognized an will leave a very frustrated member.<BR>
    The next problem is with members who rate photos when THEY ARE NOT qualifies to do so, but that is another subject.
     
  52. In one way or another, my wife and I have had our IDs stolen three times in the last six to eight months, and we're cautious people. With that said, I don't like this "RealName" suggestion and understand the issues some of the members have raised. Why can't paying members have a choice? You've brought up RealName a few times already and I really don't understand your obsession with it when members have raised concerns. Amazon let their shoppers use screen names by the way.
     
  53. Philip,

    I was so pleased to see this question addressed by you. I don�t mind saying that the rating system on this site and all the problems associated with it are a big part of why I personally do not participate much. It is also a major reason for not renewing my membership.

    I was all prepared to offer you a rant about the ratings but, quite frankly, I have written so much about this in the past that I am no longer motivated to write any more. Besides, if you are interested you can check the archives. When you do you will see that I have never complained about low ratings on my photos. I have complained about the system. But since you offered a suggestion for improvement and asked if it would solve the problem, I must say that the answer to your question is that it would help but it would not solve the problems on this site.

    Any rating must be an expression of how much an individual likes an image. Images must be divided for ratings by their genre. Gallery exposure must be separated by their genre. Abuse of the system must not be excused or rewarded.

    Ratings are a form of applause and this can be quantified. This applause can be measured and used to select the best of the best if there are controls to prevent abuse. Critiques are for educational purposes and must be clearly defined. We must recognize the difference.

    That is my opinion, I could be wrong. Thanks for asking.
     
  54. If you want a worthwhile opinion on an image eg the technical quality, the lighting, the
    composition, the subject, the mood, the colour palette, the graphic quality, does it tell a
    story, its originality etc.

    Do you ask the first 10 or 20 people you meet after you leave your home/studio? Possibly
    people who may never have commented or thought (in any depth) about these aspects of
    an image.

    Their opinion is personal and true but should you value it or is it the case that in all
    probability the chance that any of the first 10 or 20 you meet are people who can
    genuinely help you is quite poor. It may well be a representative sample of the general
    public but is that what you want?

    Will the general public help you improve the above mentioned aspects relating to your
    images.

    Not an exact parallel to photonet but close enough.
     
  55. Phillip- I found myself vacillating back and forth with regard to the anonymous verses the non-anonymous rating system as I read this entire thread. I do like your proactive choice & willingness to discuss a hot topic.

    Obviously, because I use my "real" name, I am not fond of cutesy pseudo names. I proudly display my name. Am I worried about "privacy" issues? Not really. Has anyone from PN bothered me? Not really :) Am I the average PN subscriber? I think so :)

    I use to rate & comment on lots of photos each day, once everything went anonymous, I continued to rate & comment without a lot of change. But, over the last few months, things appeared to degrade into chaos here on PN. It became as clear a mud, that member suggestions where not welcome. Of course, it depended on "which" member suggested it. I feel the average member came to feel a little alienated.

    Quip, sarcastic, one line zingers became the "in" thing to say. If you complained about the system, you were a "Whiner". If you agreed, you were a "Non-whiner". "Non-Whiners" rose to the top, "Whiners" where silently asked (via e-mail from the top) to remain silent or suffer banning".

    After belonging to this community for a few years, I came to the realization, numeric ratings are irrelevant. Only the comments made by members similar to myself (whom I could view their works on PN & decide if their comment should be taken seriously) actually helped me improve my skills as a photographer. I came to PN to improve my photography skills via honest, sincere dialog & suggestions.

    There are members on PN who are not pleasant, but they have the ability to take beautiful images. Therefore, I value their comments. There are those who are extremely pleasant that have the ability to take beautiful images. They are more fun, because one can actually ask & get a response from them about "how" they accomplish their photography.

    I would hope that "learning" is why most of us came here & to give & receive knowledge about a subject we all have a common interest, photography.

    In the end, it was a hard decision, but I vote for "real" names. I think you are on the right track. I for one, think that using ones real name makes one act more responsibly. If one can hide behind some cutesy pseudo name, then it seems to empower then to act irresponsibly. I have no problem with my real name & real rating of an image being displayed. Do I fear retaliation? No. Why, because when I give an average or below average rating, I also offer my opinion with suggestions for improvement.

    To those who say they are too busy to spend a lot of time commenting, the suggestion of a check box might be an added bonus. I somehow can't use those things, they remind me of the ever popular answering machines employed by most of the major companies now. My question, never seems to match their choices. I need to talk with a "live" person with a real name :)
     
  56. Philip

    A simple solution to solve the anonimity versus credibility problem would be to add the
    rater's personal overall rating alongside the rating given. It would preserve the rater's
    anonimity whilst giving the person requesting the rating an idea as to the ability of the
    person rating his or her work.

    Receiving a number of 3/3 ratings from members whose work is rated 5/5 overall would suggest that you need to raise your game . Similarly, receiving ratings of 7/7 from raters
    whose personal overall rating is around 4/4 would suggest that you should not become
    too excited about the quality of your work.

    I think PN could become an excellent learning medium if we are able to learn from qualified
    teachers. As it stands the rating system has no value since raters lack credibility. My idea
    would establish credibility.
     
  57. Phil,

    Mate rating is basically a positive feedback loop (in system terms), a simple way to negate it is to emplot a negative feedback loop into the system.

    My proposal is VERY SIMPLE, the further away from the median a persons average ratings are, the same statistical deviation is applied to their own average rate as shown on their profile page.

    So, if I consistently vote an average of 6, I recieve a negative handicap of 2 against the average rate of my photos. if I have an average rate of 2, I also get 2 points deducted.

    This means that I have to rate across the whole range in order to not be 'handicapped'.

    Obviously this could stimulate a slew of 4/4 rates, but we can also apply a negative feedback loop to this scenario with very simple statistical analysis (all automated).

    Finally, on our bio/profile or my workspace pages, have a CLEAR WARNING, stating why we've been handicapped and how to remedy it.

    I hope that makes sense.
     
  58. PS, I forgot to say, but the 'handicap' would affect TRP visibility too.

    So if I had a photo that scored 6/6 average, but I had a 2 handicap - it wouldnt make the TRP.

    Making the TRP seems to be the primary aim of most mate raters, and the hate raters are trying to prevent TRP entry.
     
  59. That's far too simple Ben, LOL
     
  60. 2+6 = 8 Average =?
     
  61. Why not just scrap the whole rating system entirely?

    It does not work, it gets abused, and no-one is ever happy with it and the only people who care are people who think that a couple of numbers on a website are proof of a photograph's merits.

    Find somewhere to exhibit some nice framed prints in your locality. If you sell a load of them then you get the best kind of positive feedback. Hard cash and the knowledge that a real person likes it enough to put it on a wall and look at it every day.
     
  62. Louis, I think you've completely misunderstood what I was trying to get across, I'll try and make it more clear:

    People who's overall average ratings are somewhere around 4 are rating fairly and across the board (by and large).

    Anyone who rates lowly will have an average under 4, and those that rate highly, above 4.

    My idea is to give the low and high raters a handicap which is exactly equally to the difference from 4 that they rate. so someone whose rating average is 3 gets a handicap of 1. Someone who rates on average 5, also gets a handicap of 1 - because they are rating 1 away from the middle rate of 4.

    This handicap is now applied to any photo they submit for rating - but only for TRP purposes. So if they scored 5.5 on average, they'd have their handicap subtracted and get a TRP score of 4.5 - siginificantly altering their position.

    This basically means that people have to start rating across the board - those who currently don't are the mate raters and hate raters.
     
  63. Their sole purpose is to select the top 10% or so of the daily uploads for more prominent visibility. Nobody ever learned anything from numbers. If you compel raters to leave comments, two things will happen: they will leave useless one word answers to satisfy the database, but more likely, they will not rate images they don't like.

    Most of you are discussing this from the standpoint of rates and comments you want to receive, but consider what sort of system needs to be in place to encourage those of you who have some teaching experience to participate in this forum. A system that allows any form of retaliation has exactly the opposite effect.

    If you try to handicap raters, don't you think that will encourage them to go through the RFC queue as fast possible handing out 3s and 4s to balance their average? This breeds more retaliations, which breeds more nurturing of your high rating support group to counteract them.

    Forget them numbers. Nurture the critics. It's that simple.
     
  64. In the mean time, try this sort as a source of your randomly selected front page images: photographers highest / minimum 20 rates / image must have been submitted for critique / DQ photographers who have given more than ten 6/6s (or higher) to ten or more photographers.
     
  65. mg

    mg

    Wow ! :) Great to see, FINALLY, that a decision-maker takes this issue seriously and asks for opinions - which certainly wasn't the case under the previous management. I read the whole thread and I agree with Mani Sitaraman, that "this is an exemplary feedback discussion".

    That said, as an old-timer here who really followed very closely the evolution of the rating system and threads about ratings on this site, I would like to say, that the ratings issue on PN has become a very complicated one, mostly because virtually all suggestions about it have been ignored since 2002, and because the system was adjusted so many times, that there is no consistency anymore in the Top-rated pages.

    Example: A good picture posted, say, in 2003, and the EXACT SAME picture posted yesterday, would have at least a 1 point difference (yesterday's ratings would be harsher) in Aesthetics AND Originality averages - I can guarantee you that.

    So, first of all, WE MUST LOOK AT A WAY TO CORRECT DISCREPANCIES OF RATING TRENDS THAT OCCURRED OVER TIME.

    HOW ? This is VERY difficult, and yet, quite simple in principle.

    Where did these discrepancies come from ? Answer: Rating abuses - i.e. mostly Mate-rating and systematic down-rating - and various measures to fight abuse.

    But how can we identify abusive rates and how to deal with them ? Simple again. What's an honest rater ? It's a rater who PREFERS some pictures over others, and who simply STATES HIS PREFERENCES regardless of the photographer's name and other motives. Can we find out who's honest and who's not ? Up to a certain degree of accuracy, I'd say yes. How ?

    Well, a rater who rates 10000 pictures in 2 months, with 90% of his ratings being 6s and 7s might still, perhaps, be fundamentally honest, but one thing is sure: he *HARDLY PREFERS* some pictures OVER other pictures. He basically likes everything. Why would his opinion then matter, when it comes to a calculation to SORT OUT PEOPLE'S ***FAVORITES*** ?

    So, the "main" basic principle/concept that should be used, both to correct ratings of the past and to set up a proper rating system for the future would be:

    RATERS THAT SERVE THE SITE'S SORTING SYSTEM WELL ARE THOSE THAT HAVE A PREFERENCE OF CERTAIN PICTURES OVER OTHERS WITHIN A GIVEN CATEGORY.

    (We are not interested in knowing that John Doe prefers cats pictures over human portraits: we just want to find out which cat pictures he prefers.)

    There are several ways to make sure that such a basic principle (and a couple of others) rule the rating system from past to future.

    Some proposals posted earlier in this thread made a lot of sense. What we need is to discuss them toroughly, logically. Ben Anderson's suggestion and Ray Fraser's suggestion are the ones pointing, imo, in the right direction. But I have a few other ideas that could work well combined with theirs - one of which would be to limit the number of 7s, 6s, 1s and 2s ratings that anyone could submit, following some pre-defined bell curve: i.e. the principle would be, that most pictures are average, few are exceptionally great or absolutely bad. (And I'm thinking here of some retro-active measures to counter-balance the impact, that those who "prefer everything" have had on the system over the years.)

    Then I also agree with those who thought the photographer's name should, if possible, be anonymous for a couple of days. And I agree with the general idea of having 2 categories of raters, some whose opinion "matters more" than others - based on certain criteria that would need a better definition. I agree also with those who expressed the idea that there should be top-rated photos within each separate category - and ONLY within these categories. Overall top-rated photos are nowadays, probably, an impossible dream, since too many originality ratings were in fact "category-biased" in the past - with many raters slamming entire categories. And there would be a few other things worth considering in the present thread.

    So here is my proposal.

    I've stated a principle in the present post - SEE THE 2 LINES IN CAPS. And I think I know most of the people who can really be logical, constructive and fair in a torough discussion about ratings. So I'd like to work with them on one common proposal, that we would then send to the management of the site, first with a "simple change" - as this thread was requesting -, and then later with a complete plan for a complete revamp of the rating system if necessary.

    Here are the people that I would like to "invite" to work on a common rating reform proposal - in no specific order:

    - Will King
    - Louis McCullagh
    - Carl Root
    - Ken Tallheimer
    - Doug Burgess
    - Scott Bulger
    - Robert Brown
    - Robert Woodward
    - Ray Fraser
    - Ben Anderson
    - Gungajim Downs
    - Hah Kit Yoong
    - Mani Sitaraman
    - Garry Edwards
    - Guy Scrivner
    - Cyrus K
    - John McLaine
    - Margaret Meehan (M.M)
    - Sam M-M
    - Demosthenes

    I am quite sure I forgot some people who posted come valid contributions to this topic in the past, so please feel free, anyone, to say so if you wish to join. Then there was a mathematician on this site - Jim something, I think -, who actually worked with Brian M. in 2002 to put up quite a brilliant rating system, which was never implemented. I'll need to find him again and to call him in as well...

    I'd first like to ask you, Philip, whether the idea of having a group working on this issue to come up with a common proposal is something you'd be welcoming. If you say yes, perhaps we could post another thread to start working - in a more organized way. Or we could work via emails, as you wish, but that's perhaps less convenient.

    Keen, not keen...? Your call. Regards.
     
  66. mg

    mg

    "Why not say that to be able to rate photos, a user must be
    1) registered with a verified RealName (once we get the system up and running; should work if we can send email to Joe.Smith@hp.com or some other well-known company or if an existing RealName user can vouch for the person or if the person has a credit card and can do a $1 transaction, etc.)"

    A good idea, I think.

    "2) nominated by the existing group of qualified raters, presumably based on the fact that someone among that group has seen the new user's photos and liked them"

    Another good idea.

    My previous post was an attempt at demonstrating that these 2 measures you proposed, and which are good, would probably not be enough to "clean up the rating system". We can't just be looking at the future only, we need to also look at a way to correct the past. For anyone who would doubt this, just look at the top-rated pages.

    As a side-note, may I suggest a poll ? Why not simply ask what the majority of photo.net members think about the site's top-rated pages: do they think that the TRP actually more or less present the site's best pictures or not ? (Please note that if the answers are not, globally, a YES, then that directly means that the rating system wasn't working, or wasn't used properly.)

    Example of a poll that would lead to interesting conclusions:

    Are you a) very satisfied with the TRP ; b) satisfied with SOME of the TRP (and which ones) ? c) Not really satisfied, not really dissatisfied with the TRP. d) Not satified at all.

    PS: Some statistics about various rating habits of the membership would be very useful to in order to analyze the situation properly.
     
  67. Ok, since my last comment was deleted, let me ask again, in a softer tone: why aren't you looking to the archives for the solutions to this problem? Volumes have been written, why start it all up again?
     
  68. After having read this long and very passionate thread, I will just bring my 2 cents, which I feel is a commitment for a site that brought me so much...

    First : thank you Philip for - as said upper - your proactive effort to listen for members opinions and for your commitment to improve community's satisfaction.

    Second : task is hard and ideal will never exist (PN is great !) as we are in a subjective environment (hopefully!)...I would not try to even summarize the "pro" and "con" (so far...) to improve or cancel the rating system...I don't feel I would come to a clear conclusion and I don't feel entitle to...

    Then I will just give you my feelings :

    - I AM for keeping the rating system, that makes this site original (compared to most others) and that (imho) is an incentive to improve one's skills, learn, exchange... If the rating system was nothing worth, I don't think any discussion would have even started on the matter?

    - I understand the attempt made with anonymous ratings to counterbalance mate ratings...then if the basic problem is to eliminate mate ratings , the idea to shift from "anonymous ratings" to "anonymous post" mentioned upper (used in other "critique only" sites) seems also to me a good idea, IF, in addition, you make ratings impossible to be changed afterwards (which is possible today) : identity of the photographer being revealed after 24hours.

    - finally, if PN admin has time and courage , why not to summarize all submitted ideas in this thread (reposted every day during a period of time) in a questionnaire and submit it to the community votes, say during a month, to get a democratically support for any decision ; something like :

    1/ do you want to keep a rating system : Y - N
    2/ do you want to keep anonymous rating : Y - N
    3/ do you want anonymous post implemented : Y - N
    etc...etcナ
    Web answered opinion survey, on a short list easy to sort (not easy to implement!) and (maybe?) helpful to make a decision or ,at least ,enlighten most popular wishes...

    Well, crazy idea isnメt it ? Crazy job for sure !

    Jean-Pierre
     
  69. Well, Mark has been faster than I to promote the "poll" idea...maybe not so crazy an idea...?
     
  70. Ben's idea sounds very good even if it's looks a bit difficult to make it understable. What I fear is that mate raters become half the time hate raters and vice-versa to balance their average: perverts are everywhere (BTW, only giving 3s or 7s has really no sense).<p>
    Philip, have you an idea of the percentage of members who complain about the rating system? 0.1&#137 (yes, I mean per thousand) of the whole community? 0.2&#137? 1&#137? So do you think that's really a BIG problem?<p>

    Actually, one of the main problem in the rating system is that only 3 to 7 are counted. My 2 euro cents.
     
  71. mg

    mg

    "What I fear is that mate raters become half the time hate raters and vice-versa to balance their average: perverts are everywhere"

    Well, how sure are you that this is not the case already ? :)) I mean, in my view, the typical mate-rater is exactly the kind of person who would create a second account under another name in order to down-rate many people, and to have his pictures on the TRP - don't you think ? Philip's first suggestion was also a way to prevent such things from happening.

    Besides that, here are many other ways around this problem as well - one of which is to limit the number of 7s, 6s and 1s & 2s one can ever give. See my previous post about this. An extension of this limitation, is to add an "8" rating on top of all the ratings that already exist. I'm not going into this now, but you wouldn't believe how useful it will be, knowing that 8s would then be very rare, and that all members would be asked to give a couple of 8s, retroactively, to their favorite "7"-rated pictures on the site. (I've thought about this "8" rating for a very long time, and very carefully, and it would help correcting retroactively all the nonsense that came with the massive seven-ratings of the past.)
     
  72. All the above discussion is great. Answers, of course, come with much more difficulty.

    Some of us would have to change our habits if some of the suggestions here were implemented, but I suppose we could do that. I've been here for about four and a half years, I think. I hardly ever rate any more, and when I do, it's usually a high rate to show appreciation for something I think is a great image. I never use the RR queue.

    I leave a fair amount of comments, but comments don't count toward ratings.

    What if comments DID count as something toward ratings? Whether it was a positive or negative comment might not matter as much as the fact that the photo was worth a comment. Maybe comments (other than those from the photographer who took the picture) could have some bearing on visibility (I know there is a TRP sort for most comments, but it's buried).

    I think verification is good, but I'm not sure you should have to use your real name. For paying members, the fact that you got payment should be good enough.

    If you're a paying member, you should get to rate, whether people like your photos or not.

    I am of the opinion that raters should have posted photos, but I know several who seem to leave really worth-while comments but have no portfolio, so who knows?

    But to answer your question: "Can a simple change clean up the ratings system?" - my answer would probably be "no."

    Thanks for letting us rant, Philip. Cheers.
     
  73. Interesting that somebody has to "like" your pictures before you can rate, that remark coming from a party who wants everything on photo.net his way, according to his standards. Soon we will have the English Car club, with anal-retentives swooning over their MGB's and Midgets. Then we will become like the SCCA, a spinoff from the Society for Creative Anachronism. We will examine each others ratings and cast off those that aren't like us. The whole thing sounds Kakfa-esque. I can already see the group poring over pictures of sunsets, flowers, and men with turbans. The examiners are all White Males, with coke bottle bottom glasses, smelling faintly of body odor with dirty white shirts that have soiled collars and long hairs coming out of their noses and ears. As strangers arrive with different pictures, they hurriedly gather in small groups, whispering nervously with each other and casting furtive glances of distrust and paranoia at the "intruders". With apologies to some posters and a middle finger to others, I find that the current rating system is a vast improvement over what it was before. My preferences would be some algorithms added to prevent anyone category or photographer or photo-clique from becoming too dominant.
     
  74. I have never personally posted any photos for critique or rating, having made the mistake (?) of reading about the problems others have with the ratings & the system, and decided I just didn't want to go there. Nor have I ever rated anyone else's pictures, although I did post 2 comments/critiques once, just to see how it worked.
    I think that the biggest concern people have is the mate rating and the fact that there appears to be a slew of 3/3s right after a picture is posted, rather than anonymous ratings. My suggestion is twofold. Address the mate rating issue by preventing a photographer from rating a work by someone who has just rated their work for a set period of time. In other words, if you rate one of my pictures, I can't rate anything of yours for x number of days - maybe 3 - long enough for the "revenge instinct" to cool down & go away.
    To address the issue of immediate 3/3s, why not have all new pictures "held" for a period of time - say 24 hours - before they can receive any ratings? Have a little checkbox under each new picture during that holding period and a conscientious rater can check off pictures that they'd like to rate "for real" when the holding period is over. That will allow time for thought, and someone who's just randomly assigning numbers will not want to bother going back the next day to actually input those numbers into the system.
    I also like the idea mentioned above by Ken Thalheimer about having a rater be forced to enter a comment, otherwise the rating will not be stored. Perhaps that could be random - so that sometimes a rater can just leave the numbers, other times, an explanation will be required. The "explanation" should be a minimum number of letters in order to be accepted - and that number should be kept secret, to prevent abuse.
    Definitely some good suggestions in this thread. Philip, thank you for asking!
     
  75. Philip, this thread doesn't begin to describe the hundreds of issues and proposals that have been brought forth in many threads over the years. Best you read through them, they are all archived. I would recommend the "ratings normalization" solution that Brian almost implemented but backed down because of (IMHO, misguided) criticism. There are many such.

    Your first point is fine. Clamping down on multiple bogus accounts by making it a little bit difficult to create accounts, is a fine measure. Giving new registrands read-only access to the gallery is probably another good idea. By participating in the site (posting photos, rating, commenting and forums), they can qualify for read-write access, etc.

    My advice is to reduce the reward for high ratings and the punishment for low ratings. You can probably bunch them up into buckets -- like a grade, not GPA. Don't use a strict sorting order. Jumble them up. Make the ratings average invisible in the TRP pages. This should still give you a satisfactory display of well-above-average photos to show in the TRP. Use "fairness" policies and ensure that the same photographer or the same genre doesn't hog all the limelight. By reducing the reward, you can sleep easier; trying to prevent users from gaming the system is a losing strategy.
     
  76. Others have objected to the following lines, and I will, too: "Why do we want ratings from people who just registered? From anonymous users? From people whose own quality of work we've never had a chance to judge?"

    I think the entire rating system would be greatly improved if everyone participating did so with the understanding that there is no absolute, objective way to say one photo is better than another. Ratings are opinions, and reasonable people *will* differ in their opinions. Nothing irks me more than the argument that the rating system is flawed because someone gave a 3/3 rating to a great photo. "Great" according to whom? I don't care how many people have rated a photo 7/7; if I don't like it, I don't like it--even if I can't explain WHY I don't like it.

    I agree with posters who say we need to nail down the purpose of the rating system before we try to fix it. Carl Root has, I believe, given us a start: The primary purpose of the rating system should be to determine what photos show up at the top of the TRP. But this raises the question, Who is qualified to determine what constitutes the "best" photos? Clearly Philip thinks that some PN members are unqualified to rate photos. If we start with the assumption that there's no absolute way to say one photo is better than another, then I would assert that no one us ultimately better qualified than anyone else to rate photos. I don't deny that there are people whose opinions matter more to me than others--of course there are--but ultimately I believe that the closest we can come to an "objective" definition of "good photo" is "a photo that appeals to a large number of people." If you choose the people you think are best qualified to rate photos, your *own* opinion about what makes a good photo will inevitably be reflected in their ratings.

    I think the most basic view of the TRP should be one that shows the collective opinion of the PN masses, regardless of their level of experience. (And I will continue to be very glad for the many alternative ways to view the TRP.)

    No doubt, many people will take exception to this goal. If by some miracle this is actually adopted as the goal of the rating system, however, the problem then becomes How best to ensure that the ratings reflect the opinions of the masses as unvarnished by mate raters/hate raters/bots? Many suggestions have been proposed here. My own opinion is that any approach that compromises anonymity will fail--the impetus to mate rate is just too strong (I have felt it myself). A better approach may be to try to increase the volume of rating. If every photo received 100 rates, the influence of a few mate/hate rates would be greatly diminished. No doubt 100 is an unreasonable goal, but we can certainly improve on the amount of rating that happens now.

    I would like to propose the following: Whenever someone wants to post a photo for rating, they should be required first to rate a certain number of photos themselves. Ten? Twenty? I don't know. Yes, some people will abuse this and hand out rates quickly without much consideration, but I suspect the vast majority of people in this situation--anticipating the rates they want to get on their own photos--will take some care with the process. Perhaps they can be given a quota: They need to find at least one photo to rate 3 for aesthetics, one to rate 3 for originality, eight to give 4s, etc. Allow them to click past as many photos as they want to find these. I would not require anyone to hand out 1s, 2s, 6s, or 7s (I would, incidentally, reinstitute 1s and 2s, and resign ourselves to an overall average somewhat greater than 4).

    If nothing else, this policy would greatly increase the number of ratings doled out. It would also ensure that the *majority* of rates come from people who are posting photos themselves, though it wouldn't exclude anyone from rating (I would *not* require people who rate to post photos themselves). Many people would still view it as a game and try to "beat" the system, but I believe that increasing the number of rates overall may be the best foil for such people. And obvious instances of abuse could still be reported to "abuse@photo.net."

    Finally, regardless of what approach is adopted, I would have a page explaining as clearly as possible what the purpose of the rating system is--something people need to read before they start rating, and something the moderator of this forum can point people to when they complain.

    (On a related note, I like Sylvie Lueders's idea of showing the average ratings your raters give out, but anonymously. A 6/6 means something entirely different to me from someone who hands out 4s on average than from someone who hands out all 7s. The flipside, showing the average ratings your raters *receive*, would probably also be interesting, but on principle I'd have to say it seems less important to me.)
     
  77. There are a lot of photos up for critique every hour of the day. I can't -- and don't want to -- get to even a fraction of them. What I do is scroll through the thumbnails and when one strikes me as noteworthty I make a comment and, sometimes, rate it -- usually not below 6/6. Sometimes I comment on a below average picture because the photographer says he is new or really wants advice. But then I don't rate it.

    Is that so bad?

    The largest inaccuracy of rating is that is isn't coming from a focused group, nor is it coming from the world at large (i.e. where the sales might be, if you are motivated that way).
     
  78. Yes, that's evil.

    Just kidding--it's a natural impulse, and I suppose if no one handed out anything but 7s, we could just use the number of 7s a photo gets to determine its ranking in the TRP. Actually, that might not be such a silly idea--a rating system in which people only get one positive feedback option: Yes, I like this photo.

    I agree with this: "The largest inaccuracy of rating is that is isn't coming from a focused group, nor is it coming from the world at large." I do not entirely object to having only some subset of the PN membership allowed to rate, but if that's the case, I think someone ought to just say, "You, you, you,.....you, and you. Nobody else" and get it over with (and be prepared to receive a LOT of flack over the selection). If the raters are chosen well, they could lend an air of authority to the system. Yes, this seems to contradict what I just said about everybody being fit to rate, but we would have to redefine the TRP to say it only reflects the opinions of this particular group--opinions that people *might* care about because of all their accomplishments and experience. I suspect that in practice this would be very difficult to pull off. I still think we'd be better off trying to get more people to rate overall.
     
  79. Some of these 3/3 raters are the ones who have a horse in the race and a photo on TRP. They specifically rate low because they want to stop a prospective photo coming up to compete. <BR>
    HOW DO I KNOW THIS? I do and we caught a few and reported them to abuse@photo.net which virtually has been useless. <BR>
    There was(is) a guy for instance who had OVER 30 surrogate account, many of which originating from his own IP. He would down-rate all photos on the all time TRP , one by one to bump his own photos up. LISTEN TO THIS, 8 out of 16 photos on the 'all time TRP" ended up being his, including a badly photograph photo of a flower.<BR>
    I sent the guy an email, warning him. He replied that if I did anything about this, he'd pull his photos but my photo will also vanish from the list (I used to have a photo top or second on the "all time, average, average TRP under my previous account). <BR>
    I sent him a copy of an email I was going to hand over to the administrators. He pulled his photos and put down 3 PAIRS OF 3/3 on my photo in less than 5 hours, ! My photo is no longer near that page! He is not the only one who would do such things, I have other examples, just ask me!<BR>
    The administrators DID NOTHING about his behavior, nor did they do anything about his 3 pairs of 3/3s AND THAT IS THE FACT!<BR>
    Now, Mr. Greenspun, is this what your site is all about? If you CANNOT and WILL NOT address issues like this, will there be any confidence left in the members of this community?!
     
  80. I've participated in this discussion since it was first posted. Last night I submitted a photo for critique. After posting, it usually takes a minute or two for me to see it. I checked the general critique forum and the category I listed it in. For about 2 mins. approximately I did not see it. When it finally showed up it had 4 rates. 3-3/3's and a 4/3. How is it possible for others to rate it if it isn't even showing up yet on the critique forum? This is not an unusual event as others are posting similar experiences.

    My point isn't about my particular photo deserving it or not. The point is the speed in which theses ratings show up. The history the rating system has had with bots makes it suspect. I'm here to gain advise from other photographers. Low ratings without some sort of critique is useless to me. I rate now directly and leave a comment whenever possible. Why? Because I would like the same courtesy. I wish a rating system would require all raters to do the same.

    On the issue of real names. I use my initials as I'm a private person in a small community. What I do with photography is my business so long as I don't break the law. If I wanted to identify myself as a photographer of nudes, I'd do that. I don't, so I use my initials. If your going to confirm a rater please don't limit it to those who don't mind having there full names on the internet.

    Thank you and sorry about the rant. It's just so darn frustrating for many including myself.
     
  81. Some of these posts seem a bit paranoid, but much as it may look that way, Cyrus K's is not. You would be amazed at the trouble some people will go to to warp the system to their advantage. Many of them even carry these tactics over to other sites. I was once accidentally included as part of a group email that was clearly a regular communication meant to coordinate group ratings. This is why I agree with your focus on pre-qualifying a group who will select featured images - an extended version of the POW process. Fifty would be great - so would 500 - but you have to be able to monitor their activities through programming and hands on inspection.

    Sometimes conspiracy theorists are right.
     
  82. Perhaps this has been suggested before and if so, my apologies. Would it be reasonable to normalize a rating given on a photo using the raters average rating score(s). Something like this: A rater give a rating of 3/4 on a photo and the system divides by 3.44/4.12 (the raters average) respectively, giving normalized rating of .872/.971. This means a score close to 1/1 is average, from the raters point of view. Scores lower or higher than 1/1 mean the photo is below or above average. Any validity to this?
     
  83. No.

    It takes up way too much computer processing time to run a complex calculation every time someone rates an image.
     
  84. And yet the system can calculate the averages for every rated photo for ranking and possible placement in the TRP and also update the photographers average as well as his/her average when they rate a photo. And there are many more automated calculations routinely performed. A retrieval of the raters average and then a simple division can't use that much more cpu time.
     
  85. Marc, I'd be happy to join a ratings reform group. As those of us who have been active on PN for a long time know, this is a difficult and vexing problem. While many of us will say ratings don't matter, they do matter if you want any visibility (and written critiques) for your photos. What I've found to be most troubling about the rating system is that its inequities have caused so many good photographers to leave the site over the years.

    To me, it seems like a system that handicaps (lowers high raters rates and increases low raters rates) ratings plus takes into account number of ratings (which would track how compelling a photograph is) would work for the TRP. I would also only let photos be rated in RFC queue for the first three days after they are submitted.

    Even though Brian's tinkering with the rating system didn't solve the ratings problems, it did slow down the mate rating that drove many people from the site a few years back.

    Philip, I hope you'll give Marc's idea a try. Between us, there's probably 50 years of PN experience and, like you, we all want PN to be the best place on the web to learn about and share photography.
     
  86. Marc,

    <p>

    I too would be happy to join your group.
     
  87. "I too would be happy to join your group."
    A select group of people choosing the ranking (value) for the entire collective group of people posting images, and none of you thinks this concept isn't ripe with potential for corruption? This is single handedly the worst idea I could imagine.
    You all make one flawed assumption - that the majority of people on this site want to see what the "best images" are based on an arbitrary, subjective popularity contest. I think you're wrong.
    I think it would be far more beneficial if there was a way that most people could find images that they like, not what everyone else likes.
    To that end I suggest no rating system. Get rid of it, lock, stock, and barrel. Replace it with a simple way that would allow every photo.net member (paying or not) to list their 15 favorite images. No ranking, no order, just a favorite list. Any images could be replaced at any time but the list never exceeds 15.
    Now when I find an image I like, I can look at the list of the member who selected it and it leads me to an endless link of similar minded photographers. And by default, of images I like.
     
  88. I think all you need to do is have rating like leaving a critique. No real names needed, jusy thier photo.net name. Then you can look at thier portfolio and see if they really know what they are talking about. Also, they may be afraid of incurring bad comments on thier portfolio for downing something they know nothing about. I think the rating system could be a good thing as long as they explain why they did or did not like the photo. A rating alone says nothing, but an explanation of the ratng would bring it merit. Maybe they just don't like birds and are only into nudes... Then I agree with earlier posts that a category system of rating would help. Let the bird lovers rate birds and the nude lovers rate nudes. Otherwise it's like asking a football fan about soccer.... Not their bag. What real worth can be attributed to thier input. Really hope this can be cleared up or my subscription will not be renued. I am a beginner trying to perfect the craft and you can get lots of ratings (good or bad) but rarely can you get actual advice on what you are doing right or wrong. Ratings alone don't tell you that. Hope this can be worked out. I have seen some of the best photography ever on this site, and at the same time seeing it recieve 3's. Maybe just not their bag? Maybe if we could see who's posting the 3's and look at their portfolio, we could feel more at ease.
     
  89. mg

    mg

    Other members I'd like to convey to this "group" - if the management is keen for such a group to be formed, of course -, and which I forgot in my previously posted list:

    - Marshal Goff
    - Wade Rose
    - Johnatan Charles
    - Guillermo Labarcca
    - Philip Coggan
    - Miles Hecker
    - Bob Hixon
    - G.
    - Nestor Botta
    - Niranjn T
    - Bernard Mayr

    Niranjn T, by the way, was in his earlier post here refering to a ratings reform Brian prepared (with this mathematician called Jim something) in 2002 (thread was in october 2002, I think), but which was never implemented. This was indeed the most interesting reform I had heard of todate on PNet. It was basically aiming at something close to what Ray Fraser or Ben Anderson suggested earlier. A similar reform with a few additions should do the trick for a much better system. I hope Niranj T would accept to participate in a more detailed discussion if it happens, because he was very involved and a very useful contributor to the october 2002 thread. I'll try to find this thread again and add a link to it here in a while. Regards.
     
  90. If I were Philip, I'd have a migraine by now.

    A "group"? Sorry Marc, I think most of those you mentioned are good people, but they are also mostly strongly opinionated individuals. Reminds me of a PTA group I once served on. Holy crap Batman, I fear they could never make a unanimous decision :) Sorry Ben & Will, but I don't think you fit in this group, you're too darn nice :)


    If I remember accurately, once an image has received 8 anonymous ratings through the queue, it gets placed at the bottom of the stack. It seems to me that if one wanted to manipulate the ratings, it would not be difficult. Via 8-10 bogus accounts & a few minutes of time, anyone could manipulate their image to the top & others to the bottom. This "is" the problem with the system.

    I think simplicity is the goal. Rooting out the abusers, abolishing multiple accounts & as Brian has already done, limiting "mate raters". All good behind the scenes activities.

    With the ability to have multiple e-mail accounts all over the place, AOL, Yahoo, etc. & use all kind of false names, Philip's idea of name verification via a 1.00 charge to a credit card is an excellent one.

    For those of you who don't want your real name as your ID, that's fine, chose a "handle". But, you must be registered under your "real" name.

    Possibly the aforementioned group might be better used to update the tutorial on "how" to rate. But....I fear the aforementioned discussion group may never agree on anything :) But it might keep them busy!
     
  91. mg

    mg

    "I fear they could never make a unanimous decision :)"

    Well, Jayme, you may be right. :) What harm did a try ever do, at anything...?

    Somehow, I never really understood why people would go to a fruit market just to say they don't like fruits. :)) I think they'd better do their own shopping as they fit.
     
  92. mg

    mg

    Here are the two main threads about a rating reform planned in 2002. The second one is the most important.
    <p>
    http://www.photo.net/bboard/q-and-a-fetch-msg?msg_id=003cBJ
    <p>
    http://www.photo.net/bboard/q-and-a-fetch-msg?msg_id=003opA
    <p>
    Unfortunately the links posted by Jim S (Jim Shwaiger, the mathematician we may need for this) are all gone. :-(
    <p>
    But here is some hope:
    <p>
    http://www.photo.net/shared/community-member?user_id=389767
    <p>
    That's a link to Jim Schwaiger's page; hopefully we could ask him to join our discussion about ratings and to accomplish some mathematical miracles.
    <p>
    Finally, here is the only part of his past miracle that I could still find: see mostly in the second thread ( msg_id=003opA ) the post by Jim S , oct 02, 2002; 05:14 p.m.
     
  93. Marc- it never ceases to amaze me how much information you keep tally on. Does this not wear you out?

    I think of my brain like a glass jar with a lid. Once it gets full, it's full. When I "must" add new information, something must go or.... I risk my brain exploding :) Sadly, my brain filled up about 1974. Since then, I think I've forgotten more than I've learned :) Of course, I can't be sure of that :) I can't remember!


    But I suspect, I just don't consider my images a part of me, more an expression of me. I change my expressions frequently :) So where my or anyone else's image stands in the PN TRP is kind of insignificant.


    Cheating however, should not be tolerated.
     
  94. So I am in the PTA?......LOL
     
  95. Marc- while I appreciate your zeal to select the people whose ideas you are fond of and assemble a subgroup to build the new system, I beleive at this point everybody responding to this post are the group Philip is looking for opinions from. At this point appointing your own committee is throwing the cart waaay in front of the horse; a simple suggestion that Philip choose to form a small committee here (without trying to choose it for him), I beleive, would be appropriate, but also as much off topic as this response, which addresses you and not to OPs question.
     
  96. We're not going to form a large committee to rehash all these ideas over and over. Phil is going to decide what to do based on his goals for the site, an awareness of past problems, and the practicality of the suggestions offered.

    Picking images for site visibility should be a right that you earn by leaving useful comments, by demonstrating that you have an appreciation for all kinds of photography, and by not cheating. Aren't those the kind of people you want evaluating your images?
     
  97. Wade- :) Now what does PTA stand for, I've forgotten! LOL Naw.... you also are too nice & rational for any "group". But..... your dry humor & wit would make it fun! :)

    How about having a "fun" day. Where up is down & down is up? Have the TRP start from the bottom & go forward :) Just brainstorming here :) Then a day where it starts middle & goes in either direction with each click :) Keep the cheaters guessing :)

    Retaining a sense of humor is rational. My right brain is exhausted, it can't figure out nor comprehend all those mathematical equations to create a level playing field :) Way too complicated for me. I'm getting a migraine :)
     
  98. What I have came to is that this is the Internet. Anyone and everyone can access PN So I really don't think anything is going to help. As long as you have non- paying members makes it to easy to open more then one account
     
  99. Robert Woodward: "If you don't like this, or any, rating system, don't use it.
    Exactly, it took me a week when I first joined to figure out that the rating system in place was flawed to the point that it was of no value to me in finding images that I liked. So I've never participated.
    Will King: "SP, why does this even concern you? As far as I can tell, you don't submit photos for ratings and furthermore, you have never rated other people's photos."
    Wow Will, you've been nominated to the "Ministry of Good Taste" for five whole minutes and now you're suggesting who should offer opinions?
    For the record, I've submitted images for critique many times, but I don't concern myself with the ratings only the comments.
    And also for the record, I would have a problem with any person deciding the direction of this site that insists on the deletion of an image but refuses to have a rational discussion about why.
     
  100. Let me show you an example of what I'm talking about. Let say I happen to stumble across Balaji. Where I happen to find his photo style to my liking. His list of top 15 images would most likely lead me to Saul Zelan and his list to Alexander Ziegler.
    A simple path from one set of images that I find interesting to another. The TRP has never done that for me. Why? Because I don't share the aesthetic taste of most of the people on PN. Mind you I'm not discussing the quality of the TRP, just that in my opinion, it doesn't show me what I like.
    So now tell me why anyone wouldn't like to find a whole bunch of images that suit their personal taste over seeing what everybody else likes?
     
  101. It is unfortunate that this has become so complicated. I would like to repeat my previous suggestion; no anonymous members, no anonymous ratings, and no rates/comments from anyone who doesn't enter their pictures for rates/comments. If you only submit for critique, you only get to critique. The system can track the number of 7's a member gives and it prevents giving out a 7 to someone who has left that member a 7 within a certain time frame (24 hours?) so it can certainly track the number of 3's someone gives out; perhaps there could be a maximum number of 3's a member could give within 24 hours.
    Last comment; I really would not like to see a panel of 'qualifiers' no matter how good they are; it is not needed and -to me anyway-insulting. My 2 cents (Canadian) worth-that's about 1.8 cents U.S. cb :)
     
  102. Oh for crying out loud!

    He asked questions that suggest that he hasn't been following all the nuances of the ratings system daily for the last five years (like some of us have.) Read the whole proposal again and take him at his word.

    SP, your request for a personal favorites gallery has been in effect for quite some time. Where have you been?
     
  103. "Carl Root , sep 15, 2006; 04:47 p.m.
    Some of these posts seem a bit paranoid, but much as it may look that way, Cyrus K's is not." <BR>
    Thank you Carl for understanding. It was the first ever acknowledgement of this problem by anyone! But, no one has to believe me, the photo is still there and so are the records. The photo is under my previous account.<BR>
    Yes, Ben, they go even further than that. Just look at the "top photographers" list and look at who SOME of those guys are and look at their rating habit and you WILL soon figure out how they got to be on that list and so close to the top. Just as that list is not a good representation of great photographers on this site, so is not some of the photos we see pop up top on the TRP. I have a clear understanding of how that happens too and you may ask me how!<BR>
    Now, please understand. I am not anxious as why I am not on that list. As a matter of fact, I don't deserve to be anywhere near that list. I do however, care about this site and I do want it to work for everyone. I am a paid member and I have also paid for several others on this site to become paid members, the records should be there. But, it is discouraging to see that a good site like this can be hijacked by a group that is motivated by their own personal agenda.
     
  104. SP, your request for a personal favorites gallery has been in effect for quite some time. Where have you been?
    OK, help me out, where is it?
     
  105. if you rate photos, you'll get a link on your community member page to 'photo srated highest by this member' which includes photos you've given 6/6 or above.
     
  106. mg

    mg

    Cyrus K,

    "It was the first ever acknowledgement of this problem by anyone!"

    Well, exactly the same thing happened to me. Makes it 2. :)) I believe a lot more people probably experienced the same thing. Over a couple of years I've seen so many "funny things" anyway, that I hardly take note anymore, and I surely wasn't used to having anyone listening either: you'll have to accept that too, I guess...:) This thread seems like the first time since 2002 or such, that someone at the top actually finds it worthwhile to look down there what's happening in the gallery...
     
  107. The anonymous ratings system is pnet's heart of darkness. It is the obscure, inarticulate man
    in the crowd expressing his resentment at the literate class he feels has failed him. Give him
    his voice. If you do not, he will nonetheless think his thoughts, and
    you
    will have merely deprived yourself of the only means of knowing what they are.
     
  108. The anonymous rates are allowed (and preferred in the trp) in the current system because it doesn't allow a return of the favour to the rater. If I rate someone high on a photo, it is likely that the person receiving the rate will rate a photo of mine high. Thus leading to an escalation of the average rate. Notice how the nominal average 4 is far from being the true average of the ratings, although it is intended to be.

    What I have proposed before is a normalization of the ratings by each rater. This way the mean and standard deviation of the rates by each person rating (this assumes the rater must have logged on) is reset to 4 (mean) and say 1 or 0.7 or something like that (for std) after each ratings change. This would enforce the validity of the attached meaning of the ratings (ie. 4 is truly the average). Some people seem to object to this because they will only rate pictures that are good. However, I think on average this would give a more meaningful picture of how the pictures are valued by the raters than letting them rate only 7s and 6s.
     
  109. if you rate photos, you'll get a link on your community member page to 'photo srated highest by this member' which includes photos you've given 6/6 or above.
    Yes, you are correct, but the problem I have with that it that under the current system I'm prohibited from only providing high ratings (which is exactly the images I would want to show other people). So in order to show you my favorites, I have to also rate some images low, which is not what I am interested in doing.
    Think about it, the ability to show you my favorites comes at the expense of someone else? How is that fair?
     
  110. That's not true. You can rate anything you want any way you want.

    That's one reason rates never have and never should be "normalized."

    A case can be made that the only "rates" that the site needs are the ones you use to put in your favorites pages. . . except for the problem of reciprocity which is almost impossible to avoid when you first sign up and start rating. Then you wake up one morning and realize that some people are going to be upset if you don't rate their next upload a 6/6 no matter what they offer. It doesn't matter if both photographers upload mostly "good" images. The resulting data amd image order is unavoidably based on site politics and ethics.

    This stuff is far from intuitive, which is why people who haven't rated and uploaded plausible images extensively should sit down, be quiet, and read posts thoroughly,

    . . . and above all, consider the source.
     
  111. When you allow people to opperate with out having a true Identity its just a lot of nonsense and a few other words that I will not use . Its a very simple solution . Real names , real rates , all attached to the real owner . If some one wants to be an _____ _____ , at least we will all know . They will do them selves in . Yes its really that simple , it may not work perfectly , but why make it easy for the cowards !
     
  112. Another thought , if anyone is concerned about other people getting 7/7 and you are not getting yours , well may I suggest taking some photos that will get those 7/7 s . You are in the right place to learn ! Robert :)
     
  113. That's not true. You can rate anything you want any way you want.
    Well if that is truly the case then I wasn't aware of it. I believe that I read comments from people wondering why they couldn't post a 7/7 when they wanted to. I'll spend some time searching to see if I can dig it up.
    This stuff is far from intuitive, which is why people who haven't rated and uploaded plausible images extensively should sit down, be quiet, and read posts thoroughly
    Was this comment directed towards me? If so I find your paternal tone a bit condescending, after all I've been here few a few years, following right along with you.
     
  114. The current ratings system should be abolished, not modified or fixed. The whole idea of assigning a numerical score to a photograph is silly, even insulting. The system encourages the ignorant and unconstructive attitude that photography is a kind of competitive sport.

    Still, the site needs a way of picking strong images for display to members and visitors. Some sort of rating system would seem an efficient way to do that.

    So consider this: All ratings should be INVISIBLE to everyone except the site editors and managers. Moreover, only those who have passed some type of qualifying exam should be permitted to rate. Possibly these judges should be assigned to rate only certain genres of pictures -- specialists in nature photography probably shouldn't be rating street photography and vice versa.

    It'd probably also be easier to have the ratings range from 1 to 5 instead of 1 to 7.

    To select a set of good pictures for prominent display, it's not necessary that the ratings be shown to the photographer or to viewers. Ratings are meaningless and probably detrimental to photographers attempting to improve their skills, so the site shouldn't display them at all.

    The (invisible) ratings should be explicitly treated as only a preliminary criterion for selecting photos for the TRP. The highest rated photos should be taken as a pool from which a panel of editors select which images to feature. The editors should be free to move an outstanding image from far back in the list to a more prominent position or to move a overrated photo to a less prominent position. This may sound unfair, but in the system I'm suggesting, the TRP is not a competition and fairness simply isn't an issue.

    The name of the page on which the featured images are shown should be changed from "TRP" to something else, perhaps "Images of the week." The page should be set up to randomly mix the top-rated photos in different genres of photography, so that it is not dominated by landscapes, nudes, children, or any other genre. All members should understand that there is an element of randomness involved in landing in a prominent position, and thus they should have nothing to complain about when they land there sometimes but not other times.

    If there were enough volunteer editors, they could take turns writing brief explanations of why particular images were selected. These comments would be helpful to learners.

    The system I describe would be slower and more labor-intensive than the present system. But the present system satisfies no one, and there really is no need for the featured photos or TRP to be updated daily. Once a week is enough.

    The key point is to get members to stop obsessing over ratings, thinking of them as a competition, or treating them as a source of meaningful feedback. Making the ratings invisible would instantly solve all of these problems.
     
  115. An alternative to preventing reciprocal 7/7 rates could be to apply a punitive TRP ban for 7 days instead.

    Perhaps people should not be prevented from rating any way they see fit, rather they should be prepared to suffer the consequences.
     
  116. I agree with some of the proposals you are making. How can someone who is UNFAMILIAR with let's say Macro photography and the challenges it presents , get to put a number on a close Macro and complain about "not enough DOF"? <BR>
    I am not so excited by numbers but I know a lot of members are. Even most of the ones who complain about not receiving comments, they really look for numbers<BR>
    I'm just thinking out loud and everyone can help formulate this:
    If you WERE to go with an appointed panel, What if:<BR>
    1-Everyone can comment and rate<BR>
    2-the panel will see some good critiques and fair ratings and they approve and check the names of those members.and the member get's a qualifying star next to his/her name. If you accumulate enough stars, you join the ever growing panel and your ratings will count towards photos going to the TRP, photos of the day/week or whatever you may want to call it.<BR>
    This method may not be perfect but would give members an incentive to become better critique and join the panel. I think this may work better than a permanent group of people deciding all the time.<BR>
    3- The name of the photographer should be anonymous at the time of rating and so should be the ratings of the panel but not necessarily the names of the members of the panel.
     
  117. What if Ansel Adams were still alive, was a recent registrant and just starting handing out 3/3 ratings left and right?
     
  118. Ray: with all due respect. The advice and suggestions given above are typical of a slew of folks who always pop up after a management change with their ideas of "how to set things straight" Any advice to the opposite is often regarded as negative and unconstructive, when in reality it is only checks and balances to the other remarks which have already been made. You have a tough task as a moderator. Participants in feedback forums who make remarks counter to the thread have an even tougher time. What might sound negative actually might be the very thing that you need to pay heed to. Don't be a Rumsfeld this early in the game.
     
  119. My basic viewpoint on the rating system in contribution to the stuff above. The ratings right now appear to be a statistical bell curve. Once dominant images start appearing in the peak of the bell curve, other images and genres are less dominant. When folks see what's dominant, its only too easy to start posting similar images to enhance your chances of appearing in, or close to the top of the bell curve. That, essentially defeats the whole system, and is certainly NOT what I assume both Phil G and Brian M had in Mind. Yet, that is the state of the system right now. Instead of diversity and items of interest, we see monotone repetitiveness and little discussion outside of sycophantic praise. Some of (the)convoluted practices proposed above are only schemes to enhance the viewpoints/biases of those parties, and would only make things even worse than they are now. I don't see how that will do photo.net any good, to replace one system that is fixable with some kind of convoluted idea similar to Medicare. What is basically needed is a good tuneup of the sorting engine to prevent manipulation of the end result. Every party who posts an image SHOULD have the same basic opportunity for critique and review. It should NOT be the sole domain of any one specific category. It should NOT be the domain of a couple of photographers who can magically pop themselves into the TRP regardless of what they submit. The TRP has to be tolerant of diversity, which it is NOT right now. Too many of the parties above have expressed their intolerance for certain genres because of their personal inhibitions and biases. That for me, totally disqualifies them as having the right to input to the changes in the rating system.
     
  120. I think that a lot of folks who are new are here to enjoy photography. They don't want to hear about the latest 3/3 paranoia attack, and how to go after the dreadful revenge raters. Enjoy photography. I still do after decades of it, as if its my first day with my preWWII Voigtlander. Let Photo.net worry about those things that they can control and fix. And let's make it more attractive to newer photographers young or old, by fixing the system so we don't have to look it images in reverse order to find something interesting to look at. All of us should realize that our personal photographic preferences play no role once we submit an image to photo.net. So If my images fall flat, c'est la vie. I can always take them off.
     
  121. Many valid points have been raised and discussed above. There have also been a slew of off topic unconstructive posts which I have grown tired of deleting (sometimes from the same individuals over and over again). I set this thread to expire in 90 days to allow time for a complete analysis by site programmers. Again please refrain from any additional posts - unless you have carefully read all of the comments above and strongly feel you have a constructive idea that wasn't mentioned.
     
  122. The rating thing is highly emotive but we shouldn't take it too seriously. I rarely rate now and rarely bother with the TRP, prefering to browse through the Critique Forum. Just how silly is the rating system? Well, this photo, posted today, is revealed as appreciably inferior to this one, posted in November. Occasionally I have re-posted images and the discrepancy between rates received has been pronounced. This particular photo in my folio gained massively 2nd time round. Two serial raters who first time round thought it worthy of 5/5 gave exactly the same image 6/6 on its 2nd appearance. A recent nude scraped together a dozen votes averaging 4.68. A re-post 2 weeks later sees it with 72 rates with an average just above 6. So which value should I take as the real? The 1st? The 2nd? The existence of such a discrepancy dictates that I should accept neither.Probably the best photo I've taken recently received one of the lowest marks I have in my folio. Where do I go with this rating? I don't. In the end I go instead with my instinct.
    I do a lot of work in one of the less popular genres (child) and although I hope my work might cross genre boundaries and offer more than just a pic of a cute kid, I appreciate that others might not think so. Carl is totally correct to point out that rating in genres we have no interest and knowledge in is pointless. I know nothing about insect, automobile and macro photos. Well, I know they don't interest me. Probably if I took more time I could come to appreciate the nuances involved but frankly I'm not going to. Equally I am not going to rate these photos either. How could I? What do I know about it? I wouldn't ask a plumber what he thinks of the carpenter's table, even though their toolkits look similar.
    Until the interface was changed and there was a lot of commotion about the updating, I wasn't even aware of the existence of the RR system. I think I looked at it once, decided it was a poor way to go about appraising a fellow artists work, and forgot about it. I only ever rated through the critique forum. A radical solution to the RR problems would be to simply abolish it. If a TRP and anonymity are required make the rates from the Critique Forum anonymous and use that venue to determine the TRP. Dwell on the idea for a minute of two. Think it through. It makes sense. I could expound if invited.
    Meanwhile, when the RR interface was changed the facility to easy locate photographs submitted as Critique Only was removed. See attached picture for how it used to be. I read elsewhere that Philip believes 'ratings are not core to the mission of the site, which is peer-to-peer education.' A restoration of the Critique Only facility would go part way to achieving that aim of P2P education. Photos submitted for 'critique only' are now invisible. The newbie or the shy photographer who would rather not undergo trial by rating has no other option if they wish others to see their work. P2P education through useful critiques is worth a 1000 rates. If Philip is serious in his intent, an encouragment to critique could quite easily and quickly be added to system. Provide a link to the 'Critique Only' photos from the drop down 'gallery' menu.
    Ray, I hope I have stayed on-topic and offered something which hadn't been said before. Consider leaving my contribution. It took a while to write and is meant to be of help.
    00I5bZ-32438784.jpg
     
  123. Instant P2P education
    00I5bg-32438884.jpg
     
  124. This discussion has many great comments and some less so. First off, ratings are okay if you take them for what they are. A popularity contest. One fellow suggested hanging your pictures up and if they sold, then you were good (or something to that effect). Wrong! Rating by the masses is what gives us the photographic equivalent of a Thomas Kinkade painting. Perhaps there is no way to avoid that, but at the very least there must be a way to see who is handing you the rating, and a way to see what work they have produced. I often go to the site of a rater (or a commenter, for that matter) to see how I feel about their work. I do not make an effort to retaliate. It will cause far more problems than it will solve to attempt that.

    Under no circumstances should a system be initiated in which raters are limited to a certain number of votes or inhibited in any way from speaking or voting freely. I have gravitated toward not rating stuff that bores my silly, even though I still try to be fair within the framework that PN sets out in the guidelines (how many of you have read them by the way?)

    Most raters seem to be unable to differentiate between the two scores. I don?t have an idea for solving that.

    You cannot draw the same conclusion from raters? averages. Suppose I only rate photos that I can tell I am very drawn to and which I can easily see are excellent photos and I pass on the ones that I would rate low. Then you would put me down as a high rater and unworthy of credence. Idiotic! Suppose I choose to rate a lot of really bad photos thinking I could help the poor blokes? Then I will be accused of rating too low, when, in fact, it was just a good reflection of what I was rating.

    I may be the Lone Ranger here, but I never go to TRP. It?s just a cheerleader?s dream. I have no doubt there have been some excellent images, but the real gems are the guys that hang out on the fringes, and they don?t get TRP. I follow one good thread to another, and IMHO there is far more originality (which I value over popularity) in many, many of the lesser known PN members than one will ever find in TRP.
     
  125. The corporate line used to be... "Comments left were for the photographer and Ratings were
    for the site." In that the rates given helped the site distinguish between the everyday
    snapshot and more polished work. Whamo the site can publish a TRP.

    Make all rates anonymous and only visible as an average.
     
  126. mg

    mg

    I agree with most everything you wrote, Darrell... Interesting that you pointed out this example, which clearly makes the point I was trying to make earlier:

    "Well, this photo, posted today, is revealed as appreciably inferior to this one, posted in November. Occasionally I have re-posted images and the discrepancy between rates received has been pronounced. This particular photo in my folio gained massively 2nd time round. Two serial raters who first time round thought it worthy of 5/5 gave exactly the same image 6/6 on its 2nd appearance..."

    I also agree with you that all the means available to revive a comment culture and criticism on this site will be welcome.

    We ought to realize something else. When I first joined Photo.net, there was really no other place on the internet, where I could get more, nor more valuable feedback than on photo.net. But things have changed, and there are nowadays other web sites similar to photo.net, just smaller, but which really provide more and better feedback. And if feedback is better elsewhere, people will sooner or later go elsewhere. Last month, on another site, 6576 photos were submitted and 95135 critiques were posted. That's about 15 critiques per photo. The same site, since it started, has received 242023 pictures and 4729265 critiques - i.e. an average of 19,5 critiques per picture. I doubt very much that photo.net would at this point offer the same. I'd be curious to see stats about this. And I believe it's not impossible at all toreach such an average on Photo.net. But it needs to be one of the site's primary goals - feedback quantity, but also quality of course. Therefore all kinds of ways to promote comments will be welcome, and we ought to keep this in mind as well when we will attempt to modify the rating system. The new system, no matter what it will be, MUST ALSO be able to generate more commentary.
     
  127. David Meyer wrote:

    Suppose I only rate photos that I can tell I am very drawn to and which I can easily see are excellent photos and I pass on the ones that I would rate low. Then you would put me down as a high rater and unworthy of credence. Idiotic! Suppose I choose to rate a lot of really bad photos thinking I could help the poor blokes?

    Well, people who rate honestly don't rate this way, they rate broadly, just as you do.

    If you chose to rate poor photos highly then you're not rating honestly.

    Rating lowly never helps the photographer, nor does rating highly. Critiques help photographers, not numbers. Numbers help the site.
     
  128. Mr. Anderson, you misunderstand me. Any rating system, if it is to have any meaning whatsoever, has to have a scale. My point is that if I wanted to only rate photos that I feel are outstanding in both originality and aesthetics, then logically the rates would be higher than if I rated the photos as they randomly were presented. If for some reason I decided to rate those which I felt were in need of considerable assistance, then also quite logically the rates would be low. As a matter of practice, if one rates low, it would be nice to know why. My point was that a bunch of low ratings or a bunch of high ratings does not define a person as being a high rater or a low rater.

    I already conceded the fact that ratings are nothing more than a popularity contest, and that does not always equate to quality.

    I repeat, that I would try to use the same standard on any photo whether or not I liked its subject matter or style. Just because I am bored to death with flower photos does not mean that I would not rate them in the same manner as another subject matter that really interested me. Unfortunately, I am not sure that this is the way a lot of Pn folks rate. I do think anonymous ratings are detrimental. But I will concede that there might be a whole lot fewer low ratings if you have to put your name on them.
     
  129. mg

    mg

    "My point was that a bunch of low ratings or a bunch of high ratings does not define a person as being a high rater or a low rater."

    You are correct here. Yet there is a valid definition of a mate-rater (or high-rater), and a valid definition of a hate-rater (low rater); but it's not a simple definition. Many parameters have to be taken into account. But let's not go into this too early.
     
  130. Personaly I think that no photos should be rated without a critique. Why? Because it will serve to at least curtail some of the abuse and at the same time be truly helpful to the photographer and the one rating it also. For instance sometimes I will rate a photo and critiqe it. Then when others post there opinions I see things that I missed when I offered my opinion. This helps me as much as the original person who posted the photo. If the rater knows he is going to have to explain his ratings he will be a little more careful in how he does it. As I said in another thread, at this point I am more interested in how many really look at my photo than the ratings recieved because it at least shows me that the photo was interesting enought to look at. This is not perfect but that and real critiqes are the most benifical to me. The whole point of PhotoNet is to become better photographers and yes to build a commrodary between those of us who want to achieve that end. If you want to rate a photo be prepaired to explain your rating. A critique is the best way to do this. And if the photographer who posted the photo want to ask you to explain in futher detail than they can do so. This helps everyone.

    Berryl
     
  131. I think this is precisely the time to look at that issue because if it can not be solved, then you really have to call into question the validity of the whole system.

    Brian spent a lot of time tweaking an inherited system, and even though there were some improvements, including the default TRP page, nothing happened retroactively, as you've pointed out.
     
  132. Mr. Bader,

    You may be correct and I do not argue with what you say, except to point out the. "I like it." is not a critique. And then there is the clown who puts the same comment on hundreds of photos. Even though it is complimentary, it is meaningless and irritating. So, requiring a comment may not solve the problem, but it might be better than what we currently have.
     
  133. Hi Dave, You are right in that Just saying "I like it" is not a critique, but is it helpful, Well, maybe not in a profesional sense however I think it at least builds confidence to a point. And lets face it, we all like to get compliments once in while. Maybe critique is not the best word to use as a forum heading in that it does not in itself leave any room for kudos so to speak. Its like junk food, everything in moderation is not so bad. I do agree with you though that helpful critiques done with compassion is important and the most benifical professionaly. By the way, your biography, I LIKE IT! Im sorry I could not help that. I really do like it though. And your portfolio is nice also, it looks like places I have seen before. I wonder if we live nearby. Anyway I think we both agree that helpful comments could not hurt.

    Berryl
     
  134. Brian was nervous about implementing a lot of ideas because of their impact on "legacy" data and the risk of losing backwards compatibility. Now that Philip & co. are doing wholescale changes, this will be a good time to change the ratings system (and the ratings based display) drastically. Since Philip has said that the primary goal of the site is "learning", my hope is that the whole ratings thing will be buried; the ratings based pages will not be prominent; he will come up with a better way of folks finding more interesting images; he will encourage critiques and in-depth discussions on the image pages.

    I wish we could re-start discussions on: (1) reviving "critique circles" or some such feature, (2) curators, and a curator-selected display on the front page, (3) Picture of the Day, etc.
     
  135. I think perhaps the requirement to post a critique of ,say, twelve words or more before you are allowed to rate an image would improve the credibility of the rating system a lot. This would discourage serial low ratings, and would require the rater to invest thought into why they want to rate as they do. This will, no doubt, cut down on the total number of rates given, but it will substantially increase the number of critiques exchanged and still allow a couple manners of sorting images based on subjective assesments of quality. This would also impact the "wow that's neat-o" and "blahblahblah" critique problem, as needing to type more than a couple words will encourage folks to actually say something reasonably intelligent that pertains to the image. I don't know if this change in requirements would have any deleterious effect on participation or quality, but I do know that as-is, the rating system is easily abused and only minimally useful. Thanks.

    -e-
     
  136. mg

    mg

    This post is specifically meant to explain a little better this 8th rating and idea of limiting extreme ratings.

    To retroactively correct all the negative effects that mate-rating has had on the TRP, all the site has to do is to dictate some standards for "excellent" (7), very good (6), etc. How ? By limiting the number of 7s and 6s that one single user can award. And by adding an 8th rating.

    A) Adding a new rating at the top of the 1 to 7 scale. A rating that would be like an award for an outstanding achievement. I called it "8" - but it should be a 10 or such, in fact. Doesn't matter for now... Main thing was, that this "award" could only be awarded by some people (not newly registered members for example, paying members only...), and could only be awarded a certain number of times by the same user. Example: Carl has been here, say, for 8 years; so he gets 3 awards to distribute each year - which totals up to 24. Once he has distributed his 24 awards to his 24 favorite shots on the site, these 24 pictures show up at the top of his favorite gallery; and the pictures who were on the receiving end of the "awards" he gave, will get a boost in terms of ratings as well.

    Incidentally, this idea of an "8th" award-like rating has loads of wonderful side-effects.

    1) New comers and non-paying members can't use this 8th rating, so it will be no point for anyone to create fake IDs.

    2) The more experienced a rater is, the more "8th" ratings he will be entitled to award.

    3) The 8s being limited for each rater, he will have to spend them wisely - just like you teach a kid to be careful with the limited money he gets from his parents.

    4) The introduction of the 8th rating will reduce the impact the 7s have had on the system so far. A picture that would have received a couple of 8s will end up much higher in the TRP than those which had only received tons of "friendly sevens".

    B) Another measure that would help a lot will be to limit as well the number of 7s and 6s available to each rater. And this measure would be implemented for the future AS WELL AS RETROACTIVELY. Those who used to distribute 7s like candies did not help the system, they destroyed it. Therefore they'd now have to pick their favorites among all the 7s they had awarded, and the others would automatically become 6s. Then 6s should be limited as well, and at that point, mate-rating will already be dead.

    All this needs some sophistication, but I just wanted to try to clarify a bit my earlier suggestions.
     
  137. mg

    mg

    "I wish we could re-start discussions on: (1) reviving "critique circles" or some such feature, (2) curators, and a curator-selected display on the front page, (3) Picture of the Day, etc."

    I'd strongly support this as well. Regards.
     
  138. This all sounds pretty convolutedto me. While I can appreciate the idea to try to level the feild a little, the truth is that some photographers are consistently much much better than others. While 'better' is somewhat subjective, the rating system is supposed to address that subjectivity by averaging many subjective opinions to give a guage of how a large number of people felt about an image. The problems crop up because not every image gets rated by all of the same viewers, and some images don't receive many ratings at all. If every image got 30 ratings, then a couple 3/3sor 7/7s would have little impacton its ranking. This, however, is not the case and not easily addressed.

    Adding another high rating to be doled out only by certain members, and limiting the number of 'excellent' rates a mamber can get is unlikely to solve much and places artificial restrictions on a system that is supposed to guage quality, albeit subjectively. It's a bit like saying that even if you have fifteen excellent photographs in your gallery, you are only really allowed to have nine, so the other six excellent photos are only allowed to be 'above average' or 'very good.' While it may seem like a nice idea to appease an artist whose images don't quite cut it for the 'excellent' category, it does little good to arbitrarily lump truly outstanding work with above average work because an artist is 'too good.' Thus, the mate-rating system cannot be addressed in this manner without destroying the credibility of the entire rating system by adding artificial weights and balances.
     
  139. The other part of this is that many people don't bother rating images they don't like. rather than hand out 4/4 oreven 5/5 ratings, they will usually hold out for what they really like. While these mambers' numbers may look like they're just overfriendly, they are simply showing their preference for certain images and declining to rate other images that don't measure up, in their opinions. Basically, the end numbers would look very much the same if these folks did rate average images 4/4 rather than skipping them. I think limiting the number of 6/6 and 7/7 ratings allowed from a member is likely to only increase the 4/4 ratings on lesser images, and thus compress the curve without any real effect on the overall distribution (except that some images that would have rated high will go unrated because a member is unlikely to give a lower rating than they feel an image deserves simply because they are 'out' of high praise for a period of time)
     
  140. Reading some of these hopelessly complicated suggestions has me wondering if some of you aren't putting the horse before the cart?
    The very first decision the site owners need to make is whether they want the TRP page to be filled with images that the general population of PN members likes or filled with images they feel will attract new visitors (and by default new members).
    I do not believe the two pages would very similar in content.
     
  141. Ugh sorry, cart before the horse...
     
  142. mg

    mg

    "It's a bit like saying that even if you have fifteen excellent photographs in your gallery, you are only really allowed to have nine, so the other six excellent photos are only allowed to be 'above average' or 'very good.'"

    I disagree. Firstly, that's another reason why there should be an 8th rating. This 8th rating would then allow you to to select a few "OUTSTANDING" pictures above the "excellent" ones. Secondly, in any fantastic artist's portfolio here, I will personally always have my favorites: the outstanding shots. Then, even among the ones that I'd be tempted to call "excellent" - and once left out the outstanding ones -, I will always have my preferences; those that I'll then prefer should be the ones I will rate 7s, the others should get 6s. Thirdly, please keep in mind, that nobody is forcing anyone to rate ALL the shots in somebody's portfolio. In short, my proposal is based on the idea that the site should encourage (and even force) people to be as discriminating as they can. Why so ? Well, for 2 reasons. A) Because "the ratings are for the site, not for the photographer", as we have been told; so the ratings MUST be useful to the site in order to sort images. Therefore people who "prefer everything" are actually no help to the site, because they can't state a preference. B) Because I believe it's very educative, for anone at any level to learn how to be more discriminating when looking at any set of images.
     
  143. Marc, your 8th rating sounds very Spinal Tap to me. But I do understand what you would want it to achieve.

    One problem area I can envisage is that it would pretty much skew some of the gallery views toward newer photos which had been given some 8's - whether that's a bad thing I don't know.

    David Meyer responded to my earlier post by restating his explanation of a persons ability to only rate images they perceived as poor, or conversely one which they perceived as good, I totaly understood his original point, but perhaps did not convey my thoughts in a very clear manner: Those being, I don't beleive that people adopt this type of rating regime. Hands up anyone who does - we can easily check.
     
  144. Ben, there are many different rating patterns and philosophies, high, low, and middle. Some of us have changed over the years, so the posted average you're looking at could be meaningless.
     
  145. I'm just clutching at straws Carl, don't shatter my illusions :)
     
  146. Straw's mite not be a bad Idea get rid of the rating system and Let every one get one straw aday the picture with the most straw,s makes it to the top of the TRP
     
  147. mg

    mg

    What does "Spinal Tap" mean ? Sorry for asking, but I don't know this expression.

    Besides that, you wrote: "One problem area I can envisage is that it would pretty much skew some of the gallery views toward newer photos which had been given some 8's - whether that's a bad thing I don't know."

    I think you are not being logical here, Ben. You would be correct IF there was a majority of "mate-raters" on photo.net, and IF these mate-raters had been here for a very long time. Remember that in my proposal, the number of 8s available to any single rater would depend on the number of years he's been around. Also, something I didn't mention, but which is obviously part of the plan, is that any user would need to have submitted the correct number of 7s (i.e. not more 7 ratings than what the new limit would permit)n before any mate-rater can use 8s.

    So, here's the calculation: let's assume there are 20% of mate-raters on the site - there are far less imo, but never mind... Lets also assume that these 20% of mate-raters would use their 8s to promote each other's images - which they are probably going to do... Then their 8s will be promoting pictures which are already in the TRP, whereas 80% of the members of this site will genuinely use their many 8s to promote the pictures they fell are worth it.

    80% vs. 20%... You get the idea...?
     
  148. Spinal Tap is a movie spoof about a rock band... one of the more famous concepts is to be louder, all their amps turning up to 11 instead of 10. One of those 'you have to be there' jokes.
     
  149. I guess I do not understand the significance of a numerical rating. Who cares? I don't. What I'd like to receive is a written critique from a peer photographer about how to improve the shot. Yeah, yeah, if someone gives me a 7/7 I smile. But on some of my photos, they don't deserve this. So officially as of now, I'm going to use "critique only." If you don't like my stuff, tough luck, you gotta write about it or ignore it, but you can't give me that anonymous 3/3 anymore.
     
  150. The easiest thing, I believe, would be to have a system that the criquer's average of all photos rated. This would allow you to calibrate a particular critiquer's rating. You'll be able to tell if that 3 3 is from someone who typically rates photo's lower than other folks or someone who rates higher (or possibly only rates photos that they really like).

    Even better would be to calculate the average and 3 sigma limit from the average but this is probably getting a tad too complicated. Can you tell I am an engineer? :) (PS visiting sister and logged in under her account Therese from MD)
     
  151. I have a simple suggestion, and i dont know if any one has introduced it so far. I know that this rating topic is one that dwells in all of our minds, so i suggest this:
    Make the rating process SO HARD so that it will deter the unseriuos rater. For instance, have like 5 criteria (asthetics, originality, DOF, ...) rated and beside each one have a space for the rater to fill an explanation for why the user doing the rating choose a specific value.

    I know it seems a bit naive, but in a sense it would be like a form u fill over the net and because they are asking for too much information, u ll just leave it and close the browser:) hopefully only the unseriuos raters will be the ones closing the browser.

    Hope that makes any sense:)

    regards,

    Rafik Kamel
     
  152. Looks like this thread is still alive and having read that far, let me put in my 2 cents worth as well on an unfortunately rather complex issue.

    I don't mind anonymous raters and what is this with people who just registered? One has to start sometime; and just because I just joined means I can't have an opinion? Why people find enjoyment in rating photos low is, I am afraid, beyond my comprehension. High ratings should be appreciated by anybody, or not? There is a complete stranger that tells you he likes your photo, so what is wrong with that?

    0) nomination is nonsense. What makes for a qualified rater? I can always go to the persons on photonet site and see what she/he has posted and then make my own evaluation on how serious I take the rating received from that person. If there is no post, well, that tells me something too.
    1) real names will not make the 3/3 go away.
    2) I don't see the need for more numbers, 5 is about as much as I can handle. FOR ME a picture is either below average, average, above average, way above average, or outstanding; that makes five.
    3) originality can go; this is useless the way it is defined. One point of view is that about everything has already been photographed once, so nothing is original. My view is, every shot is original since the photographer took it at that particular moment; the exact same shot cannot be repeated again.
    4) there will never be a "perfect" numerical system, so don't try for one. Having the choice, abandon the current one; it is not a real one anyway.
    5) having only experts rate the photo: besides the point. Asked for the aesthetics of a shot, all I have to say is that I like/hate it. "Objective" rating by experts would require such things as technical, exposure etc. and in the days of digital and PS, that's obsolete.
    6) not having the photographers name displayed when rating a shot is by far the best suggestions I have seen in this forum; this actually might accomplish something.
    7) I don't learn from an anonymous 3/3 rating, and I wouldn't from one that has a real name to it. That person's taste and mine are just different, so I don't care. Consequently, now only people who have posted in the same category can rate there? No, this doesn't work either; I may not be into, for example, fashion, but I can still say whether a shot is appealing or not. Why I would go into the category and rate every shot in sight I don't know, but apparently some people do.
    8) it might help to force people that want to leave a below average (and maybe average) rating to leave a comment as well. I am afraid that this will not terminate the abuse: a 3/3 will be left and the comment will be: don't like the shot. Hence the only solution seems to be to a "positive" rating system: on a scale of 1 to 4 (or 5), one could rate according to like, like a lot, exceptional, and far out there.
    I am only posting photos in photo.net for a very short period of time and admit enjoy most the verbal critique other give, in particular if it is helpful advise. I got occasionally upset with the 3/3 ratings and hope they will disappear. Without comment, they are useless and even with a comment, I don't care to learn that so and so does not like that particular photo and I certainly don't expect a constructive critique. Easy solution to the 3/3 abuse, abandon that rating. Then there is the abuse of the high ratings. I am afraid that I don't have an answer here; I just don't understand the people who create a thousand accounts just to rate their own shots high. I enjoy outstanding photos taken by somebody else and after all, at the end of the day, there is no gain from being on the first or the second page of TRP. I do, however, value the comments I receive from others.
     
  153. I want to add the following suggestion to fix the clearly broken rating system. Keep the anonymous ratings separate from the ones that were submitted with names attached and don't include them in the overall average. Using the "rate recent" or "rate category" features actually invites the 3/3 abuse. For example, one doesn't see comments, one cannot see the larger version, one cannot leave a comment, and apparently people think they have to rate every photo.
     
  154. As a former teacher, I did not grade or rate "anonymously".
    I also know that some of us go on "anon" and work through frustrations. That in my experience is a dangerous way to approach rating. I think that makes for a biased view [ whether high or low on the rating scale].
    One would think that if we want to keep ourselves hidden then we are not honest -or - truly have no idea how to back up our ratings. That being said, this is a real shame because the rating has no meaning and should not be given. If one does not have a way to back up their "opinion" then just don't rate it.
    I personally do not like the anonymous ratings but I signed up knowing that was part of the game played here.
    Let's not forget that some us may just give out 5's or 6's with reckless abandon.
    Sometimes it is hard to be objective if we know who took the photo or maybe we have an affinity for that photographer. etc. etc.
    Psychology has done many studies to show how we are influenced without realization.
    I read on a PN page - a photographer was disillusioned by the "mate rating". I am not sure what she means by that phrase by I suspect it is something like, "scratch my back and I'll scratch yours".
    No matter, it sounds like a very negative practice indeed and in no way represents a true critique to help one excel in artistic pursuits.
    Let's not forget:
    Art is the pursuit of excellence - not matter the commerce, opinion, or cost.
    I am sure I have vented on this matter before...
    vallery b
     
  155. ..Now that I have been on PN for a little while I see why people get upset over ratings...It is easy to tell yourself they mean nothing but harder to convince yourself they mean nothing....
    besides how do you stop "collusion"......
     
  156. I also agree that to stop the 3/3 abuse the rate should be abandoned and only the "positive" rates should stay. That way we get rid of the negative aspects the 3/3 ratings give - it`s not a good thing if it`s demotivating.
     
  157. What good does a 4,5,6 or 7 rating hold if there is no 1,2, or 3 rating? Not all photographs deserve an above average rating.
    I think the move for a "Utopian" rating system is a very unrealistic proposition. If all photos got the same ratings (or all GOOD ratings) then that would cause ill will as well.

    What motivates a person to create a better piece of art is different from the next.A passive approach to life never created a masterpiece.

    I am of the camp that ratings should follow with a comment as to why/why not -AND - this includes the above average ratings as well.
    Many above average ratings are based with comments e.g. - "I like your pictures, too". A solid constructive criticism is what we seek here. A good rating can be followed with criticism, a poor rating can be followed with the positive points. If you have time to rate many photos with reckless abandoned, then you have time to make a comment or two along the way.
    There are many ways to look at the ratings. Many feel they are too broad (too many numbers!). Less numbers would probably slow us down to really think about the rating we giving the photograph (not the photographer!).
    A shorter, succinct number system is easier but we would find fault with that as well.
    This discussion will never end on PN. It is interesting to see others opinions on this topic.
     
  158. The idea wasn`t to keep 4,5,6 and 7, but rather change the ratingsystem and what they mean so that we get fewer numbers and a more positive meaning attached to them. Obviously the 3/3 is unpopular because of its meaning. I certainly agree to that the rates need to balance each other.
     
  159. I was trying to say that if you have any rating system - one of those numbers would represent a below average rating. Any person that "abuses" the ratings system - will use that new assigned number for below average to wreck havoc in the same manner and the so it goes......there needs to be above and below average guidelines in order to gauge[sp] any rating.
     

Share This Page