Another ratings change: Goodbye to individual ratings, hello averages

[steveshinn]

<p>Oops, guess I'm getting old. Looks like I'm repeating myself... goes with being near 70 I guess.</p>

[joshroot]

<blockquote>

<p>Oops, guess I'm getting old. Looks like I'm repeating myself... goes with being near 70 I guess.</p>

</blockquote>

<p>It looks like you clicked "back" in order to edit your post. That results in a second submission of the same (or almost the same) post.</p>

[ardenpress]

<p>Steve,<br>

I sent Josh an e-mail with the same information , but only about the 1 and 7 rater. Haven't come across the other...yet.</p>

 

[snigam]

<p>One of my image was rated by somebody who has never posted any image, the profile says nothing and he has given out over 1000 ratings. </p>

<p>What is to prevent somebody from creating 10 different yahoo email addresses, create PN accounts with those addresses and start rating his own photos. I would think if somebody has never received a rating, system should not allow him to rate others.</p>

[lex_jenkins]

<blockquote>

<p>"What is to prevent somebody from creating 10 different yahoo email addresses, create PN accounts with those addresses and start rating his own photos."</p>

</blockquote>

<p>Admin (which doesn't include me) has access to all account info and will fix those problems as they occur. Problems with actual, proven abuse are routinely solved.</p>

 

<blockquote>

<p>"I would think if somebody has never received a rating, system should not allow him to rate others."</p>

</blockquote>

<p>Why? That opinion comes up often. How, specifically, would it influence your opinions and actions in regard to the ratings system and top rated photos?</p>

<p>This isn't a juried competition with any prizes to be won. So it's not comparable to a juried competition in which the judge or judges will usually have accomplished a body of their own work that has been recognized as critically successful.</p>

<p>There's nothing to be won or lost on photo.net's ratings system. In fact, over the years the system has been modified to help ensure that even lower rated photos receive some visibility on the TRP. I've tested this several times by submitting my own photos for ratings. Even when my average ratings were less than 5 my photos still appeared on page one of the TRP when using the category specific view option (usually under the street, documentary or other categories to which I submitted the photos for ratings and critiques). So the only thing to be gained from higher ratings is a warm fuzzy feeling of being praised by ones peers.</p>

<p>Asking viewers to have their own body of work displayed on photo.net before being allowed to rate photos would be akin to American Idol contestants demanding that every audience member stand up and sing a song before being allowed to vote.</p>

<p>If something of material significance was available to be gained or lost through the ratings system and TRP then, sure, it might be reasonable to expect that viewers who rate photos also have their own portfolio hosted on photo.net. But the ratings system and TRP have never been anything more than a rough gauge of popularity.</p>

<p>And if the goal is greater web prominence - which does have a certain material value in terms of internet currency - the ratings system and TRP have less influence than you might believe. Some of the photo.net members with the highest Google visibility are not those whose photos receive high ratings. The most visible members are those who understand how Google (and similar search engines) works. For example, a photo.net member who wants higher Google visibility for his/her travel or stock photographs would be better off using tags and informative photo descriptions as keywords, along with constructive participation on the forums and written critiques. Worrying about high ratings isn't the most effective way to translate photo.net activity into the sort of visibility that may enhance the material value of your portfolio. For that, a member might be better off entering legitimate juried competitions where placing highly in the rankings can look good on ones resume.</p>

[derek_hofmann]

<blockquote>

<p>When I looked at the "photos rated highest by this member", I found that <strong>all of the 7's in both cases were only of nudes</strong> and, IMHO, not very good ones at that!</p>

</blockquote>

<p>That's a good reason to use a ranked pairs voting system. Rather than assigning a numerical score, the person would indicate which of two photos he or she likes more.</p>

[lex_jenkins]

<p>Ranked pairs voting is similar to the ELO grade type system that's been proposed. It's a worthwhile goal but doesn't resolve the most immediate problem. Such a system might be worth considering more seriously if used in conjunction with a ratings "competition" in which there is something tangible to be gained for participants.</p>

<p>A simpler way to resolve perceptions of unfair, inappropriate or uninformed ratings would be to sample a larger group. Again, let's go with the presumption that most photo.net members are here because they're fans of photography, not fans of trolling. That being the case, a larger sampling of the photo.net population at large should present a more representative rating. If members who asked for ratings were required, in turn, to rate a certain number of photos first, the sampling group would be closer to statistically significant.</p>

<p>And if you still don't like the ratings you're getting it probably means nothing more than "Take more photos that appeal to popular taste." Since I don't take many photos that I would expect to have popular appeal, I would be surprised and somewhat disappointed if a statistically significant sampling of photo.net members rated my photos 6 or higher. Our membership is too diverse to expect certain genres of photos to consistently receive high ratings.</p>

<p>Meanwhile, prohibiting ratings by members who don't have portfolios will only serve to decrease the sampling group.</p>

[gregverena]

<p>Well, one more comment on the continuing saga. Out of curiosity,I was clicking on the names of those who rated one of my pics. One, Justin Goodwin, has ratings he gave either all 1's, 2's and 3's and 7's. No 4's, 5's or 6's. All the 7's were nudes. In other words, he doesn't like anything but nudes. Which is fine. He also has posted no photos at all. Which is also fine. That's the way it's set up.<br>

But. Now when I see his name, I will figure he gave a low mark and therefore skewed the results. This is good, except I don't know if there other similar folks since the ratings are not linked to the names.<br>

If, as Josh's stated goal is to give a rough idea of how a photo is received and the way he feels to do this is to only allow an average to be viewed, I have to point out that in cases like this ( a compulsive 3 rater for no apparent reason) it skews the average to the point of making it useless.<br>

To solve this, simply link names to ratings, then the ratee will immediately know whether to give any worth to a person's rating and would have a better idea of the real overall reaction of raters.<br>

Again. There is no financial value to these ratings. Not even ego. But if they are to be of any worth, in other words, is what I did here pleasing to somebody? not? so so? the average has to have some semblance of meaning.<br>

I know Josh, you have stated your mind is made up and you won't be influenced by the masses. And voting is sometimes overrated ( viz. congress and dwts!) but still, think about it. Please. It was a nice try, and it seems to be a good step by adding names, but one more step of adding the ratings given by those names (can't be that hard) would solve a lot of problems.<br>

With all due respect,<br>

Greg</p>

 

[joshroot]

<blockquote>

<p>To solve this, simply link names to ratings, then the ratee will immediately know whether to give any worth to a person's rating and would have a better idea of the real overall reaction of raters.</p>

</blockquote>

<p>Allow me to more or less repeat the answer I gave previously to the same question:</p>

<p>This has been tried and was a spectacular failure for many many people. The response was harassment, insults, revenge rating and general childish behavior. It drove people from the site and generally made life unpleasant for anyone who dared to try and give an honest opinion that was anything but a high rating.</p><p>

I suppose one can "never say never", but I can say with fairly high certainty that as long as I am the one running photo.net, we will never return to such a system.</p>

[Michael R Freeman]

<blockquote>

<p><em>"To solve this, simply link names to ratings..."</em></p>

</blockquote>

<p>Been there, done that. As Josh says, it was a <strong>spectacular failure</strong>. In fact, such a system resulted in what was essentially a hijacking of the "Top Photos" pages by a select clique of ego stroking members; and harassment, revenge ratings, and sabotage followed for many who were not part of the elite ass-kissers fraternity. That's why we have the current system.</p>

<p>Any anonymous system is open to abuse, but so is any system, and the names-linked-to-ratings period was subject to blatant abuse. The current system, while not perfect, allows for honesty without reprisal and discourages dishonest mate-rating abuse. Josh and company have tools / are developing tools to root out the abusers (ratings are not really anonymous), so let's give the system a chance to work.</p>

[gregverena]

<p>OK Josh - I get it - it is sad that folks react like that making the system less viable for others.<br>

I did want to say, though that the real angels are those who take the time to leave a comment and the arch angels are those who actually give a constructive criticism - that's why we're here - to learn.<br>

Here's an idea, how about a more comprehensive rating system instead of just like or not like, how about a range of clear criteria e.g composition , focus, mood, lighting, balance, color rendering, b w rendering. you get the idea. All rated on your scale. Could be a box to click like on surveys. I'm not a pro - somebody with real background would know what one looks for. That way, name or no name, you would at least know why the 3's or 7's were given (why scale of 7 anyway?) Surely a programmer would be able to figure out how to do this without too much trouble. Then you could even drop the names or leave it as an option for the rater.<br>

Just an idea.<br>

Cheers,<br>

G</p>

[derek_hofmann]

<blockquote>

<p>Ranked pairs voting is similar to the ELO grade type system that's been proposed. It's a worthwhile goal but doesn't resolve the most immediate problem...to resolve perceptions of unfair, inappropriate or uninformed ratings...</p>

</blockquote>

<p>Sure it would, because you can't have unfair, inappropriate, or uninformed ratings if you don't have ratings at all!</p>

[steveshinn]

<p>Just a reminde;<br>

Let's not forget that the current system DOES allow anyone who wants to put the time on it to know, rather exactly, who rated what number by simply observing each successive post of each rater and subtracting the old average times the old number of raters from the new average times the new number of raters.<br /> <br />It is being done, identities are known and yet there are no riots in the streets.<br /><br />We HAVE, however, discovered some rather nefarious characters who seem bent on changing this site to "<a href="http://porn.net/" target="_blank">porn.net</a>".</p>

[joshroot]

<blockquote>

<p>Just a reminde;<br /> Let's not forget that the current system DOES allow anyone who wants to put the time on it to know, rather exactly, who rated what number by simply observing each successive post of each rater and subtracting the old average times the old number of raters from the new average times the new number of raters.</p>

</blockquote>

<p>Incorrect. That only works if you aren't rater 1-5, as no average is shown before there are 5 ratings and those names are all released at once. And as soon as I finish checking it out, raters 6-∞ will be released in groups of three to stop the childishness that has once again reared it's head at the slightest possibility of tracking down someone who gave a low rating and harassing them for that fact.</p>

[steveshinn]

<p>Good move but I'll bet that they will still resort to mindless calculations, etc ... ah, the drama of it all... <br>

I hesitated in posting the current math games on this public forum but the practice seems, from what I'm hearing, pretty well known. Interestingly enough, I have heard no instances of threats, retribution or the other childish crap that was going on before the recent changes.<br>

I have received very respectful emails asking why I rated some particular image as I did and am now trying to take the time to include the rating in my comment section. I might get blasted for it but I never figured that I would ring the gong with my stuff anyway... unless I start shooting really raunchy nudes.</p>

 

[joshroot]

<blockquote>

<p>Interestingly enough, I have heard no instances of threats, retribution or the other childish crap that was going on before the recent changes.</p>

</blockquote>

<p>I have. Within days of the change being made. Multiple instances.</p>

[lex_jenkins]

<p>Derek, you must have studied journalism. You've managed to top some of my own efforts at quoting comments out of context through clever use of ellipses. Nicely done. You should write movie reviews.</p>

[bmm]

<p>I think I've seen it all in life, but sometimes things make me realise just how naive I am.</p>

<p>Seriously Josh, people are doing calculations of averages in order to persecute low raters as they appear? Wow.... thats tragic. Now starting to realise what you guys have on your hands, and it makes me appreciate even more what you deal with for us.</p>

<p>Going back to the previous discussion of 'remediation' - that whole attitude really troubles me too because it is exactly upside down. How convenient that if a photograph is rated average or low, within this philosophy, it is the fault of the rater rather than the fault of the photographer... hmmm doesn't work for me. And even if it made conceptual sense - which it doesn't - how do you 'train'/'remediate' while not imposing a awful standardised banality and formulaic approach to appreciation of works posted on this site.</p>

<p>Just some remarks in general... but as both a giver and receiver of ratings that are more or less arranged around a mean of 4.5, with expected bell-curve shape, I almost can't believe that this isn't just the accepted sensible norm that the vast majority expects.</p>

[jcg23]

<p>so I thought that the average would be marked as 5 until 5 marks were obtained. This isn't the case, it looks like my averge is changing every time I get a new rating. In both cases the number of raters were under 5. It would help if things were kept consistent</p>

<p>Thanks..</p>

[joshroot]

I'm not sure I understand what you are saying Jane. Can you elaborate a bit more please?

[jcg23]

<p><strong>Josh see this.. you aren't suppose to get an average until 5 ratings have been given</strong></p>

<p><strong>''Why isn't the average showing up yet?</strong> - The average (and the names of those who have rated your image) will not show up until there are enough ratings to be meaningful. Currently this number is 5.''</p>

<p>I see my average being calculated even after 2 and changing after that.. so hence the question..</p>

[jcg23]

<p>FYI<br>

I just had a look on the rate photos section and even though each photo had one rating they had different 'averages'. </p>

[joshroot]

<blockquote>

<p>I see my average being calculated even after 2 and changing after that.. so hence the question..</p>

</blockquote>

<p>Ah, I understand now. You must be talking about rankings on the "Top Photos" pages.</p>

<p>That is merely a result of the fact that this is a large complex system and we can't change everything at once. We're working on it and things will straighten out in time. But they may act a little strange while we do so.</p>

[Guest]

<p>Hi Josh,<br /><br />since some day's I notice very high ratings for nude images. Because this was curious to me I did some investigations. Thanks to the new list of each rate I found out that there are a lot of new members which have no photos uploaded and they gave only 7 ratings across the nude rating section. <br /><br />Is it possible to disallow the rating for new members, especially if they have no own photos in the rating. I good way could be to count only the ratings of members with more than 100 ratings given with an average between 4 and 6. They should also have received 100 ratings for you own photos with an average within the same range. <br>

The ratings of new members should be flagged to exclude it from the average of a given picture.<br>

But they should count for her own statistic from the beginning.<br>

Such abuse destroy many for photo.net, because it's rating system is one of the best, in any case the best I know, and the main reason for me to post my photos here.<br /><br /></p>

[bjcarlton]

<p>I like these changes, even though it's been awhile since I sought numerical ratings, and even longer since I gave any. I eventually couldn't decide what they meant when I was giving them. But I write because something way back earlier in this thread caught my eye, which was the idea that the ratings system might be a good way for someone -- a rater -- to develop a critical eye. For the reasons stated in the thread, I doubt that any tweaking of the ratings system could accomplish that. However, I'm wondering if it might be fun to come up with a different sort of feature that would be designed to encourage critical debate. What if you had a panel of volunteers (you could vet them for qualifications) who would agree to criticize (and especially debate) the merits of an image. The panel would take an image (say one a week), and the members would each offer an opinion on it with respect to its photographic merits (or lack of merits). There might or might not be ratings; the important thing would be the specific explanations of why each panelist thought what he or she thought about the image. Once the image and its comments were posted, others could enter the debate as they are in this very thread. Perhaps people could submit their images for possible selection by this panel, or perhaps the panel could select images on its own (then seek permission of the photographer for a critique). Panel members could be rotated every so often to keep things fresh. It would be like the POW, only with an institutionalized, and hopefully illuminating, debate.</p>