3/3s and 4/4s--the inferior methods of skipping

Thanks to the great improvement made by Photo.net site workers,

without a doubt most users have witnessed the betterment of the rating

system. However, on top of the renewed discussions of advantages and

disadvantages, I think there is one persisting issue that must be duly

dealt�Xthe methods of skipping, because the pure intention to skip one

or two is not uncommon when the photos fail to interest the raters,

yet the raters may have chosen the bad methods of skipping by giving 4/4s.

 

When a rater reviews and judges a batch of photos from the categories

that actually encompass an enormous variety of topics, it is not

possible that all of those topics are interesting or meaningful to the

specific rater. Those photos may justify 6/6 or unfortunately, 3/3,

yet probably not a dozen of 4/4s, which dominate the calculation of

average rating and dilute the ratings of other raters to which the

contents of those photos are really connected because of either shared

experience, similar culture, analogous identify or so on. Sometimes

the rater may stop, lean back and ponder, other times they may simply

want to skip it and continue to rate the next.

 

Maybe we can make the button [>>] bigger and brighter, with a line of

notification like ��There is nothing wrong to skip by clicking the

button, really.�� Does any one have an idea about this?

 

 

Best regards,

 

Elsen

I think you'll find it all balances in the end. The 3/3s and 4/4s will bring the whole aggregate down, they're just saying 'this is average quality' or slightly-below. It would make sense now to reinstate the 1s and 2s, as those ratings are automatically silenced now. The images in the TRP won't change substantially, though, with more people rating, as it seems that numbers of ratings are rising pretty much across the board and so probably will affect the highest rated photos as much as the most average. Theoretically in a population ranging from way below average (1/1) to way above average (7/7) the distribution of ratings should resemble a bell curve, where most fall around the average (4/4), with fewer and fewer members deviating from the mean as you approach the extreme ends (1/1 and 7/7). Check this out:<BR><A href="http://mathworld.wolfram.com/NormalDistribution.html"> Normal distribution</A>

I see what you mean and thank for the website. However, according to what I learnt in statistics for business research, usually we would need a sample size of at least 30 to make the assumption of normal distribution hold and the "bell" shaped. I am not sure what percentage of all photos submitted have received at least 30 ratings.

Further, it is inappropriate to apply the assumption of normal distribution to a discrete 7-point likert scale.

In my long years on this website, I have never complained about the ratings - until now. I've read countless complaints, arguments, etc. about mate ratings, unfair ratings, etc. Well, add me to the list. I have uploaded several photos for critique in the last couple weeks, and I can't even get a 5 average, when I know my work is of higher caliber than "average". I will no longer be submitting for ratings critique - it will be critique only. Ratings have gotten out of hand and I will tolerate no more.

Jenifer, you should have stuck with your practice of not complaining about ratings. Your most common rating, by far, is 6, with 5 being the next most common. Your average is in the high 5-6 range. "5" is not average, it is "Good", and 6 is "Very Good". The vast majority of ratings *SHOULD* be in the 3-5 range, and you are among the rare people who are getting a majority of ratings above this. Being a good photographer doesn't mean that everybody is compelled to find your photos exceptionally good. It just means that more people will think them very good than otherwise. But people who think otherwise will exist, and they are entitled to their opinions.

I'm not usually a complainer, Brian. But when I get irritated to the point that this so-called rating system has gone on and gets worse, I'm going to speak up. I've never had a problem with ratings until the changes that have been inplemented recently, and I'm referring to the photos just since the end of March I've uploaded. The number of ratings has increased, yes, but so has the amount of 3s and 4s. My overall average is high 5s, yes. But if the 3/4 ratings continue, it affects my overall and will bring it down. I guess I'd rather have 9 ratings on a quality image than 20 ratings with an average of 4. This isn't about my low scrores. I know a lot of PN regulars have gone to giving a critique only and no rating anymore. I guess I should do the same.

Jenifer, you will be most welcomed in the Critique Only Forum. Unfortunately, management here is doing what it can to keep the Critique Only Forum hidden from view -- maintaining this moronic rating system at any cost.

Jen, if you know your work is of a higher caliber than "average," then don't even look at the ratings - are you shooting for you, or for thousands of anonymous visitors of this international community?

 

I've followed your gorgeous shots over the years, and they're definitely above "below average" and "average" when compared to what else is "on average" posted to this site.

 

But the way the rating system here works, as Brian has said over and over, it's merely a way to sort the tons of uploads each day for presentation - not a contest. Find that switch in your head, and throw it - the rating will stop bugging you. You have no idea if the person rating it has published 15 calendars of lighthouses, once fell off a lighthouse and crippled himself, or is a purist b&w shooter. All those people might not rate highly, and so what? Even if it's a petty rate just to be a nudge, blow it off. Means nothing.

 

At least the way the rating system is now, you're apt to get a few new peeks at your image, and maybe find another shooter to share and learn with.

"But if the 3/4 ratings continue, it affects my overall and will bring it down." Yahhhh! So what? You live by ratings you die by ratings. Why not just post images and have fun?

Jenifer shoots commercial, so I guess her ratings equal visibility to some degree. Not all of us can just shoot for fun all of the time, or not all of us have the ability to shoot and gain by promoting our photography. We don't all have the same goals.

 

Personally, I would not count on the ratings for visibility, but you have been around the site longer than I have and know it probably better than I do. Your clients are local people and probably don't even look at Photo.net.

 

The reduction in ratings values in the last few weeks must be some sort of anomaly. I equate the whole ratings thing to a pendulum swing. Its going into its back swing now, I think. Ratings should slowly even out and get back to 'normal' some day, hopefully. When a 5 really meant very good. A 7 has to very exceptional if graded on a real scale. Or, maybe I just dream.

I am also tired of this rating system.It doesn't upset me when I get 3/3 from an anonymous but the fact that he doesn't explain what is wrong with the picture is very annoying.No,I don't want to see his name in order to rate back in revenge.I just want to know why he rated it so low,so I can learn something and improve my photos.One more thing:I strongly believe that good photographers(professionals or not) don't rate anonymously.They would have no reason to waste their time doing this.And I also think they only rate and comment the really good photos and ignore the rest.As a conclusion,I guess those anonymous raters are persons that don't know so much about photography and waste their time rating photos.Of course,I can't know this for sure,but I am convinced that many of them have not taken any photos during their lives.

So correct me if I'm wrong

Ratings don't mean much. You shouldn't care about them.

 

 

However I've noticed that sometimes when I rate an image (directly) 6/6 or even 5/5 that image gets one or two more ratings (usually lower) instantly !

 

 

It's like photo.net has implemented a software to correct my ratings!...so for example if I like the images by Mr. X and usually rate them, the system probably assumes that I am a mate rater and add one or two additional rates to them to balance it out (based on a curve probably which would be flawed mathematically as well as based on algorithm (sp?) ).

 

 

I think I noticed the phenomena couple of days ago. I rated a photo 6/6 and it had only 14 ratings...but as soon I submitted my rating, it had 17 instead of 15 and the additions were two 4/4 ratings anonymously. So I tried it with another photo of the same person and it happened again....so from now on I either should stop rating Mr. X photos or poor guy actually will get a lower average and wonders why!

 

 

To be sure I just tried it again and it happened again...but this time after my voting it does show that the images has got "two new ratings" instead of "one new rating" but it doesn't show the amount of the extra rating even as anonymously . So the software is getting improved.

 

 

If this happen to be the case here (which sadly I think it is) then photo.net assumes, it knows the best and even its software has better understandings of the quality of the images than an actual viewer...that means not only the received ratings are useless but also giving ratings out as well. (for photographer the least to say.)

 

 

Do you think government does that in their voting system too?

 

 

"Well, we assume that majority of the votes in Florida should be republican so let's for every democrat vote add one or two automatic republican votes as well!" ...probably!?

 

 

I hope that I am wrong....I don't see how I could be.

I fully agree with Jenifer...If some people think that ratings doesn't matter, then why to maintain a rating system ? Isn't a rating system precisely set up to rank pictures and photographers ? If these don't care about then why don't they stay away of this discussion, and keep on shooting for fun and posting, but please, don't give lesson to people that think rating system can be a serious way to promote or judge their own work and improve their skills. I think PN is a way for photographers not just posting "for fun", to promote their work internationaly, and get some feedback and why not business opportunities ?

 

Then, when you post a photo, and after having pressed the button "confirm", you already have 4 or 5 anonymous ratings 3/3 or 4/4, in less than a minute, do you really think that it is serious and gives PN any credibitility...NO, definitely NO...

 

What Jennifer says is true, most of the best photographers (among which I don't consider to be...)have stopped posting their shots for rating critics...just critics only : I am speaking about many photographers of which we do not see the work any longer in the TRP : is it an improvement ? for me, NO, it is not...

 

Jenifer says " I will no longer be submitting for ratings critique - it will be critique only. Ratings have gotten out of hand and I will tolerate no more"...Great answer, Brian : "Jenifer, you should have stuck with your practice of not complaining about ratings "...so you ignore complains and are satisfied by yourself and by silent people ? May I remain you what is the forum subject, as indicated on the screen : " This forum is for questions, suggestions, criticisms, complaints, or just comments on the photo.net site itself, its user interface, and its mechanics "...

 

Then instead of always answering to complains, "you don't have to complain, my system is improved, be happy or go away...", you should ask yourself why all these complains are suddenly raising, what is wrong no that was not before....?

 

I never complained in the past, I found PN to be the best site for photo critics, learning, sharing its passion with serious people etc...Today, sorry, I complain with the rating system changes,OK...! and obviously ,I am not the only one...easy way to solve the point, Brian : delete this post, as you have done it previously on similar...

The rating system is a joke. Personally, I don't understand why anyone would rate through the rate recent queue. I've been using PN for 6 months and have never felt the need. The critique forum provides a mechanism for viewing and rating photos which allows you to actually see them. The RR interface compresses images and encourages instant judgement. On several occasions I have received two or three ratings just seconds after submitting. Now, I don't know how long it takes you to craft the work you present here but it would take me at least 10 seconds of contemplation to form an opinion on the overal quality of that work and i certaintly would not insult someone by giving a rating unless I was willing to really look properly. Yet a snap judgement of a compressed image seems to be the desired method of determining which are the highest rated (and therefore more visible) photos on this website. This raises the question as to who is it making the ratings. I suspect the ratings are given by fellow photography enthusiasts and not, say, car salesmen who happened to drop by. Therefore the administration are hardly to blame. They simply create the mechanism. That we use it on each other is our choice. I propose that that those complaining of the rate system don't use it or at very least don't use it via the RR queue which, i believe, is little more than an insult to the works. It is also worth bearing in mind that with the recent change in interface a photo will now receive a lot more of its rates from the RR queue than before. The ratio previously was about 50% RR and 50% Category view. Now it is two thirds from the RR. This will have the overal effect of increasing the number of ratings but reducing the 'score' received (ie. your sunset photo passing across more eyes which think sunsets are boring and are worth 3 or 4 at that most) Add to this the general confusion about the scale. you may rate 1 or 2 but 1 and 2 are suspended. The scale is 7 but really 5 but really 7 as the 1's and 2's are archived and may be restored at a later date-are you following. The rating system has some value as it is but nor much. Forget about it. Write as many critiques as you can and try to build a circle of like-minded individuals who will return the favour.

BTW- MH's comment above is disturbing. Would anyone from admin care to point out to him what's causing the strange effect his 6/6 rates have?

"Jenifer shoots commercial, so I guess her ratings equal visibility to some degree. Not all of us can just shoot for fun all of the time, or not all of us have the ability to shoot and gain by promoting our photography. We don't all have the same goals." You got me there. My goal is not to care about what some joker here on PN rates one of my photos, low or high. I am interested in how many people take the time to look at my portfolio. I would rather have them see the forest instead of just the trees.

"you should ask yourself why all these complains are suddenly raising, what is wrong no that was not before....?" How long have you been around?

"I hope that I am wrong....I don't see how I could be" Anecdotal evidence will trump an in depth statistical study every time.

Concerning M.H.'s effect, I looked at all the 6/6 ratings that M.H gave in the last 14 days.

 

April 9, photo 4166833. M.H's 6/6 rating was followed after an hour

by a 3/3 rating, and after another 3 hours by another 3/3 rating,

and after a further five hours by another 3/3 rating.

 

April 2, photo 4191568: M.H's 6/6 was a direct rating which was done

in the middle of a run of Rate Recent ratings. Apparently, the photo was near the front of the queue at the time. The 6/6 was followed 21 seconds later by a 3/3 and about a minute later by a 5/5. However, all the other ratings before and after M.H's 6/6 were 3's, 4's, and 5's. M.H's 6/6 was the outlier.

 

April 1, photo 4278804: M.H's 6/6 was followed 12 hours later by a 6/7

 

April 6, photo 4285667: M.H's 6/6 was followed more than 30 hours later by a 6/6

 

April 3, photo 4292341: M.H's 6/6 was followed 6 hours later by a 7/7.

 

April 4, photo 4295847: M.H's 6/6 was followed 3 hours later by a 6/6.

 

April 9, photo 4309035: M.H's 6/6 was followed twenty minutes later

by a 5/4. Most of the ratings on the photo were 5's and 4's. The photo was getting only a few 6 ratings.

 

Apirl 9, photo 4312982: M.H's 6/6 was followed 2 minutes later by a 5/5 and then ten seconds after that by another 6/6

 

April 9, photo 4322382: M.H's was followed nearly 24 hours later by another 6/6

 

 

There were several other 6/6 ratings by M.H on other photos, but in all other cases, these were the last ratings on the photos, and weren't followed by any other ratings at all.

 

If anybody can see in this anything remotely resembling the pattern that M.H describes, they are sharper than me. I can only find one case in the last two weeks where an M.H 6/6 rating was followed quickly by lower ratings, and that was a case where M.H. happened to rate the photo in the middle of its "prime" time at the front of the R.R. queue, and all the ratings before and after M.H.'s rating were lower because M.H thought more highly of this photo than just about everyone who rated it. So, the "evil" pattern that M.H saw fit to mention in the Site Feedback forum, comes down to one case and the evil explanation for that one case is that M.H was over-rating the photo, if you go by what everybody else thought of it. I think this is what one might call "anecdotal".

"In depth statistical study"? Really!? Where?

 

Show me the one that breaks down rates into characteristics of the raters instead of merely reporting gross numbers and averages.

The kind of characteristics of the raters that I had in mind would include factoring in the bona fides of the three 3/3 raters and comparing them to M.H.'s. People come to this forum begging for feedback from photographers who actually know what they're talking about, so to marginalize M.H.'s rate when in fact he may be the only one who understands what gives this photo some value makes a mockery of the whole process.

 

The number one photo at the moment on the default TRP is a close up of a flamingo with a black background - the work of the collective wisdom of the PN raters.

Carl, you are missing the point. Even if it is granted that the one 6/6 rating on the photo was the most "qualified" and more valid than every other rating on the photo, it was still the outlier rating on that photo and the explanation for the so-called pattern was simply that M.H. rated a photo higher than everyone else and that rating arrived in the middle of a run of rate-recent ratings, all of which were lower.

 

M.H. was "observing" that there was some kind of weird cancelling of his/her 6/6 ratings going on, and this turns out to be an extrapolation from one single case, which is only one case shy of complete hallucination or fabrication.

 

I suppose M.H. thinks that somehow someone is observing his/her 6/6 ratings and is jumping in instantly and down-rating the photos. That is just flat-out stupid. Sorry to be blunt. Or maybe M.H. thinks that the site has singled him/her out and is running a robot to cancel his/her 6/6 ratings by creating bogus 3/3 ratings. Or maybe he/she thinks the "site" has predetermined how photos should be rated and compensates automatically for any ratings that are out-of-line.

 

Why the site would need to create bogus ratings to cancel out ratings that it deems illegitimate, I don't know, since we make no secret of the fact that we simply delete ratings where there seems to be evidence that they are illegitimate. I don't know why M.H thinks the site would create an elaborate charade of a rating system if it has enough time and people secretly to look at every photo and decide where it should be in the Top-rated photos.

 

But people seem to want to think that the rating system is some kind of evil hoax by me and "the site". In that case, my bringing forward inconvenient little things called facts is going to be dismissed. If I can fabricate ratings to cancel M.H's 6/6's, I can invent facts to contradict his/her "observations", people will say. So why even bother?

I'm not missing the point at all. I know full well that crying "bots" is silly, but it comes from a place where the validity of the ratings in terms of which images get rated at what values is often way off kilter. Otherwise they wouldn't come up with these conspiracy theories. The image with the three 3/3s and the flamingo image are not isolated examples.

Carl, I've got to agree with you about the flamingo. However the seagull which is 5th or 6th is even worse. Yet it has over 50 ratings averaging out at 6/6. People just don't look before they rate. Which makes me wonder why they bother.