An Answer to the Ubiquitous Frustration of Ratings?

Seems everyone on PN has a love/hate relationship with ratings.

 

I teach photography to high school and junior high students in Tempe, Arizona.

This year I incorporated as part of the curriculum a critiquing session on PN.

I have been astounded and bemused at some of the ratings as well as having

gained some insight as why the not-yet-mature mind comes up with the ratings it

does.

 

First, they take the criteria of "esthetics" quite seriously. Most haven't even

heard of the word, so we look at a dictionary definition. Now if the image is

a well-executed, well-composed composition of an old lady with exquisitely

wrinkled skin, it almost always gets a low rating. Why? "If she is ugly, then

the picture is ugly." By definition esthetics equates to beauty. Perhaps skin

deep matched to skin deep thinking. Hey, they're kids! Pretty girls in

bikinis get high ratings almost without regard to composition; beautifully

composed, deeply moving images of the homeless man gets twos and threes. Sigh.

 

I put together a display of about thirty top rated PN images of "ugly" people

and "ugly" scenes, and called it "beauty where there is none." It kind of

helped, but the immature mind makes few judgments with the frontal lobes and

relies instead on the impulsive amygdala. Frontal lobe isn't fully developed in

most people until about age 25. Like I said, they're kids.

 

"Originality" becomes a short term memory, precedent-based knee-jerk reaction.

Somebody is doing a study on smoke trails and offers two or three images as a

thematic contribution, the latter ones score lower on originality than the

former ones in spite of my protests.

 

Having been a member of professional associations in the days when I was

operating the family studio, I am used to a different set of criteria. Instead

of esthetics, we judged more on impact. Instead of originality, style was more

the issue.

 

Then my brain went into brainstorm mode, almost involuntarily. How do we bypass

the less mature, amygdala-centered judges? By qualifying the judges! Why not

have a set of moderators take applications from those (who are subscribers

perhaps?) that demonstrate in their own portfolios at least a modicum of skill.

This is exactly how the professional organizations do it.

 

I am thinking that maybe the judges would be obligated to comment at least on

scores very low or very high, maybe even on every image.

 

But who should have their works judged by this elite panel? Maybe as a revenue

source, perhaps only subscribers should get the benefit. Qualified judges would

get icons by their names. This would encourage both subscribing and

participating.

 

I'm not saying to do away with the old system. I am saying that the two systems

can peacefully co-exist.

 

Any thoughts?

The ratings are on 2 things-originality and beauty. There is nothing for how good it is overall, nothing for how difficult it was to get, or how important a moment it was to your family. So if a picture is poorly done of a boring subject with the wrong exposure and the camera shook during that exposure, just how high a rating does a photographer expect? I read comments like "I put a picture up and immediately it got a 3." Well that is not a surprise. Most of them are barely worth a 3 rating. There are a lot of great and very good pictures up for critique, but so many are so bad you wonder why people are complaining about the ratings. If you don't want honest opinions, don't submit your work.

I don't buy your amygdalae conclusions as there are simply too many exceptions among the young throughout history. (A partial list on request). Besides, I find people who use big words to be mordacious.

Bruce, you are certainly entitled to your opinion here. However, I am observing two things: 1) your ratings are atypical in that you have given almost a flat line, non-gausian distribution of ratings between 1-5. 2) You haven't demonstrated you know anything about making a quality image because you haven't posted any. Seems one could conclude you can critique but can't perform, doesn't it?

 

Art, the amygdala statement is a matter of neuroscience and pedagogy. Those who don't buy it are simply not familiar with the latest research in those areas. That, and they haven't been exposed much to the adolescent mind as say, a high school teacher?

I was an English teacher before I turned photographer, and dealt with MANY adolescents, and I still disagree.

Children should be kept in barrels and fed through the hole until they are adults. Then the hole should be stopped up.

Bruce, while I agree with your assessment that most photos getting low marks deserve them...and the people submitting sub-par work shouldn't be surprised when it gets the rating it deserves. The problem lies in the fact that all one has to do to become a qualified art critic who stands on equal footing with the finest photographers on this site...is to be able to somehow find the site, or have someone find it for you if you're too smoked up on some "really good dope...man", and have enough focus and energy to push the little key that has the nice "3" on top. Can you honestly call that an "honest" critique? Unfortunately, the more correct statement would be, "if you don't want a DISHONEST critique, don't submit your work". Even pedophiles get to come face to face with their accusers. On this site...they remain lurking in the shadows. What's most disturbing to me is that they are ALLOWED to do so. And I really don't care to hear any further lame excuses about why nothing can be done. As I've said before, all that lacking to change is a LACK OF WILL on the part of those who run the site.

P.S.- Bruce, I took the time to look over the ratings you've handed out and found nothing that would indicate that you're anything less than fair...not that you need or care of my opinion. I'm curious - the comment someone made concerning your own lack of submissions of your own photography. May I ask, why not? Whatever the reason, I don't think it devalues your ability or right to critique. You seem just as quick to compliment good work as you do to point out poor work. I'm just a novice photographer, but I don't hesitate to assert that although I may not yet be able to make a photograph that would deserve high marks...I've been a lover of the art since I was a wee child and put considerable time and thought into the process before remarking on the work of any photographer...equal weight given to novice and master. Since I disagreed w/ some of your comments, I wanted to be clear that at least to some degree, I can't go along with some of the comments made by others...about you. I don't know your credentials, I haven't seen your work...but at least I know your name, which you attach to all of your comments/critiques/ratings. I commend you for your HONEST critiques. I would bet that if I receive 3s from you on a future posting, I will know why and that's all i need. I need to be challenged in every respectful manner to constantly assess/reassess my work...not so that it will eventually meet up to some one else's standard...but to be certain that it meets up to what I am trying to achieve w/ my work. Best regards, John

"Even pedophiles get to come face to face with their accusers. On this site...they remain

lurking in the shadows."

 

 

Are you saying there are pedophiles lurking in the shadows of photo.net ?!?

I suppose a group of "qualified critiquers" would be preferable to the random, anonymous bunch doing it now.<P>

 

I've been a member here now for, I dunno - maybe 5 years? I never rated a photograph that I can recall. Why? Personal feelings about the whole thing, I guess. But the big two reasons are that I don't feel particularly qualified and I'm not convinced it helps.<P>

 

What may sometimes help are useful comments. If I don't like someone's photograph I just don't leave a comment (let alone a rating) because someone else might love it.<P>

 

To me, it's a lot like music. We all have different tastes and critiquing it is almost crazy. It's funny - I used to read Rolling Stone Magazine and for a while there, if they gave an album a crappy review, I just might buy it because I so disagreed with the stuff they liked.<P>

 

<I>As I've said before, all that lacking to change is a LACK OF WILL on the part of those who run the site.</i><P>

 

Have you considered lack of funding, the coding changes to the site, who at Photonet would drop their existing responsibilities to organize the proposed group and keep it staffed, would this person be paid, if so, how much? Would the reviewers be paid, if so, how much? Etc, etc. I suppose you could contact the owner if you have the time and inclination and experience. It *is* easier said than done to change it.<P>

 

Lastly, I'm curious Steve and I'm honestly *just* curious: Were you concerned that the parents of the children in your class might object if their kids were exposed to nudity on this site while taking your class? Again, I don't ask from a puritanical bent at all and I think your class sounds pretty cool. I just know that, where I'm from, a teacher might get in some pretty hot water over their students viewing some of Photonet's content in such a class. Good luck!

You've got to be kidding...i hope...Robert. I was trying to convey that on this site anyone can "accuse" us of bad photography and there's no accountability. I was in no sense literally suggesting that there are pedophiles on PN...but I'm sure there are the same percentage of pedophiles here as there are in the general population. Were you offended by that... or just wanting clarification Robert?

No one has ever given me a reasonable explanation why the anonymous ratings can't simply be done away with. The "standard" answer is that if we identify ourselves and give someone a low rating then they may retaliate and get their buddies to do so also. That sounds like a valid assumption,but giving a low rating isn't the only reason someone might want to sabotage someone. It may be a personal issue completely away from this site. It may be because of a comment in a forum...and many of the low marks probably just come from some neurotic nut-job. If we know by name who is giving us what we perceive to be undeserved low marks- marks that aren't consistent with the majority of the critiques, perhaps there could be some type of review by staff. But even if this would be too complicated and unfeasible for PN to do...just knowing who is giving me the marks is enough for me. If I see of pattern of unfairness I would know to ignore the remarks. I saw a photo in here today that received 33 7/7s out of 64 ratings. Most of the other ratings were 6s. One lone person rated it a 3/3. So, let's just say that person gives me a 3/3 and the majority gives me an average of 5.0 or better...and I see that this guy rated the above mentioned photo w/ a 3/3 as well, it's pretty easy to see a pattern. One more thing - require a comment for any marks below 4. If a rater has to take the time to write a minimum of one sentence for originality and one for aesthetics, most of the neurotics was cease. Repeatedly pressing one key takes little time or effort. Let them start writing several pages of comments every time they want to screw with people and it would come to a grinding halt. "Lack of will on the part of those who ruin this site"...yea, when monkey's fly out our collective asses.

"You've got to be kidding...i hope...Robert. I was trying to convey that on this site anyone

can "accuse" us of bad photography and there's no accountability. I was in no sense

literally suggesting that there are pedophiles on PN...but I'm sure there are the same

percentage of pedophiles here as there are in the general population. Were you offended

by that... or just wanting clarification Robert?"

 

 

I was part kidding and part offended by the incredibly inappropriate analogy. If I read it

right, you were not talking of wrongly accused innocents, and you were comparing the act

of someone [i presume this would normally be the victim] reporting a child-abuser to

someone anonymously giving a low rating to a photo on this site. It seems like you are

saying that someone has accused you of taking a bad photo, and that if you were a

pedophile then at least you would know who it was who had turned you in.....

 

It's pretty crass.

 

As to the "problem" - If you look back over the forum comments, about every tenth post

since the year dot has been along the lines of "3/3 RATERS SUCK - HOW TO DEAL" or

similar. It used to be 1/1 ratings. Then they changed it. The way forward is to learn not to

care about it. To me, all of my photos that I love are 7/7 and i don't give a **** what

anyone else might rate them.

 

 

RX

The "system" of anonymous ratings isn't fixable because there's no accounting for taste. Which is worse, the 3/3 guy or the 7/7 guy ? Of course the 7/7 guy is going to get a lot fewer complaints. I agree with the OP that the only legitimate system would be one that employs a panel of judges, or, just eliminating the anonymous ratings altogether.

 

Perhaps we could take a cue from Fred Miranda's site where photos are submitted in the appropriate forum and all critiques are posted in that thread, no anonymous comments and more constructive criticism with the chance for the poster to respond.

 

Or, stick with the current system and just not worry about it, safe in the knowledge that most of us really aren't qualified to critique anyone else's work. Like Robert X said, I take pictures for myself.

Good luck.

To John Gaylon and Steve Lowther (and others): Thanks for your comments. I rate the photos because that is the only way I have discovered to see the next one. Also it helps me to be self-critical about my own work. If you want to see my pictures they are on my site www.brucescameras.com. I did not mention this before because the monitor on this site removed some of my previous postings that listed it. I do not pretend to be a great photographer, but am a pretty good one who made a living shooting for many years. I am sorry that I rate so many pictures so low, but if a person posts boring or badly done pictures, I cannot lie about it and say "great work". That person will only grow with helpful criticism. You are welcome to post a rating of my pictures on this forum. Rate them as low as you feel justified. I will not be offended. Being of the older generation I cannot put them on because I do not even know enough about how to work a computer to do so. I do know that since I shoot film each one would have to be scanned first, and I do not have any interest in doing this. They lose too much, as you can see from the photos on my site. Hope I have not offended anyone. That really was not the intention. I do not teach formally, but have several photo and painting students who visit me with their work regularly for help. FYI: I am a retired photographer who runs a small camera store for the fun of it.

John-- Don't you find the ratings where people do leave their names often as biased as the

anonymous ratings? I often see ratings that have gotten 4s and 5s anonymously, and

seemingly quite fairly, get 6s and 7s by named people. Then when I go to those named

people's portfolios, I see them getting 6s and 7s abundantly in return. I rarely hear

complaints about that.

 

I know it's easy for all of us to complain, but while we're at it, why not only allow each of

us to rate in 3 or 4 categories. It's pretty clear that, for the most part, landscapes and

flowers overall get more and higher rates and more "critiques" than do street,

documentary, and portrait work. I love landscapes but also have come to realize that

people with limited esthetic experience assume that

automatically a pretty sunset is to be rated higher than an old woman, no matter how

much more uniqueness and character the old woman has than the pretty sunset. Those

folks who do portraits and are familiar with what actually makes a good portrait should be

rating the portraits, not those who think beauty is limited to oversaturated skies and

threatening clouds.

I'm new around here... posted my first photo 3/12/07 so I'm still getting a feel for it and

wondering about paying for a subscription. My only previous experience of membership

on a photo site is flickr. My first impressions of photo.net were that it would be a step up

from flickr in terms of the quality of photos AND the quality of interaction.

 

Regarding the rating system, it seems a bit too simplistic. Particularly originality. It seems

to me that the vast majority should get a an average rating. Regarding low ratings, I know I

have gotten my fair share! I'm far from professional... I'd consider myself an enthusiast

that has advanced a few steps beyond typical snapshots. I don't mind a low rating but

would be very helpful to know why and that brings me to comments/critiques. I see far

more ratings on photos and very few comments/critiques. This seems to be the norm and

it's not very helpful. I know it is easier to rate than it is to critique and I have done the

same. That said, I've tried to comment at least 3-4 photos for every one I upload to the

critique forum. Ideally I'd like to make sure that at least 50% of my comments are critiques

that offer helpful information. If I am going to rate a photo below 4 I'm going to offer

critique as well because that's what I would like for low ratings on my own photos.

 

I guess it comes down to quality of information which reinforces sense of community.

Simple number ratings don't do much by themselves but they are more convenient...

they're cheap. I'm not familiar with other photo sites and how they implement rating/

critique systems, I just know that I don't care for this system and I think it could be better.

It seems to me that photo.net has built an infrastructure to offer a meaningful medium

that will encourage the development of the skills of photographers but that it needs

refinement. Would it be possible to combine the form for ratings with the critique form. If

both are offered with the same click more people might be inclined to offer critique. What

about requiring comment/critique for ratings below 4?

Good comments. 1-7 is not enough. I often wish for a 5 1/2. Also some credit should be given for difficulty of getting the shot. For instance if someone came up with a picture almost impossible to get(like an eagle eating a hamburger in flight, or Saddam Hussein kissing a nun) although the exposure might be off and the quality poor, the photo deserves more credit than it might get in the present system.

First...Robert, let's try this one more time. I was saying that there are those w/ dishonorable intentions "lurking in the shadows", taking advantage of the fact that they can rate anyone on any shot as low as they please and there's no accountability. Perhaps I should have said persons "accused of being a pedolphile", but then anyone facing legal charging is assumed innocent until proven guilty ...etc., etc., etc. Perhaps I could have said, not that pedophiles "have to face their accusers" instead of "get to face their accusers". But I think that if you want to use phrasiology (is their such a word?) and semantics as a reason to get your panties in a wad...then you'll have to deal w/ that. No one else in the room seemed to be confused about what I was saying but you. As to the comments about my comments being crass... I think it's not so much crassness but lack of political correctness that you're offended by. This is the big boy room. If you prefer the little boy room, I think it's down the hall, through the door with the picture of the sandbox on the front. You're attempting to be far more philosophical than is needed...reading far more into my comments than would be thought possible. It's not just about me and my photos, it's about how everyone's photos are affected. I somewhat agree with you about photos you feel are good being 7/7s. Forget the ratings thing for a moment, if the photograph has said everything I want it to say that's good enough for me. But in balance, I sure do pay close attention to what people like Bruce and Fred say when commenting on my work, because the reality is that even though I may be satisfied, if they say somethings wrong...it probably is. I don't think anyone who has ever posted a single photo on this site can say that they do not care what someone else thinks about their work, or else they wouldn't post it. If you really don't care about the critiques/ratings of others...why do you bother to offer your comments in this forum? Incredibly inappropriate analogy? I'm just about as concerned about your being offended as you say you are about low ratings. "It's pretty crass". That's your sensitive little opinion Robert. Unless you can show me your badge stating that you're now the Crass Police...then I guess once again, you'll just have to deal with it. If I had intended to offend you Robert, I assure you that you wouldn't have to have asked what my comments meant...there would be no doubt. Have a good evening...Mr.X

Good comment/good ideas Fred. I agree...and I think it's just as objectionable when 7's are dished out dishonestly. I think the reason that people concentrate on the unfairness of undeserved low marks as opposed to undeserved high marks is that the majority of members who post their work are not advanced professionals...they're at every stage in between the novice(such as myself) and those who have mastered their craft. I'm more concerned about beginners who consistently receive these mindless ratings with no helpful comments. If someone got a 3, but attached was a helpful comment on what went wrong and how it could be improved upon...I think they'd be encouraged. The anonymous 3's with no explanation, seen over and over could be very discouraging. Fortunately for me, I was warned before I ever came to the site by a present member that if I concentrate on ratings only...I'll want to throw up my hands and quit. I think your idea of limiting the numbe of categories we rate in is a great idea. Something I think would be immensely helpful would be an article by PN staff that would address "how to critique a photo". No propaganda attempting to direct our taste,but simply addressing things like - you don't have to "like" a photo to appreciate it and therefore rate it on it's merits, not your personal taste. By the way Fred, the more you and I speak, the more I feel like you're becoming a mentor. You keep me grounded and I appreciate it. It's not difficult to see that I'm a bit of a wild hare...ask Robert.

P.S.- Robert, I'm just blunt perhaps to a fault at times...but one thing...you'll always know where you stand with me. Im really not quite as mean-spirited as I might seem. I offended you - you pissed me off. We're even unless you want to make it a pissing match...but if you do, let's carry it on via e-mail, the room needs to be for productive commentary,,,,not our petty battles.

Steve, I'm hurridly rushing to check out Fred Mirand's site. It's exactly what I'd like to see here. Sounds great to me! Gotta say that it's another case of I agree in part with what you said - "most of us aren't qualified to critique the work of others". Someone with no expertise in photography, whose mind hasn't been propagandized for endless years with all kinds of theory about how we "should" critique a pic, can often offer a much more valid critique. They see with more honesty. Amidst the majority of good people on this site you'll find some really arrogant asses who quickly remind you you don't know anything and your work is juvenile trash...with their work being the standard for excellent photography. I'd rather receive an, "hey mister that's a cool shot!" from a 10 year old, than an 8 from those folks. Of course someone will probably say something like...well, that's a reaction...not a true critique. Whatever. It's a constant game of one-ups-manship isn't it? :(

Bruce, I loved reading your response...and you're welcome. Your comments conveyed humility while relating enough information about your career in photography that I had this instant sense of respect for you and as well, a sense that I've been addressed by someone who has DONE what most of us only hope to do. While I just commented to Steve about "anyone" being able to critique a photograph...I believe there's truth to that in a certain way of thinking, but for me, the balance is that, for instance...you undoubtedly have technical expertise that no ten year old will have, nor that many or most of the members of PN possess. Should your critique carry more weight? It certainly does to me. I wanted to say this personally to you, because I didn't want you to misconstrue the intent of my comment written in my clumsy manner of trying to express myself. It's my pleasure to have read your remarks and look forward to seeing your work! John

John - at the risk of you thinking I am trying to turn this into a pissing match, I will

respond. It's true that I did say I was "part offended" by your analogy. This isn't to say that

I was looking at my screen all upset and thinking "how dare he be so incorrect" or

anything. My first comment was a joke, really. Sorry that was missed. I wasn't even slightly

confused by your comment.

 

The reason I was reading and then posting in this forum was just to see what was being

said this time about the ratings system. If you search photo.net for something like

"improve ratings" or "change ratings system" you will get thousands and thousands of hits.

This is a topic that repeats and repeats and repeats and repeats and repeats. Always with

the same kind of things being said. I admit that I like what Steve is trying to do with his

students, though the idea of having PN approved 'judges' would probably end up being

just as flawed in the long term and people who didn't get rave comments would just get

pissed off at the judges, instead of the anonymous raters.

 

It seems that you have already found the way that the system works for you. You lend

more weight to the critiques of members whose opinions you respect. Ratings can never

really give any help alone and to try force someone into giving a critique will mean you get

less of both, I imagine. Comments can not be anonymous, so it's easy to see which

comments you should pay attention to really - you just navigate back to their page and

see what their photography is like. Maybe you will even make some friends this way.

 

As far as my "sensitive little opinion" goes, or my "political correctness" then I am sorry but

I think neither applies. I don't see that the "big boys room" or "little boys room" (there are

those little boys again!) has anything to do with it. Simply saying that my pointing out a

daft remark means that I am not mature enough to be in the discussion does not affect the

daftness of the original remark.

 

It was my intelligence that objected to the analogy, not my morals. I know you were trying

to use an analogy that was extreme to attempt to illustrate your point, I just felt that the

one that you chose was so ill-judged that it actually worked against you.

 

Mr X

Ya know Robert, so many misunderstandings originate with w/ our attempting to say much, in few words. If we had spoken to one another in person there probably would have been no issue. In writing, we don't see the person's expression or hear the intonation of their voice, etc. Please forgive my comments...no excuse, but for a little background, I have a lot of Irish blood coursing through my veins and I'm a bit reactionary by nature. I actually love to "slug it out" on here, but not for the sake of a fight, certainly not to be offensive for the sake of being offensive, and never to play devil's advocate. I don't really look for a fight, but unless one tends to be very meek and quiet...you'll usually find one. Just like you...you said just what was on your mind, which I (for what it's worth) commend. You said that my comment was "daft", essentially saying it was "silly"...which I still contend is incorrect. So now you've said that my comments were "daft" and "crass". I have no problem w/ you stating that...but try saying that to a perfect stranger face to face and you might get your clock cleaned. We all get a bit too brave and big-mouthed in here at times. All I can say is, the analogy didn't work for you...but ya can't deny you're the only one who took issue w/ it. As for it working against me, well acutally...I don't see any evidence to that being a fact. You understood the analogy even though you found it objectionable...so it worked. You understood it. No one else needed an explanation and no one else found it so deft and/or crass that they felt the need to bring the matter up. Two other persons responded to me and to other comments about the original question. You said your morals didn't object to the analogy, but your intelligence. I'm not really sure what that means...seriously. Your intelligence doesn't have emotions, so I'm not sure how one would know when the intellgience would be offended...perhaps Spock might be an acceptable comparison. But then he just seemed puzzled when someones's words or actions didn't seem logical. Do you always react in this manner when someone acts/speaks from a level of intelligence that you evidently feel is inferior to your own? You admitted that you understood my point....and you say you're not offended on a moral basis...so what's the issue? So if you are offended by the fact that I don't act or speak as intelligently as you...that's simply arrogance. Understand, that I'm not angry in saying this...I'm smiling, smoking a clove cigarette, and drinking a really good cup of coffee. Not a pissing match...just really curious as to where you're coming from. Maybe someday soon we can just talk photography? :)