scott bulger Posted February 14, 2003 Share Posted February 14, 2003 I had contributed earlier to this thread, but noticed that the activity had picked back up and didn't want to leave Marc hanging by himself. I feel like the illness has improved due to the defaults put in place by Brian on the top images pages but it's not cured. It still takes some searching to find the great ones, but at least the top pages are more interesting than they used to be. I know I've said it before, but I think the ratings are valuable so I will say it again. I think that the saviour of the rating system will be curators in combination with some sort of weighted rating system. I don't pretend to understand the math that Jim Schweigger went into in his proposal, but I think the basic idea is sound. Sure, some people will continually attempt to abuse the system no matter what, but lets not give in and give up. It's inevitable that crimes will occur in society, but that doesn't mean we should just give up and throw open our front doors to the criminals. Link to comment Share on other sites More sharing options...
jay_dixon Posted February 14, 2003 Share Posted February 14, 2003 Here is my honest opinion on how to generate more overall ratings and critiques which may or may not shift the emphasis away from clique driven ratings. 1)As of now, navigating newly posted images is amzingly unclear. Having a recently loaded page that goes back strictly chronologically would really make finding thumbs more straightforward and you could browse back through the pages. (Sounds like photosig without the cumulative point structure.) 2) Possibly putting critique requests in a similar thumb based page system. As of now the only way for one of my images to get noticed is to request a critique. I notice that many top-rated images have no critique requests, and arrive there from the early clique driven method. Once this happens, I agree with Marc, if you rate lower, people tend to get testy. Last month, I had the most viewed, rated on and commentented on image on the entire site outside of the POW's, and I know that requesting a critique played the strongest role in that. It just jumped to the front of many different search variations. So it had an unfair advantage by the end of the first week. 3) As Marc said, limiting a users highs for a given time period may reduce the overall clique driven inflation somewhat. 4) Navigation is the key to seeing images not restricted to just a small percentage of the photographers. It has to be opened up somehow, in order to spread the wealth. I like the site, but am restricted to the front pages for any easy navigation, and the front pages are in large part essentially clique driven. Link to comment Share on other sites More sharing options...
mg Posted February 14, 2003 Share Posted February 14, 2003 Your description of the situation and its causes makes perfect sense all the way. I'll just have a closer look at the end of your post, the rest being understood...<p>"So, it looks like we should just "eliminate" friend-rating -- somehow."<p>Well, that's exactly what I think and why I proposed to simply limit the number of 7s, but based on what you wrote above, some may not like it, that's pretty clear as well. Who ? Of course those who distribute 7s like sweets - who else...?<p>"But, will we be driving people out of the system entirely?"<p>By bet would be that it would drive SOME of the mate-raters away, and they aren't THAT many, but I'd suppose that it would bring back a few people who have reduced their participation on the site, or who have left or stopped ratings. I doubt it would make a significant change in the participation rate - but I could be wrong of course...<p>"The friend-rating problem (...) were created by making ratings non-anonymous in August 2001. (...) However, Gallery participation, as measured by photos submitted and number of ratings, has also doubled since August 2001."<p>I am not sure whether these 2 facts are related. Maybe they are, and to a certain extent, I am sure they are... Mate-raters are typically those who upload the most and who rate many images - almost all 7s & 6s, and that's why it works -; they also invite reciprocity and other people will return the favors, hence increasing again the number of ratings in the statistics. But all this is what I would call "artificial participation", meaning that it serves a personal ambition, but not the learning community - who can't learn much from all these one-liners and 7s...<p>"So, friend-rating may undermine the statistical value of the photo rankings"<p>That's for sure, yes.<p>"...but it may also be the factor driving increased participation in the Gallery."<p>Yes, but I'd say "increased artificial and meaningless participation", whereas real participation - i.e criticism and sincere cooperation - goes down the drain.<p>Basically, it's a very old problem: QUANTITY or QUALITY - of participation...<p>Obviously, I'd vote quality, but let me explain why shortly: simply because quality brings quality, then brings a larger quantity of quality-driven crowds. Whereas it is my belief that mate-raters will bring mate-raters only, and kill quality - after which, there's just no point of having a place called photo.net.<p>If this site's goal is ONLY to make profit, then I say "go ahead" as it is, and sell 7s as well, as Bob humoristically suggested elsewhere... But if the site's objective is to survive financially AND to help photographers grow, then, I think it is time indeed to "eliminate friend-rating -- somehow."<p>Limiting the number of 6s and 7s available for all users seems to eliminate the worst effects of mate-rating while reducing mostly the impact of mate-raters on the results, but friends will still be able to exchange A FEW 7s. So, friends will remain friends, but friendship won't damage the rankings all that much anymore... That's why I thought this limitation had the merit of being an easy, soft, and light solution... It shouldn't annoy the mate-raters to the point where the site would collapse - imho. Best regards. Link to comment Share on other sites More sharing options...
z_z1 Posted February 14, 2003 Share Posted February 14, 2003 Felix, at first I too was put off by these anon. people hurling low scores around like feathers flying in a chicken coup when the fox attacks. But then again I realized I was being thin skinned and who cares about these putz's anyway. I don't give a rats behind what my "score" is on photo.net. I do however enjoy comments from people. On the odd occasion when I've had a bad day and feel like crapping on someone I do actually email a reviewer who has given me an unusually low score compared to what others have given me and asked them to elaborate on their numericial critique of my photo. In most cases they just said did't get into that type of subject matter or some other crap like that. So in those cases they should not even bother rating as all they are doing is really saying "I don't like pictures of churchs" I would have rather seen a picture of someone with purple hair and pins through their nipples. So don't put to much in what others say about your art. If it pleases you and you are able to convey the message you where trying to, that should be good enough. I am always tickled that people are willing to share their art with me. I try and respect this gift. I do rate photos and I think I'm being fairly honest when I do. After 30+ years of doing this, many of them as a professional, I think I have some relavent input to share. If it were up to me I would get rid of the numerical rating system. Have you asked Farber to explain his judging criteria? Link to comment Share on other sites More sharing options...
bobatkins Posted February 14, 2003 Share Posted February 14, 2003 Getting rid of numerical ratings is a good idea. However I'm sure it would result in loud screams from a subset of users, since there would be no way to present the "top" images or "top" photographers, in short no way to win the virtual photo.net image contest. The only way to win this game is not to play. Read comments, ignore ratings. Let those who want to fret over scores play their game, you play your game. Link to comment Share on other sites More sharing options...
z_z1 Posted February 14, 2003 Share Posted February 14, 2003 I have my suspicions that some of the daily "top rated" photographers, you know the ones, they're on the first page day in and day out, have it set up with each other to rate each others work at the highest scores possible thus insuring their continued place amoungst the clouds. If people are that starving for admiration I feel sorry for them. This is not to say some of the images are very good, it's just that anything they do seems to get on the front page, good or not. I have not bothered to put much on this site for several reasons; don't have a very good scanner, don't have a lot of time to piss around with PS. I live in the darkroom, and while I make a good living selling my prints, the web representations are very sub-standard. I don't want to have some wet-behind-the-ears type telling me my image is fuzzy or muddy just because the scan is bad. But then again as I've already mentioned, who cares what they say. Link to comment Share on other sites More sharing options...
bob_pictaker Posted February 15, 2003 Share Posted February 15, 2003 The mate raters that are so prevalent on the top pages are most certainly aware of the many threads such as this one where discussions have raged concerning their practices. They seem unwilling to participate in these threads to offer a differing opinion or to defend their actions. They continue on their 7/7 way knowing full well that it goes against one of the stated purposes of this site as I understand it. In so doing they are effectively snubbing their noses at the site itself, the sites administrators, and the community in general. The result of this is they have taken over a segment of this site to suit their own purposes. So, it looks like we should just "eliminate" friend-rating -- somehow. But, will we be driving people out of the system entirely? - Brian Mottershead I agree that we should limit the number of 7's anyone is allowed to dole out. Is this going to anger some people? Sure, but mostly the people who hand out too many 7's. Will some of them leave? Of course. But is that really such a bad thing? As Marc pointed out, even if every mate rater abandoned this site tomorrow we are still only talking about a handful of people. Insignificant I would think relative to the total number of Photographers who post images. And how many people I wonder leave this site because they have become discouraged? Because they have posted images they feel in their heart are at least as good as many of the images they see in the top rated section, but these images sit in their portfolio with 0 ratings and 0 comments. Their images may not be realistically viewed as "top images" but it is this inequity that is the most discouraging I would think. I'll bet that human nature is playing two roles in this issue. The most notable role being the vanity on display on the top rated pages. Less noticeable, but possibly more wide spread is the tendency for people to ignore a problem, and hope it goes away. It's the easy way out. I've certainly done it, and I imagine that mate rating is a problem for a great many people that they are just hoping will go away. It takes a great deal of effort to push for change. To be the squeaky wheel. "The only way to win this game is not to play" - Bob Atkins That's very true! This site is after all meant to be a fun diversion for most people. When the game is no longer fun the easiest solution is to just stop playing. Unfortunately I would think this to be detrimental to the general health of this site. And again goes against the sites stated purpose - its reason for being. If this site wants more people to play the game as intended, a level playing field is a must. No system will every be perfect, and to be honest I don't need perfection. A few adjustments to this system could make enough of a difference to alleviate this problem to a acceptable level for most of us. You might lose some players from the game, but we might just see a bunch of other people getting off the bench and back into a much healthier and more enjoyable game. Link to comment Share on other sites More sharing options...
bobatkins Posted February 15, 2003 Share Posted February 15, 2003 When I suggested the only way to win is not to play I meant that not playing the rating game is the way to win. Just ignore the numbers. Keep participating, get critiques, give critiques, just don't play the numbers "game". You CANNOT stop "rate-mates". Suppose Edward Weston and Ansel Adams had joined and they really only wanted to rate each other's images. Giving each other 7s would have been reasonable most of the time. This would be totally fair and above board. It I wanted to screw the system, I'd simply join under 20 different user ID's, upload 5 images as each user and give myself the maximum allowed ratings from each of my accounts. Whatever scheme you come up with, I can come up with a scheme to beat it. You can make it slightly more difficult for me, but you can't stop me. I can open as many photo.net accounts as I need to and upload as many images as necessary to make certain that my "real" gallery wins the game. Short of absolute account ID verification (drivers licence, SSN, bank account, credit card, DNA profile) and/or charging me a fee to give ratings that's high enough to stop me taking out multiple accounts, nothing else can stop me. It's a game, change the rules and I'll just find another way to beat them. Link to comment Share on other sites More sharing options...
bob_pictaker Posted February 15, 2003 Share Posted February 15, 2003 Bob - when I quoted you I understood your meaning. I wasn't trying to misrepresent what you were saying I was merely attempting to draw an analogy. Sorry if I wasn't as clear as I should have been. I agree with your comment in regards to the rating situation as it is now. I do try to ignore them. Lately I've made an effort whenever time allows to comment on more images than I rate. In fact I will comment on an image I feel is over rated, and do so honestly, then deliberately not rate the image so as not to promote it farther up the top rated page. And you are right that people will always find a way to work around the system. But why not make it harder for them? There will always be people determined enough to find a way to manipulate the ratings. But there will also be people who do not have that kind of determination, who will in essence remove themselves from the equation. So by making mate rating more difficult we may begin to see less of it. Again, it will never be perfect, but it could be better. Why not try? Link to comment Share on other sites More sharing options...
bobatkins Posted February 15, 2003 Share Posted February 15, 2003 I don't see any of the proposed solutions working. Sure you can limit the number of 6s and 7s you can give out, but what if you are the sort of person who only wants to rate good images? Limiting ratings is so artificial. Every time you change the rating system you make all the old ratings invalid. If you start limiting 6s and 7s, the old images with "too many" 6s and 7s will be even more likely to be at the top of any list, so what have you gained? You can tweak and tweak and tweak, and still end up with a mess, not to mention taking time away from making other improvements to the site which are badly needed. So please come up with a scheme that will prevent me and my friends from boosting each other's ratings without invalidating all current "scores". I'm willing to go to quite a bit of trouble because my ego knows no bounds and if there's a way to win the game, I'll take it. Otherwise you're just wasting your time. It would probably be easier and cheaper just to give me free psychiatric help than come up with a rating scheme to prevent me from cheating. Link to comment Share on other sites More sharing options...
jim schwaiger Posted February 15, 2003 Share Posted February 15, 2003 Marc pointed out this thread to me so I thought I might drop in a few suggestions. It looks like incremental changes are the best we might hope for, so let me start with a few relatively simple ideas (simpler that startng from scratch at least). 1) Drop Originality/Aesthetics and make it one simple rating for each image. This has been discussed to death and it seems there is a consensus that they aren't used consistantly enough to be used for scoring. Instead, just ask "how good" the image is on the scale. 2) Make subscribing much more valuable. 2a) Limit storage space based on subscriber status. Lets say that non-subscribers get 1 or 2MB of storage (enough for 10 or 20 images) and subscibers get 10 or 20MB (10 times as much). You could even sell more storage in 10MB increments to those who wish to have large portfolios on the site. Along with this you could also strictly limit image sizes to 100kB for non-subscribers and 200kB for subscribers. That way those who wish to be allowed better quality JPEGs will be pleased as well. At the same time, members who compress to a reasonable quality will inherently be allowed more images than those who always use maximum quality. 2b) Another annoyance is the "Medium" image size. A resaved JPEG at 75% is the first impression people get of our images. A better solution is to simply force the desired file size and pixel dimensions to start with. If you want 600x600 images less than 80kB, then specify that and show the original image that we upload. And why not enforce some limits on pixel dimensions. To keep it simple, just set a limit on height+width. And maybe make it different for subscibers. So, let's say non-subscibers get 1000 pixels (500x500) and subscibers get 1600 (800x800). This allows for panoramics as well. If an image is outside the pixel dimensions or the image size limits, either reject it during the upload process or tell the member that you have to resize it to meet the limits. For those of us who abide by the rules and try our best to submit a nice looking JPEG, let the others see what we actually submitted. The result would be that people would take more care in compressing their photos to a reasonable size so they can upload more images and at the same time, the value of subscribing would be much greater (10 times more images instead of 2 times more). And, subscibers would be allowed to have larger, better quality photos on the site. 2c) Make uploading and submitting photos separate actions. Let us upload and delete images in our portfolios without ever automatically submitting any of them to be rated (no New Uploads page). Then allow members to submit any image from their portfolio into a Ratings Gallery (see below) for comment and critique. Non-subscibers may get to submit 1 or 2 images a week while subscibers may get 1 per day. These might go to a "New Submissions" page which would replace the "New Uploads" page, but it would have much fewer images. 3) Create Rating Galleries Portfolio images would be submitted into a Ratings Gallery, much like the existing Categories. The difference is that the score from within a category is kept separate from other ratings. So, one image may be submitted into multiple galleries and receive different scores from each. A landscape-portrait photo may be a great portrait, but a mediocre landscape image, so it might get 6/7 from the Portrait Gallery and only 4/7 from the Landscape Gallery. Galleries could be anything at all, specific or general, and it they could be added as we go along. So, we could have general categories like: "General Photography", "Snapshots", "B&W", "Landscapes"; and at the same time have specific ones like: "B&W Baby Portraits", "Bird Photography", and "Beach Sunsets". One image could be submitted to the "B&W" gallery one day and then submitted to the "B&W Portraits" gallery the next day. Scores could be kept for up to 5 or so categories for each image and maybe a non-Gallery score and a "Total Score" could be calculated as well. Rating would occur inside the Galleries, so in the "B&W" gallery you are simply rating how good the image is as a "B&W" photo. The best of each Gallery could be displayed on a page and the a couple of images from each could be placed on the "Top Photos" page. If you get a lot of categories, you could simply use statistics to determine which photo's make the Top Photo page. This does a few things that would help the site. It vastly reduces the visibility of non-submitted photos and limits the number of submissions per member. Rating from a portfolio page (where mate-raters live) would be a non-Gallery rating, so it would not count towards the Gallery Ratings at all, even though it could affect the total image score. It also spreads out the competition quite a bit. Instead of only one Top Photo page, there is one for each Gallery and only an image or two from each will make the "Top Photos" page. The existing scores can be kept as the initial non-Gallery rating and we all start from scratch with the Gallery Ratings. 4) Comment Galleries It also might be nice to have a few comment Galleries for people who have real questions about improving their skills. Maybe a "Beginner" gallery, a "Lighting" Gallery, a "Composition" gallery, etc. Members would not be able to rate anything in the "Comment Galleries". ------------------------------------ I know that even those ideas would take some effort, but much less than what I proposed before and they can be added to the existing system without big changes. Link to comment Share on other sites More sharing options...
mg Posted February 15, 2003 Share Posted February 15, 2003 I like a few ideas proposed by Jim, but rather than disserting any further in this thread, I'd like to clarify something about what you wrote Bob... <p> "I don't see any of the proposed solutions working. Sure you can limit the number of 6s and 7s you can give out, but what if you are the sort of person who only wants to rate good images? Limiting ratings is so artificial. Every time you change the rating system you make all the old ratings invalid. If you start limiting 6s and 7s, the old images with "too many" 6s and 7s will be even more likely to be at the top of any list, so what have you gained?" <p> Good question and valid objection at this point. <p> "So please come up with a scheme that will prevent me and my friends from boosting each other's ratings without invalidating all current "scores". I'm willing to go to quite a bit of trouble because my ego knows no bounds and if there's a way to win the game, I'll take it. Otherwise you're just wasting your time." <p> Right. Makes perfect sense. <p> So, what I want to clariy is this: is photo.net interested in a more complete proposal along thse lines, including limitation of the number of 6s & 7s or something of that sort to limit damages caused by rating inflation ? <p> Is PN prepared to help a group of people who would try to come up with such a proposal, by answering a few simple questions along the way, and provide some rating statistics when necessary and when possible ? <p> If so, I have no doubt that we could come up with something that (hopefully) may work. <p> The reason why I'm asking this is that Jim, for example, already spent hell of a time to build a proposal that - to many of us - would work. But he had no reply from PN about it. <p> No doubt there are people here willing to give a hand to solve this problem, but we might need some information and at least we would want to know we wouldn't be doing all this in vain... <p> Last but not least, maybe it would be good to hear from PN why Jim's first general proposal was apparently disregarded. Was it because it was too complicated ? Or were there other reasons ? Would photo.net maybe just describe briefly what are the things the site would never want to consider in such matters...? <p> What I am saying is that I am glad to see some sort of a dialog suddenly taking place here thanks to you Bob, and to Brian. All we may need at this point is to understand a bit more what sort of changes can and can not be considered... what are PN's criteria to accept a reform proposal basically... For example, has PN entirely dropped the idea of Curators...? Of having members favorite picture galleries...? ...of having categories on the site...? Any guidelines ? Any tips ? We could probably work from there. We might nevertheless need a little help from PN at the start, and a little feedback from PN along the way... Is that possible...? Link to comment Share on other sites More sharing options...
jorge_gasteazoro4 Posted February 16, 2003 Share Posted February 16, 2003 Not long ago I was taken to task for expressing the same thoughts Felix did in the LF forum. I think his concern is not the "7" or "1" numbers, but the fact that many of these people who consistenly rate low or high depending their mood or whose photograph they are rating <b>do not have a single image uploaded</b>. I know of at least one member here in photo.net who has upwards of 13000 critiques, yep you read right more than 13000 critiques and does not have a single uploaded image. Now, that in itself is not bad, but the problem rises when many of these "critiques" and ratings are harsh and sometimes down right nasty. <p> One must consider the source of those critiques, if the person giving the ones or the sevens has some uploaded images and everybody is able to see them and judge <b>their</b> technical ability then fair is fair. But I feel there is something fundametally unfair when someone has done many "ratings" or critiques which are low and does have the same courage as the persons they rate or critique to put their work out there. This is the reason I removed my images from photo.net and why I refuse to rate or critique anybody. Since I dont have any images I dont feel is fair I comment on those who do. <p> I dont know why is so hard to request that people who want to critique also upload images. Some sites do a 3 for 1 deal, you upload an image, you get to critique 3 images, etc, etc. <p> On a more personal response to Felix, buddy, dont worry. If someone is nice enough to write a comment, take into account. If it is just a number, high or low from a "no image" person, dismiss it, probably their opinion is worth as much as their images. Link to comment Share on other sites More sharing options...
garry edwards Posted February 16, 2003 Share Posted February 16, 2003 I think Jim Schwaiger has come up with some very sound suggestions but I question (again) the whole point of providing numerical ratings. I once received a 1&1 from someone, without comment, and it later turned out (when someone alluded to this individual in another post) that he objected to my subject matter, apparently on religious grounds!<br>O.K., he can no longer gives 1's without comment but he can still give 2's...<br>My whole point is that ratings without comments mean little or nothing and could easily either inflate or depress the self-confidence of the receivers, regardless of whether or not the figures are artificially manipulated in some way.<p>Comments, by contrast (as long as they don't just say 'Wow') can provide meaningful and extremely helpful feedback, not only on what the reviewer thinks of the photo but also on the way that s/he thinks generally, and so makes it possible for the recipient to make a judgement on whether or not that particular opinion should be taken as seriously as others.<p>Personally, I don't post many photos, and when I do post them they're largely ignored (maybe they're no good)but when I do post them I always add the line "All comments appreciated" - and I mean it. I don't ask for ratings and I pay little regard to any ratings I do receive - it's comments I want!<p>One small point, I also agree with Jim that there should be more catagories - none of my own pics ever seem to fall within the categories available. Link to comment Share on other sites More sharing options...
bobatkins Posted February 16, 2003 Share Posted February 16, 2003 Again, I do not speak for photo.net on this subject. I just throw out ideas. I have zero influence over what's done in the gallery section. Here's a scheme: (1) All comments MUST be at least 25 (50?) words. Anything less is probably a pretty useless comment anyway. (2) To give a 1,2, 6 or 7 rating you must first leave a comment (3) Anyone can give a 3,4 or 5. Now this sounds great and addresses the issues of useless comments, inflation of ratings and maybe even "mate rating:. HOWEVER - It would cut down on the number of comments posted, and people complain all the time that they get no comments. ALSO it would mean that the existing 479092 images, especially the "top" images would have a rating advantge over any new images. It's going to be a LOT more difficult to get a 6 or a 7 so your chance of ever getting an image in the top 100 (or top 1000) is probably slim to none. So my guess is that it would piss off as many (maybe more) people than it pleases. This quest for universal fairness isn't as easy as it seems! Link to comment Share on other sites More sharing options...
jim schwaiger Posted February 16, 2003 Share Posted February 16, 2003 To respond to the original question and Garry Edwards, there are many reasons that no uploads are required to rate/comment. First and most obvious is that good photography skills and good critiquing ability don't always correlate. A great photographer can be a poor judge of others' work and a poor photographer can be a great judge. Second, many members simply do not have the right equipment to digitize their work. For example, someone who shoots slides needs a decent film scanner to present a good image. And finally, some people just don't want to take the risk of their web quality images being stolen. Let me also go ahead and say that I am totally against requiring comments along with any of the ratings. I understand the intent, but in practice, it simply doesn't work. It doesn't take much experience here to realize that forcing comments just doesn't do what it was intended to do. I've gotten things like "???" myself and it really just wastes space and everyone's time. Likewise, highest ratings often get comments like "Great" and "Wonderful" which really aren't any more informative than the rating itself. Why would someone give 7/7 if they didn't think those things. Forcing the comments to a minimum length might discourage some of the low ratings, but at the same time it will punish those who are helpful and concise. A comment like "Nice try, but lighting is way too harsh. Try late evening or an overcast day." would not meet the minimum 25 words that Bob suggests, yet "a a a a a a a a a a a a a a a a a a a a a a a a a" would. While that is obvious, a member could simply come up with a 25 word sentence that they paste into the comment box of all images that require a comment. I think you can build a better scale, but the numbers and the "Very Bad" rating need to go. Noone wants to receive those kind of ratings, they serve no purpose and provide no benefit to anyone. A quick fix would be to eliminate 1 & 2 altogether and make the scale go from 3-7 where 3 means "below average" or more simply "poor". Limiting how often the highest rating can be given has some merit. For example, we could give one 7/7 per week to those who rate 50 or more images in that week. That means that any one member can only give a 7/7 2% of the time. In an entire year they get to give 52 7/7 ratings maximum and even then must give 2600 other ratings. If this were implemented, the existing photos would have to be adjusted somehow to account for the fact that 7/7 are currently unlimited. One simple method would be to simply add a new rating of "8" that is the new highest rating possible. Then the existing ratings could be left alone. I think we haev to keep the system as open and free as possible. I think raters would bear a system where they can earn the usage of the highest rating, but I think requiring comments or certain comment lenghts will indeed run people away. Especially those of us who generally try to be helpful and concise with our comments. Make a system that is friendly to honest members, not limiting based on the dishonesty of a few. I think we can eventually get to a point where we have so many honest users that the efforts of the abusers are neglegible. Link to comment Share on other sites More sharing options...
bobatkins Posted February 16, 2003 Share Posted February 16, 2003 <em>Forcing the comments to a minimum length might discourage some of the low ratings, but at the same time it will punish those who are helpful and concise. A comment like "Nice try, but lighting is way too harsh. Try late evening or an overcast day." would not meet the minimum 25 words that Bob suggests, yet "a a a a a a a a a a a a a a a a a a a a a a a a a" would. While that is obvious, a member could simply come up with a 25 word sentence that they paste into the comment box of all images that require a comment</em> <p> Obvious attempts to subvert the 25 word minimum comment requirement would result in the loss of posting and rating ability for 7 days (or longer), plus result in removal of the comments and rating by a moderator. Obvious abuse is easy to deal with, it's the subtle sneaky, non-obvious and unprovable stuff that's difficult. <p> If you can't come up with 25 original words to comment on an image, it pretty much indicates your opinion isn't worth posting. <p> There certainly are images posted that deserve a 1 or 2 rating and giving a range of 3-7 would simply shift the mean from 4 to 5, so rendering all ratings on the previous 479,091 images invalid. Sure you can tell people that a "4" means average, but they don't have to listen to you. Link to comment Share on other sites More sharing options...
niranjn Posted February 16, 2003 Share Posted February 16, 2003 The way I see it, overly high ratings is more pervasive than overly low ratings. However, as Brian M points out, it is exactly these mate-ratings cliques that have enthusiastic participants in the Gallery part of the site, and Photo.net should not turn them off COMPLETELY. Right now, the vanity rewards are too great for mate-raters to ignore. By rewards, I mean the exposure that they get by being featured in the top pages. I am convinced that a way to limit mate rating is to reduce but not eliminate the rewards that go with a high rating. Limiting user behavior by reducing the number of 7's or other such schemes can be easily gotten around. How to limit the rewards? I don't know. Vague thoughts swirl in my head about using the ratings to select a small number of featured photographs of the month and rotating them, not repeating the same photographer for a while. So, an overrated photograph may get exposure, but the same photographer will not get his/her turn again for a while. In a nutshell, don't use the ratings as raw numbers in any query but to aggregate when displaying "recent uploads" pages. Second, making the list of photographs that a user has rated to be publicly viewable would be an excellent way to expose undesirable behavior for all to see. Third, the various navigational search methods do get you different top pages, but the most common ones used such as "number of ratings" or "average rating" are prone to abuse. Maybe we should have a "total length of comments" (NOT "number of comments") as a default method. POW's would automatically be more represented in those pages, I would imagine, and also a lot of old gems. Link to comment Share on other sites More sharing options...
mg Posted February 16, 2003 Share Posted February 16, 2003 So much to say, so little time... Anyway, I'll try...:-) "Here's a scheme: (1) All comments MUST be at least 25 (50?) words. Anything less is probably a pretty useless comment anyway. (2) To give a 1,2, 6 or 7 rating you must first leave a comment (3) Anyone can give a 3,4 or 5. Now this sounds great and addresses the issues of useless comments, inflation of ratings and maybe even "mate rating:. " - Bob Atkins <p> Agreed. But if I may, I'd propose to add this: after a picture has received its first 5 ratings, I'd also like to havea 25 words comment for any 3, 4 and 5 that are more than 1.5 points below or above a certain picture's existing average - if that's technically possible. That would for example solve the problem Felix was refering to at the beginning of this thread, and further limit abuses. What PN needs to realize is that a 3 or a 4 on a picture worth a 5.5 is no less an abuse than a rating of 2 on an image worth a 4. Some people just systematically rate PS works for example a 3 or lower, some people never give more than 4 to a sunset shot, etc - that has to stop. By the way, categories would help a great deal in that sense too, as well as the abolition of the originality rating... <p> "HOWEVER - It would cut down on the number of comments posted" - Bob <p> I don't think so, Bob. It may cause the "number" of comments to drop a bit, but not much, because more people will now have to write instead of just dropping a number. But the trouble is for those who can't express themselves well in English... Other than that, I'd also say, that I prefer a 25 words comment than 25 1-word comments ! <p> "ALSO it would mean that the existing 479092 images, especially the "top" images would have a rating advantge over any new images. It's going to be a LOT more difficult to get a 6 or a 7 so your chance of ever getting an image in the top 100 (or top 1000) is probably slim to none." - Bob <p> Yep. Agreed. But that's not a major problem to me either. For example, Photo.net could "stop" the new uploads for a wek or 2, and ask all members to rate and comment each 30 shots from the top 300, which would certainly change quite a bit the look of the top-rated pages. Each member would HAVE TO rate 30 of these pictures, before being allowed to re-upload images on the site. (I still believe that the number of 7s and 6s available MUST be limited somehow, and would need to be limited as well during this process.) <p> " For example, we could give one 7/7 per week to those who rate 50 or more images in that week. That means that any one member can only give a 7/7 2% of the time. In an entire year they get to give 52 7/7 ratings maximum and even then must give 2600 other ratings. If this were implemented, the existing photos would have to be adjusted somehow to account for the fact that 7/7 are currently unlimited. One simple method would be to simply add a new rating of "8" that is the new highest rating possible. Then the existing ratings could be left alone. " Jim S <p> That sounds very smart to me. Could you explain a bit how the 8s would fit in, Jim. That may be the solution for this limitation. <p> "Obvious attempts to subvert the 25 word minimum comment requirement would result in the loss of posting and rating ability for 7 days (or longer), plus result in removal of the comments and rating by a moderator. Obvious abuse is easy to deal with, it's the subtle sneaky, non-obvious and unprovable stuff that's difficult. If you can't come up with 25 original words to comment on an image, it pretty much indicates your opinion isn't worth posting." - Bob <p> Agreed. <p> "There certainly are images posted that deserve a 1 or 2 rating and giving a range of 3-7 would simply shift the mean from 4 to 5, so rendering all ratings on the previous 479,091 images invalid." - Bob Atkins. <p> I'm not sure whether I understand this, but if I do, I disagree...:-) PN could simply convert ALL the previous 1s and 2s to become 3s. By the way, it would do a lot of good to many great pictures I've seen, which were hit by obviously biaised ratings of 1/1s or 2/2s... <p> "Limiting user behavior by reducing the number of 7's or other such schemes can be easily gotten around." - Niranjan <p> Duh ?? Is that really so obvious ?!? Please help me out here: HOW can people get around it ? Fake accounts ? If so, I have an idea against that, and Bob had another one which was to open ratings to suscribers only. What am I missing besides that ? <p> "Second, making the list of photographs that a user has rated to be publicly viewable would be an excellent way to expose undesirable behavior for all to see." Niranjan <p> Now, this, yes !!!! I had this idea long ago, and I think it is really necessary. That way, those who distribute 7s to any bad image will appear what they are ! Hiding their ratings is to protect them. <p> " Maybe we should have a "total length of comments" (NOT "number of comments") as a default method. POW's would automatically be more represented in those pages, I would imagine, and also a lot of old gems." - Niranjan <p> Another very interesting idea... Link to comment Share on other sites More sharing options...
mg Posted February 16, 2003 Share Posted February 16, 2003 http://www.photo.net/photodb/ratings_breakdown?photo_id=1303436 <p> Best picture on photo.net, obviously...:-) Link to comment Share on other sites More sharing options...
bobatkins Posted February 16, 2003 Share Posted February 16, 2003 I don't see how eliminating the 1 and 2 scores (and making them all 3s) does much. Instead of a scale from 1-10 we went to a scale of 1-7. Now we propose to go from 3-7. What's next 4-6? Then everything is a 5? I just hate to see ever more complex rules and restrictive rating schemes - based mainly on the impossible desire to get "fair" rating numbers. It just strikes me that if you have to go to the trouble of averaging and renormalizing all the time, based on a set of arcane rules about who can give what to who and how many they can give, and how many they must give before they do something else that the whole system is so fundamentally flawed that no amount of tinkering will ever do any good. So I'd stick to a simple proposal. 25 (though I prefer 50) words of real comment to give a 1,2,6 or 7. No comment, limit to 3,4 or 5. It's not just a mathematical way of doing things, it's actually a REASONABLE way to do things. If you have a strong opinion on an image, voice it. If you can't voice it, you haven't thought it through. No need to renormalize ratings, no need for complex accounting schemes of who gives what to who and how many times. However if I were running the site I'd have to carefully consider if this scheme would actually end up annoying more users than it would please. If 90% of the users want to be able to rate from 1-7 and give one word comments, why should they be denied that ability? Of course I'd also add that if I had to actually do all the programming for this, I might well have a different opinion, based on what else needs doing on photo.net, how much time and effort I could devote to each project and what the priority of each project was. Link to comment Share on other sites More sharing options...
mg Posted February 16, 2003 Share Posted February 16, 2003 Well, Bob... I don't think that there are absolutely simple perfect solutions to such a complicated problem. But I tell you what, I've just joined those who don't care anymore... Let's have Clifford.net. <p> The site is a complete mess with nothing fair at all and where top-rated pages are a real joke - as you well know -, so let's dance... <p> The day anybody at PN would care to actually do something reasonable AND EFFECTIVE about this, without being worried to lose a few egomaniac members, then I'll be glad to contribute. If it's all fine as it is, then let me shut up once and for all. I've been a very tiring member for way too long, and I'm tired. Then, I've got a thousand 7s to distribute, so that will keep me busy for a while... <p> Cheers. Link to comment Share on other sites More sharing options...
jim schwaiger Posted February 16, 2003 Share Posted February 16, 2003 While long thoughtful comments are nice, I know several members who can express their thoughts and/or provide advice in less than 25 words. Forcing a minimum comment size will have more of a negative impact than I think you guys realize. I wish I knew the statistics, but my guess is that a large number of comments are relatively short. But my primary objection is that you are making it harder for a lot of honest, sincere members to comment if you force a minimum comment length. I disagree with Marc about limiting which ratings are available without comment as well. The first 5 ratings could be mate-raters and that would force others to comment before giving their honest rating. Again, it could easily encumber those who are honest and sincere. And I think Marc knows that there are many images that are 2 points overrated. Marc, the 8 would simply be the top end of the scale from that point on. Obviously members can go back and re-rate any photo they think is deserving. Suddenly limiting 7's would not be fair to new photos as they could never approach the highest rating as many images now do. Bob, Obviously there are some terrible images on photo.net, but tell me what benefit anyone gets from a 1 or 2 rating? My only point was that we could make one rating for all the below average photos. That means that a dishonest 3 has less effect on the good images and the fact that it is not a good image is still conveyed. My proposal gets rid of the numbers altogether, and makes a broad "Poor" rating to do this same thing. Eliminating 1 & 2 is a quick fix, but I think it would help minimize some of the abuses and make things easier for the management who have to police the abusive low ratings. The numbers are arbitrary anyway, so why would a 3-8 scale be any worse than 1-10 or 1-7? The numbers on the rating interface could also be eliminated and the scoring scale could still be 1-7 or any other scale you like. I'm with Bob about keeping the system unrestrictive. But I think that forcing comments in the first place and forcing a certain length comment are the type of restrictions that will run members away. People who are commenting to be helpful should not be restricted at all on the site. And people who are commenting because they are forced to are likely to not be helpful in their comments anyway. Ultimately I think we have to have a system that simply has enough participants that the mate-raters become insignificant. The limits Bob is talking about will discourage participation. If rating and commenting are strongly encouraged we get more active members and with other needed changes, more subscribers. Link to comment Share on other sites More sharing options...
bobatkins Posted February 16, 2003 Share Posted February 16, 2003 I'd say again that if you can't find 25 words to describe your feelings about an image, then you simply are not qualified to offer an opinion and/or you simply haven't given that matter any thought. It's 25 words. People have written pages, if not whole books, about a single image. If all you have is some instinctive gut reaction that the image is good, average or bad, and you can't be bothered to analyze why you feel that way then 5,4 and 3 ratings are more than adequate to express that. You can still add a one word comment of course, if 25 words are just too much for you, but you just can't give a score over 5 or under 3. Comment's such as "Wow", "Beautiful Image", "Wonderful" are useless ego stroking. If that's what people want, fine, but let's not pretend in any way that these comments are useful or in any way give the photographer feedback of any instructive value. Link to comment Share on other sites More sharing options...
seven Posted February 16, 2003 Share Posted February 16, 2003 Who <b>moderates</b> PN Problems & Feedback? Why is it that you allow multiple threads regarding dissatisfaction with ratings to be posted week in and week out (photosig style) - instead of referring posters to existing threads on the subject? Is it not Forum policy to check "has this question been raised before?" <p>Clearly the thread has deviated from Felix's specific query into another ratings rant - covered over-and-over elsewhere (and featuring the same old unhappy posters.)<p> Please pull the duct tape from your eyes and start nuking these repeat threads, as you would elsewhere on the site. My thanks and regards. Link to comment Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now