I have another blog that doesn't suck.
Yelp asks me to give restaurants a rating of between one and five stars. This suits me. People sometimes ask me to rate things on a scale of one to ten, and I never feel like my discrimination is delicate enough to warrant so many different grades. What's the difference between a 6 and a 7? I'm not sure.
I have a standard system to convert my overall impression to stars. The most common rating is three stars:
☆☆☆★★ “OK, I guess”
Yelp doesn't allow fewer than one star, which is fine with me.
Going in the other direction, we have:
My impression of Yelp ratings is that this is atypical and that ratings are heavily weighted toward four and five stars, and that by giving three-star ratings to restaurants I like, I am doing them a disservice. That is unfortunate for everyone. Here are the rating distributions of a user A, on the left, and myself, on the right:
Some upward skew is to be expected. There are plenty of mediocre restaurants that I don't bother to review, but when the food is good I get excited to tell people about it.
But clearly, the two of us mean something very different by five stars. When I give a restaurant five stars, you know it means I like it better than 82% of the other restaurants I've reviewed. When use A gives five stars, all you know is that it's better than the worst 30%.
I've often wondered how hard it would be to weight different users’ ratings, so when user A gives out five stars two-thirds of the time, their stars are not worth as much as those of user B who awards five stars only one time in six. And conversely, user A's rating of one star seems to be more significant than mine, because a one-star restaurant is in the bottom 3% for user A, but only the bottom 10% for me.