A while back, I mused about the shortcomings of ratings systems.
Photo by Jack Rydquist from Flickr used under Creative Commons
The short version is that a scaled system trends toward averages or extremes, falls victim to social pressures and provides too few/too many options. Problems exist for an up/down voting system which fails to capture any nuance and restricts the voter.
Nothing has changed, but I have an idea.
When asked to rate something, we’re generally presented with a scale, e.g. five stars, one-ten scale, percentage.
When rating, the mind wants context, which causes problems for a single instance. I might rate a beer four bottle caps on Untappd, but I’ll immediately wonder what other beers I’ve rated four stars for comparison.
Is this one better? How can I compare a porter with an IPA? What other factors are involved, e.g. was it hot day when I rated that pils four stars, was that hoppy ale a freebie, etc.
If you’re not into beer, try that scenario with movies instead, i.e. the Netflix rating conundrum.
There are so many variables involved, each with shifting weights. Even presented with the same scenario dozens of times, I’d probably make dozens of different ratings.
I think that’s the key because given a large enough sample, my true rating should emerge. Just like the photo a day project reveals changes over time that you wouldn’t notice in each instance.
Back to beer, I might even change my rating as I drink it, or shortly after I finish, or when I drink it again.
Great, easy solution, simply prompt the user several times for a rating, and offer a comparison, e.g. you also rated this beer four stars, are they equally awesome?
Not so much. It wouldn’t take long for users to get annoyed with that system.
The issue here is that the ratings just aren’t that important. Ratings are generally a tacked-on attribute for the real object of value.
An alternative to creating recommendations is to use an algorithm, as foursquare did for the 3.0 release. It’s an interesting approach, but with no human intervention, the recommendations cannot be tweaked. The assumptions may not be right, which could lead to poor results.
So, what’s my idea?
You need a dedicated system for rating. If foursquare has proved anything, it has shown the value of focus on a single unit of work. When your goal is to recommend something, you need a good rating system.
You’ll need to offer comparisons with other similar objects, with similar attributes and ratings.
And finally, you’ll need to account for the fluidity of rating. Ask me to rate something a few times because maybe my mind has changed or the newness/glow has disappeared.
Sounds cumbersome to add to an existing system, which is why it’s an independent entity. Once you establish the human interaction, you can layer on algorithms to make it smarter, but always asking for tweaks.
Rating could be social, but that’s optional.
The way to make this shine is to pipe in various objects for rating, e.g. movies, beers, restaurants, anything. Then expose the rating to the object providers.
All of a sudden, you layer personal recommendations all over the web, based on what you’ve rated.
So now anyone up for it?
Find the comments.
