Saturday, March 7, 2009

Amazon "Helpful" Reviews (And now for something completely different...)

This post is nothing like most posts on this blog, so feel free to sick the "articles totally out of line with blog theme" police on me.

Have you ever been on Amazon and wondered how they determine what is a "helpful" review and what is not a helpful review? Sure, there are those voting buttons, but it's not obvious how "helpful" votes and "unhelpful" votes contribute to ranking reviews. For example this review currently has 32 out of 38 helpful votes, but is considered more helpful than this review, which has 18 out of 20 [90% helpful!]

But it is not just a matter of having more helpful votes, after all this review has over 450 helpful votes, and it is considered less useful than this review, which (at the time of writing) has fewer than 100.

And the above two example are nothing compared to the observation that this review, which so far not one person has said is helpful, is considered more helpful than this one, which is currently batting above 50% at least [13 out of 24].

Welp, before I turned to theology, I was trained in deductive reasoning... and I still work as a mathematical modeler professionally, so one day I decided to figure out what is going on here.

I'll give a few different versions of increasing complexity: [Note, some reviewers get bonus points directly, and others appear to be penalized by Amazon...however what is described below is accurate 99% of the time.]

Super-Simple Version
Each review starts off [in Amazon's mind] as having a value of around 65% [just a little higher]. each time someone votes for your review, it nudges that score up some, each time they vote against it, it nudges it down some. The more reviews you've had, the less each nudge is. Also, if your score is really high, a "yes" vote is not going to help you as much as a "no" vote hurts you (and vice versa if your review has not hitherto been appreciated.)

Medium Version
Reviews are rated based on a percentage of how many of your "votes" have been "helpful." Except every review starts out with 13* "helpful" votes and 7** "unhelpful" votes that Amazon counts in addition to the actual votes people cast.
So, if your review gets 8 "helpful" votes right out of the box, Amazon sees your review as about 75% helpful [the 8 "real" votes you've gotten plus the 13 "fake" ones gives you 21 "helpful" votes, and you have 7 "fake" "unhelpful" votes as well, so there are 28 votes total, of which 21 are "helpful."]

*Actually it is more like 7.08
** Actually more like 12.92

No-More Mister Nice Guy Version

Let X be the number of helpful votes.
Let Y be the number of unhelpful votes.
Calculate a review's score as [X + 13.08]/[X+Y+20]

Reviews are ranked based on their score.

No comments: