Computing & AI · the experiment
Beat the algorithm
A question on a film site's homepage, 551 guesses, four algorithms. Four questions, from beginner to expert, on when your own forecast beats a recommender, and when it does not.
1 · Name the best forecaster
Beginner
Who best predicted the rating members later gave a film they had not seen?
2 · Measure the gain from asking
With all the machine information, the mean squared error of the forecast is 0.921. Adding the member's guess brings it down to 0.685.
Intermediate
By what percentage does the guess cut the error?
3 · Pick where asking pays least
A long-time member with a very long rating history is asked about a decades-old classic that a great many members have rated.
Expert
What does the study suggest about the member's edge here?
4 · Weigh the member's certainty
Besides the expected rating, members said how sure they were, from 1 to 5.
Expert
What does adding that certainty to the forecast bring?
Algorithm beaten
—