Computing & AI · the experiment
Make the score speak
A game store that asks for a review after three hours of play, 21.6 million reviews, and a score that still ranks but warns less. Four questions, from beginner to expert.
1 · Measure the effect of the question
Beginner
After Steam's question arrived, what happened to reviews written just past three hours of play?
2 · Follow a bad month
Pass-through is the share of the monthly movement in reviews written in the first two hours that shows up in another group of reviews. Just above the three-hour threshold, it goes from 0.73 before the launch to 0.18 after.
In a given month, the score of the reviews from the first two hours drops by 10 points.
Intermediate
How much does the score of the group just above the threshold then drop, before and after the launch?
3 · Explain the preserved information
Each added review responds about half as much as a voluntary one to a bad month. Yet, at the volumes actually observed, the stream of reviews above the threshold lost no detectable information compared with the stream below (its agreement with the first-two-hours reviews changed by −0.03 relative to the stream below, 95% interval −0.13 to 0.06).
Expert
What explains it?
4 · Find what stays visible
Expert
Which bad news still clearly pulls the score down after the question arrived?
Score decoded
—