Computing & AI · the experiment
Assemble thirty AIs
A Fermi question, a team of AI answers, and their average. Grow the team, mix the models, and see what the crowd can correct, and what it cannot.
1 · What the average erases
Beginner
When you average one model's answers to a question, which part of their error can be erased?
Grow the team, with the shared share measured on realFP, up to thirty answers.
2 · Mix the models
Intermediate
Two answers drawn from different models share less error: about ρ = 0.591. With that shared share, how many independent agents is a team of thirty worth?
3 · A thousand agents
Expert
Go back to a single model, ρ = 0.906, but with a team of a thousand answers. How many independent agents is it worth?
4 · What would have to change
Expert
For a team of thirty to be worth five independent agents, what would have to change?
Check it: lower the shared share until the team of thirty is worth at least five independent agents.
Team assembled
—