CLIMATE MODELS: THIRTY YEARS OF COMPUTING, AND RAIN IS STILL HARD

Climate models provide much of the evidence behind climate policy. Since 1995, an international programme called CMIP — the Coupled Model Intercomparison Project — has run them side by side, generation after generation, under common rules. Each generation has used finer grids and more physics, and therefore more computing. What nobody tracks, according to a team of 24 researchers led by Ioana Colfescu (University of St Andrews and the UK’s National Centre for Atmospheric Science), is how much energy this costs, and what it buys.

159 models, one error score

The team recomputed a single error index for 159 models from five CMIP generations, using their archived output: how far each model’s winter climate (1979–1999) strays from observations, for three variables — air pressure at sea level, surface air temperature and precipitation. Only these three, because the second generation archived nothing else. Lower is better.

The median error falls from 1.56 in the first generation to 0.61 in the sixth. Over the same period, the computing needed for a typical model, estimated from its grid resolution, grew about 29-fold.

A steady rate of return

Generation by generation, the error drops by 21, 28, 21 and 14% — an irregular series. But measured against how much computing each step added, the pattern becomes regular: 13.5% less error per doubling of computing, with a spread of only 0.7 points across the three transitions where the comparison works.

That rate gives a simple, sobering calculation. Halving the error takes about 4.8 doublings, “because 0.865 to the power 4.8 is 0.5”, the authors write — and 4.8 doublings mean about 28 times more computing than today’s models. Meanwhile, the growth in computing between generations has slowed, from 2.2 doublings to 1.1.

The authors are careful about what this means. Better physics also improves models, and the two cannot be separated, so 13.5% measures the joint return on computing and science. And it describes past progress, not a law of nature.

Rain lags behind

A single score hides big differences. Between the first and sixth generations, the error on sea-level pressure fell 3.5-fold and on surface temperature 2.7-fold. On precipitation, it fell by 7%. Three decades of added resolution and complexity have barely improved the variable that matters most for climate impacts, by this measure.

Energy that nobody measures

Energy reporting is patchy. The eight institutions that published complete data for the latest generation account for 1,692 tonnes of CO₂. Across the 13 models with measured energy, the energy per simulated year varies 65-fold, and it barely tracks resolution. Of 30 institutions, nine publish standardised energy figures, and none reports energy linked to a skill score.

AI weather models such as GraphCast, Pangu-Weather, FuXi and GenCast now match or beat the European Centre’s high-resolution forecast. They are said to be far cheaper to run. But the team found no published measurement of the energy needed to produce their forecasts, so they reconstructed it from reported hardware and run times. Then they added training: about 2,900 kWh for GraphCast, 62,000 kWh for Pangu-Weather. Over a year of four forecasts a day, training is more than 99% of each system’s energy, and Pangu-Weather falls from second to last in skill per kilowatt-hour.

A frontier instead of a ladder

The roadmap proposes to judge models on a skill–energy plane: skill on one axis, energy on the other, and the set of systems no other beats on both. A score, skill divided by energy, can rank systems within a class, but the authors warn that it should never be used alone: a nearly useless model that uses almost no energy would score well. They also propose five levels of energy reporting, from “reported” to “complete and audited”, and four ways to do more with less: better-designed experiments, new hardware such as GPUs, less data shuffling, and targeted AI components.

They also flag the risks: cheaper runs may simply mean more runs, and training large weather models remains concentrated in a few organisations and countries. Their first step is modest. Report the energy next to the accuracy, with clear boundaries, before the next generation of models — CMIP8 — gets under way.