A POWER METER THAT CATCHES HIDDEN AI TRAINING

Agreements to govern artificial intelligence may one day limit how large a model can be trained. Enforcing them would require an outside verifier able to tell whether a monitored computing cluster is training a model — even when its operator is trying to hide it.

Most ways of checking require trust: logs kept on the chips, taps on the network between them, interviews with staff, whistleblowers. All rely on hardware the operator controls or on the operator’s people. A physical sensor outside the chips does not. A power meter clamped on a facility’s input cable could be retrofitted to an existing datacentre and sealed against tampering.

Earlier work showed that training leaves a trace in power. But those studies used probes on a single board, sampling millions of times a second, or the chips’ own telemetry. A realistic meter would sit at the rack (server cabinet) or facility level, about a million times slower, behind a power supply that smooths and dilutes the signal.

The heartbeat of training

Tom Kimpson, of the University of Melbourne and the MATS programme, Mauricio Baker, of the University of Oxford, and Emlyn Graham, of MATS, start from a simple fact. When many chips train a model together, each step alternates a high-power compute phase with a low-power phase in which the chips exchange their results. That happens about once a second, and it draws a narrow line in the spectrum of the facility’s power — break the power signal down into its rhythms, the way a prism splits light, and a once-a-second beat shows up as a sharp peak at a single frequency.

Spotting that line is harder than it sounds. Real training does not keep perfect time: slow components, overheating chips and temperature changes make the rhythm wander. And an operator can make it wander on purpose, or smear it out.

So the team borrowed a tool from passive sonar and from searches for continuous gravitational waves: a Viterbi tracker, which follows a faint line as it drifts slowly through a spectrogram (a map of which rhythms are present, moment by moment).

Tested on a simulated datacentre

There is no public power data from a real training facility, so the authors built a model of one, based on a published model of training power used for grid stability: a main training run plus smaller jobs in the background. The alternative, “innocent” scenario replaces the main run with declared inference — serving a model’s answers to users — which the authors deliberately made noisy, to make the test harder.

Against an operator who makes no effort to hide, the tracker detects at least 96% of training runs in five-minute windows, with 1% false alarms, at most four percentage points below the best any test could theoretically do. Classic tests that look for a line at one fixed frequency collapse as soon as the rhythm drifts.

Three panels of frequency versus time: a steady training line, a drifting training line followed by the tracker, and inference with no line.

The tracker (blue) on three five-minute traces: steady training (a), training whose rhythm wanders by 0.4 Hz, or 0.4 beats per second (b; dashed, the true rhythm) and inference (c), where it only strings noise together and stays below the detection threshold. — Figure 4, Kimpson et al. (2026), arXiv:2610.30182.

The meter does not need to be fancy: it must sample at least twice a second, average over no more than half a second, and the power supply must not filter out the rhythm too deeply.

Graph of detection rate versus drift of the training rhythm: the tracker stays near 1, a fixed-frequency filter collapses, another test stays near zero.

Detection rate at 1% false alarms as the training rhythm drifts: the tracker (blue) stays close to the theoretical ceiling (dashed). — Figure 5, Kimpson et al. (2026), arXiv:2610.30182.

The price of hiding

Then the team played the cheat. They tried eight ways to hide: adding random idle time, shifting the rhythm, reshaping each step, diluting training among other jobs, and more. Against fixed-frequency tests, most worked. Against the tracker, almost none did — it caught at least 95% of runs in 43 of the 46 settings tested.

Only one approach really worked: varying the amount of work in each step. Its strongest form hid training in up to 73% of five-minute windows, at no cost in speed. But it has another cost. In a small real training run — a 30-million-parameter language model on a single A100 chip — reaching the same quality took 93% to 267% more computing, depending on the variant.

And time works against the cheat. Watching a full day, with a false-alarm rate of 0.1% per run, the verifier caught 87% of runs using this best trick, 44% within the first hour. About one in seven escaped.

Real meters next

The authors list clear limits. Every trace is synthetic; testing on a real multi-chip facility is planned but requires equipment they do not yet have. The meter detects a regular rhythm, not training as such: other periodic jobs, or a chip’s own power-management loop, could trigger it. And a large enough energy store between the chips and the meter could flatten the signal — though the verifier could then simply place a second meter closer to the chips.