THE MORE ACCURATE THE RESTAURANT APP, THE MORE IT SEPARATES RICH AND POOR

Maps rank the restaurants nearby, food apps suggest where to eat next, and language models now answer questions about where to go. Unlike a printed guide, the systems behind these suggestions learn from the visits they steer. They are retrained on where people went — including the places they recommended themselves. That is a feedback loop between people and algorithms.

Platforms usually judge these systems by their offline accuracy: how well they predict visits people made before any recommendation. That test never lets the suggestions change where people go, so it cannot see what the loop does over time. A team from Northeastern University, the Bruno Kessler Foundation in Trento, the ISTI-CNR in Pisa and New York University set out to measure it.

Why a city is not a playlist

Feedback loops have mostly been studied for songs, films and products, which any number of people can consume without affecting one another. Restaurants differ in three ways. People choose among places within reach of where they are. A restaurant can serve only so many customers. And a recommendation to one person also changes whom the other customers meet.

The team therefore built a simulation on real movements. Anonymised GPS data from opted-in users, recorded between October 2016 and March 2017, gave the daily schedules of 4,000 residents in Boston, San Francisco and Detroit and 8,000 in Chicago, drawn equally from the four income quartiles (four equal-sized bands, from poorest to richest) of their home neighbourhoods. On each trip to eat, a recommender suggested 20 venues within a realistic distance that still had room; each venue’s weekly capacity was set at 1.5 times its real visits. The system was retrained every week on the previous four months, for about three years. The real visits of everyone else in the data — up to 290,977 people per city — were kept, so the simulated residents shared venues with them.

Four algorithms were compared: one that recommends the places a person visits most often (UR-POI), two that learn from similar venues or similar people, and one that follows the moves of the whole population.

Measuring who meets whom

The team measured experienced income segregation: for each person, how unevenly the time spent in venues is shared among people of the four income groups. A score of 0 means an even mix; a score of 1 means meeting only one group. Mixed places matter because ties across incomes predict upward mobility. Restaurants are a good test bed: they are about half as segregated as the neighbourhoods where people live.

Accuracy against mixing

The result is a trade-off. Ranked by offline accuracy and ranked by the segregation they produce, the four algorithms come out in the same order. Choosing by accuracy picks the most segregating algorithm in every city. The most accurate one, UR-POI, left people 15–21% more segregated than in their real visits. It wins on accuracy precisely because people return to familiar places — and by sending them back there, it keeps them among their own.

The ranking, not the map, is to blame. Picking at random among the same reachable venues gave the lowest segregation of all in Boston. And the burden falls hardest on those already most isolated: the lowest-income residents were the most segregated group everywhere, and in Boston their score rose from 0.360 in reality to 0.413 under UR-POI.

Restaurants that vanish

The loop also empties venues. Under every algorithm and in every city, at least a fifth of the restaurants lost all their simulated visitors — between 21% and 71% of them, against 5% to 30% without weekly retraining. Retraining itself caused 55 to 75% of the loss: a venue that receives no visits drops out of the data and is no longer recommended. The losses concentrated in less active, lower-income neighbourhoods.

Mixing and survival can pull apart. The two algorithms that segregate least learn most from other people’s visits, gathering everyone at the same venues — an overlap 4.4 to 6.2 times that of real visits — and so leave more venues empty, UserKNN the most of all, in every city.

A nudge in the ranking

The team tested a simple fix called tail-boost, added on top of any algorithm: each venue’s score is blended with a bonus that is larger for less visited places. In Boston, while giving up at most 15% of offline accuracy, it cut venue disappearance by 54 to 75% under every algorithm and lowered segregation under three of the four; under UR-POI, the excess segregation fell from 21% to 13%. It did so largely without sending people to places where their own income group is a small minority. Offline accuracy alone would have rejected it.

The simulation keeps people simple — in the main runs, everyone follows the recommendations — and it measures income by neighbourhood, not by person. The authors release their code and argue that platforms and auditors should simulate the loop before deploying an algorithm, in the spirit of the European Digital Services Act. Recommendation research has long treated each suggestion as a matter between one user and one item; in a city, as the paper puts it, every followed recommendation “also changes whom other people meet and which venues stay visited.”