ONE IN FIVE CHATGPT HEALTH ANSWERS RATED HIGH-RISK

ChatGPT has over 700 million weekly users, the authors note, and health questions are a routine part of that use. Adoption is growing fastest in low- and middle-income countries, where doctors are scarce — so an AI answer is more likely to replace professional care than to add to it.

Yet almost everything known about the safety of AI health advice comes from invented cases: vignettes written by clinicians, synthetic prompts, simulated patients. And safety depends on who is asking. Fasting, lifting heavy weights or a home remedy are harmless for most healthy adults and dangerous for someone with an ulcer, recent surgery or an eating disorder.

202,590 donated conversations

Shreyasi Roy Chowdhury, an independent researcher, and Kiran Garimella, of Rutgers University, recruited 1,252 ChatGPT users in India, Nigeria, Brazil and Pakistan. Each one exported and donated their complete ChatGPT history, with their age and gender; personal details were stripped on their own computer before upload. That gave 202,590 conversations, from December 2022 to February 2026, of which 15,555 were about health.

To grade them, the team used a published safety rubric and an AI from another provider as judge — Gemini 3.1 Pro — so that ChatGPT’s answers were not scored by a model from the same company. Each answer received a score from 1 (dangerous) to 7 (safe); 3 or below counted as high-risk. The judge scored every answer twice:

blind, looking only at the conversation, as standard safety tests do;
with context, knowing the durable health information the user had revealed in earlier conversations — and asking whether the answer was safe for that person.

One in five, and most users

Scored blind, most answers were safe: the median score was 6. But about one health conversation in five was high-risk — from 20.6% in Nigeria to 26.3% in Pakistan. And because people come back, 51.6% to 61.7% of users received at least one high-risk answer.

Risk concentrated in the most clinical questions: 40 to 48% of conversations about conditions and care, and 33 to 40% about symptoms, were high-risk, against about 8 to 12% for general health information.

Reading 128 risky conversations, the authors found recurring failures: no referral to a professional, wrong or harmful advice, unsafe self-treatment or dosage, encouraging dependence on the chatbot, and risky self-diagnosis. Their examples include:

discouraging tuberculosis screening for an infant exposed to the disease, contrary to guidelines;
giving medication doses to a pregnant user with no advice to consult a provider;
listing only harmless causes for chest pain during exercise;
recommending diet and exercise for severe anaemia (haemoglobin 6.6) instead of urgent care;
positioning itself as the main support for someone in severe, long-term mental distress.

Who is asking changes everything

With the user’s history in view, the same answers looked much worse. The high-risk share rose by 13 to 17 percentage points in every country — in India, from 29% to 44% of the conversations analysed this way. Strikingly constant across countries: about one in four answers rated safe in isolation was unsafe for the person who actually asked.

Fitness coaching flipped from one of the safest topics to one of the riskiest. Heavy core exercises for someone who had had intestinal surgery; raw garlic, lemon water and ginger for someone with a stomach ulcer. In most of these cases, the authors write, the information needed to avoid harm was already in the user’s chat history. The model used it to personalise tone and content — but not its caution.

Newer was not safer

Each conversation records the model that answered. Scored blind, the high-risk rate rose with each upgrade from GPT-3.5 through GPT-4o and GPT-5 to GPT-5.1, in all four countries — in Brazil, from 6.2% to 34.6%. The “mini” versions were riskier than the flagships; the newest, GPT-5.2, fell back in India and Nigeria. The trend held for the same users across consecutive models, although the authors cannot fully separate the model from the moment it was used.

A judge, not a doctor

The authors are careful about what this shows. The participants are a convenience sample of online gig workers, young and mostly male in India and Pakistan, so the rates are not national figures. The judge is an AI, not a clinician, and it measures risk, not actual harm to anyone. The risky conversations were coded by a single author.

They draw a parallel with pharmacovigilance, the mandatory monitoring of medicines after approval: AI health advice has no equivalent yet. Consented data donation, like theirs, could be a first prototype — provided that safety tools reading users’ profiles do not turn into surveillance.