HomeArtificial Intelligence (AI)US state sues OpenAI over ChatGPT’s medical advice

US state sues OpenAI over ChatGPT’s medical advice

The US state of Iowa’s Attorney-General Brenna Bird has sued OpenAI, alleging the company misled consumers about its product ChatGPT’s safety and reliability, including the risks of receiving inaccurate or dangerous medical advice from the chatbot, reports Becker’s Health IT.

Seven other states have filed separate lawsuits against OpenAI, all broadly challenging ChatGPT’s safety and OpenAI’s representations to consumers, while Iowa’s petition specifically includes allegations involving medical advice.

The 91-page petition alleges that OpenAI violated the state’s Consumer Fraud Act by misrepresenting ChatGPT as a safe and trustworthy product despite evidence of potential harms to users.

It specifically raises concerns about ChatGPT’s performance when providing medical guidance.

The petition cites a 2026 study published in Nature Medicine that evaluated ChatGPT Health’s ability to recommend appropriate levels of care for medical emergencies.

According to the filing, researchers found that the tool under-triaged 51.6% of emergency cases tested, recommending delayed care in situations where physicians would have advised patients to visit an emergency department.

The lawsuit also references a separate legal case involving a Californian teenager whose parents allege ChatGPT provided unsafe advice about combining kratom with the anti-anxiety medication Xanax.

According to Iowa’s petition, the teenager’s parents alleged that ChatGPT provided advice about drug combinations and failed to warn about potentially fatal interactions. The teenager died after consuming a combination of Xanax, kratom and alcohol, the filing states.

The allegations involving ChatGPT’s advice have not been established in court.

Beyond medical advice, Iowa alleges that ChatGPT’s design can encourage unhealthy emotional attachments, reinforce delusions and expose vulnerable users, including minors, to harmful content.

The lawsuit also challenges OpenAI’s testing of advanced AI models, alleging that the company failed to implement adequate safeguards against potentially harmful autonomous behaviour.

Iowa is seeking civil penalties of up to $40 000 per violation of the state’s Consumer Fraud Act, reimbursement for affected consumers, and additional safety measures. It also wants the court to prohibit further testing of advanced AI models until adequate safeguards have been implemented and independently verified.

The Iowa lawsuit adds to growing legal scrutiny of OpenAI’s consumer-facing health tools.

In July, Scott Winters, from Florida, sued OpenAI and CEO Sam Altman, alleging ChatGPT-4o downplayed his worsening symptoms and advised him to remain at home rather than seek medical attention. Winters claims the advice contributed to a near-fatal pulmonary embolism. His lawsuit alleges negligence, defective product design and the unlicensed practice of medicine.

In May, a Texas couple sued OpenAI after their 19-year-old son died of an overdose. The family alleges ChatGPT advised him that combining Xanax with kratom was safe. The lawsuit also sought to delay the rollout of ChatGPT Health pending a safety review.

In a previous statement to Becker’s, an OpenAI spokesperson said the company sees AI as a way to improve access to health information but emphasised that ChatGPT should not replace medical professionals.

“ChatGPT is not a doctor and should never be used as a substitute for medical care, diagnosis, or treatment,” the spokesperson said.

OpenAI has also said its newer AI models have improved on health-related tasks, including recognising when users need urgent medical attention and communicating uncertainty.

Becker’s has contacted OpenAI for comment.

 

Becker’s Health IT article – Iowa sues OpenAI over ChatGPT’s ‘dangerous medical advice’ (Open access)

 

See more from MedicalBrief archives:

 

ChatGPT sued over health advice that discouraged medical care

 

US state sues AI firm over chatbot posing as doctors

 

Poll shows 50% of AI health query answers are problematic

 

AI medical diagnoses may include fake health info – US study

MedicalBrief — our free weekly e-newsletter

We'd appreciate as much information as possible, however only an email address is required.