A US man has launched a first-of-its-kind legal action against ChatGPT, saying it offered inaccurate medical guidance to him that resulted in delayed care for his pulmonary embolism. This comes as the large language model AI tool, used by hundreds of millions of people every week for health advice, launched Health in ChatGPT, a dedicated space for users to upload medical records such as lab results and medications, and connect Apple Health to allow the platform access to health and fitness data.
The lawsuit, filed in the Superior Court of California in San Francisco, accuses OpenAI and CEO Sam Altman of negligence, unlicensed practise of medicine, defective product design and other claims, reports Becker’s Medical Review.
It is among the first to allege that a chatbot’s health advice caused physical injury to a user seeking guidance about a medical condition, according to The New York Times, and the plaintiff has asked the court to stop ChatGPT Health from operating until independent evaluators verify its safety.
According to the complaint, ChatGPT-40 told Scott Winters, a pastor, that early signs of his health problems were “not something dangerous” and discouraged him from seeking medical care, urging him instead to trust that “God did not design your body to endlessly fail”.
Winters (55), claims ChatGPT manipulated his own language and beliefs with the knowledge that he was a pastor.
He began using ChatGPT in 2024 for everyday questions and general health queries. Over time, the lawsuit claims, the chatbot’s disclaimers urging him to seek professional care fell away, and it began offering direct guidance instead – including, when he described dizziness and blood pressure instability in June 2025, advice to stay home and remain “recliner-bound”.
Winters followed the guidance and limited his movement.
“ChatGPT-40 positioned itself as an authoritative partner in a medical journey it was simultaneously designing for Scott,” alleges the complaint, which includes screenshots of Winters’ exchanges with the chatbot.
In July 2025, Winters turned to the chatbot when he developed pain in his groin, which ChatGPT dismissed as “very likely another minor piece of the long story” involving previously discussed health issues with his gut and pelvic floor.
The next morning, Winters was admitted to the ICU with a massive pulmonary embolism due to blood clots in both lungs, which physicians attributed to weeks of immobility, according to the complaint.
Drew Pusateri, an OpenAI spokesperson, said that ChatGPT’s terms of service state that the chatbot is not meant to be used for medical diagnosis or treatment.
Even so, he said the company takes seriously its responsibility to make its answers to health questions as safe as possible, since people do turn to the technology for that purpose.
“Treating chatbots as the whole story behind people’s medical decisions or outcomes oversimplifies a much bigger challenge, and risks getting in the way of people accessing powerful new tools that can aid them in their health journey,” he told The Times.
He added that OpenAI’s newer models are an improvement on the one Winters had used, with newer models that seek missing information more effectively, signal uncertainty and recognise when someone should seek professional care.
The lawsuit seeks monetary and punitive damages, a pause on ChatGPT Health pending independent safety audits, and injunctive relief requiring OpenAI to hard-code refusals for medical diagnosis and treatment questions and to automatically end conversations when emergency care is needed.
Earlier this year, the safety of ChatGPT’s specifically trained healthcare AI came into question after researchers found it had considerable and potentially dangerous flaws, the British Medical Journal reported.
It said a study, published in Nature Medicine,1 found problems with accuracy and safety, as well as race bias.
“We tested ChatGPT Health on 60 clinical scenarios across 21 specialties, each run 16 times under different conditions, varying patient race, sex, whether labs were included, if a family member minimised symptoms, whether they were babysitting and couldn’t go to a doctor, and so on,” the trial’s lead researcher Ashwin Ramaswamy told The BMJ.
“ChatGPT Health is most reliable when the clinical decision is least consequential, and least reliable when it matters most.”
The team found the AI called a severe asthma exacerbation “a moderate flare” and recommended an urgent care rather than emergency department visit in 81% of attempts.
OpenAI says 300m people already turn to ChatGPT with health-related questions weekly — something that Tanzeem Choudhury, the chief of health innovation at Cornell Tech, says is reflective of the state of healthcare in the U.S.
Choudhury, who has been working at the intersection of artificial intelligence and health for more than 15 years, said AI can be a democratising force in healthcare at a time when many Americans face barriers to access, but that "we just need to think about the risks and have better protections in place."
Choudhury said users turning to AI for health information need to understand the risks and trade-offs associated with sharing personal data with a large language model such as ChatGPT.
AI can hallucinate and give misleading, inaccurate or even dangerous medical advice. An OpenAI spokesperson told CBS News that "ChatGPT is not a doctor and should never be used as a substitute for medical care, diagnosis or treatment." A few months ago, a California teenager died after ChatGPT told him it was safe to consume what turned out to be a fatal concoction of substances. OpenAI said in a statement to CBS News at the time that "this is a heartbreaking situation, and our thoughts are with the family," and said that version of ChatGPT is no longer available to the public.
In its statement Thursday announcing Health in ChatGPT, OpenAI said ChatGPT can make mistakes and "does not replace the care and judgment of qualified medical professionals."
CBS News has reached out to OpenAI for comment, and asked whether ChatGPT is equipped to recognize when to advise a user to seek help from a medical professional.
As it stands, Choudhury said, the onus is on the user to determine when to escalate concerns to a medical professional — and when not to trust advice given by a chatbot.
"It depends on the user's state and capacity to actually make that right decision," she said, adding that some users — such as a person who is elderly or cognitively impaired — might be susceptible to misunderstanding the risks of interacting with a chatbot.
Aside from a likelihood of inaccuracy, the issue of sharing sensitive, potentially deeply personal information with a chatbot is also something users should consider, said Vishal Misra, a Columbia University computer science professor and vice dean of computing and artificial intelligence.
"The same risks that exist sharing health data with anyone transfer here," he said. "You are sharing something very personal."
OpenAI said physicians around the world helped develop realistic health scenarios and test Health before its release. Health in ChatGPT contains extra safeguards, OpenAI said, including additional encryption protections, and will check in with users before taking action in a connected plugin, like sending an email, to make sure that it is properly matching the user's request. OpenAI said records shared with ChatGPT will not be used to train its models or target ads.
For some users, the benefits of turning to ChatGPT for advice might outweigh the risks, Misra said, adding that he has used ChatGPT to help track some of his own health data.
"It could be a good thing if done carefully because it can do a lot of things," he said. "It can find connections that maybe a really competent physician might miss."
Using data provided by a user, AI analysis can help users track their health information over time, which can then be shared with a physician to provide a more comprehensive picture of a patient's health. While ChatGPT might provide helpful information, Misra says to resist treating it like a doctor — and to keep a real one in the loop.
"Whatever it says, take it to an actual human physician," he said. "Do not act. The only action that you should take is to consult your physician."
Becker’s Hospital Review article – Man sues OpenAI for unlicensed practice of medicine (Open access)
BMJ – ChatGPT’s health AI has dangerous flaws, study warns
CBS News – ChatGPT now has a space for sharing medical records. Should you?
See more from MedicalBrief archives:
ChatGPT Health fails to ID emergencies – US study
US state sues AI firm over chatbot posing as doctors