OpenAI's ChatGPT Health, a specialized chatbot launched to provide health guidance, has been found to frequently underestimate the severity of medical emergencies, according to a study published in the journal Nature Medicine.
Source:
nbcnews.comThis first independent evaluation of the AI tool raises critical concerns about its safety in making urgent medical decisions.
Source:
mountsinai.orgResearchers tested ChatGPT Health's ability to triage medical cases by feeding it 60 different scenarios that ranged from mild conditions to serious emergencies.
Source:
nbcnews.comThe responses were then compared to those of three independent physicians who assessed the urgency based on established medical guidelines.
Source:
mountsinai.orgThe study aimed to determine whether the chatbot could accurately advise users on whether to seek immediate medical attention.
Source:
theguardian.comThe results were concerning: ChatGPT Health under-triaged 51.6% of emergency cases, advising users to wait for 24 to 48 hours instead of recommending immediate care.
Source:
yahoo.comThis included critical situations such as diabetic ketoacidosis and respiratory failure, which can be life-threatening if not treated promptly.
Source:
yahoo.comDr Ashwin Ramaswamy, the lead author of the study, emphasized that any trained healthcare professional would recognize the need for immediate intervention in these scenarios.
Source:
nbcnews.comWhile the chatbot performed well with clear-cut emergencies, such as strokes, it struggled with more nuanced cases where clinical judgment is essential.
Source:
mountsinai.orgIn one example, ChatGPT Health identified signs of respiratory failure but still advised the patient to wait for further evaluation.
Source:
nbcnews.comThis pattern of under-triage raises fears that users may receive a false sense of security in critical situations.
Source:
theguardian.comAdditionally, the study revealed that ChatGPT Health over-triaged 64.8% of non-urgent cases, often recommending unnecessary appointments for conditions that could be managed at home.
Source:
nbcnews.comFor instance, the bot advised a patient with a three-day sore throat to seek a doctor's appointment when home care would suffice.
Source:
yahoo.comThis inconsistency in triage could lead to unnecessary healthcare utilization, further straining an already overwhelmed medical system.
Source:
youcanknowthings.substack.comPerhaps most alarmingly, the study highlighted the chatbot's inconsistent responses in scenarios involving suicidal ideation.
Source:
theguardian.comWhile ChatGPT Health is programmed to direct users to the 988 Suicide and Crisis Lifeline when users express suicidal thoughts, it failed to do so in high-risk scenarios where patients also provided normal lab results.
Source:
yahoo.comThis inconsistency could have dire consequences, as Dr Ramaswamy pointed out that the failure to activate crisis intervention mechanisms in serious situations is more dangerous than having no safeguards at all.
Source:
theguardian.comThe results of this study have prompted calls for more rigorous testing and safety standards for AI tools used in healthcare.
Source:
mountsinai.orgDr John Mafi, an associate professor at UCLA Health, emphasized the importance of controlled trials to evaluate the benefits and risks of such technologies before they are widely adopted.
Source:
nbcnews.comDespite the potential benefits of AI in healthcare, experts underscore that these systems should not replace traditional medical judgment.
Source:
theguardian.comDr Ethan Goh, director of ARISE, noted that while chatbots can provide valuable information, they cannot substitute for professional medical advice.
Source:
nbcnews.comThe findings from this research serve as a reminder that while AI can assist in healthcare delivery, it is crucial to ensure patient safety remains the top priority.
Source:
yahoo.comOpenAI has acknowledged the study and expressed a commitment to improving the safety and reliability of ChatGPT Health before expanding its availability.
Source:
nbcnews.comAs technology continues to advance, ongoing evaluation and updates will be necessary to ensure that AI tools effectively support, rather than endanger, patient health.
Source:
mountsinai.orgIn conclusion, while ChatGPT Health offers a promising approach to providing medical guidance, significant concerns about its triage capabilities and response to critical situations must be addressed to prevent unnecessary harm and ensure patient safety.
