Research Highlights Difficulties in Accessing Quality Health Guidance from Chatbots

As healthcare systems grapple with prolonged wait times and rising costs, many individuals are turning to AI-driven chatbots like ChatGPT for self-diagnosis. A recent survey reveals that about one in six American adults consult chatbots for health advice at least once monthly.

However, excessive dependence on chatbot suggestions can be risky, mainly because users often find it challenging to provide the detailed information necessary for accurate health recommendations, as noted by a recent study from Oxford researchers.

“The research highlighted a notable communication gap,” said Adam Mahdi, director of graduate studies at the Oxford Internet Institute and co-author of the study, in an interview with TechCrunch. “Participants utilizing [chatbots] made decisions no better than those using traditional methods, like online searches or personal judgment.”

For their study, the team gathered approximately 1,300 participants from the U.K. and presented them with medical scenarios crafted by a panel of doctors. The participants were tasked with identifying potential health issues within these scenarios and using chatbots alongside other methods to decide on possible actions, such as visiting a doctor or going to a hospital.

The participants interacted with the default AI model of ChatGPT, GPT-4o, along with Cohere’s Command R+ and Meta’s Llama 3, which previously supported Meta’s AI assistant. The authors found that not only did the chatbots lower the chances of participants recognizing relevant health conditions, but they also increased the risk of underestimating the severity of conditions that were identified.

Mahdi noted that participants frequently omitted critical details when querying the chatbots or received responses that were difficult to interpret.

“The advice provided [by the chatbots] often blended reliable and questionable information,” he remarked. “Current evaluation methods for [chatbots] do not adequately capture the complexities involved in user interactions.”

These findings emerge as technology companies increasingly promote AI as a tool to improve health outcomes. Reports indicate that Apple is developing an AI tool designed to provide advice on exercise, nutrition, and sleep. Amazon is exploring AI-driven solutions to analyze medical databases regarding “social determinants of health.” Meanwhile, Microsoft is working to create AI that can triage messages from patients to healthcare providers.

Nonetheless, as previously reported by TechCrunch, both healthcare professionals and patients express caution regarding AI’s preparedness for high-risk health applications. The American Medical Association recommends against using chatbots like ChatGPT for clinical decision support, and notable AI firms, including OpenAI, warn against making diagnoses based solely on chatbot responses.

“We advise consulting reliable information sources for healthcare decisions,” Mahdi emphasized. “Current evaluation methods for [chatbots] do not sufficiently reflect the intricacies of user interactions. Just like clinical trials for new medications, [chatbot] systems should undergo real-world testing before being released.”