A study led by researchers from the University of California San Diego School of Medicine has demonstrated that ChatGPT, an artificial intelligence (AI) chatbot developed by OpenAI, outperformed human doctors in terms of bedside manner.

To reach this conclusion, the researchers assembled a panel of licensed healthcare professionals to assess the replies of both human doctors and the chatbot in response to medical scenarios.

One such scenario tasked human doctors and ChatGPT with responding to a scenario wherein a patient has swallowed a toothpick.

The response from the human doctor was: “While many people have swallowed toothpicks without issue, if two to six hours have passed, chances are it’s already in the intestines, and therefore not easily retrievable. In case you develop a stomach ache, then don’t hesitate to seek out an emergency room, and remember to point out the swallowed toothpick.”

Meanwhile, the response from the chatbot was: “It’s natural to be concerned if you have ingested a foreign object. While it’s possible for a toothpick to lodge in the throat or puncture the digestive track, causing serious injury, it’s unlikely to happen with a “dull, cooked toothpick that is only two cm long."

“However, any discomfort, like abdominal pain, difficulty swallowing or vomiting would merit medical evaluation. It’s understandable you may be feeling paranoid, but try not to worry too much.”

The response from the chatbot was considered by the panel to be considerably more empathetic than the response from the human doctor, according to the researchers.

Following a number of similar medical scenarios posed to both the chatbot and human doctors, the researchers determined that the panel preferred the chatbot’s responses to roughly 200 inquiries almost 80% of the time.

The AI system is being eyed by researchers to potentially help alleviate the overload of work on healthcare professionals, possibly improving both workflow and quality of responses to patients.

The study, “Comparing Physician and Artificial Intelligence Chatbot Responses to Patient Questions Posted to a Public Social Media Forum,” appears in the journal JAMA Internal Medicine.