New Delhi: Artificial Intelligence-driven chatbots are providing unreliable or problematic medical advice that is neither prescribed nor recommended by experts, suggests a new study published in the medical journal BMJ Open. The research highlights the increased risk of artificial intelligence and technology to health, as they are increasingly interfering in day-to-day life, with people relying on them without any expertise.
Researchers from the United States, Canada, and the United Kingdom assessed five widely used platforms—ChatGPT, Gemini, Meta AI, Grok, and DeepSeek—by posing 10 questions in each of five different health categories. The study, published this week in the medical journal BMJ Open, found that nearly half of the responses were problematic, with around 20 per cent classified as highly concerning.
Artificial intelligence gives misleading advice
During the research, scientists found that chatbots reacted relatively better on closed-ended prompts and questions related to vaccines and cancer, and worse on open-ended prompts and in areas like stem cells and nutrition.
Answers received from chatbots were confident, and certainty was delivered through each prompt; no chatbot was able to produce a full list in response to any prompt, said researchers. There were only two refusals to answer a question, both from Meta AI.
The results of the study showed growing concern about how people are using AI platforms to generate answers and seek advice, which aren’t licensed to give medical advice and lack clinical judgment to diagnose.
The researchers noted that the results “highlight important behavioural limitations and the need to reevaluate how AI chatbots are deployed in public-facing health and medical communication.” They added that these tools can produce “authoritative-sounding but potentially flawed responses,” underscoring the need for more caution in their use.