ChatGPT‘s striking ability to answer people’s questions in instantly generated, easy-to-understand answers could be a game-changer for people searching for medical advice, but a new medical journal essay says it’s not quite that simple. And so does ChatGPT. Peter Lee, a vice president at Microsoft, has spent the last six months experimenting with the latest edition of ChatGPT, an artificial intelligence platform designed to answer user questions in a conversational tone.

Like other so-called large language models, ChatGPT, made by San Francisco-based startup OpenAI, is trained on vast amounts of data available on the internet, in books and from other sources. It uses algorithms to generate words in sequence based on the frequency with which they appear together online. Microsoft‘s Bing search engine now uses ChatGPT. He published an essay Thursday in The New England Journal of Medicine about the potential uses of ChatGPT for recording medical notes, offering diagnostic possibilities, educating clinicians and patients and analyzing and summarizing data.

Particularly notable, he said, was the interface’s ability to provide empathetic, human-like responses. We asked Lee, also co-author of the forthcoming book “The AI Revolution in Medicine: GPT-4 and Beyond,” about his enthusiasm and concerns about the latest version of ChatGPT, now ChatGPT-4, which was recently released. We also posed the same questions to ChatGPT, but because it’s availability is still limited, the responses are from ChatGPT-3.

Question: Does ChatGPT provide reliable medical information?

Lee: I personally would not trust models at that level of sophistication for my own personal health advice or information and I would be concerned if doctors and nurses were making it for medical decision-making.

ChatGPT: ChatGPT is a language model trained on vast amounts of text data, including medical information. However, it’s important to note that ChatGPT is not a medical professional, and its responses should not be considered as medical advice. While ChatGPT may provide information on medical topics, it’s essential to verify that information with a qualified medical professional before making any decisions regarding your health. There is also a risk that ChatGPT may provide inaccurate or outdated medical information, which could potentially be harmful.

Question: What kinds of mistakes does ChatGPT commonly make and why?

Lee: GPT models generally are very bad at information retrieval. It’s almost by design because the GPT models don’t want to regurgitate their training data. If you’re asking it for a specific source, like a reference to a paper, it’s not able to resurrect the exact source, so it makes something up, it “hallucinates.”

ChatGPT: ChatGPT may make mistakes such as providing irrelevant or incorrect responses, repeating itself, or producing responses that are inconsistent with the context of the conversation. These mistakes can occur because ChatGPT is trained on vast amounts of text data, including unverified and potentially biased information, which can lead to incorrect or outdated information. Additionally, ChatGPT’s responses may be affected by the quality of the input it receives, which can result in irrelevant or off-topic responses.

Question: How is ChatGPT-4 better than ChatGPT-3?

Lee: We get to a different level of intelligence. The fundamental thing is you have this AI model that has a lot of cognitive capability and you have a chat interface. It ends up being really natural to interact with.

ChatGPT: It’s reasonable to expect improvements in areas such as scale, accuracy, multilingual capabilities, contextual understanding, and consistency.

Source: https://eu.usatoday.com/
AND
https://www.nejm.org/