‘One of the key tenets of At the Limits has been to provide medical education to the widest possible audience, irrespective of global location or wealth. It’s a way in which we can contribute to the democratisation of healthcare. And that’s something that will become an ever more important subject in future years as the impact of AI becomes more ingrained into healthcare provision. In fact we had the privilege to welcome Dirk Smeets, the CTO of Icometrix from Belgium to our Multiple Sclerosis at the Limits meeting last September, where he presented on the subject of demystifying AI and embedding it into clinical practice. (He’s at the MS Nurses meeting in November 2024 reprising the topic for our nurse audience). Here’s the video:
A further example of AI development was revealed last month, when the results of a test of the Articulate Medical Intelligence Explorer’ (AMIE), an algorithm written by Google developers, was made available online along with an associated paper entitled ‘Towards conversational, diagnostic AI’. To cut a long story short the system is a text-based chat, but that mimics the discussion a patient might have with a human doctor.
The process is fairly straightforward, the chatbot asks the patient relevant questions to try elicit evidence about their medical complaint. The system compiles details of elements such as family history, medical histories, medication etc. to then try and diagnose what the patient is suffering from. The chatbot was created by using LLM’s which are (apparently!) large language models developed by Google. It looks like Google started with a diagnosis and then used an LLM to generate a clinical vignette very similar to the ones that doctors will be posed with at medical exams.
Pretty straightforward so far, but then a second LLM uses that data to create a simulated discussion between doctor and their patient. A third LLM then plays the role of critic to decide whether the dialogue was correct, believable, and most importantly, accurate.
And the results are in
Once all of that was in place, it was a question of testing to see if the model had achieved its aim. Researchers enrolled 20 participants to take part in the studies. They weren’t actual patients but actors who have been trained to report on specific symptoms. They then engaged in discussions with the chatbot using a text-based interface although in some cases it was an actual doctor the actors were chatting to, but they had no idea which, the computer or the doctors themselves. They simply recorded their experiences and how they felt about the responses they were getting.
And this is a wake up call. The algorithm demonstrated superior accuracy in diagnosing respiratory issues and cardiovascular conditions (amongst others) than a human doctor. Additionally, as far as the patients were concerned, one of the real differences was the fact that the chatbot displayed a higher level of empathy and friendliness than real life doctors!
The Google system either matched or surpassed the doctors diagnosis across a total of six medical specialities and in 24 out of 26 criteria for conversation quality, including politeness, symptom explanation, treatment, honesty, thoroughness and engagement, AMIE outperformed the humans.
I don’t think any of us expect artificial intelligence to replace our doctor or specialist anytime soon, but a process to accelerate healthcare with a general diagnosis being triaged far quicker obviously gives the opportunity for greater benefits. But how much would you trust your confidential medical data with Google (and others of course).
Who decides?
The question will really arise when AI starts to decide on the best treatment available for your particular condition. Being able to draw on vast amounts of data from similar patients’ outcomes for specific drugs, balanced against costs, the future will be a very interesting one for pharmaceutical companies. It will simply be a question of delivering the data and letting the computer decide what’s best. No more salesforces required!
