While the actual “study” is a little old and it is possible that AI is better now, but I’m not holding my breath. While it is great people can do basic research on their own medical conditions, stuff like this does not bode well. Cliff’s Notes: a made up disease was “published” with tons of clues along the lines of an April Fool’s joke, and it got picked up by AI and was then starting to be referenced in other journals.
2 Likes
This was several years ago, but my health scare was a possible glioma. The first thing the surgeon said to me is “if you have questions, call me…don’t seek answers online because they’re probably wrong”.
It’s OK to educate yourself, but there is no substitute for the doctor who’s looking at your images personally. If you don’t trust your doctor over “the google”, get a new doctor. And educate yourself with real articles (like NIH), not some “.net” blog, or a site that is trying to sell you on something.
7 Likes
Problem with AI and its recognized susceptibility to manipulation, hallucinations and whatnot is that these phenom are only really obvious in a field where any user is already knowledgeable and experienced. That isn’t confined to healthcare either.
Regardless, its use still continues to be seductive to folk who want to score points in any discussion or to pretend to insight they don’t actually possess …oftentimes without even bothering to read the vomitus a Google Bamboozle fetches up. Matters not whether AI is getting better or worse, I’m not convinced the majority using it bother to wonder if it makes a difference.
150,000 I.U. Vit D…
2 Likes
Better, worse? I guess it’s a matter of perspective. It might be making a difference in all the wrong ways.
1 Like
The problem is that at this stage LLMs are idiot intelligent. Good to get information but I would not trust my life to them. In the searches for covered calls I’ve posted I have detected at least two errors. Funny thing, when I asked about one of them Google AI confirmed the error. What this tells me is that Google’s LLM does not cross reference its output with all the “knowledge” it has accumulated.
It’s still early AI days yet. 
The Captain
3 Likes
If I had an AI doctor, I bet I’d still have my leg.
intercst
3 Likes
That surprises me, too. It should be pretty easy to check if a footnoted study is bogus or not.
intercst
I think the primary problem is that current LLM’s are all about “pattern matching”, and not about ACTUAL LEARNING. They don’t know how to REASON. They are simply (very impressive) stochastic parrots. And there is a reason why we don’t teach our children this way.
6 Likes
The job of pattern matching is just that and it’s what our subconscious does but we have a second brain, a boolean brain that does the thinking. Some boolean software (Python?) has been added to LLMs but they needs much, much more.
The Captain
1 Like
But missing half your liver and your dominant arm.
For healthcare, AI speeds up routine tasks (data processing, pattern recognition), but human expertise is required for strategic decisions and trust.
While I’m wholeheartedly in alignment with this point of view as a complete, correct reference point, I have considerable concerns about how this is presented to the patient, office and clinical support staff as well as the decision making role in the process as well.
For patients using this incorrectly, I believe they can:
- Suffer imagined or supported (incorrect) theses while using AI tools at home
- Improperly influence care plan and decision processes (unnecessary scans, unnecessary appointments, undue influence to clinical team, etc.)
- Push out availability to others who truly need care, thus removing available resources.
- Potentially lose the services for a provider as a bad customer
If the customer is always right, then the wrong human is in the loop!
For clinical staff incorrect use, I think we all understand:
- Fatigue in process. As thought (and attention) requirements decrease, clerical and logical errors will rise (human nature). If AI is not capable, or not enabled for these aspects, those errors will propagate.
- Declining capability. We are seeing this with transactional workers in customer service now (medical and other fields). So much so, that the “Drone behind the counter” reference are now internet memes.
- Increasing systemic risks. Problem is not well handled, requiring even more steps and rework between specialists and other clinical staff. I predict people will be labeled hypochondriacs with increasing frequency, ironically NOT because of their use of AI to inquire about their conditions.
Anyways, these are not significant problems in the long run…
I’m sure that when AI is finally ready, it will have solved the problems…
.
.
.
2 Likes
Checking to see if a footnoted citation is actually from a reputable journal is “pattern matching”.
Same thing with bogus citations in legal briefs. It should be easy enough to run the output against the Lexis+ database to check if it’s actual case law.
intercst