AI Helping Doctor Take Notes

[64K]

This is my first encounter with such a thing but today at my regular Dr appointment she asked if it would be alright to use an AI aid to record our conversation and help her to take notes. She assured me that she would still be taking notes by writing some things down but that it would save her some work in not taking so many notes. I didn't see the harm in it but now that I've had a chance to think about it I'm not sure if I want to approve of that in the future. I'm not sure how what I'm saying is being prioritized or understood correctly and in proper context to what is being discussed and if this AI is drawing some medical conclusions either. Has anyone else encountered this yet?

 

[cvaldes]

Apart from the AI assistance this is nothing new.

Before the pandemic I accompanied a family member to a doctor's appointment. He was wearing a pair of Google Glasses and asked if he could use it for the transcription. That family member looked at me for my opinion and I said "decline." So the doctor took off his Google Glasses making it very clear that he would have to do the note transcription himself.

At the time (2019 and earlier) third parties were used for appointment note transcriptions. Google Glass was being used to do an audio recording of the visit to be transcribed later.

I've been using Google Voice since before it was acquired by Google (it was called GrandCentral in the early years). Based on my experience with Google's voicemail transcription feature, I still think automated/AI processing is crap.

Today AI is doing the same thing (much like corporate meeting focused otter.ai is doing). The problem is you don't really know how well the security is between the medical clinic and whatever service is running this technology.

At some point I will probably cave in but there is no way in hell that I will be a guinea pig for early efforts. There are always early apprehension sentiments for any new introduction of medical technology, especially when it relates strongly to patient privacy issues.

Worse, if AI transcribes a voice message/note, there still needs to be someone who picked out the important parts. When a human does the transcription, if they are summarizing, they will focus on important elements. AI is still too stupid to know what's right or wrong, when someone is just babbling or joking.

The biggest problem with these note taking transcription services is that they distance the caregiver from the patient. Yes, they are more efficient but they don't actually improve the quality of healthcare. They mostly help bottom lines (financial).

This isn't specific to healthcare data. I see the same thing with my investment portfolio monitoring. It has been years since my copy of Quicken 2017 downloaded stock quotes & brokerage transactions from the Internet. So I manually enter these myself and find myself questioning things more closely.

Today I have an Excel spreadsheet that provides a high level snapshot of my investment portfolio. But I balked on trying to figure out how to automate the stock price updating. I'd rather manually type them in which forces me to look at the numbers carefully.

At some point having AI do this for me will likely be a better solution. But not in 2024, at least on a consumer level. So much of AI is at this awkward stage right now: not ready for primetime.

 

[Superzuber]

Today I have an Excel spreadsheet that provides a high level snapshot of my investment portfolio. But I balked on trying to figure out how to automate the stock price updating. I'd rather manually type them in which forces me to look at the numbers carefully.

this can be done in MS Excel if you have source with API, which shouldn't be issue

https://www.codementor.io/@benniejayguevarra/importing-public-web-api-to-excel-no-coding-required-14bzfy4nev

you could also ask MS Copilot to fill your uploaded sheet, maybe just the tickers without the actual numbers to keep it private, but the first solution with API is straightforward and without thirdparty apps or AI

 

[leeamtheone]

Oh, when I saw your post on doctors, notes and AI, I too was a bit like "Is this really going to make things easier, or am I just handing over control?" But, at the same time, I understand why they want to experiment with new things.

I mean, I’ve had my moments of doubt too, but eventually, I did cave in and decided to give AI a try just out of curiosity. And honestly, I'm becoming more of a fan of the automation process when I see the time it saves me.

There's a balance to be struck, for sure (especially with privacy). But, I'm trying to look at it from a positive perspective.

 

[cvaldes]

At my annual physical last month my primary care physician asked if she could use an AI transcription service to take appointment notes. I declined for pretty much the same reasons I listed in my earlier response.

I will likely continue to decline until there is more disclosure about the security protocols used in the process and reasonable assurance that the data isn’t handled by some faceless third party provider.

There’s already so much of that in the healthcare industry but I have no interest in being a guinea pig at this stage in AI development. I’ll probably reconsider my stance in five years without any expectations that my opinion will be different.

TRUST IS EARNED.

 

[64K]

At my annual physical last month my primary care physician asked if she could use an AI transcription service to take appointment notes. I declined for pretty much the same reasons I listed in my earlier response.

I will likely continue to decline until there is more disclosure about the security protocols used in the process and reasonable assurance that the data isn’t handled my some faceless third party provider.

There’s already so much of that in the healthcare industry but I have no interest in being a guinea pig at this stage in AI development. I’ll probably reconsider my stance in five years without any expectations that my opinion will be different.

TRUST IS EARNED.

That's what I will do in the future. I didn't think about it until later and realized what a problem an AI being involved in something as critical as my health care and the accuracy and importance of some notes over others and already numerous examples of what is now labeled as "AI hallucinations" which is a chic way of saying the program derped because you can't program common sense. Well anyway, potential future problem averted.

 

[Steevo]

They have been trying to implement this in the medical field for notes since the early 2000s.

I manage the IT for a few Drs offices and their voice recognition software for medical terms has been very lackluster. AI is actually suited for this, as long as it’s used correctly in voice recognition on device. Most of them stopped using the implementations they have. (Wow, AI assisted words salad) One hired a temp to check it’s work and found too many errors

 

[cvaldes]

They have been trying to implement this in the medical field for notes since the early 2000s.

I manage the IT for a few Drs offices and their voice recognition software for medical terms has been very lackluster. AI is actually suited for this, as long as it’s used correctly in voice recognition on device. Most of them stopped using the implementations they have. (Wow, AI assisted words salad) One hired a temp to check it’s work and found too many errors

This basically confirms all of my concerns in my original response.

Voice transcription is still garbage in 2024. I have a bunch of medical providers who speak in accents. I have no problem with that but voice transcription technology is still too undeveloped to handle this.

I have zero assurance that current voice transcription is on device and accurate. I already witness the outright butchery that Google Voice does to my voice messages. It's pathetic. And I'm to trust some voice transcription service to handle my physician's notes? HELL NO.

I know that AI already has shown great promise in enterprise usage cases, particularly workloads that are math or physics related. 2024 AI is still crushingly lame in voice-driven tasks. I believe AI will revolutionize the healthcare industry but not in all workloads at the same time. It will do better at math/physics/chemistry based workloads (like pharmaceutical research) earlier than it can handle more nuanced tasks like voice transcription.

As always, much of this will end up being up to the consumer to decide whether or not to accept things. There are laws that govern electronic healthcare recording keeping. Basically when you allow a medical provider to use AI voice transcription service, you are waive your protection.

Worse, once I approve AI transcription approval, I don't know if I've given carte blanche for having some AI bot plow through decades worth of medical records.

I'm all for progress in implementing AI in healthcare but the last thing I will give up is my own personal medical information. Maybe in 2030, 2035 but certainly not next month, next year, or in 2026.

 

[damric]

A mediocre AI would probably do a better job than my primary care provider

 

[ir_cow]

Google is better than most of the doctors I've been to. I'll take AI Doc if it is free.

 

[cvaldes]

Google is better than most of the doctors I've been to. I'll take AI Doc if it is free.

But that's not what this is. This is just a service that transcribes what your doctor says. It's not an alternative healthcare provider.

I don't care whether my primary care provider has 2 or 20 years under their belt if they are going to use a voice transcription service. I just don't want the latter right now.

If your doctor sucks, get a new one. Medical schools churn out new ones every year. Keep trying until you find one you like. Remember that ALL old doctors started as rookies. Best of luck.

 

[R-T-B]

Google is better than most of the doctors I've been to. I'll take AI Doc if it is free.

I say this sadly being in conpletely the same boat bear in mind: this is entirely a "your doctor" personal issue that should never be the case.

If only changing doctors was remotely easy...

 

[Hyderz]

Taking notes is fine, as long as your doctor still go through the proper procedure to examine you and give you feedback and prescribe you the right medication then it’s all good. The notes is there for doctor to refer to on your medical history, the world has a shortage of doctors and nurses. If the ai can help in taking notes and make their job a bit easier then sure

 

[Redwoodz]

Since when do we need AI to record a conversation?

 

[64K]

Since when do we need AI to record a conversation?

From what I understood from my Dr. it's more than that. The AI is supposed to help her organize her notes as well. It's the word 'organize' that I'm not comfortable with. I'm going to call her office in the morning and ask the name of the AI program she's using and maybe post some info about what it does for doctors here.

 

[Redwoodz]

Sounds like data mining to me, so they can replace the doctors in the future in many instances. Funny how all these AI applications are really designed to replace the people that are using them.

 

[JIWIL]

In the late 90's I worked for a company that digitalized, summarized, and made chronologies of medical records for litigation. We employed more than 100 RN's who would double check each other's work, a dozen or so senor Nurse's who would do a third read, and four English majors to edit the summaries. I can totally imagine an AI doing a years worth of work in a few minutes these days.

If you've ever looked at a hand-written prescription and gone 'wtf?'... just picture a warehouse full of boxes with that handwriting. I'm not sure what scares me more.. the past or the future.

 

[Steevo]

In the late 90's I worked for a company that digitalized, summarized, and made chronologies of medical records for litigation. We employed more than 100 RN's who would double check each other's work, a dozen or so senor Nurse's who would do a third read, and four English majors to edit the summaries. I can totally imagine an AI doing a years worth of work in a few minutes these days.

If you've ever looked at a hand-written prescription and gone 'wtf?'... just picture a warehouse full of boxes with that handwriting. I'm not sure what scares me more.. the past or the future.

Most offices won't allow Dr's to have a pad anymore here, it's all digital, they have to find the name, dose and regimen that gets sent to your pharmacy. The last one that gave an actual prescription it was printed and had a QR code verification as well and on special paper. The paper had some of the QR code and the letterhead and the rest was printed along with the patient name. It didn't last long as the printing was an issue since it came out of a large copy machine and got made into copies and other stuff printed on it.

Also you can buy docuguard paper and still print it yourself, so I'm sure whoever came up with the idea thought it was revolutionary.

Dragon Medical One - #1 Clinical Documentation Companion | Nuance

Go beyond speech with Nuance Dragon Medical One, a comprehensive medical dictation solution that can aid, assist, and advise, from pre-charting through post-encounter documentation.

www.nuance.com

This is the one they have been using.

 

[Assimilator]

My company has a service department that uses an LLM to summarize agents' conversations with customers. It works well until it starts hallucinating, at which point we have to nuke the LLM instance and spin up a fresh one. This happens about once a month, and it's acceptable because we aren't dealing with any conversations that might affect a person medically.

But for doctors to be using this? Holy shit, that's terrifying. LLMs are dumb-as-rocks next token predictors. They predict wrong because they don't have context to do it right. And context, especially in medicine, is vital. How long before an LLM hallucinates and puts something completely nonsensical in the notes, the doctor ends up dispensing the completely wrong medication as a result, and the patient dies?

The sad thing is that there is no way around this. Doctors will start using it allows them to see more people in the same amount of time, and thus make more money. Those that don't will be missing out, so nobody won't. And thus we get to the next level of beautiful capitalist dystopia whereby every doctor uses a LLM for notes, it's impossible for patients to opt out, and any resultant patient deaths from LLM hallucinations are viewed as normal.

Welcome to capitalism, where the health and welfare of the masses is sacrificed on the altar of worship of unsuitable technology, solely because that makes a handful of people really really rich.

 

[cvaldes]

Since when do we need AI to record a conversation?

We don't. There have been voice transcription services for decades. The conversation is recorded and someone does the audio-to-text transcription after the fact. Or someone does the transcriptions in realtime. Shorthand is a mostly archaic form of writing specifically for this practice. In courts, they are call stenographers. They have been around for a long, long time.

And there have been simple voice to text transcription computer based services (like visual voicemail) for years.

However these healthcare AI transcribers will eventually categorize things. Ideally the AI would figure out who's talking, what's an observation, what's a recommendation, what's an opinion, etc. and populate a patient office visit record with the correct information in the proper fields. It's not supposed to be basic voice-to-text translation. These are all things the physician has to do themselves during/after the patient visit if they aren't using a transcription service.

I don't want to be a guinea pig for this. I want the least amount of my healthcare data being sent off to some third party. Right now, there isn't enough disclosure and visibility about how much is being sent, to whom, and how securely. That's why I continue to decline when my physicians ask if they can use it on me.

At some point, the technology may end up being more reliable than manual human efforts, just like anti-lock brakes in a car or autopilot in a transoceanic jetliner. But I'm willing to wait until these service providers get it right. Once again...

TRUST IS EARNED.

 

[Count von Schwalbe]

However these healthcare AI transcribers will eventually categorize things. Ideally the AI would figure out who's talking, what's an observation, what's a recommendation, what's an opinion, etc. and populate a patient office visit record with the correct information in the proper fields. It's not supposed to be basic voice-to-text translation. These are all things the physician has to do themselves during/after the patient visit if they aren't using a transcription service.

And who does all this? Some intern looking for a buck? Some third party company?

I don't want to be a guinea pig for this. I want the least amount of my healthcare data being sent off to some third party. Right now, there isn't enough disclosure and visibility about how much is being sent, to whom, and how securely. That's why I continue to decline when my physicians ask if they can use it on me.

Not enough is disclosed, AI or no AI.

 

[cvaldes]

And who does all this? Some intern looking for a buck? Some third party company?

It wouldn’t be an intern or some random schmuck, HIPAA compliance is a little too tricky for amateur hour.

Undoubtedly the companies offering AI-assisted transcription services are a mix of startups and established companies who have been employing humans (in some cases for decades).

Silicon Valley is deep in startups who want to carve out a chunk of any industry. While it is a meeting transcription service, Otter.ai is one example of an AI-powered transcription service in startup mode that is undoubtedly looking to get acquired by a bigger player.

The essence of Silicon Valley is the “we can do this way better than the way it’s been done right now” attitude.

Naturally some of the existing companies who have been using humans are looking for a way to reduce headcount and speed up processing by leveraging AI. If you ran one of these companies you better be eyeing AI because for sure the competition is headed down this path.

 

[cvaldes]

Yet another reason to say no to AI-assisted medical transcription in October 2024:

Researchers say an AI-powered transcription tool used in hospitals invents things no one ever said

Whisper is a popular transcription tool powered by artificial intelligence, but it has a major flaw. It makes things up that were never said.

apnews.com

A lot of AI "features" are still alpha or early beta quality, even enterprise-focused workloads. Someday it will get there and be more accurate and faster than humans. But not today, not tomorrow, not next week, nor next month. Even if a experienced senior transcriber reviews all AI-generated transcript, forget it. Count me out for now.

I'm not interested in being the guinea pig, thank you very much.



Note that I still consider all consumer-facing AI features to be alpha/early beta quality. There is no AI for consumers that is remotely close to being release quality. When you have this mindframe it's easier to understand how much to AI, when to say "no" and laugh at the results.

LLMs have zero common sense. They are modeled after human beings and even before LLMs it was clear that a substantial portion of human beings have very little common sense at all. Programmers have an old acronym for this: GIGO (Garbage In, Garbage Out).

If you base your LLM on Redditors' answers, well, there's one reason why AI chatbots are more of an SNL comedy skit than a credible tool in 2024. A human being with half a brain will figure out that a Redditor's advice to use glue as a pizza topping is a joke. AI chatbots do not understand humor, don't have any common sense. But they are good at solving math and some engineering problems.

It's one thing to use an AI chatbot and chuckle at the answers it spits back. It's something else to hand over your health data to some cuckoo LLM.

 

[Assimilator]

They are modeled after human beings

They're not, though. Being trained on human data doesn't mean they're modelled on humans. And the reason they get stuff so hilariously wrong is that, as I've mentioned before, they are correlation without causation; they know that 1 + 2 = 3 because the data tells them this, but they don't understand the mathematical principles as to why the answer is 3, so they are unable to answer the general question "what is M + N?" unless that same equation and answer appears in their training data. If it doesn't, they fall back to correlating i.e. looking for cases in their data containing M and/or N, and if they find any they analyse those and try to figure out which is the most likely combination that will give the correct answer, and a lot of the time they choose wrong.

If this sounds incredibly stupid to you, that's because it is. Why would you try to solve M + N with statistical data analysis, when you can solve it with basic mathematical principles that any human child with schooling understands? And yet this is what LLMs do because they are incapable of understanding anything except correlation. This is why they are not the next big thing that will change the world for the better, because they are and never can be more than frequency analysis and extrapolation on steroids. And don't get me wrong, there are scenarios in which that is an incredibly useful tool, but do you know what is even more useful? Context.

 

[eidairaman1]

This is my first encounter with such a thing but today at my regular Dr appointment she asked if it would be alright to use an AI aid to record our conversation and help her to take notes. She assured me that she would still be taking notes by writing some things down but that it would save her some work in not taking so many notes. I didn't see the harm in it but now that I've had a chance to think about it I'm not sure if I want to approve of that in the future. I'm not sure how what I'm saying is being prioritized or understood correctly and in proper context to what is being discussed and if this AI is drawing some medical conclusions either. Has anyone else encountered this yet?

I wouldn't trust it, To me it's opening HIPAA vulnerability and can lead to a HIPAA violation.

Most offices won't allow Dr's to have a pad anymore here, it's all digital, they have to find the name, dose and regimen that gets sent to your pharmacy. The last one that gave an actual prescription it was printed and had a QR code verification as well and on special paper. The paper had some of the QR code and the letterhead and the rest was printed along with the patient name. It didn't last long as the printing was an issue since it came out of a large copy machine and got made into copies and other stuff printed on it.

Also you can buy docuguard paper and still print it yourself, so I'm sure whoever came up with the idea thought it was revolutionary.

Dragon Medical One - #1 Clinical Documentation Companion | Nuance

Go beyond speech with Nuance Dragon Medical One, a comprehensive medical dictation solution that can aid, assist, and advise, from pre-charting through post-encounter documentation.

www.nuance.com

This is the one they have been using.

Carbon copies ftw man, just like facsimile data transport.