Launching Health in ChatGPT to US Users(openai.com) |
Launching Health in ChatGPT to US Users(openai.com) |
Recently I had Claude analyze an MRI of my knee and it gave me five different potential problems, but once I got the radiologist report - none of those problems were present, it did not detect that it was chondromalacia that I found out from the radiologist report.
Health and the current iteration of LLM's just doesn't seem like a great match.
It's hard for patients to interpret medical research and evidence though... ChatGPT recommended treatment for some of my family members based on studies with 60 patients. It wasn't exactly established medical practice.
I think someone needs to teach these models the old "horses not zebras" adage, or we're gonna have a lot of people insisting on having rare cancers.
I have witnessed a non-radiology case (a STD on the face) where the LLM got it right and the doctor didn't (he said it was diabetes related). It may have saved that person's life by causing them to go to a different doctor with the LLM's diagnosis and getting the proper treatment.
What I don't understand is why don't doctors use LLMs as a tool for a quick second opinion before arriving at a determination? They have my data. They can put it into GPT 5.6 Pro or Fable, read the output, and choose to disregard it if they really want to, but if there's a 20% chance that it spots something they missed, why not?
It's a world of difference to upload the radiologist/pathologist report and go from there than uploading the raw images.
The first is a great idea, the second reckless and dangerous.
Maybe someone who has used it can give some feedback.
When my first kiddo was an infant any google result for information related to even a sneeze said it was an indication of some super tragic situation. Any advice about a behavior or trait ended with basically "your child is destined to be a serial killer".
For any new parents out there, just call your doctor. Stay far away from the internet and AI when it comes to your baby's health.
That said, Claude did say “I’m not a doctor, if in doubt make sure you see one”. So, that’s good… but then, what’s the point in using it?
The real main problem is that you need to wait to get help, may it be a therapist or an eye doctor. But ChatGPT is instant.
This part worries me the most.
https://www.auditor.on.ca/en/content/specialreports/specialr...
I had Claude analyze a blood panel and provide potential diagnoses, medications, dosing and contraindications for a fairly esoteric condition. It gave me the questions to ask a doctor (being completely forthright that I'm yet another layman regurgitating AI output) who agreed with the approach and even admitted that he hadn't seen a case study it referenced but that he wish he knew about sooner.
Will doctors still be assmad that years of their study is now democratized? Probably. I am (or was) a software engineer, welcome to the club.
https://www.auditor.on.ca/en/content/specialreports/specialr...
https://en.wikipedia.org/wiki/List_of_countries_by_total_hea...
Also, what is HIPPA?
Sure, doctors’ advice is best, but sometimes I wait 6 months to get to see a specialist who seems hell bent on getting through our appt as quickly as possible.
Models are infinitely patient and can see a lot more context about my health.
But, the privacy is a big question mark. The page extensively reassures on privacy (no model training, encrypted, no ads, can be deleted). There’s only one part that stands out to me:
> Memory remains in your control. To personalize future chats, memories can be created from your Health conversations. Memories are not created directly from connected medical records and Apple Health information. If you’d like to have a conversation without creating memories, you can use Temporary Chat (opens in a new window) or turn memory off in Settings > Memory.
Great that you can turn it off in settings but I wonder how many people will do that and how much sensitive context leakage will come out of your conversations and suddenly you’ll get ads for rashes and whatnot.
It doesn't apply to information the patient supplies directly.
As a patient, I can take out a street sign displaying my health records if I want. It's my data.
If I want to upload it to a strange AI or any other app, that's completely fine. HIPAA isn't involved, for either me or the company I upload to.
In fact, quite the opposite. CMS has been going to great pains over the past decade or so to create standards to help patients get access to their records and own them themselves.
Different states may have additional protections in this case, and I know there has been some discussion at the federal level about regulating these "wellness" apps.
As an interesting technical detail not many people know, HIPAA only applies to Covered Entities and Business Associates engaging in the electronic exchange of information using "standard transactions", and was intended to regulate insurance and billing transactions.
A doctor that keeps paper charts and bills cash directly to the patient wouldn't fall under HIPAA at all.
Question: Why don't Claude or ChatGPT get message timestamps?
Go to any conversation you’ve had with Claude or ChatGPT in their chat apps and ask them what time they sent their last message, and you’ll find they don’t see timestamps for any of the messages in your chat history. They might guess or hallucinate an answer but they’ll be wildly off and if you probe further they should admit they don’t see timestamps at all, they only know the current time. I’ve also noticed generally that if you pick up conversations over multiple days, it doesn’t know that days have passed in between messages.
——
Me: How much time has passed since your last message?
Claude: I don't have a way to measure the gap between my messages—I don't have a clock running between turns, and nothing in our conversation timestamps the messages for me. So I can't tell you how much time has passed since my last reply. Could've been seconds, could've been days.
If it's useful, I can check the current time, but that would only tell you now, not the interval since I last wrote.
——
This is very bizarre to me and seems like an obvious flaw in the product, but both major AI companies are doing this and not implementing the obvious solution of surfacing message timestamps. Does anyone know why they don’t do this?
For an example of the type of issue this causes, I’ve been using Claude for advice while my dog is going through medical treatment and it’s very concerning because I’ll say I gave him some medicine and then later ask questions about a symptom he’s displaying and Claude doesn’t seem to know whether the medicine was minutes ago or days ago and usually doesn’t ask for clarification either. I’ve learned to either be explicit about how much time has passed or just not ask it questions where timing would be a factor. But I’m surprised because this seems like it should be an easy thing to address.
So if the harness appended a timestamp to each message the model sees, I wouldn't be surprised if the model often responded with "given that it's 12:30 pm on a Sunday..." even in cases where that's absolutely not relevant.
Maybe you could instead give the model a tool it could call that would return timestamps?
What do you think the LLMs are trained on?
I agree the current state of US healthcare is a mess. But that doesn’t mean anything is better. The current state of LLMs are so agreeable I worry what they’ll suggest to users who provide even the slightest pushback.
“Are you sure I need this test?”
“You’re right to question that. It’s unlikely to surface anything you don’t already know”
…while a chronic condition develops under the surface
https://www.nysenate.gov/legislation/bills/2025/S7263/amendm...
The engineering aspect is quite hilarious. I have a PE license, but don't use it. But the PE definition in NY is wide open. Technically, if you do the right work, even writing software can fall under a PE license, though its incredibly rare. So this could theoretically ban LLM Code assistants.
-USA
Even if they say they won't, they will, and there is no telling what they would do with the data itself...
- They avoid doctors because it's expensive and they don't want to burden their families.
- There aren't enough doctors, regardless of social spending.
- There isn't enough social spending.
Even if this only increases bandwidth of existing doctors, or allows lesser-trained nurses to cosplay as a doctor during triage to increase bandwidth, it's a great thing.
With ChatGPT at least I control what I enter in the prompt to minimize my exposure.
There are a wide variety of things, including communication with providers, that OpenAI can do without being regulated by HIPAA. The provider, however would most likely be covered, so they would be under an obligation to not disclose any PHI.
As an example, consider a hypothetical "email my doc" app. This app could take health information from the patient and email it to all of their doctors without falling under HIPAA at all.
_per_person._OECD_countries_and_more.png){kind=link}