Watson could never really deliver on the AI healthcare integration and diagnostic support. Basically it became a status symbol for hospitals to have, but it was dead weight. https://www.statnews.com/2017/09/05/watson-ibm-cancer/

I think this might also be supported by looking at some of the better funded / successfully exited health tech startups.

I’m thinking of things like Flatiron Health, BenevolentAI, Quartet and Spreemo, and surely many more.

At heart these companies seem to shy away from being “really about” medicine, and try to be a more watered down data company, looking at health records, social media, and other data to provide recommendation, service matching, comorbidity analysis, hospitalist tools, cost tools, assistance program allocation tools.

I’ve been very interested in ML applications in health, but a lot of the businesses that seem capable of getting funding seem like they are super light / elementary on the statistical modeling side, and the value add is just a claim to modernize crappy claims data or unify a bunch of previously disparate health data sources.

Maybe more advanced use of modeling will come later, though I am skeptical just given the way that once big enterprise customers dig hooks into essentially consulting services for health records, they won’t let go.

Another thing that creeps me out is when you see e.g. an ex Palantir board member joining health record data companies. It’s not hard to understand why a big insurance company, or at worst even government agencies, want decision tools on top of huge stores of health record data. Do we really benefit from some super new / unproven startup building up that data set & tooling?

Yeah, to be clear, Watson failing is good news. This is marketing people losing their minds because they didn't understand the products they were selling, and hospitals failing to do due diligence.

I've talked to literally dozens of people / companies / academics running at problems in healthcare using deep learning and large data sets, and the recurrent theme is cowboys who pretty consistently fail to follow basic practices of experimental rigor. ML is frankly an area where more rigorous and clear regulation is desperately needed, because the potential to cut corners / cheat is so easy and the challenges of vetting these systems are specialized enough that most organizations won't successfully be able to catch fairly basic experimental mistakes.

To be fair, it did more than participate. It dominated.

I hear what you are saying, but feel like it also requires the caveat of "for in-demand roles where one has leverage in the negotiation."

At this point, a junior engineer may have some of that, but I'm not sure you can really apply that to all levels of experience across all functions. I agree with the sentiment and logic behind it, I just am not sure that is actually something most people could consider.

Interesting article, but I'd bet a big part of the reason physicians don't know it's that do much is dependent upon information that lives "out of the system".

How much time will a patient spend on rehab, and will they actually perform the exercises effectively? What's their diet like, not just calories in but in terms of individual nutrients? How active is their lifestyle?

Recovery time seems like something that should be knowable by educated doctors, but the best poker player in the world can't tell you what card is coming up on the river.

"Hand-coded" may be an exaggeration, depending on your field.

ML systems need good training data (where "good" might mean "extremely voluminous") and careful tweaking of parameters. I think an ML expert would call a system where a substantial amount of time is spent here "hand-coded".

Yes. Well, at the very least, they don't involve only hand-coded things.

The ones that aren't hand-coded tend to act poorly, or racist, or worst.

Unfortunately, from my experience the biggest barriers to sharing data are very deeply embedded incentives.

I was at a talk recently by an executive in charge of managing data for a massive health system. Think tens of millions of patients. He said they dont share data because if people learn their real quality metrics it would give payers, the public, anyone with an interest that isnt aligned w the hospital system ammunition against them. Have heard this sentiment echoed by many. Open data is their enemy because then patients could choose based on outcomes, not on market power or how nice the lobbies are

The EMRs designed a closed garden into their systems DNA. If data is shared freely then it's easy to switch systems, and they will have to compete based on cost and quality and couldn't charge hundreds of millions for installations and maintenance contracts

Call me cynical but this is all based on conversations with people well placed in the industry

I am working on a medical dictation app and the lack of any interoperability between 1000s of EHRs is mind boggling to me. There is absolutely no standards and like you mentioned its completely deliberate. Working through these platforms over the years has made me less and less optimistic about having a better health care system in the future.

As a patient, I'd love to be able to archive/hold my own comprehensive copy of all of my health records.

I disagree too. I think that humor should be allowed on hacker news. I mean as long as the jokes aren't at the expense of the ideology of the majority of hacker news's users. In that case, that humor should be downvoted and removed.

I agree that there's room for humor. But for that, it should be well-written satire that makes that point.

"[T]hey renamed it Watson Health due to the fact Watson is going to run the entire division itself" is a complete non-sequitur, just filling up space.

> The FDA is not the primary source of regulation in medical software, it's CMS

It's totally dependent on what you are doing. Sure, HIPAA / PHI concerns are the primary issues for people who generally make software for the medical space, but by "medical software" I mean software that FDA might classify as "software as a medical device" AKA FDA regulated software. If your software falls under this designation, as most diagnostic medical AI systems like Watson would, then your primary regulatory concern is the FDA and you are held to much more stringent software development requirements than someone that has to deal with some PHI concerns.

In this context, my bulk de-regulation comment is 100% correct, as the FDA has basically dictated almost everything in the space to be unregulated (all back office products, general wellness products, enforcement discretion products, MDDS, even CDS which almost certainly should be regulated). They have literally gutted Class 1. You basically have to be writing something that is going to provide a diagnosis or treatment plan, control an existing medical device remotely, or mimic an existing regulated medical device in functionality in order to be regulated now. Reasonable people weren't scared to enter the space because they might have to deal with PHI for HIPAA concerns, as it is well known how to do that at this stage; they were scared to enter because there was (and continues to be) some uncertainty as to how novel product classes will be regulated by FDA.

> Your concerns on the sales side also do not mention "RISK" even once. You don't speak even a little bit of our healthcare space. Those doctors want risk management. What happens if/when the software screws up? Whose fault is it that the software is literally response for killing patients?

All FDA regulated software is mandated to have robust risk management planning prior to approval (ie ISO 14971 compliance). Other than that, the vast majority of products that a patient will be interfacing with can't do them real harm if they are buggy. That is precisely why the FDA has chosen not to regulate them.

> You also don't mention what doctors care about: does it meet CMS regs? Does it meet state regs? Will I be able to submit my obscure XYZ state form? Will I be able to submit to BCBS, Aetna, Medicare? Will they accept? Etc etc. You will be writing custom software for customers all over, because the regulations that states and even cities like NYC might impose could be imposed on a single customer of yours only.

I don't mention doctor wants because doctors aren't the buyers most of the time. Payers are the buyers, hospitals are the buyers, or patients are the buyers. How many pieces of software that you know of are sold directly to the doctor?

Learn MUMPS and move to Wisconsin.

You get at the door of companies in the healthcare space that need to get up to their game and tell them what you can do for them. That's what I did.

I was recruited right out of college to work at Cerner. Its probably still fairly trivial to get a job there, but you have to live in Kansas City.

No, Theranos started to fail as soon as anyone with real medical expertise without a stake in the company started to probe it.

Theranos was extremely successful at fleecing VCs and embarrassing political board members.

It's a good thing that it got so far, because it proves just how bad VCs are at evaluating successful companies and how much of the funding game is who you know.

Arguably, the big deals and optimism indicate that there is a lot of pent up demand for successful things, and people are willing to take some risk to get solutions. It’s not all doom and gloom, solutions just have to work.

I worked on a project that did a little Watson work as well. Same result as you described.

The shame is decision tree based expert systems can be pretty powerful, but the tools these guys were using were not that great.

Epic still dominates the EHR space.

They have deployments slotted for 3 years.

If you have ever worked with epic, they offer "managed" solutions, which you either do the epic designed way, or don't go epic... They have enough market share and business to dictate how things will work in a hospital env

For diagnostic imaging (x-rays, CTs etc) which we make software for, alot of equipment is FDA regulated.

We chose to go for class 2 designation in FDA terms as it shows you comply with uptmost regulations, which is a selling point. Some hospitals depending on state gets rebates for meeting compliance.

It depends on what space you are going into. If it touches a patient, more than likely FDA regulated in the USA.

Other countries have similar orgs ( Canada has... Health Canada).

We also work with DoD/VA... Don't get me started on compliance with them.(the security stuff is a plus though, while initially challenging to meet, it has us pushing the code into commercial space products, everyone benefits from the extra security hardening. )

What it entails depends on the product. FDA has 2 regulatory classes which require a premarket notification, class 2 and class 3. Class 2 submissions generally require you to comply with a set of software development standards governing testing and validation, requirements generation at multiple levels, interface design, risk management, traceability, design, and actual development. You are required to handle customer complaints, safety problems, and maintenance in specific ways. You are required to produce customer facing documentation in specific ways, have installation plans, etc. Finally, you need to register yearly with the FDA and have all sorts of training and infrastructure planning. Proof of all of this is to be constantly generated (think every test report) and it all gets put into a massive file for FDA review. There are also some writeups you need to do explaining why you are class 2 as opposed to class 3. This generally involves finding a class 2 piece of software which resembles yours and saying your software is basically doing the same thing. This is typically referred to as a 510(k) submission.

There is also a pathway you can apply for if there is no existing device with a class 2 designation which resembles yours, but you believe your device is still fairly low risk. That is called a de Novo submission.

A class 3 submission generally involves even stricter requirements for development plus a clinical trial to prove efficacy and safety. The most common one if these is called a PMA which stands for premarket approval.

Also, if you are FDA regulated you still have to comply with CMS regs. It is an extra layer of regulatory scrutiny.

I've been doing a lot of work lately with automated Abductive Inference, and medical diagnosis has been cited as one of the key uses of same. Perhaps there could be some opportunity to collaborate (everything I do is open source as well, btw). If you guys would like to talk, feel free to hit me up at prhodes@fogbeam.com