How one patient found errors in the algorithm making transplant decisions

How one patient found errors in the algorithm making transplant decisions(ft.com)

43 points by mrtedbear 2 years ago | 22 comments

isaacfrond 2 years ago |

As far as I can tell the algorithm was not flawed. At least not in the sense that is produced different outputs than intended. The algorithm was designed to favor older patients, and it did just that. The result was that older patients received more livers. As was the intention.

So it not an algorithm that is flawed, it is a policy that is flawed. A policy that flawlessly and fairly executed by algorithm exactly as it was designed.

I'm not saying the policy was bad or fair. I have no idea to be honest. If you have one liver and two patients, then it's always going to be hard choice. But I don't think it is helpful to say the algorithm was misbehaving when it was not.

In fact, as mentioned in article, the outcomes of the algorithm are regularly checked by humans. And when they found a genuine bug (misclassifying people with liver cancer) the algorithm was fixed. Isn't that more or less exactly what you want. Humans thinking about policy, then having a computer executing the policy, while humans regularly check its output to see if the algorithm aligns with the intention.

paulryanrogers 2 years ago | |

Don't medical professionals strive to maximize years of life/health, at least in zerosum situations? Hence the whole triage thing.

Favoring older patients could be better at that or worse. But I'd suspect it's worse unless there is data proving otherwise.

bitshiftfaced 2 years ago | | |

That's how I'd like to think that they'd go about it. I'd be interested in learning the details. Say for example that younger people tend to be able to go without for a longer period than older people. I don't know if that's true, but if so you could prolong the lives of older people now but then get to the younger folks later, but not necessarily vice versa.

dpz 2 years ago |

Just finished reading this, I get why it's done but its really tragic and sad that an algorithm is deciding who lives or dies. Could you imagine working on something like that? I don't think I'd have the stomach.

GaelFG 2 years ago | |

I'm glad and don't see better solution than to use an algorithm to decide this over personnal choice. I understand the choice of using software. What I don't understand is why this software is not open source, heavily documented and the algorithm annually monitored, publicly reported and updated if necessessary by an open committee.

I personally prefer avoid being involved as much as possible in human life and death inducing procedure because that awfully stressful, but I don't see big problem on working on the software coding itself. What is problematic in this case is that the software was used to deinvolve everyone from the process and for what I understand the software developpement team was apparently tasked to make up the decision algorihtm and no human basic oversight was keeped during the attribution process.

dghlsakjg 2 years ago | |

It sounds like the algorithm in question was designed by a committee of experts to make a better triage process. Given that the previous process for deciding who gets an organ sounded a lot like individual doctors just picking patients based on proximity and how good the patient advocacy was, I think this is a much fairer alternative.

At the end of the day, triage is always a hard thing to do, but it is based in the idea of optimizing for the best global outcome at the expense of specific individual outcomes.

I personally would have no qualms about implementing the algorithm design in code (the organ score is going to be computed by hand or computed by code regardless, might as make a tool that saves time and mistakes), and would have no problem participating in the algorithm design if I had the appropriate expertise.

At the end of the day I'd rather live in a world where there is a known, predictable process for these decisions instead of an informal network of professionals making game-time decisions.

jowea 2 years ago | |

Does it matter whether it is a bureaucrat executing the algorithm with paper or software on a computer? The issue must be decided somehow.

lacrimacida 2 years ago | | |

At least there’s a hope the person doing it has conscience and does a good job, vetting the data, etc. They could use tech but in the end it’s a human’s decision over an algo. Casual mistakes are okay as long as they’re not embedded in an algo that applies the same mistakes over and over.

JudasGoat 2 years ago | |

At a recent physical, my doctor informed me that "according to the drug company's algorithm" I should begin taking a cholesterol reducing drug. I had to wonder about the transparency of such an approach.

sodii 2 years ago | | |

'algorithm' in that context is not a computer program, but a flow chart or decision tree made by distilling the result of various medical trials.

RecycledEle 2 years ago | |

You are correct. Those of us with moral values avoid this kind of work. Only the sociopaths do this kind of work. They only care about gaining power.

Maybe genetically modified pig organs will eliminate the supply shortfall in organs and get rid of the power trip these jokers enjoy so much.

orbz 2 years ago | | |

This feels unfair to say.

People with moral values don’t and shouldn’t avoid this kind of work to make sure that they care about accuracy, edge cases, that they continuously refine how they define utility and try to view it from a place of empathy.

Doctors and first responders make these calls often when they perform mass casualty triage, or when to declare someone’s death to enable collection of life saving organs.

Suzuran 2 years ago | | |

The "supply shortfall" is due to the requirement that the DNA of the organ has to match up with the DNA of the recipient in order for the organ to function and the fact that the organ has a very short "shelf life" when outside of the donor's body. You can't just randomly stuff organs into people and expect a good outcome, and you can't just stockpile them.

Until we can customize the DNA in the organ for the intended recipient and manufacture them on demand with a short lead time, we are going to have some kind of lack of availability issue. Our issues are not a bunch of power-tripping sociopaths sitting on a hoard of organs deciding who lives or dies, they are trying to rapidly identify who is physically close enough and who genetically matches in the hours you have when an organ becomes available, and is there any opportunity for a cascade donation to be brought into the mix (is this organ compatible with someone who has a willing but incompatible donor who is compatible with someone else, and does that person have a willing but incompatible donor, and so on down the line)

(Edit: I have been informed that the paywalled article, which I cannot read, is about a UK system; I am only familiar with the kidney system in the US. This post probably does not reflect UK practice)

dado3212 2 years ago | | |

Or an alternative framing: using data can help to remove human biases and increase overall lifespans (which the studies have shown). How is it sociopathic to work on software that demonstrably saves lives?

kameit00 2 years ago |

Is there another article, where we can read the content? Or can somebody summarize the article? I have no FT subscription :(

sidkhanooja 2 years ago | |

https://archive.is/PzN0p

jowea 2 years ago |

I didn't get it from the article, even through it was written by the artificial intelligence editor, did the algorithm use some machine learning something technique or is it a manually programmed calculator? Although in both cases it seems it has gone through a lot of the stuff that can wrong with this sort of thing.

helsinkiandrew 2 years ago |

I believe (but am not 100% certain) that this is the calculator mentioned in the article:

https://transplantbenefit.org/