IBM's Watson For Business: The $1 Billion Siri Slayer(fastcompany.com) |
IBM's Watson For Business: The $1 Billion Siri Slayer(fastcompany.com) |
That said, the approach and subsequent utility might not live up to the hype that IBM is pumping out. It's one thing to search very quickly. Being able to discover patterns that lead to new levels of understanding and predictable relationships is another thing entirely. IBM is more search vs predict in part because they only have so much data to work with. All of the medical books in the world are a drop in the bucket in terms of algorithmic understanding. Watson has mastered working with all available information. Collecting and processing massive data sets is another challenge that IBM hasn't been willing to tackle yet.
IBM is billing Watson as the all singing, all dancing solution to the world's data problems. They're tackling a lot of problems in diverse areas. I hope it works out, the world needs as much help as it can get. But IBM has shifted their core mission to be consulting and I wonder if Watson's purpose will be to support that more than becoming a Super Siri type software project that could do the most good.
It may only be better than humans at buzzing in, but being as good as humans at natural language search, but faster and more consistently (doesn't make mistakes when tired; works just as well in Kampala as New York; can be audited when it makes mistakes) is already better than humans.
I don't get this, Watson might not be able to provide new levels of understanding but it still does a much better job than current search engines, so why do you think it cannot live up to the hype? Why do you think that it is not a significant improvement? What makes you think the approach is wrong? I need some clarifications :)
Does it?
Humans are error prone, even when doing things they know and are good at.
Artificial intelligence is marketed as being a machine that is as smart as a human, but somehow we infer that because AI is a machine it will not make human mistakes. Mistakes are what produces learning.
The question becomes, do we only release AI for public use when it is assigned to a narrow range of problems and trained to 99.9% accuracy? Or does a consumer just throw AI at unknown, or even non trainable, problems and we take the result with a grain of salt? (Non trainable being something like predicting the value of the S&P 500 in 24 months.)
Perhaps a new words will be formed to describe AI, its behavior, accuracy, and experience? For now there is a lot of "one size fits all" and "holy grail" seeking. Big companies with armies of sales people seem to prefer this.
This sounds a lot like my present job description.
As part of that I've been leaning as much as I can about how Watson actually works.
The most useful information can be found by Googling "Deep QA" which is what IBM has dubbed their question answering pipeline.
A slide deck like [2] is a good place to start if you are interested in this.
[1] Yeah, I know that is kind of a crazy thing to work on. It's actually even more stupid than you may think, because I want it make it self-hostable, with the ability to keep your own data separately to the rest of the application (ie, enforcing privacy).
[2] http://www.cs.hku.hk/news/2011/WatsonHongKong_talk_ppt.pdf
Do hit me up if you'd welcome help with this. Email is in HN profile
Real life example: in the 90s, journalists reported on the "Great Hacker War," a "virtual gang war" between two competing hacker groups, LOD & MOD. In reality, the event was a scuffle between some hackers in a chat room, which resulted in some minor hacking, name calling, and prank phone calls. But that didn't make for a great headline.
I'm afraid Watson is just a PR stunt. Was it oversold by IBM engineers to their executives? Or by the executives to the PR team? Or by the PR team to the press? I don't know. But they lost control of it.
At least 20% of full-stack programmers job is to figure out how people want the computer to behave and all we have to work with is chaos of words that flow from their mouths and fingers.
I asked Siri "Did Michigan win its bowl game?" and it gave me the right answer and said "Michigan lost to Kansas state in the Buffalo Wild Wings Bowl.
Wolfram Alpha, with the same query, just gave me information about the state of Michigan.
Bing gave me search results about Michigan Football, and Google's top results were an article about the actual bowl game. I don't have an Android phone to try Google Now.
Comment 1 - Most likely it's not a full "Watson" but rather a network appliance that slots into a rack. Watson is powered by 2,880 8-core IBM POWER7 processors, which AFAIK haven't received a core bump or a die shrink since their introduction in 2011.
Comment 2 - POWER7 (which came out in 2009) was replaced by POWER7+ in 2012. IBM shrunk the lithography but kept the die size the same, so they used the extra space for more cache, a crypto accelerator, a compression/decompression accelerator, and some other goodies. There were able to bump up the clock speed as well. Core for core, POWER7+ is about a 20% improvement, but you're right, no more cores per socket so there is no way they would see the kind of shrink described in the article if they kept the same amount of compute power. IBM did come out with a new blade design (Flex Systems) with denser packaging, but that combined with the faster CPU will still only get them about 2/3rd of the way there (still impressive).
It would be more convincing if Watson accomplished some real knowledge archievment such as finding a cure for a specific disease, or publish some papers enhance our understanding of some research topics ...
Unix starts small, it works. Google starts small, it helps us tremendously. Haven't seen something starts as a big business plan can success greatly? even Microsoft started small ...
I'd personally want to be diagnosed by a panel of experts with access to said AI.
http://www.managedcaremag.com/archives/0109/0109.predictive....
Here's the original article:
50 years of successful predictive modeling should be enough: Lessons for philosophy of science (2002) by Michael A. Bishop , J. D. Trout: http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.217....
In the end, we'll rely on the AI. We'll _want_ to rely on the AI. Doctor's roles will be reduced to that of nurses. Care will improve. See it all described in the 2006 movie "Idiocracy":
Exactly. Neither one nor the other alone, even though today the panel of experts seems the best bet
To see how the AI can go wrong, just try to diagnose something with flu-like symptoms using Google.
There's also an issue with incomplete tests. Oh the AI can improve the diagnostic using exam X but exam X is too expensive/too invasive/risky etc, someone needs to play "middle ground"
For example software outperforming experts at xray interpretation seems pretty easy to imagine.
However outperforming a general practitioner by asking questions of a patient seems more difficult because the problem space diverges so much...
I agree that a combination of a panel and AI would make me feel better about a diagnoses.
http://online.wsj.com/news/articles/SB1000142405270230488710...
In 2012, Memorial Sloan-Kettering Cancer Center in New York began work on an adviser to recommend cancer treatments. Dr. Mark Kris, a Sloan-Kettering oncologist, said an early version of the Watson tool could be used on patients later this year if it passes tests.
At his office, he pulled out an iPad and showed a screen from Watson that listed three potential treatments. Watson was less than 32% confident that any of them were correct. "Just like cancer, it is much more complex than we thought," Dr. Kris said.
We're meeting mental health professionals who were trained in the 60s, 70s, and 80s who proudly claim they know little about the brain. I don't blame them entirely - when they were trained we only knew about the human brain from stokes and open head trauma. To get them up to speed requires a whole new training. That's very hard when you spend all week seeing patients - that's your livelihood.
I don't think this is true, but if so, please contact me (details in my HN profile).
I'm sure there are plenty of reason to believe that this cannot be generalized to real world problem but no one here has given these reasons. So I am not really sure why people claim that it might not live up the the hype. (And I would genuinely like to understand).
Watson, I think, will be the same way. Say for medical diagnoses - you don't just feed it observations and prescribe whatever it says. But if you want to ask about an unusual combinations of symptoms you've never encountered, it'll come back with something you can then go research. How you could make that connection before systems like Watson or Google (or similar medically-focussed systems if they exist) is beyond me - but they'll probably never replace the doctor's judgement. They're just a tool and should be treated that way.
Google isn't designed to diagnose symptoms, so I don't think that's a very good analogy.
A few issues: There is a lot of information available to the doctor in a typical diagnostic setting that is not currently codified in machine-readable form, and asking the doctor to do custom data entry per patient is not likely to improve uptake. Ideally the systems should integrate with other patient-information-management systems, and such patient-information systems might need to be augmented with new or differently coded data collection. Perhaps equally or more importantly, if the AI system is going to be a component of the diagnosis rather than handed over full trust to make the diagnosis, it should ideally produce "white-box" diagnoses with justification for its reasoning and human-readable explanation of what it thinks the situation is, not just black-box predictions.
Or that an AI runs through a doctor surgery's collection of patient medical records and highlights patients that probably would benefit from an intervention?
Assuming it was more reliable, of course.
There are around 500 000 biology and medical research papers published per year (http://www.stm-assoc.org/2012_12_11_STM_Report_2012.pdf), and it's completely impossible for a doctor to keep up with all of the latest knowledge. A search engine can help a lot in discovering otherwise unnoticed connections.
I'm not sure how insurance companies are going to approach this - but "the computer agrees with my decision" seems to be a good defense argument in a malpractice case. And if the end result is statistically better, then insurance companies will adjust their prices.
So, expert systems are already here and helping make decisions...
With that history you should have been seen urgently. There are a number of serious things that are possible with those symptoms.
Plus, the nurse was relying on your accurate reporting of symptoms. Things are often very different face to face than over a telephone. People very often under/overplay their symptoms.
Also, presumably, if I was having any shortness of breath, it could be determined just by talking to me. They may have also taken my age into account (32) and decided that 32 year olds don't have heart attacks in the way I was describing.
I came in for a checkup a week later, and all was well (except for a RSI issue that was contributing to the completely numb left arm)