In fact, the stark difference between Italy and Germany would provide support to the paper's conclusion.
My hypothesis is that this comes from bigger inter-generation connections in Italy.
* assume false positive rate of 10%, assume true positive rate of 100% (it's not but let's be generous)
* maybe 1/10 of the population (30 million) gets tested
* lets say 100,000 people have COVID right now (50x the official number)
number of positive tests = 30 million * 0.1 + 100k * 1 = 3,100,000
fraction of postive tests that actually have the disease = 1 / 31 = 3.2%
problem is FPR (1) depends on the population tested (i.e. p(covid | positive test) != p(covid|positive test, some symptoms) != ...), (2) we need very accurate measurements of FPR because:
lets say we constrain FPR to 10% +/- 1% ==> 10% uncertainty in FPR -- that means our inference of the number of infected people is:
n_infected = (n_positive_tests - FPR * n_tested)
which is: -200,000 to 400,000
so...not very useful.
My wife is a doctor (and I've learned a lot from her). In med school, they're specifically taught to evaluate diagnostics on their specificity and sensitivity - which essentially covers false positives and false negatives. If you hear a doctor talk about the "accuracy" of a test, it's likely because they're simplifying the concepts.
"Error rate" or "accuracy" is not used at the scientific level in medicine. Partly, for the reason you defined. It doesn't convey enough information about the outcome of the test.
A "99% accurate test" is pretty meaningless without understanding the specificity and/or sensitivity components. In fact, I've seen some headlines where they incorrectly refer to only one component as the "accuracy".
If something is rare, it has a low base rate. That even means a test with excellent specificity and sensitivity could still be wrong most of the time.
Decisions on test accuracy simply cannot be made coherently when ignoring the base rate. To make an intuitive example, suppose that one in a thousand people have a disease. A test for the disease has 90% specificity and 100% sensitivity. It will always correctly give a positive result if the person has the disease, and has a 99% chance that a given positive test is valid. Pretty good, much better than most tests.
Now suppose that 1/1000 of people have the disease. A person with a positive result has a 1% chance of not having the disease. If everyone is tested, then 1/1000 people will get true positive results. But, (999/1000 * 0.01) ~ 1% of people will get false positives.
Thus, a given person with a positive result has nearly a 10x chance of it being erroneous compared to it being accurate! As I said, the frequentist techniques that you describe and are taught in medical schools do not help with this.
Yet this is endemic in medicine. This sort of thing is why in a recent meta-study of 54 landmark cancer trials, only six could be replicated. That is frankly terrifying.
(True positives + True Negatives) / number of all tested
Similar concept comes up in measuring accuracy of computerized image segmentation, where you ignore the true negatives
true positive / (true positive + false positive + false negative)
where it is called intersection over Union (IOU).
I can’t ever remember the names, and just rebuild whatever metric I care about in terms of true vs false and positive vs negative.
Applying all this to the real world is tough because of the over fitting problem. Even if you got the test to be 100% accurate in your tested population, it doesn’t mean it won’t be wrong on the next person it tests. Generalization is hard. So doctors have to guess based on their understanding of the tested and untested population and the sensitivity and specificity of the test. You can go meta and give the doctor a sensitivity and specificity also.
Can you elaborate on this a little more...?
Here is someone else explaining it.
https://betterexplained.com/articles/an-intuitive-and-short-...
Basically, imagine that it's a decade from now, and no one in the world has COVID-19 anymore. The test still has a 1% false-positive rate, though - so if you test a few thousand people, a few of them will test positive. Given that setup, every single one of the test positives will be false positives.
The same holds true if there's one infected person and you test 1,000,000.
In this case, every positive test you observe is around 10x more likely to be a false positive than a true positive.
There could be many more people that never even get to the testing phase.
First, the country has NOT been completely shut down. I went shopping today and bought milk, yeast and flour. We didn't need toilet paper, but the store had plenty. All schools and most of the public sector closes down for two weeks on Monday. Some business (like restaurants, movie theaters and fitness gyms) are closing down on their own accord. But you can - if you will - still go shopping for clothes, gardening stuff, electronics and most importantly food.
Second, although the right to forced entry into private property was in the original draft it was removed before vote. Entry still follows the known rules of needing approval by a judge.
You are right, however, that forced testing, forced treatment, forced vaccination (if/when possible) and forced quarantine is mandated as per discretion of the public health authorities.
Our infection rate has grown dramatically in the past few days, and not as a result of increased testing AFAIK. Testing capacity has been limited, but is being drastically increased as of today. So maybe the already high growth rate will increase further as a result.
“ Dr. Tedros noted that only 1 percent of cases in China are reported as “asymptomatic.” And of that 1 percent, 75 percent do go on to develop symptoms.”
https://arstechnica.com/science/2020/03/dont-panic-the-compr...
If false positives dominate true positives then you’d expect total positives to depend primarily on number of tests given, right? Which sounds wrong to me, but I’d be interested in hearing other thoughts.
But, it would suggest downsides to more general testing.
I am curious if this also could indicate a false-positive problem with non asymptomatic people as well.
False-positives are also why the CDC tests had to be shipped back, although that was because it was showing false positives in other diseases it was testing for, not COVID-19.
If s/he is in the hospital in quarantine you must give blankets and food, probably a nurse to check the temperature and symptoms two or three times per day, a medical doctor one a day just to be sure. Perhaps a blood analysis from time to time?
Luckily you don't have to handle visitors because they are in quarantine. (Or there are some visits? What if one patient tries to escape?) You must give an official reports for the family. Now you can assume the patient can send a WhatsApp message to the family saying s/he is fine, but you need probably still an official report. Paperwork, there is also paperwork.
How isolated are them from each other. If they are all together, you can transform the overcrowded false positives in real patients.
On the one hand, no matter what measures politics takes here (the Netherlands), every political party is clamouring for more. That suggests to me that we'll probably end up doing too much.
On the other hand, the scenes from Hubei and Italy are horrific, we're going to get them here as well, it could be far worse, we absolutely must act now.
On the third and fourth hands, the total number of deaths so far wouldn't even make a serious flu season in a single country, and fatality rates are all over the place depending on the country.
Very confusing. Still, better too much than too little.
If you mean the day-to-day numbers, those will always show variations, and paradoxically those will even show a substantial uptick once the spread slows down.
The most vulnerable are affected first and most severely.
EDIT: All else remaining the same. See AnthonyMouse's comment below for important clarifications and corrections.
Meanwhile you also have the opposite happening for the same reason -- if even a small percentage of asymptomatic people are actually infected but not being tested, a small percentage of "asymptomatic people" (i.e. nearly the entire population) could represent a very large proportion of those infected and cause the fatality rate estimates to be much higher than the true number.
It can be that you have somehow managed to avoid infecting older generation. I have longer discussion about this here http://www.savespain.eu/italy-vs-germany
ie, if someone was a false positive the first time would they still have a 1% chance of getting another false positive, or is it possible there's something about that individual that will always give them a positive result?
If the false positive rate is higher than the expected rate of disease in a given community, then the majority of positive tests will be false positives.
Does this relate to COVID in any way? Since the rates among affected communities seem to be growing rapidly. Would appreciate your thoughts.
Not necessarily. In theory all the deaths could have some other cause, i.e. some fraction of people with a different underlying fatal condition had false positive tests for this coronavirus and then died of the other condition.
That's probably not what's happening, but it's theoretically possible. (It's also probable that some of the reported deaths are that, but who knows what percentage.)
(Edited to fix a silly mistake: The phone rang while I was posting, so I ended up being hasty.)
Edit the 2nd: Even in the simplified case q=0, you can't easily tell.