Is this test result real?
A yes from a good test feels like an answer. How common the thing is changes what it means, often more than the test does.
Try it with your own numbers
9.2% of the people that test positive really have the condition.
A screening test for a condition 1 in 100 people have. It catches 90% of real cases and false-alarms on 9% of the rest. Made-up numbers.
About 1 in 100 of the people checked really have the condition, before any test.
Catches: of the real cases, the share it says yes to. False-alarms: of the rest, the share it says yes to anyway.
- 9 real, and flagged
- 89 false alarms
- 1 real, but missed
- 901 rightly cleared
1,000 people, one dot each, rounded to whole dots.
- When it says yes
- 9.2%
- the chance it really has it
- When it says no
- 0.11%
- the chance it still has it: 1 in 900
- False alarms
- 90.8%
- of all the people that test positive
- Says yes overall
- 9.8%
- of all people checked
Picture 1,000 people: 10 have the condition, and 9 of those test positive. So do 89 of the 990 who don't. Only 9 of the 98 flagged are real.
How this works, step by step
Start with how common it is.
Before any test, the best guess is how often the thing happens among everything being checked. If 1 in 100 people have a condition, a random person has a 1% chance. This is the base rate, and it is the number people forget once a test says yes.
Split everyone four ways.
Every case is real or not, and the test says yes or no. That makes four groups: real and caught, real but missed, not real but flagged (a false alarm), and not real and cleared. Counting people instead of percents makes it easy to see: of 1,000 people, 10 have it, 9 of them are caught, and 89 of the 990 others are flagged too.
caught = base rate × catch rate · false alarms = (1 − base rate) × false-alarm rate
Look only at the ones flagged.
You got a yes, so you are somewhere in the flagged group. The question is what share of that group is real. In the screening example, 9 real out of 98 flagged: about 9%. A test that is right 9 times in 10 still leaves a yes that is wrong 9 times in 10.
chance it is real, given a yes = caught ÷ (caught + false alarms) · 0.009 ÷ (0.009 + 0.0891) = 9.2%
Rare things let false alarms win.
When the thing is rare, the group without it is huge. Even a small false-alarm rate on a huge group makes more false alarms than there are real cases. That is why the fraud alert is wrong most of the time while the spam filter is almost always right: spam is common. This way of updating a chance with new evidence is called Bayes’ rule.
P(real | yes) = P(yes | real) × P(real) ÷ P(yes)
A second test moves it a lot.
After one yes, the chance is about 9%. That becomes the new base rate. A second, separate test that also says yes lifts it to about 50%, which is why a positive screening is followed by a different test before anyone decides anything. This assumes the second test’s mistakes have nothing to do with the first one’s.
0.092 × 0.9 ÷ (0.092 × 0.9 + 0.908 × 0.09) = 50.3%
The words doctors use
How often a test catches a real case is its sensitivity. How often it clears a case that is not real is its specificity, which is 100% minus the false-alarm rate. The chance a yes is real is the positive predictive value.
The base rate depends on who gets tested. A test given to people who already have symptoms starts from a much higher base rate than a screening of everyone, so the same yes means more. For learning only, not medical advice: a doctor reads a result with everything else they know about you.
For learning, not financial, medical or betting advice.