Preparing to learn
Seth Godin said reading helps us prepare to learn, experience helps us actually learn. For years, I’ve read articles and did tutorials on applying bayesian logic to medical tests. The main [thinking] point of these pedagogical exercises, usually, is that tests aren’t 100% accurate and depending upon their level of accuracy… their results should be interpreted in different ways. When you test positive, depending upon the reliability of the test, the probability that you are actually positive may be lower than you think. But, to Godin’s point… tutorials only get you halfway to learning. This summer, I had the opportunity to actually learn how to use bayesian logic with regard to medical tests.
Probability of lyme disease
In late August I received a positive test result for Lyme Disease. My first thought was “oh, it’s probably a false positive.” So I pulled up a Bayes theorem calculator. All I needed to do was plug in the numbers. If the output showed me that the probability of actually having Lyme disease was not very high (because of test inaccuracy), I’d have peace of mind for the night. I wanted that peace of mind! So, the numbers and the equation…
It looks daunting, but here’s what we want to know; the left hand side of the equation would tell me the probability that I don’t have Lyme disease given that I just tested positive. The lower this number, the better – I really want the test to be wrong, I really don’t want to have Lyme disease.
First, I need the numbers…
- P(B|A): A quick Google search yields some information saying that the false positive rate is very high at somewhere around 60%. This feels good already!
- P(A) : This is the prevalence of Lyme Disease (or rather, the inverse of it). I often call it the base rate or prior probability. So, I look at how many cases of Lyme Disease are reported annually in CT and divide that by the total population. It’s very low at less than 1% (this means the probability of not having Lyme disease is 99%).
- P(B) : The probability of a positive test (all the false positives and the true positives). From a few quick Google searches I estimate it’s roughly 60+%.
Right. So plug the numbers into the equation, calculate…
…. and the result is that, given I received a positive test result, the probability I don’t have Lyme Disease is 99%. That’s insanely high. I knew it. It’s almost certainly a false positive. I can stop worrying so much. It’s almost certainly not Lyme Disease.
I tell my wife, who is an ED nurse, that despite the positive result I don’t think I actually have Lyme Disease. She looks at me with a puzzled look and says “you definitely have Lyme Disease.” How is she so sure despite my maths? Maybe I have a garbage in garbage out problem…
Insane assumptions
- A panicked mind is a confused mind. This isn’t true for everyone, but it’s true for me. When the course of my health or a loved one’s health seems like it’s about to change forever (for the worse), I don’t think very clearly. I was happy to grab at the first numbers I saw as inputs to my problem. The craziness of them actually made me more willing to use them as inputs.
- Lyme disease tests are more accurate than we think. It’s problematic that the way Lyme antibodies develop in the system leaves a window of about 4-6 weeks after infection where tests won’t detect it. Outside of this window, and I have good reason to believe I was outside of this window, the tests are quite accurate. I updated my equation with a false positive rate of 5% rather than 60%.
- We are good at “reasoning away” information. I was running a lot over the summer in the hot, humid weather. I assumed my aching hips, glutes, and quads were because of that. In reality, that was the first reason why I should have understood that using the entire population of CT as my population when determining a prior was dumb. The other two reasons; I spend a lot of time in the woods. B) I have a dog. C) I rarely used any bug spray. I don’t entirely know what this means for my base rate, but let’s at least say it’s likely / probable at 51% (barely more likely than not).
Updating the calculation shows that the probability of me not having Lyme Disease, given the positive test, was about 6%. And if you want to think about it as the probability of having Lyme disease given the positive test, it’s 94%. A much different number than <1%.
‘Tis better to know of it rather than think you know it
Let’s close with the good news and a takeaway heuristic. Antibiotics put me on the swift road to recovery ✅. The heuristic for this article might have to do with double checking your assumptions or recognizing the distortion of your thinking under stress. However, I think the most useful heuristic relates back to the Seth Godin wisdom about preparing to learn and actually learning. I think it is better to know of it rather than think you know it. I could have walked through articles on the Bayes equation another dozen times and never really got it to sink in. Only when I had the opportunity to use it did the components click together. So, cast a broad net in your preparations to learn, and open yourself up to more opportunities to learn.