This text popped up in our Facebook feed recently. It was a copy and paste from the above comment on a British Medical Journal (BMJ) and luckily for us there was a handy link to the original.
Now this looks pretty official, right?, it’s a reputable medical journal, easily recognised by people around the world. If you click the link it takes you to the BMJ website, and you see the formal-looking page above.
The text suggests that there might be a link between flu shots and the current Coronavirus outbreak. (We are writing this in late July 2020). This sounds scary. We had our flu shots back in November, late last year. Are we really at risk?
If true, this would be huge! I must have missed the flurry of class action lawsuits on the issue…
The text in the post looks medical and when we follow the link to the BMJ page we see the text next to the details of a ‘Retired pediatrician’. But wait… Things aren’t quite what they seem. The page is actually a comment on a BMJ article, not the BMJ article itself.
But lets look at what it’s saying, maybe it’s a worthwhile comment…
The commenter states “figures yield an odds ratio of 4.91 (CI 1.04 to 8.14)”. Whereas the paper he references (from the journal Clinical Infectious Diseases) concludes: “an increased risk of virologically-confirmed non-influenza infections (relative risk: 4.40; 95% confidence interval: 1.31-14.8)”.
It appears the commenter is cherry picking one particular subset of numbers from a set of results. This is unusual, for example, looking at the same data, the rate of Rhinovirus is much lower in those with the Flu vaccine.
This doesn’t mean the flu vaccine has cured the common cold (AKA, Rhinovirus).
The reason we see these fluctuating effects is that we are looking at a tiny sample of a large population. It’s like taking 2 M&Ms out of a whole bucket of them, seeing the 2 are green, and assuming all M&Ms are green.
The clue in both the statistics mentioned above is the very wide CIs or Confidence Intervals. 1.31-14.8 is large (and so is 1.04 to 8.14) and this indicates a great deal of possible variability in the result.
Regarding the defining an infected person, the definition in this study of an Acute Respiratory Infection could include someone with a headache and a sore throat. Or muscle aches and a head-ache. That definition seems a little loose.
Better studies, covering much larger groups, have produced evidence suggesting there is no cause for concern. These larger studies are much less prone to the statistical problems inherent in above study. You can find out more here: Claim that flu vaccine increases coronavirus infection is unsupported, misinterprets scientific studies.
Now, it’s useful to think through the steps we took here:
Step 1: We didn’t just assume the evidence was correct, and we didn’t become overwhelmed at the medical nature of the post.
Step 2: We followed the link and read the original ‘evidence’.
Step 3: We noticed it was a comment on an article and not a BMJ article. And that comment linked to a report in a different journal.
Step 4: We examined the medical study that the comment quoted. We have enough statistical knowledge to read that confidently. It takes practice to develop that confidence.
Step 5: We applied our statistical knowledge and debunked the post.
You may not have the time to go through all these steps. But you can get an initial impression of the quality of the evidence by step 3.