A post linking to this study was shared across our social media recently. The study published results of a poll showing that people in general have “a wildly overstated view of Covid-19’s impact with respect to total infections and deaths as a percentage of the population”.
It took less than 2 minutes to find a FullFact article highlighting the floors in the study.
And this is where the discussion should have ended but it didn’t.
In fact, it continued on for no less than a further 46 responses as various respondents went back and forth discussing the maths.
Here are some quotes from our responses and an approximation of the queries we received, not quoted because these were personal comments intended only as friendly back and forths and not for public broadcast.
“the study reports a “mean average”, rather than a “median average”.”
“Median is preferable to mean? I’ve not heard that before. Could I ask what your educational and career background in datasets and research is please?”
We explained giving a text book example and added our relevant career experience (which was an extra and unnecessary step since we had referenced FullFact as our evidence and were not asking the respondent to believe in our own maths expertise):
“I think you have answered your own question there, “assuming there are no extreme outliers, the data isn’t skewed”. The reason you use the median for continuous data is to reduce the effect of just that: the outliers and the skew.”
Let’s take a simple contrived example, I ask ten people to estimate the fuel economy [in mpg] of my car, I might get the following responses:
50, 51, 52, 52, 55, 55, 56, 57, 58, 110 (like real life, some people would guess a wildly inaccurate value)
That would give central tendency values of Mean: 59.6 and Median: 55
“So it’s clear that if I wanted to show what the typical person thought, the median is a much better statistic to use. Most people provided results close to the median. While the mean was above 90% of the estimates.”
“You don’t have to trust me, FullFact have debunked this.”
“And as to your point about skew, that was mentioned explicitly (below the big graph displaying the skewed data): “While most people said that fewer than 2% of the population had died, the few people who answered with high figures skewed the mean upwards.””
They replied that the study was still a great study with valid conclusions.
“While the ‘map’ is used to support point (4) in the document that “people significantly over-estimate the spread and fatality rate of the disease”, the maps and associated statistics grossly misrepresent the scale of the issue, which consequently undermines their point.”
They said my criticism was my opinion and their belief in the study was their opinion.
We disagreed, pointing out that:
“The study used the incorrect measure. This is mathematical fact and not my opinion. I can say that my comments are factual because I have referenced a factual source (full fact), and provided a simple mathematical example that debunks the use of mean, in this instance.”
This dialogue shows that even when engaged in a polite discussion of the evidence that our ‘friends’ are choosing to deliberately ignore mathematical fact and continuing to view the study as valid and supportive of the position they hold. They supplied no further evidence during the discussion.
This is confirmational bias, their view has been supported so they choose to keep hold of the evidence and their cognitive bias refuses to allow them to see the debunked evidence in the light of established mathematical fact.
We believe that engaging in evidence-based debate is hugely important. When we share our ideas and their evidence, that evidence is challenged in ways that we possibly hadn’t considered, and this helps us to reconsider that evidence in a new light with less of our own cognitive bias. We all have cognitive bias and we are all prone to confirmational bias.
However, we choose not to encourage our children to engage in discussions like these (yet) because they are frustratingly 1-sided. The respondents are neither accepting mathematical facts, putting forward counter-argument or offering alternative evidence to support their view-point.