Covid Inquiry Rapped by Statistics Regulator for “Misleading” Use of Neil Ferguson “23,000 Deaths” Modelling Claim
The statistics regulator has written to the Covid Inquiry to rap its knuckles over the “misleading” use of a Neil Ferguson modelling claim that locking down a week earlier would have meant “23,000 fewer deaths”.
Writing to Ben Connah, the Secretary of the Covid Inquiry, Ed Humpherson, Director General of the Office for Statistics Regulation, said that “the Inquiry’s Executive Summary of the modelling does not sufficiently communicate the level of uncertainty associated with the analysis”. He invited the inquiry to “set out the steps you are taking in response to our findings, to support the future communication of statistics and analysis by the UK COVID-19 Inquiry”.
The problem, Humpherson explains, arises in the Executive Summary of the Module 2 report, where it is categorically stated that:
Had a mandatory lockdown been imposed on or immediately after March 16th 2020, modelling has established that the number of deaths in England in the first wave up until July 1st 2020 would have been reduced by 48% – equating to approximately 23,000 fewer deaths.
Humpherson says this statement in particular risks being “a misleading representation of the underlying analysis”. He explains:
Uncertainty is inherent in statistical models, some of which is quantifiable as an interval around an estimate, but there is also broader uncertainty due to unavoidable assumptions made in the modelling process. The use of the words “established” and “would have been reduced” overstates the confidence in the 48% figure and has the potential to mislead the public about the certainty of the finding.
The full text of the letter is available on the OSR website and is reproduced below.
Dear Ben,
Thank you for taking the time to meet with the Office for Statistics Regulation (OSR), the regulatory arm of the UK Statistics Authority. We provide independent regulation of all official statistics produced in the UK, and investigate concerns raised with the Authority.
As you know, we were contacted about the presentation of modelling in the Executive Summary of the Module 2 report of the UK COVID-19 Inquiry, specifically the section which says that “Had a mandatory lockdown been imposed on or immediately after March 16th 2020, modelling has established that the number of deaths in England in the first wave up until July 1st 2020 would have been reduced by 48% – equating to approximately 23,000 fewer deaths”. Evidence given to the inquiry by Professor Neil Ferguson from Imperial College London and from an academic paper (Knock et al.) provided the figure of a 48% reduction in deaths. The 23,000 figure was calculated by the inquiry by subtracting the estimated number of deaths from the actual number of deaths.
While the inquiry report does not fall within our statutory remit, we considered the issue on an advisory basis, focusing on the communication of these figures in relation to the Standards for the Public Use of Statistics, Data and Wider Analysis, within the Code of Practice for Statistics.
I want to emphasise that we are not questioning the inquiry’s conclusions, including those on the timing of advisory and mandated lockdowns, nor would it be appropriate for us to do so. Our focus is the presentation of statistical evidence, and the risk that the wording chosen for the Executive Summary is a misleading representation of the underlying analysis. We thought it would be helpful to share our findings with you because in future modules of the Inquiry report, you may wish to present modelling or other analytical evidence and it will be important that this is communicated clearly and accurately.
Firstly, while steps have been taken to communicate uncertainty in the main body of the report, we consider that the inquiry’s Executive Summary of the modelling does not sufficiently communicate the level of uncertainty associated with the analysis. Uncertainty is inherent in statistical models, some of which is quantifiable as an interval around an estimate, but there is also broader uncertainty due to unavoidable assumptions made in the modelling process. The use of the words “established” and “would have been reduced” overstates the confidence in the 48% figure and has the potential to mislead the public about the certainty of the finding.
The second aspect concerns what the “23,000” figure represents. The inquiry’s Executive Summary for Module 2 says the “23,000” is an estimate of the effect of bringing the mandatory lockdown forward a week. However, in investigating this case, we have confirmed with Professor Ferguson that the ‘counter-factual’ (the hypothetical scenario which the model aimed to describe) was that both the mandatory lockdown and the voluntary measures announced on March 16th were each brought a week earlier. While we acknowledge that the Knock et al. paper did not explicitly set this out as the counterfactual, there is a risk that the Executive Summary is misleading about what the “23,000” actually refers to.
Given the weight put on the “23,000” figure in the Executive Summary and in the subsequent media coverage, we consider it is important to have clarified this point about the counterfactual.
We invite you to set out the steps you are taking in response to our findings, to support the future communication of statistics and analysis by the UK COVID-19 Inquiry.
Yours sincerely
Ed Humpherson
Director General for OSR
To join in with the discussion please make a donation to The Daily Sceptic.
Profanity and abuse will be removed and may lead to a permanent ban.
Please, could somebody explain to me why we still care about anything this Neil Ferguson guy says?
Seriously, I just don’t get it.
His models didn’t work.
His predictions bore no resemblance to reality.
He even broke the lockdown rules that he himself proposed.
And yet here he is, acting as an expert.
Why?
I think we care about the quality of the modelling. Did the code implement the model described in the academic papers or is it bug ridden? Is the model quantitative or qualitative? Why were the results this code gave for Sweden so different from the predictions?
Why has Ferguson got so many forecasts so very badly wrong, all anticipating excessive deaths and never under-estimating. Could it be he is a bad statistician or does he use unjustified assumptions and bad data intentionally. Either way he has no place in a University.
We don’t know whether his model is qualitative or quantitative. The epidemic modellers have a whole series of models called chain binomial the first of which, Reed-Frost, is by the authors own admission a toy model. i.e. Qualitative.
Why do we care?
Because we need to make sure this person is never trusted again.
The one thing I agree with him about; Far more important for people to maintain their normal lives than observing lockdown. It does demonstrate his raging arrogance and hypocrisy, of course.
We don’t care – in fact, the guy should be locked up for crimes against humanity – but the system does. The system needs him so it can justify its past and future actions.
Its because when you need a Prof to prove something that didn’t or couldn’t possibly have happened, he’s your man.
Probably because he tells people what they want to hear.
As soon as I saw Ferguson was being cited as a prognosticator I knew the whole thing was a complete lie. So in that sense he was useful.
“the Inquiry’s Executive Summary of the modelling does not sufficiently communicate the level of uncertainty associated with the analysis”
A very polite way of saying something blunter… For example, I could say that I am the Queen of Sheba and you might say that I have not sufficiently communicated the level of certainty associated with my statement.
This is an age old trick to get people to believe there’s some hard data behind a guess. State something to a couple of significant places and make it not a nice round number, people assume that there is something solid behind the number. So 48%? 50% ish would not be as convincing – there must be some serious maths to get it to 48%. up to 23,000? Why not up to 20,000 or up to 25,000? Because 23,000 sounds as if it has some accuracy behind it. We get it in climate change as well with statements like “there is a 36% greater chance of severe weather”. Really? 36% of an unknown number is still an unknown number. Whenever I see this sort of accuracy coming out of statistical models I know it’s rubbish – I’ve used this trick myself in a previous life.
They don’t talk about accuracy, or tolerance. Alien terms in their view of the world.
The Germans have a word for this: “Scheingenau”.
Someone once coined it “spurious specificity”.
If I were a cynic (moi?) I’d argue that the findings of the inquiry so far have been ‘bent’ by the presentation of modelling in the Executive Summary of the Module 2 report to get closer to the politically desired outcome.
Here is some data and as if by magic we get the right answer, perhaps.
It’s right to complain about the inquiry’s use of Ferguson’s useless modelling.
It’s also right to complain about the inquiry in general being a foetid heap of dingos’ kidneys that does nothing other than muddying the waters of the whole Covid debâcle and its disastrous effects on the world and the UK.
I think that’s what their remit was at the start?…
Ferguson et al’s Report 9 was dated 16 Mar.
Don’t read the report or think about it, just do what we say.
Arrogant fools.
Gotta say, the ONS was largely scrupulously excellent throughout The Hysteria and now the OSR is putting in a good shift in the aftermath.
Go Stattos
Not only were the outputs of the model taken way too seriously, it was also a very poorly written model.
https://staging.dailysceptic.org/2020/05/06/code-review-of-fergusons-model/
https://staging.dailysceptic.org/2020/05/09/second-analysis-of-fergusons-model/
The Covid Inquiry is just Whitewash which will, and is intended, to absolve the entire Establishment from the consequences of the appalling tyranny.
So this letter is just a plea for the Inquiry not to overdo the Whitewash …. and nothing else.
But it will leave all the extra powers they took firmly in place, just in case we need them again.
Has even one shred of truth been uncovered in this “covid inquiry” otherwise known as the scamdemic inquiry? How much has it cost the taxpayer to date? Who received all this money. How many lives has it saved? Why aren’t the key gov’t health advisors in ‘19, ‘
’20, ‘21, and prime minister and his cabinet and his “ modeller” in prison for crimes against humanity. Does anyone know? What an odd country this is. To ignore all that is right in front of you over and over again. No wonder Britain is in the state it is in.
Are they still wasting our money on this pile of get out of jail for the Politicians and advisors crap? We can’t police the borders, we can’t afford to defend the country but arse covering and career protection? well we can spend multi millions on that.
It all depends on who’s arse needs covering… and in this case it’s powerful people, so it’s all ok…
Had the lockdown happened a week earlier it would not have made a blind bit of difference to me. I read the Health Protection (Coronavirus Restrictions) Regulations, established that the occupation of myself and 2 contractors could be said to be ‘essential’, generated a risk assessment that showed using the minimal data available – later updated with that from Diamond Princess – that risk/consequence was low and worked 5 or 6 days a week. Never once stopped by police or LA officials. It’s all archived for the ‘next time’
I agree the timing of the lockdown was largely irrelevant. An article here just last week showed that infection rates were falling before the lockdown was imposed.
The inquirey is busting a gut to justify lockdowns
This letter shows how far we are removed from free and straightforward speech.
The letter could have been just a few sentences.
Dear so and so,
Your conclusion of earlier lockdowns preventing 23,000 deaths is based on a flawed model. Please remove it.
Kind regards,
Office for Statistics Regulation
Lockdown could only slow the spread of the virus. Even if we believe this claim of 23,000 deaths, lockdown would only have postponed the deaths for a few weeks.
Except that, according to an article published on this site a week or so ago, the infection rates were falling before the lockdown was introduced, so we could probably have done without it altogether.
Does anyone think that Dame Heather Hallet will be able to understand let alone do anything about this letter?
The glib naivety of the so called Covid Enquiry needs calling out. A wonderful example of confirmation bias. Start with your opinion and only credit evidence that supports it. This expensive exercise in obfuscation and delay will only benefit the lawyers. A fat pension all round….. as if the whole debacle hadn’t already bankrupted the country, we are now pouring some extra borrowed money down the drain.