UKHSA Suddenly Discovers There is No Good Evidence on Lockdowns

Share this:

In the early summer of 2020, masks were introduced in the U.K. Thereafter, there was a mishmash of weird and wonderful measures that didn’t prevent further restrictive measures or lockdowns. According to Rishi Sunak’s statement to the Spectator, not even Cabinet Ministers were aware of the rationale for this merry-go-round.

As most of the important people making decisions, or at least communicating them, flip-flopped at approximately the same time on both sides of the Atlantic, were we treated to (or should we say, were we victims of) some clockwork change of course?

But why did this take place? Perhaps the question for Trust the Evidence is: on the basis of what new convincing evidence did the flip become a flop? We are unaware of any new high-quality evidence apart from the feardemic the flip-floppers induced.

Now that things have calmed down, we look closer at the efforts to retrofit the evidence to the policy decisions. Keith Duddlestone comments that some folks want to forgive, forget and move on. Some understandably want to do so, as they have suffered enough; others, though, are well aware that these measures will remain on the menu if we do not hold those responsible for their actions.

The reintroduction of masks will depend mainly on the politics of the day. More so the case in the U.S.: the CDC is back recommending face coverings in areas where hospitalisations are high, and some universities announced face coverings would be compulsory for staff and students on their campuses.

Because of the mismatch between policy and evidence, we promised to track the flip-floppers, escape artists, retrofitters and those who have just collected their honours and want to move on.

So, let’s start with the U.K. Health Security Agency (UKHSA). We have documented its dreadful reviews on masks that include the wrong type of evidence to answer the question, including just about anything its researchers could lay their hands on. The garbage in its reviews was presented in Parliament as evidence of effectiveness of mask use. It led us to question whether they had read some of the studies.

However, it now looks as if the UKHSA intends to update and enlarge its evidence dragnet. Now we are told:

There is a lack of strong evidence on the effectiveness of NPIs to reduce COVID-19 transmission, and for many NPIs, the scientific consensus shifted over the course of the pandemic.

Did you note the flip-flop? Consensus shifting, hey. Apparently, “there are specific limitations to the evidence-base for NPIs effectiveness”. Who’d have thought it?

The UKHSA has mapped available evidence by intervention and study design. The map is interesting. It is a cosmology of every non-pharmaceutical intervention (NPI) they can think of. The typology is unclear, but that is not the point. They called it an Evidence Gap Map.

If you go to the page and click the arrows in the boxes at the top, you obtain a detailed list of interventions under each heading.

The map is part of a more detailed 91-page review of ‘What evidence exists of the effectiveness of the NPIs as implemented in the community in the U.K. to control the COVID-19 pandemic?’

The rationale for deconstructing the evidence solely to focus on the U.K. doesn’t make sense. There are certain situations where this might be appropriate. If the issue is genuinely setting specific, e.g. waiting list initiatives, or if the intervention is a U.K.-specific public health intervention alone. But masks are masks, and their deployment and effectiveness in developed countries should be the same as in the U.K.

The boxes in the map show the availability of evidence for each intervention and are almost all bare except for the model row. As no one seems to venture into conducting randomised controlled trials because of the politicisation of the whole issue, we wonder how the UKHSA plans to fill the boxes. With more models? With observational studies or ‘systematic reviews’ carried out by friendly and trustworthy academics carefully directed by the agency?

Or is UKHSA just playing draughts using a chequerboard of its own design? Whatever, it must get a move on, as more people are asking questions and getting fobbed off, like Philip Davies MP.

The UKHSA concludes the evidence was “primarily based on modelling studies (100 out of 151 studies) and there was a lack of experimental studies (two out of 151 studies)”.

It goes on to report that:

Whilst this body of evidence overall provides weak evidence in terms of study design (study designs at the lower end of the hierarchy of evidence such as cross-sectional studies are at higher risk of bias than studies at the higher end of the hierarchy of evidence such as RCT) and potentially in terms of study quality (although critical appraisal was not performed), the wider challenges of the pandemic should be acknowledged, including the limited resources that were available.

We did not perform a critical appraisal. Why? Because we didn’t have the resources. This is an astounding statement; it means the UKHSA doesn’t know how to do a critical appraisal.

Furthermore, it states:

It is also worth noting that the traditional evidence hierarchies and corresponding ‘low level of evidence’ and ‘low or very low certainty’ of the GRADE framework were developed to inform clinical practice where RCTs are feasible, and linear causal pathways are more often the norm.

For this reason, the ROBINs I tool was developed to assess the risk of bias in non-randomised studies of interventions. It’s only been available since 2016 and is the preferred method for Cochrane reviews.

UKHSA says the “next steps are to critically appraise and synthesise the evidence identified on the effectiveness of individual NPIs implemented in community settings to reduce the transmission of COVID-19 in the U.K.”

Despite a lack of resources at Trust the Evidence, we think we can do the job of critical appraisal for them as we’ve done in the past.

The UK Health Security Agency Review – our Trust the Evidence Critical Appraisal