Scientists Against Science: RCTs Which Give the Wrong Answer on Masks Are “the Worst Way to Answer the Question”

Paradigm shifts in science are rare, but it seems we may have just had one. The RCT (randomised controlled trial), an experimental method used to test if medical procedures and drugs work, has long been considered the gold standard method of establishing the relationship between cause and effect. But it may just have been knocked off its perch and usurped by a new approach to seeking evidence.

Based on the first such study by James Lind in 1774 when he rid the Royal Navy of scurvy, the method in its simplest form involves giving one group of people a treatment and withholding it from another group and seeing if the treatment group fares better than the other (control) group. There is no evidence that Lind randomised the sailors on which he tested lemons as a cure for scurvy; randomisation, to avoid bias in who does and who does not receive treatment, was introduced much later. There are many modern variations on the theme of the RCT but, essentially, they are all designed to achieve the same thing.

As an experimental method for trying to settle whether treatments worked, the clinical trial took a while to catch on with the first RCT being published in 1948. Until that time, what was purported to work was based on power and opinion and, therefore, largely on who said it. Other, weaker designs based on observation and correlation abounded but, eventually, were superseded by the RCT.

Of course, not every RCT produces the same results due to an annoying phenomenon called ‘regression to the mean’ whereby observed effects are often obtained one day and inverse effects are obtained on another day. To account for regression to the mean, it is considered necessary to combine the results of similar studies to be able to pinpoint, at any time, where the true effect lies. Thus, the science of meta-analysis arose which does precisely that and the most rigorous repository of such analyses is considered to be the Cochrane Collaboration.

Well, forget all the above. It seems we have been following the wrong lines of investigation — especially when it comes to the use of face masks to prevent the spread of respiratory infections (e.g. COVID-19) — and that we should simply have asked the experts what they thought all along. In view of what we have witnessed in the past few years, what could possibly go wrong?

I may be doing them a disservice, but that is my interpretation of a recent article in STAT of May 2nd titled: ‘Do masks work? Randomised controlled trials are the worst way to answer the question.’ STAT is a newsletter that purports to be “Reporting from the frontiers of health and medicine” and the authors of the article are Baruch Fischhoff, Howard Heinz University Professor in the Department Engineering and Public Policy and Institute for Politics and Strategy Carnegie Mellon University, Martin Cetron an infectious disease epidemiologist who has worked for the CDC and Katelyn Jetelina, an epidemiology, data scientist, and science communicator who publishes a Substack, Your Local Epidemiologist.

Such is their faith in experts (and I assume they see themselves as such), with respect to RCTs on the use of face masks they “believe that many of these studies should never have been done at all, reserving resources for studies that could improve health outcomes”. The recent pair of Cochrane meta-analyses of studies on the use of face masks concluded that:

Pooled results of RCTs did not show a clear reduction in respiratory viral infection with the use of medical/surgical masks. There were no clear differences between the use of medical/surgical masks compared with N95/P2 respirators in healthcare workers when used in routine care to reduce respiratory viral infection.

Our paradigm-shifting team is not impressed, however: “Both meta-analyses have been widely misinterpreted as showing that face masks don’t work.” I cannot be certain but, had the Cochrane review shown a positive result regarding the use of face masks, my guess is that they would have been proclaiming that from the rooftops.

With apologies for lengthy quotes, they also say that:

What if it is so difficult to conduct scientifically sound randomised trials of mask wearing that even the best studies reveal little? Such studies can confuse people who want to know how effective face masks are, while emboldening people who are already completely convinced that face masks are ineffective — and are looking for grounds to sow doubt about them.

Clearly they see themselves as being above the misinterpretation of data to satisfy their own predilections; being “convinced” can work both ways.

With reference to the much-quoted and much-maligned DANMASK study where the difference observed between mask mandates and no mask mandates was “not statistically significant”, they conclude — presumably as the outcome, inconveniently, did not fit their prejudices – that “The designs of most clinical trials are too weak to answer the question that they pose — namely, whether an intervention succeeded”. Plus: “RCTs have value only when researchers can be sure that the treatment is administered as intended.” Perhaps they meant to say: “RCTs have value only when they show us what we want to see.” Besides, knowing whether an intervention works in practice rather than when done perfectly is valuable information from a public health point of view.

However, they do not leave us with no hope and inform us that: “Today, we have strong evidence regarding the effectiveness of face masks in the form of laboratory studies, theoretical analyses and RCTs that involved health care personnel. It has not come from RCTs of face masks distributed to the general public.” They do not trouble us ignoramuses with any details of this strong evidence, unless their hypertexted link to some words in an earlier sentence are meant to do the job. I guess they did not expect many people to go past the various subsequent links to read the material. It is fascinating stuff. The link eventually enables the explorer to download a ‘Rapid Expert Consultation on the Effectiveness of Fabric Masks for the COVID-19 Pandemic (April 8th 2020)’. This is a document written by experts which relates a series of studies on the likely effectiveness of face masks. No method is applied to the selection of studies which show a 100% publication bias. They have all been ‘cherry-picked’ to show exactly what the authors want them to show: that face masks could work.

Conveniently, the authors of the STAT article fail to refer to Cochrane reviews of precisely the kind of studies they advocate. A 2015 review of studies titled ‘Gloves, gowns and masks for reducing the transmission of meticillin‐resistant Staphylococcus aureus (MRSA) in the hospital setting’ concluded: “The effects of gloves, gowns and masks in these circumstances have yet to be determined by rigorous experimental studies.” In a 2016 review of studies titled ‘Disposable surgical face masks for preventing surgical wound infection in clean surgery’, the authors concluded: “From the limited results it is unclear whether the wearing of surgical face masks by members of the surgical team has any impact on surgical wound infection rates for patients undergoing clean surgery.”

Naturally, readers of the Daily Sceptic have the humility to admit that absence of evidence is not conclusive – though a null result from an RCT is not really absence of evidence but evidence of absence within the bounds of the trial’s limitations. In these circumstances, is it acceptable to impose a costly, polluting and potentially harmful intervention on the public? I think not.

Dr. Roger Watson is Academic Dean of Nursing at Southwest Medical University, China. He has a PhD in biochemistry.