…that even a poorly designed and executed RCT outweighs any amount of "outside" evidence. This frees the advocate from the need to understand and address expertise and findings from other fields, and allows them to stake their claim to the resulting "terra nullius".
This brings us back to the unfortunate conflation of EBM (a useful shortcut under the right circumstances) and science. The mistaken belief that RCTs magically have universally greater evidentiary value than all other approaches leads to the erroneous-but-convenient assumption…
…in the success of the intervention, or that is not completely transparent and free from outside interference (such as a publicity campaign arguing that the drug is ineffective, and trial participants should not take their assigned medications).
A drug company could easily generate a trial in which their competitor's product was not effective - but it wouldn't mean very much. The same is true for a trial employing a protocol that wouldn't be accepted by experts with a clear interest…
Critically, this also means that once drug efficacy has been demonstrated, a new study claiming to show lack of efficacy with a different protocol merely demonstrates that the protocol was not well-designed.
It can't tell you that the drug wouldn't have worked at a different dose, or on a different schedule. This is directly analogous to the famed parachute study, in which parachutes showed no effect on survival - when jumping from an altitude of 0.6m. www.bmj.com/content/363/...
Objective To determine if using a parachute prevents death or major traumatic injury when jumping from an aircraft. Design Randomized controlled trial. Setting Private or commercial aircraft between...
A drug trial that shows clear evidence of efficacy is a positive demonstration that the drug can work. A trial that does not, *cannot* show you the drug can't work - the results only hold for the specific protocol tested.
This is the crucial limitation of RCTs for this category of problems - they are big, they are expensive, they answer only a small number of questions (very often only one) and if you don't see an effect, often can’t tell you why - a significant drawback vs mechanistic approaches.
An RCT tests only the latter, but all too often ambiguous language is used to imply that the results then say something general about the first two (and - perhaps not so coincidentally - extend the range of authority claimed by the speaker).
A particular failing in discussion of RCTs around respiratory protection is lack of clarity around what exactly is being tested - is it "masks", "community mask use", or "a very specific protocol for the use of masks under very specific circumstances"? en.wikipedia.org/wiki/Obscura...
