WASHINGTON - Nov. 16, 2020 - Moderna Vaccine Maybe Not 94.4% Effective
Epidemiologists Can't Infer Precise Results From Tiny Sample Size

WASHINGTON, D.C. (November 16, 2020) - "Moderna's COVID-19 Vaccine Is 94.5% effective" say many headlines, but claiming the precise effectiveness, supposedly accurate to one-tenth of a percentage point, is misleading at best, like claiming that a political poll shows candidate X with 54.3% to 45.7% lead when the poll has a margin of error of +/- 4.0 percentage points, says Professor John Banzhaf.

Moderna's statement, making a claim of effectiveness down to one-tenth of a percentage point, is based upon the report that, among a test population of some 15,000 who received the vaccine, only 5 became infected, whereas of the 90 volunteers in the control group who received a placebo, and presumably were subject to the same exposure risk, 85 became infected.

While it's mathematically true that 85 divided by 90 does equal 94.44%, the number in the vaccinated group who became infected could as easily have been 2 or 3 higher or lower, simply because of random statistical fluctuation plus inherent inaccuracies of tests for COVID-19.

Thus, for example, if the number in the vaccinated group who became infected was 7 instead of only 5, the effectiveness would drop to 92.22%.

And, if the number who actually became infected was 9 instead of 5 - a number which is quite possible given the random uncertainties of medical tests, plus the false negatives of some COVID tests - the effectiveness would drop to 90%, which is the same effectiveness claimed by Pfizer for its new COVID vaccine.

This, of course, doesn't mean that the two vaccines are likely to be equally effective, but rather that a number like 5, out of a total of some 15,000, hardly seems sufficient to draw a reliable statistical conclusion that the vaccine was 94.4% (and not 90% or 98%) effective, argues Banzhaf, a mathematician and creator of the Banzhaf Index.

That's because a change of only a few detected infections in the test group - something which could easily be caused by a random statistical fluctuation and/or by inherent inaccuracies of the test for COVID - could cause a significant change in the number of volunteers who apparently became infected, and therefore in the claimed effectiveness figure itself.

That's why most epidemiologists and statisticians would probably not draw any firm conclusions from such a small result, especially after factoring in the limited reliability of some of the tests.

http://banzhaf.net/ jbanzhaf3ATgmail.com @profbanzhaf

Contact
GW LAW