Translational Failure: Nature Medicine Inmate Study Removes >99% of the Data, Is Used to Conclude that COVID-19 Vaccines Reduce Transmission. PR Score: F-

Study restricts consideration of close contact inmates (who shared cells). As if non-close contacts do not transmit, and as if prison cell settings are relevant for the world outside the Big House.

This is not an academic note. Your representatives, family members, and school boards must know that science on efficacy has been warped just as much as science on safety.

A hugely important part of epidemiologic studies is whether the study sample - the group(s) of people being studied - is representative of the general population about which generalizable knowledge is sought.

This issue is so important that when I read an observational study, such as this one in Nature Medicine, entitled “Infectiousness of SARS-CoV-2 breakthrough infections and reinfections during the Omicron wave”, the first part of the study I read the section that describes the inclusion/exclusion criteria and compares the outcome of the exclusions to the general population for which knowledge is sought. With a title as general as this, one would think that the study has general relevance to the question of whether COVID-19 mRNA vaccines and infections protect against infection and re-infection.

For such a study, I then look to see whether the study found a difference or association (or no difference or association), and then only could report a difference following statistical adjustment for variables. It is truly important to know how the various results were chosen; which variables were selected as confounders, and then, importantly, whether the confounders are suspected (weak) or if previous functional relationships of the type necessary to use the label “confounder”, rather then “covariate”, are known.

Study Earns a Popular Rationalists Score of F- on Generalizability

The study in question involves data from prisons in California’s penal system (hardly a representative population or setting relevant to the rest of society). The following inmates were excluded:

• those who were not held in cells with other inmates

• those who did not have housing or prior COVID-19 test result data

• likely to have tested positive for a variant other than Omicron

• housed in a small institution

• negative PCR test during the infectious period (risk: false negative results)

• had contacts w/inmates w/positive test +/- 2 days after first exposure

• inmates without “valid” contacts

• no negative test for PCR +/- 2 days of first exposure

• no follow-up testing data

• could have been exposed to >1 infected case

After Exclusions, the Study Used Only 0.79% of the Data

Starting with over 155,000 inmates, the study was based on a grand final total of 273 unvaccinated + 953 “vaccinated” = 1,226 inmates. “Vaccinated”, of course, was restricted to inmates only after 14 days after their first dose; anyone who developed COVID-19 or who died on days 1-14 after injection was excluded.

The study is relevant for 0.79% of the inmate population and is not expected to be relevant for 99.21% of the prison population in California. Thus, the PR score of F-.

Even after all of this, the initial (unadjusted) results were reported as

“Unvaccinated index cases had a 36% (31–42%) risk of transmitting to close contacts, whereas vaccinated index cases had a 28% (25–31%)”

In other words, the 95% confidence intervals overlapped (31/31). No difference. Fine.

Side note: I have dealt with the issue of adjusting for covariates as confounders - over-adjusting - before, way back on Sept 28, 2015, when I first started reading en masse all of the studies on vaccine safety I could find:

DISEASE EPIDEMIOLOGISTS AND PUBLIC HEALTH POLICY MAKERS: PLEASE STOP “ADJUSTING” FOR RACE, INCOME, AND OTHER COVARIATES WITHOUT STUDYING INTERACTION TERMS https://jameslyonsweiler.com/disease-epidemiologists-and-public-health-policy-makers-please-stop-adjusting-for-race-income-and-other-covariates-without-studying-interaction-terms/

Not to worry - the authors are just warming up. Adjustments await the reader of the Nature study:

“Adjusting for the duration of exposure between index cases and close contacts, close contacts’ history of vaccination and prior infection, facility effects, and background SARS-CoV-2 incidence via a robust Poisson regression model, we estimated that index cases who had received ≥1 COVID-19 vaccine doses had a 22% (6–36%) lower risk of transmitting infection than unvaccinated index cases.”

Now, if you can divine what “facility effects” are from the information provided (I could not), and if you think the duration of exposure does not matter necessarily must be precisely the same between vaccinated and unvaccinated (I do not), and if you believe that the immunity of inmates’ close contact does not matter (it does), you can just use this handy percentage (22%) - and that is the percentage that is being used to claim that the vaccine works.

And of course, that is how the study is being interpreted. It is being misinterpreted as if it provides definitive proof that vaccination reduces transmission (in general) - as if the association can test the hypothesis of causality (it cannot).

But that inference should strictly be limited to assessing the risk of infection in people who have two doses, and then only 14 days after the second dose, or more specifically, inmates who are housed in close quarters with other inmates, and who also meet the exclusion criteria, and is relevant to 0.79% of the inmate population.

And because of the artificiality of the setting, sampling bias of the inmates, and peculiarities of their make-up and their behavior, It is not likely relevant to the general population.

Neil de Grasse Tyson and Bill Maher both said recently that differences between populations differ, Tyson, arm-waving and high-voicing yelping (about how correct he is) tried to use population differences to try to downplay the importance of the miracle of Sweden (as if parts of the United States do not have the same population density!), to which Maher correctly adjusted:

“You just said that we can’t make any judgment (on whether the lesson of Sweden shows our response was wrong) because don’t live in another universe where the United States handled it differently, and I’m saying there are other places that did handle it differently - and that does matter.”

Good for Maher.

The problem for the WHO, CDC, FDA, NIAID, Fauci, and people who message and think like these organizations is this: you cannot debunk reality.

Yes, Sweden matters. The data show that Sweden is doing far better than the United States, and is suffering far less as a result of their response. And using inmates, Tyson, as if they are representative of the rest of the US population - well, he should be just as excited about that issue as he is about debunking the reality of Sweden.

Sign up below for my Spring 2023 course, How to Read and Interpret a Scientific Study. Empower yourself.