ECOLOGICAL AND EPIDEMIOLOGICAL FALLACIES IN VACCINE STUDIES
In this article, updated and re-posted from jameslyonsweiler.com, I outline the top Ecological and Epidemiological Fallacies in Vaccines Studies.
I AM A HOUND FOR SCIENCE. I don’t read hundreds of studies a month, I read thousands. As a lifelong methodologist, I focus first on the study design, then the design of analysis, and THEN the results. In teaching study design, I always emphasize the use of logic and the importance of mapping a critical path to try to insure that given the variables measured and unmeasured, students and scientists do not rigorously fool themselves into an erroneous conclusion.
Fallacy #1. Association or Correlation Tests Causation. Most vaccine studies employ retrospective association studies. The search for the association of vaccination (V) with any negative health outcomes (E, for ‘Effect’) could be a fair test of the presence of an association or correlation. This could be functionally due to either a direct causal path (V->E) or an indirect causal path (V->Eo->E). Such tests as applied in vaccine studies fall far short of a test of causality, for several reasons. The first is that associations and correlations can also be due to shared causes (V->E, V->Eo). One indirect causal pathway that is unlikely is a cause C leading both the V and E (shared cause) but see Healthy User Bias, below.
Association and correlation are nowhere near sufficient tests of causality. The second is that the results of analyses are rarely applied symmetrically; that is the weight of the evidence of a negative outcome (no association) is higher than the weight applied to evidence of a positive outcome (association with adverse health outcomes). This is also a type of Interpretation Bias.
Fallacy #2. Accepting Negative Results When Power is Low. Statistical power is the probability of rejecting a null hypothesis when it is, in fact, false. In the case of Vaccine safety science, the null is usually “no difference in rates” of a specific type of adverse event. A flaw of many clinical studies, including those that do not involve vaccines, is the publication and interpretation of negative results from studies that have such low sample sizes that the expected result is to fail to reject the null. Systematic reviews and meta-analyses that cite low-powered studies as if they were, in fact, sufficiently powered (large enough) to confidently interpret a negative result as evidence inherit and perpetuate the flaws of those studies.
Fallacy #3. Adjusting for Risk Factors and Interpreting Resulting Negative Results as “No Effect”. Smoking, the number of sexual partners, and the use of a condom are all known risk factors for HPV infection. Studies of the risk of infection by non-vaccine targeted HPV types that use adjusted odds ratios (aOR) after adjusting for known HPV vaccine risk factors often conclude there is no risk of type replacement at the population level. Type replacement is the increase in non-vaccine targeted types following the removal of the vaccine-targeted types. In the unadjusted analyses, the rates of infection by non-vaccine targeted types have been shown to increase, and the loss of the significance of the test of association is taken by some authors to mean that vaccination does not contribute as a causal factor to type replacement. The fallacy is made clear by comparing these two sentences:
“Our adjusted odds ratio was not significant, suggesting that vaccines do not contribute to type replacement” vs.
“Our adjusted odds ratio was not significant, suggesting that vaccines do not contribute to type replacement unless you smoke, continue to have multiple partners, and fail to use a condom”.
Population health policies on vaccination that leave those at the highest risk of infection due to other factors should not relegate the entire causality of risk to behavior; in fact, the population-wide rate of infection of non-vaccine targeted types is increasing beyond that expected without the vaccination because its prevalence is increased. That means, quite simply, if one is going to be infected with an HPV, it is becoming less likely that the HPV will be vaccine-targeted. This expectation exists even without ecological niche competition – all that is needed is an increase in the prevalence of HPV-vaccine targeted types. Individuals who are immune to vaccine-targeted types are not immune to all vaccine-non-targeted HPV types (although some cross-protection has been detected among a few non-targeted and targeted types).
The inherent risk of misinterpreting type replacement studies is that rarer, more dangerous types of HPV could increase in prevalence, leading to more aggressive HPV-linked cancers.
Similarly, studies of the link between vaccines and autism that adjust for birth weight, gestational age, mother’s age, and mother’s income, interpret the loss of significance of association as showing there is no risk of autism from vaccines at the population level. Instead, they should conclude that their analyses support that low birth weight or poor nutrition may be co-factors, or risk factors, for the development of autism after exposure to vaccines. The CDC has published a White Paper that mentions correcting for confounders using a path diagram that could identify “back door” influences (Link). The problem with this entire paradigm is that the back door influences actually do exist and might make vaccines riskier for persons with those risk factors.
In reality, any such causal pathway diagram designed to help test hypotheses would have every factor feeding arrows to Vaccination and then to Environmental Exposure (source of antigen/trigger). Fixing the lower diagram for some of these errors, the missing path Vaccine->Person->Environmental Exposure (antigen source)-> asthma diagnosis is seen:
The fact that leadership in vaccine science mis-specifies the causal critical pathway, and then trims out likely co-factors, is unsettling as it represents an example of Dogmatism. The removal of variation introduced by covariates that are, in reality, co-factor, and may be useful as predictors of vaccine adverse events, is not warranted unless the interaction term has been tested, and found to be not significant, and then only if the study was sufficiently powered to test the interaction terms.
Fallacy #4. Affirming the Consequent (Unwarranted Generalization)
MMR vaccines are not associated with autism.
Other vaccines are vaccines.
Vaccines do not cause autism.
Based on what?
Similarly, the CDC website:
“In fact, a 2004 scientific review by the IOM concluded that “the evidence favors rejection of a causal relationship between thimerosal-containing vaccines and autism.'”
That does not mean that all vaccines do not cause autism. In reality, not all vaccines have been tested for association with autism.
It is therefore fallacious to conclude that VACCINES do not cause autism.
Fallacy #5. Healthy User Bias
Healthy user bias is a serious limitation of all observational studies that seek to examine the impact of one medical intervention, and the risk is particularly high in prevention medicine.
Many parents stop vaccinating due to early, bad experiences with vaccines (such as HepB) with their child, or with their older sibling. In a study of later vaccines, such as MMR, these individuals would be marked as “unvaccinated” against MMR but would in reality be partly vaccinated. Also, individuals who can tolerate vaccines would tend to have more vaccines and would thus be more likely to receive the MMR. The net result is total confounding of health care choice with both vaccination status and health outcomes. If the MMR was going to tip the toxic scale of a child and push them over their personal limit, other toxins in the environment still accumulating, the “unvaccinated” child might still regress into autism. Due to the careful nature of their parents, however, the child may not have been given further vaccines, and thus they would appear as “unvaccinated” and not regress into autism. The vaccinated population also may include individuals more likely to participate in other health care programs, not studied by a study of the risk of a single vaccine, and the non-vaccinated population may include individuals more likely to avoid other sources of toxins.
The net effect is a non-random tangled web of confounding variables, potentially leading observational studies to biased odds ratios. The key here is that the effects are non-random and impose selection biases. For non-vaccine and vaccine examples of Healthy User Bias, see this review.
Fallacy #6. Argument from Ignorance
In 2012 and 2014, IOM reviewed available studies (but ignored animal studies) and concluded that there was insufficient evidence to conclude that vaccines cause autism. They used terminology such as “Evidence is inadequate to accept or reject a causal relationship … The committee does not conclude that the vaccine does not cause the adverse event …”
Some reported this as “IOM found no evidence” to support the hypothesis that vaccines cause autism…. and they leave out the corollary that IOM also found insufficient evidence to say that vaccines do NOT cause autism.
Cherry-picking the literature is an example of argument from ignorance. A good example of Cherry-picking is the exclusion of studies that reported positive results. When AAP sent their list of studies to POTUS, they conveniently left out several key studies, such as these provided by JB Handley:
Neonatal hepatitis B vaccination impaired the behavior and neurogenesis of mice transiently in early adulthood
Hepatitis B Vaccination of Male Neonates and Autism Diagnosis, NHIS 1997-2002
https://vaccinesafetycommission.org/pdfs/26-2010-Hep-B-Autism.pdf
Hepatitis B triple series vaccine and developmental disability in US children aged 1-9 years
https://vaccinesafetycommission.org/pdfs/35-2008-Hep-B-EIS.pdf
IL-4 mediates the delayed neurobehavioral impairments induced by neonatal hepatitis B vaccination that involves the down-regulation of the IL- 4 receptor in the hippocampus
And, the link below is JB’s article explaining the implications of study #1.
https://jbhandleyblog.com/home/hepatitsb2018
Other studies left out include
One-sided targeted analyses from biased websites are not useful for understanding the full body of literature on the vaccine/autism question. And meta-analyses that ignore studies are also guilty of cherry-picking, and their fallacious conclusions will naturally be suspect.
Fallacy #7. Assuming a Random Population
If a genetic subgroup with higher susceptibility exists, the population under study is non-homogeneous. Studies that ignore genetic or familial risk will tend to be unable to detect an effect at the population level. Enrichment studies that accrue patients to groups based on familial risk and genetics are needed. High-risk vaccinated vs high-risk unvaccinated would be an important comparison. Until we actually test the relevant hypothesis of risk, we cannot rule out that vaccines might be harming a genetic minority of individuals, who have rights of equal protection.
This fallacy leads, in translation to clinical practice, to the Either/Or or Black/White, False Dilemma, or Excluded Middle Fallacy, in which the “sometimes” option is replaced with the binary option (yes/no). The correct hypothesis is not whether vaccines cause autism in everyone (“I was vaccinated, I am fine”), but rather do vaccine cause autism in some people. Importantly, if there is genetic risk to vaccine injury, those at risk are entitled to protection from harm; those who are injured should be heralded, not scoffed at, and we as a society should do everything we can do to insure that people who fall into the at-risk category are protected.
Conclusion
In reality, the use of observational studies to assess vaccine safety is fundamentally flawed. Randomized clinical trials with total health outcome awareness must be used to insure full knowledge of the risks of vaccination. Those who argue that it would be unethical to not vaccinate some children and deny them protection from infections are committing the fallacy of begging the question: they are assuming the truth of the conclusion, instead of conducting Science to test the question of relative safety (circular reasoning): their argument requires that their desired conclusion be true.
The real health risks of any individual vaccine, to >1 vaccine per day, or to exposure to the entire CDC childhood schedule is unknown due to
reliance on passive vaccine injury collection via VAERS, which requires users to acknowledge that no entry can be used to assume causality, and due to intentional steps during vaccine studies to exclude adverse events that the investigators have determined to not be due to the vaccine – both addition examples of the begging the question.
Consist efforts to bias results on studies of safety toward the null hypothesis of n increased risk of adverse events.
Over time, I will add more common fallacies in the vaccine safety science literature. In the meantime, post your favorite flawed studies and let the readers know which fallacy – or fallacies – the studies suffer from.
Empower Yourself with Knowledge. Consider taking this course w/Dr. Cathy Stein (Spring 2023) @ IPAK-EDU! First day of class is 1/26/22. Class is Thursdays 2:30-3:30
I think they could call the current way they do vaccine science “ Wishful thinking” and have done with it.
So many people say, “ Nothing happened to me, my child or my neighbors child, therefore all vaccines are safe for everyone. “
This ignores so many things, chief among them is that all people are not the same due to a host of factors, genetic and environmental.
I wish there was acknowledgment that for some people, vaccines are infinitely worse than the illness they purport to protect people from, and that one size fits all mandates will cause irredeemable harm to that group of individuals that can almost never be recovered from.
If you point this out to those avowed vaccines proponents that logically, all people are not the same, and that we know from past vaccination usage that some people will suffer unanticipated harms from the shots, they respond that the risks of harm are very low.
As if that answer, “ the risk of harms are very low” is a reasonable justification for the potential infliction of suffering that can occur to untold people.
We might as well say these vaccine proponents have a “ Compassion Deficit “, since the unnecessary suffering of affected others is ok, unproven, ecetera in their book.
This is just what they did in the covid shot trials, too.
"The problem with this entire paradigm is that the back door influences actually do exist and might make vaccines riskier for persons with those risk factors."
They said, "Oh, those people who had heart attacks or strokes during the trials were at higher risks for those things anyhow because they were on medication that increased their chances or had a physical condition that increased their risk."
So? You are expecting every single person in the country to get these shots. How many of them also fit that profile? Have you proven that their heart attacks or strokes were just incidental, that they would have happened anyhow because of those other risk factors? Or could it be that people with those other risk factors are at greater risk of those events because of the combination of those factors and the shots?