Interpreting Autoimmune Risks in Covid-19 Vaccine Literature: A Two-Phase Qualitative Coding Protocol on Biological Mechanisms and Epistemic Integrity

1. Introduction

The global COVID-19 vaccination campaign has been unprecedented in speed, scope, and ambition. Endorsed by health authorities worldwide as a necessary exit strategy from the COVID-19 crisis, the rollout was justified on the grounds of reducing viral transmission, severe illness, and death - as the Lancet Commission on lessons for the future from the COVID-19 pandemic put it: "Vaccines are the single most important medical technology to bring the COVID-19 pandemic under control" and their “early distribution in high-income countries offered the hope of a return to normalcy" (Sachs et al., 2022) (p. 21). However, as with any mass medical intervention, the assessment of benefits must be weighed against potential risks, especially when vulnerable populations, often excluded from initial clinical trials – such as individuals suffering from autoimmune disorders - are targeted for intervention precisely given their vulnerability (American College of Rheumatology, 2021).

In a recently completed scoping review of 109 peer-reviewed articles, we documented substantial and diverse reports of associations between COVID-19 vaccines and postvaccination autoimmune disorders, both among individuals with autoimmune disorders – in the forms of flares/relapses and new autoimmune disorders – as well as in individuals with no prior autoimmunity (Chaufan et al., 2025). Studies included well-documented clinical case reports, case series, and observational studies. Importantly, multiple studies provided mechanistic explanations for how COVID-19 vaccines could plausibly trigger autoimmune responses. These included established pathways such as molecular mimicry, bystander activation, epitope spreading, polyclonal activation, and adjuvant-induced immune stimulation - mechanisms historically observed in post-vaccination autoimmunity involving other vaccines (Cohen & Shoenfeld, 1996; Molina & Shoenfeld, 2005; Shoenfeld & Aron-Maor, 2000).

The study outlined in this protocol was prompted by the need to examine whether - and how - the literature on COVID-19 vaccines and autoimmunity has maintained consistency and transparency in its treatment of the risks and benefits of vaccination. While the biomedical literature has focused mostly on vaccine benefits – relying largely on modeling studies to assert that COVID-19 vaccination has saved “millions of lives” (Imperial College London 2022; Mellis 2022; Prof Peter Hotez MD PhD [@PeterHotez] 2025; Watson et al. 2022) - an emerging body of research actually points to COVID-19 vaccine harms – such as autoimmune complications, at least in some individuals (Guo et al., 2023; Jara et al., 2022; Lim et al., 2025; Mahroum & Shoenfeld, 2022; Watad et al., 2021).

The growing literature on autoimmune disorders postvaccination notwithstanding, consistent gaps remain. First, in the course of conducting our scoping review, we observed a disconnect between the presence of mechanistic plausibility and the conclusions drawn in many studies, with many authors documenting findings that supported a plausible vaccine-autoimmunity link yet concluding that no causal relationship could be established or implied. While such caution may be warranted given limitations of study design or data, it also raises questions about consistency in interpretive practice - precisely given study design and data limitations, a similar caution should apply to vaccine benefits, often claimed in the conclusion. Second, we found a tendency to apply different evidentiary thresholds to statements of vaccine benefits and vaccine harms, even within the same article. This epistemic asymmetry - wherein benefit claims are often asserted without corresponding rigor or explicit data, while harm claims are accompanied by hedging language and disclaimers - prompted us to explore this dynamic more systematically.

One other salient observation in our scoping review examining the relationship between COVID-19 vaccination and autoimmune disorders was that the Bradford Hill criteria – long recognized as foundational in evaluating causal relationships (Hill, 1965) – were notably underused or inconsistently applied. These criteria include key elements such as temporal sequence, strength and consistency of association, biological plausibility, and dose-response relationship. Their absence, combined with improperly supported benefits and safety claims and rhetorical minimization of risks, raises concerns about the standards being applied to interpret vaccine-related harms. This project aims to shed light on those interpretive standards, assess how the literature discusses the possibility of causality, and whether it does so with integrity.

This planned, two-phase follow-up study builds directly on our scoping review dataset and adopts a critical realist lens to explore both the biological and epistemic dimensions of the COVID-19 vaccine–autoimmunity literature. The first phase of the study will map the mechanisms of action described in the literature and assess whether these are consistent with the interpretive claims made by authors. The second phase will apply a structured coding tool to evaluate the epistemic integrity of each article, focusing on how consistently authors apply evidentiary standards to claims of benefits versus harms.

Although methodologically and conceptually aligned, each phase is self-contained. Together, however, both phases will trace a continuum - from biological plausibility to epistemological judgment - revealing how the interpretation of evidence about vaccine safety may be shaped not only by data, but by contrasting rhetorical, institutional, and epistemic norms.

2. Aims and Objectives

The two-phase planned study aims to evaluate how the biomedical literature addressing COVID-19 vaccines and autoimmune disorders frames questions of risk, benefit, and causality. Building on a comprehensive scoping review, it will seek to clarify how mechanistic plausibility and evidentiary reasoning are treated in the interpretation of findings. Phase 1 will assess whether the biological mechanisms reported by study authors are consistent with their interpretive conclusions. Phase 2 will apply an epistemic integrity framework to examine the standards of evidence applied to benefit and harm claims. By integrating mechanistic and epistemological analyses, the planned study will contribute a critical empirical and conceptual lens for assessing the evidentiary logic of COVID-19 vaccine safety research.

Objectives include:

• Phase 1 – Biological Mechanisms and Interpretive Consistency
  • To identify and categorize mechanisms of action proposed in peer-reviewed literature linking COVID-19 vaccination to autoimmune disorders.
  • To assess whether these mechanisms are treated as supporting causal interpretations or are contradicted by study conclusions.
  • To examine rhetorical strategies used to reinforce or downplay mechanistic findings.
• Phase 2 – Epistemic Integrity Evaluation
  • To assess how scientific publications frame vaccine-related harms and benefits.
  • To classify each article according to a four-tiered typology of epistemic integrity (high, moderate, low, neutral).
  • To identify patterns of evidentiary asymmetry, rhetorical minimization, or double standards in causal reasoning.
  • To offer a reproducible model for evaluating epistemic framing in biomedical safety research.

3. Methods

Study Design

This protocol outlines a qualitative, two-phase, document-based study grounded in a critical realist philosophy of science. It builds directly on 109 studies previously selected and analyzed in a comprehensive scoping review using PRISMA-ScR guidelines and stored in full-text format (Chaufan et al., 2025). A qualitative content analysis approach will be applied to extract, categorize, and interpret data from these 109 full-text scientific articles and to appraise how these articles report, interpret, and contextualize associations between COVID-19 vaccination and autoimmune disorders. Qualitative content analysis is well suited for both phases of this project, as it enables systematic coding of textual data and facilitates the breakdown of published literature into manageable analytic units, while preserving contextual meaning (Hsieh & Shannon, 2005), thus aligning well with the study’s critical realist orientation by foregrounding the relationship between empirical observation and theoretical interpretation. It also allows for a clear mapping of how claims are framed relative to the data presented.

Each phase will employ a tailored methodological framework aligned with its respective objective. In Phase 1, this method will allow for the transparent classification of immunological mechanisms and the assessment of their interpretive treatment by study authors. In Phase 2, it will provide a basis for evaluating epistemic integrity using a reproducible typology that classifies articles based on internal consistency, evidentiary rigor, and framing asymmetries. This combination should support a robust and transparent examination of how scientific literature on COVID-19 vaccine safety represents uncertainty, causality, and risk.

• Phase 1 – Mechanisms and Interpretive Consistency

This phase will extract mechanistic explanations provided by study authors and classify them into immunological categories such as molecular mimicry, bystander activation, cytokine dysregulation, and ASIA (autoimmune/inflammatory syndrome induced by adjuvants). It will also extract authors' conclusions regarding causality or lack thereof and evaluate the extent to which these are consistent with the biological mechanisms proposed. The goal is to examine whether the logic used in interpretation is coherent with the reported data. Attention will also be given to rhetorical strategies - hedging, disclaimers, or narrative distancing - in authors’ interpretive statements.

• Phase 2 – Epistemic Integrity Evaluation

This phase will re-analyze the same set of articles using a structured typology of epistemic integrity. Quotations related to autoimmune harms, safety and efficacy claims, causal reasoning, and rhetorical framing will be extracted. Each article will be classified according to a four-level typology of epistemic integrity (high, moderate, low, or neutral), previously developed through inductive and deductive iteration during the scoping review. Coding will assess internal consistency, evidentiary rigour, and the symmetry of standards applied to benefit and harm claims.

A visual summary of the study design, including both phases and their respective goals, methods, and outputs, is provided in the flowchart below:

• Data Source and Storage

The data for this planned study have already been collected as part of a completed scoping review conducted in 2024–2025. The full dataset consists of 109 peer-reviewed articles, all of which are stored in full-text PDF format and have been archived for analysis. These texts represent the empirical foundation for the coding and classification phases described earlier.

• Coding and Analysis Plan

Each phase will involve systematic coding of the 109 peer-reviewed articles selected in the published scoping review. Each article will be closely read by two independent coders, who will follow structured extraction templates tailored to each phase, with reflexive memos maintained to ensure transparency.

• Phase 1 – Mechanisms and Interpretive Consistency

Articles will be coded for:

• Mechanisms of action proposed to explain associations between COVID-19 vaccines and autoimmune disorders, classified into predefined immunological categories (e.g., molecular mimicry, bystander activation, cytokine dysregulation, epitope spreading, ASIA).
• Descriptions of autoimmune outcomes (e.g., flares, new-onset conditions) and affected populations (e.g., pre-existing autoimmune disorder, no prior autoimmunity).
• Interpretive conclusions related to causality (e.g., explicit denial, assertion, hedging, or neutrality).
• Rhetorical framing strategies, such as minimization ("rare"), emphasis ("severe"), disclaimers, or qualification.

An Excel sheet will be used to document both mechanistic content and interpretive alignment. The analysis will examine whether mechanisms are used as support for causal reasoning or are rhetorically neutralized. Of note, it will not treat the absence of causal conclusions as evidence of bias. Rather, it will examine whether the level of caution applied to mechanistic findings is consistent with the level of certainty applied to claims of benefit within the same article. Quotations to support this analysis will be extracted verbatim and include page references.

• Phase 2 – Epistemic Integrity Evaluation

This phase will apply a previously developed four-level typology of epistemic integrity (high, moderate, low, neutral) to evaluate how each article frames vaccine-related benefits and harms. Quotations will be extracted verbatim and recorded into an Excel sheet for:

• Descriptions of autoimmune harms or adverse events
• Statements about vaccine efficacy and safety
• Causal framing and rhetorical qualifiers (e.g., “rare,” “anecdotal,” “safe,” “highly effective”)

The classification will be based on operational indicators across four domains: harms framing, efficacy claims, causal reasoning, and epistemic symmetry. Illustrative examples from the dataset will be included to show how the classification was applied. Table 1 presents a typology summarizing the key indicators for each tier.

Finally, some statements may be relevant to both phases - for example, when conclusions reference mechanistic evidence while also asserting benefits or minimizing harms. In such cases, the same quotation may be analyzed in both phases but evaluated through distinct coding frameworks: Phase 1 will assess alignment between the author’s interpretation of their mechanistic evidence and their conclusion, while Phase 2 will assess the consistency of evidentiary standards applied across claims of benefit and harm. Coders will be trained to apply phase-specific criteria to ensure analytical separation.