Using Measles Outbreaks to Identify Under-Resourced Health Systems in Low- and Middle-Income Countries: A Predictive Model

1. Introduction

Measles is a vaccine-preventable infectious disease; however, outbreaks continue to occur. The disease mainly affects children, and complications of infection with this airborne virus can be severe and include pneumonia, encephalitis, ear infection, neurological complications and death [1]. Since the introduction of the measles vaccine in 1963, there has been a significant reduction in the number of cases and deaths associated with the disease [2,3]. Measles vaccination has averted 56 million deaths worldwide between 2000 and 2021 [3]. Despite this, measles remains a public health burden. In 2021, there was an estimated incidence of 9 million cases and 128,000 measles-related deaths globally [4]. The number of large and disruptive measles outbreaks (defined as an incidence of at least 20 cases per million population over a period of 12 months) has increased consistently since January 2022 [5,6]. The burden of measles varies substantially across regions, with particularly poor outcomes in low-and-middle-income countries (LMICs) [7]. In fact, 99.7% of all reported measles cases in 2022 occurred in LMICs [8].

The burden of measles is driven by its high transmissibility, with a basic reproduction number (R0) estimated between 12 and 18 [9]. Therefore, 95% vaccination coverage with two doses of a measles-containing vaccine is necessary to achieve herd immunity [10]. The accumulation of susceptible individuals can lead to outbreaks of measles, particularly among those without complete vaccination [11]. Even when high national vaccination coverage is reported, marginalised, vulnerable, or conflict-affected communities may be missed by vaccination programs, resulting in variation at sub-national levels [12]. Vaccination coverage is not necessarily equivalent to the level of protection in a population, particularly in LMICs where vaccine effectiveness is sub-optimal [13]. Variation in immunisation coverage is driven by health inequities, arising from differences in the distribution of health system services [14]. Institutionalised disparities in privilege between groups (e.g., racism, homophobia), known as systemic inequities, can influence the allocation of health services as well as social and environmental infrastructure [14]. The Gavi 2021-2025 strategy [15] aims to address these health inequities, by focussing immunisation programs on missed communities and zero-dose children (children who have not received the first dose of the diphtheria-tetanus-pertussis vaccine) [16]. Furthermore, the recent COVID-19 pandemic has led to a decreased uptake of routine childhood vaccines in LMICs, exacerbating challenges to achieving high, uniform coverage [17]. Variation in immunisation coverage both locally and nationally plays a critical role in the emergence of measles outbreaks, as it creates pockets of susceptible individuals [11]. Countries or regions with sub-optimal measles immunisation coverage have a higher likelihood of measles outbreaks, posing the risk of increased morbidity and mortality as well as severe social and economic outcomes [3,18].

Measles is often described as a ‘canary in the coal mine’ of under-resourced health systems [12,19,20]. Outbreaks of the disease may serve as an indicator of health system weaknesses, as populations which are un-immunised, under-immunised, and under-served by the health system are more likely to be impacted by measles outbreaks [12]. Populations with fragmented health systems often lack high-quality data on immunisation coverage and other health system performance indicators, hindering effective prioritisation of resources [21,22,23]. Applying measles outbreaks as a tracer of health system weaknesses could address this issue. The World Health Organization (WHO) Immunisation Agenda 2030 [12] and the Measles and Rubella Strategic Framework 2021-2030 [19] establish that measles outbreaks reveal vulnerabilities in the health system, owing to the high transmissibility of the disease. Similarly, John [20] reported that measles may uncover gaps in immunity, in turn indicating underlying problems in health infrastructure. An article from WHO [24] discusses the overwhelmed state of health systems following the COVID-19 pandemic, creating a ‘perfect storm’ for disruptive measles outbreaks, which may serve as a warning for the emergence of other vaccine-preventable infectious diseases.

Robust health systems are crucial to achieving high, uniform measles immunisation coverage, as well as preventing and responding to outbreaks [12,25]. A successful response to the emergence of measles depends on the strength of communication in the health system, a strong and numerous workforce, as well as surveillance systems and diagnostic capacity [25]. In under-resourced settings, where health systems lack sufficient infrastructure to deliver effective vaccination programs and control outbreaks of the disease, measles can become a considerable public health burden [24]. These critical components of the health system are described in Monitoring the building blocks of health systems: a handbook of indicators [26], published by the WHO. This handbook provides an overview of the health system using six distinct building blocks: health service delivery, the health workforce, access to essential medicines, health information systems, health financing, and leadership & governance [26]. The building blocks framework has been widely used in the health system strengthening literature [27,28,29], while also serving as a core component of the WHO Primary health care measurement and indicators operational framework [30] and the WHO Health systems resilience toolkit [31]. The health system can also be described on a national, community or individual level. Although under-resourced health systems exist in both high-income countries and LMICs, the latter are disproportionately affected, with an estimated 8 million deaths annually due to conditions that should be treatable in a high-quality health system [32]. Therefore, this analysis focusses on health systems in LMICs.

Despite reports of measles as an early indicator of insufficient health systems, to our knowledge there has not been any formal quantitative analysis on the topic. An understanding of this topic may reveal the most vulnerable aspects of the health system, allowing public health coordinators and policymakers to identify priorities in health planning. Consequently, countries may be able to prevent future measles outbreaks and other public health disasters, while strengthening the health system more broadly.

This study aims to determine whether under-resourced health systems are associated with an increased probability of large measles outbreaks in LMICs. In addition to the public health significance of large measles outbreaks, comparing them to smaller outbreaks provides a more robust analysis than comparing outbreaks of any size to the absence of outbreaks. Understanding the association between under-resourced health systems and large measles outbreaks may help public health professionals to identify those health systems which are most at risk of a large measles outbreak and apply appropriate interventions.

2. Materials and Methods

Our analysis used an ecological study design to understand, at a national level, whether under-resourced health systems are associated with an increased probability of large measles outbreaks in LMICs. Using publicly available secondary data, we fitted a predictive model to estimate this association. Ethical approval was not required, as this project only used publicly available, non-identifiable data measured at the group level.

2.1. Directed Acyclic Graph

First, we developed a directed acyclic graph (DAG) using the DAGitty software [33] in consultation with the literature and expert knowledge, to identify potential sources of selection or information bias.

2.2. Data Collection

2.2.1. Measles Outbreaks

We defined a measles outbreak as two or more laboratory-confirmed measles cases with rash onset within 7-21 days of each other, in addition to being epidemiologically and/or virologically linked, as per the WHO Measles Outbreak Toolkit [34]. Using these criteria, we searched the websites ReliefWeb [35] and WHO Disease Outbreak News [36] for past measles outbreaks in LMICs, and removed any duplicate outbreaks. Each measles outbreak corresponded to one observation in our dataset. We included outbreaks in LMICs with a start date between January 1, 2010 and March 11, 2020 (the date the COVID-19 pandemic was declared), including ongoing outbreaks [37]. During the COVID-19 pandemic, measles surveillance systems were disrupted, so we limited the outbreak date range to reduce potential bias arising from under-reporting of measles cases [38,39]. For ongoing outbreaks, we used the most recent outbreak reports as of 12/09/2023, to ensure case numbers were as accurate as possible. Where multiple outbreaks had occurred in the same country across non-overlapping time periods, all outbreaks were included in the data.

Subsequently, we collected data on the number of cumulative cases for each identified outbreak, using WHO regional epidemiological reports, and supplemented by ReliefWeb and Disease Outbreak News reports [35,36,40,41,42,43,44,45]. When necessary, we collated case numbers over multiple (non-overlapping) reporting periods, and included all laboratory-confirmed, epidemiologically-linked and suspected cases.

2.2.2. Health Systems

We represented the capacity of health systems using three health system indicators aggregated at the national level. To select indicators, we devised a suitable indicator corresponding to each health systems building block, guided by the WHO building blocks of health systems.22 To minimise bias, we excluded building blocks with limited indicator data across a large number of LMICs (access to essential medicines, health information systems, and leadership & governance) [26]. The final indicators were: the proportion of births delivered in a health facility (corresponding to health service delivery), the number of nurses and midwives per 10,000 population (health workforce), and domestic general government health expenditure per capita in US$ (health financing) [26].

For each identified measles outbreak, we collected data from the WHO Global Health Observatory [46,47,48] on the above indicators for the outbreak country. To ensure temporality, we collected health system indicator data measured in the calendar year prior to the start of the measles outbreak where available, otherwise for the most recent year data was available before the outbreak.

2.2.3. Gini Coefficient

We used the Gini coefficient as a proxy for systemic inequity. The Gini coefficient quantifies income inequality, in addition to other forms of inequality, and can be compared across countries [49]. For each observation, we collected Gini coefficient data from the World Bank [50] for the most recent calendar year prior to the start of the outbreak in the corresponding country.

2.3. Data Processing

To prepare the data for analysis, we clustered the observations by each outbreak country, using the mean of each variable as a summary statistic in each cluster. Clustering methods were used to account for the possibility of non-independence of observations for outbreaks which had occurred in the same country, arising from similarities in their health system characteristics. We then created a binary variable for large measles outbreaks, derived from the cumulative cases data. We defined a large outbreak as having an incidence equal to or greater than 100 cases per million population over a 12-month period, and calculated this incidence using population sizes from the World Bank estimates [51]. Although this definition uses an arbitrary threshold of 100 cases, the literature does not present a consistent definition of a large measles outbreak. Since our analysis compares outbreaks across various countries, we did not use country-specific thresholds for the definition of a large outbreak. We conducted a sensitivity analysis for different definitions: incidence greater than or equal to 50 cases, 150 cases and 200 cases per million population over a course of 12 months.

We did not scale the health system indicator variables, since this study is not focussed on the interpretation of individual regression coefficients, but rather, the broader association between under-resourced health systems and large measles outbreaks.

2.4. Analysis Methods

Stata version 17 (StataCorp, TX, USA) was used for analysis. In the initial analysis, we calculated measures of central tendency and spread for all variables. The dataset was randomly allocated into training and testing datasets, using an 80:20 training:testing split. The training dataset was used to fit a regression model, whereas the testing dataset was used to assess the predictive performance of the model.

2.4.1. Regression Model Fitting

We constructed a multivariable logistic regression model, with health system indicators as inputs and large measles outbreaks as the output. We conducted diagnostics for linearity, goodness of fit, and influential observations to ensure our model did not violate the assumptions of regression. We removed any influential observations by assessing standardised residuals, deviance, and leverage. Additionally, we tested for collinearity between the health system indicator variables.

2.4.2. Regression Model Testing

We used the regression output to predict the odds of large measles outbreaks for observations in the testing dataset. Subsequently, we assessed the external validity of the model by comparing the predicted odds against the observed outcomes, using a plot and the Brier score [52]. A model which perfectly predicts the outcome has a Brier score of 0, whereas a model with perfect inaccuracy has a Brier score of 1 [52]. A model which gives a 50% chance of the outcome (and is therefore non-informative) gives a Brier score of 0.25 [52].

2.4.3. Effect Modification and Missing Data

To test for effect modification, we constructed the logistic regression model with an interaction term between the Gini coefficient variable and each of the input variables, then assessed the output of the interaction terms. In addition, we tested whether missing data influenced the results. We handled all missing values using multiple imputation and fitted and tested the logistic regression model using the imputed dataset.

3. Results

3.1. DAG

The DAG for this study is shown in Figure 1. In the DAG, we included measles immunisation coverage, given the critical role of the health system in reaching high immunisation coverage and the impact of immunisation gaps on measles outbreaks [11,12]. Health inequities were also incorporated into the DAG, acknowledging that such inequities stem from the allocation of health system resources, leading to variation in immunisation coverage and an increased likelihood of measles infection [12,14]. Systemic inequities influence the distribution of health services, both directly, through institutionalised attitudes towards groups of people, and indirectly, through differences in social and environmental conditions [14]. We also considered the measurement of the health system and large measles outbreaks, through both health system reporting and surveillance systems. In the analysis, we did not condition on surveillance systems, as this is a collider variable. Similarly, we did not adjust for measles immunisation coverage nor health inequities, as both these variables are mediators, with measles immunisation coverage also being a collider variable.

3.2. Descriptive Statistics

In total, we identified 78 measles outbreaks which had occurred in LMICs over the specified time period, corresponding to 55 clusters, after collapsing the data by outbreak country. Across the clusters, 30 (56.6%) were classified as large; two outbreaks were not classified due to missing cumulative cases data. Among countries with large measles outbreaks, the median proportion of births delivered in a health facility was 74.9%, compared to 88.6% for non-large measles outbreaks (Table 1). The median number of nurses and midwives per 10,000 population was also lower for large outbreaks (11.2) compared to non-large outbreaks (19.4), as was health expenditure.

3.3. Regression Analysis

Forty-four observations were randomly allocated to the training dataset, although only 38 of these were used to fit the logistic regression model due to missing data. Eleven observations were randomly allocated to the testing dataset, all of which were used to predict outcome probabilities. The likelihood ratio test for effect modification due to systemic inequities found weak evidence against the null hypothesis of no effect modification (p-value = 0.03). Therefore, the Gini coefficient was not included in the final model.

From the output of the multivariable regression model (Table 2), all coefficients for the health system indicator input variables are close to the null value of one. The regression model returned a pseudo-R² estimate of 4.7%, meaning that only 4.7% of the variance in the large outbreaks variable could be predicted by the health system indicator inputs. We interpret this with caution as it is as pseudo-estimate.

After making predictions for the testing dataset, the model obtained a Brier score of 0.241, indicating that the model does not effectively predict large measles outbreaks, and therefore lacks external validity. Figure 2 shows the predicted odds of large measles outbreaks for each observation in the testing dataset, grouped by the observed outcome (large outbreak or non-large outbreak). Outbreaks which were observed to be large had a higher median predicted odds of being large, compared to outbreaks which were observed to be non-large. Despite this result, observed non-large outbreaks still had a median predicted odds higher than the null value of one. Consequently, the health system indicators in the model could not effectively predict large measles outbreaks.

3.4. Sensitivity Analysis

When fitting regression models for different definitions of a large measles outbreak, it was observed that all models produce similar model coefficients and validation results to the baseline definition. These results confirm that the selected definition of a large measles outbreak did not impact the results of the model. Results of the sensitivity analysis can be found in Appendix A.

3.5. Missing Data

When fitting the regression model on imputed data, we obtained similar model coefficients and validation results to the output of the non-imputed data. In addition, the results of model validation were also consistent with those of the baseline model, indicating that the model does not effectively predict the outcome. We expect that missing data did not affect the results.

4. Discussion

4.1. Key Findings

We found no association between the selected health system indicators and the odds of a large measles outbreak. The selected indicators could not effectively predict large measles outbreaks. Our results show that measles outbreaks may not be an effective indicator of under-resourced health systems in LMICs.

Considering these results, it is important to review our characterisation of health systems, for which we only used three indicators as inputs to the model. When selecting indicators, we did not identify any indicators which sufficiently characterised the health information systems or leadership & governance building blocks. After reviewing potential data sources, we found a lack of complete data for Average availability of 14 selected essential medicines in public health facilities, which was our selected indicator for the access to essential medicines building block. Therefore, this study only considered indicators for health service delivery, health workforce, and health financing building blocks [26]. The importance of these particular health system components in the control of other infectious diseases has been emphasised in the literature. A systematic review by Shoman et al. [53] found that the health workforce (including nurses and physicians) is paramount to Ebola outbreak control, as is health service delivery. Importantly, this review noted that successful health service delivery depends on the success of the other building blocks, and therefore by including an indicator for service delivery, our model has encapsulated the health system [53]. Such a relationship between the building blocks also suggests that including additional indicators in our model may have led to collinearity, so selection of only three indicators is justified. However, this selection may have obscured some aspects of the health system from the predictive model, attenuating the association between under-resourced health systems and measles outbreaks.

Another possible explanation for why our findings did not demonstrate an association is that our model did not incorporate the effect of fragile and conflict-affected settings on the health system. The presence of conflict may disrupt health services and access to healthcare, while increasing the likelihood of measles cases due to overcrowding and poor sanitation, accelerating the spread of the virus [54]. Conflict and insecurity is often localised to a sub-national level, which poses challenges in incorporating meaningful measures of conflict into our analysis, which was conducted at the national level [54]. Failing to adjust for conflict may introduce potential confounding, although given we are not making causal inferences on the association, this effect is of less concern.

This project used an ecological study design with health system indicators measured at the national level, which may mask associations occurring at the sub-national level. Under-resourced health systems may lead to varying measles vaccination coverage between communities and regions, resulting in increased measles susceptibility in some areas [55]. Certain marginalised groups often face physical, cultural and social barriers to accessing healthcare, in addition to unequal distribution of health services and resources by the health system [55]. Missed communities have become a key focus in the current approach to immunisation delivery, as they are a priority population in the Gavi 2021-2025 strategy [15]. This issue is especially pertinent to the high transmissibility of measles, as outbreaks occur in pockets of susceptible individuals [11]. Further research is required to determine whether under-resourced health systems are associated with measles outbreaks on a sub-national or individual level.

We focussed on comparing large measles outbreaks to non-large measles outbreaks. Therefore, our study has only somewhat ascertained whether measles outbreaks can be used to identify under-resourced health systems in LMICs. The components of weak health systems may still be significant in determining the presence or absence of any measles outbreak, which was not explored in our analysis. This association is largely dependent on outbreak prevention, particularly the presence of herd immunity, achieved through 95% vaccination coverage with two doses of a measles-containing vaccine [10]. Vaccine effectiveness is critical to achieving herd immunity, requiring strong health systems to ensure that an immune response is induced [13]. This is important as variation in measles immunisation coverage can lead to measles outbreaks [11]. Conversely, the size of existing measles outbreaks depends on both prevention and control measures, such as reactive vaccination campaigns [12,56,57]. Once a measles outbreak is established, an effective and timely response is critical to circumventing a large outbreak [56]. Consequently, the capacity of health systems to implement preventive measures and avert measles outbreaks may have been attenuated in this analysis. Therefore, the concept of measles as a proxy for under-resourced health systems, as put forward in the Immunization Agenda 2030 [12] and other public health papers [19,20], may still be relevant to the application of health systems strengthening.

4.2. Strengths and Limitations

In this study, there were challenges in ensuring that all measles cases stated in the epidemiological reports belonged to the same measles outbreak, resulting in potential misclassification of non-large outbreaks as large outbreaks. However, by capturing all suspected, epidemiologically-linked and laboratory-confirmed cases in the measles case data, we reduced information bias resulting from the possibility of weaker surveillance systems and diagnostic capacities in countries with under-resourced health systems. As we restricted the data to outbreaks which started before the COVID-19 pandemic, our findings are not generalisable to post-COVID-19 settings, given the effect that the COVID-19 pandemic has had on health systems and vaccination programs [17]. Despite this, our study produced evidence on a topic of global interest which, to our knowledge, has not yet been investigated empirically.

4.3. Implications

Measles outbreaks have been put forward as a “canary in the coal mine” of broader health system weaknesses, suggesting that they may be used to identify health systems which are most vulnerable to infectious disease outbreaks and other health issues [12,19,20]. However, our analysis has found that large measles outbreaks may not be an effective indicator of under-resourced health systems in LMICs. Given these findings, we cannot make any recommendations on the aspects of the health system which warrant specific attention due to their indication of vulnerability to measles outbreaks. We recommend further research to investigate the association between under-resourced health systems and the presence or absence of a measles outbreak. We also recommend that future research explores the possibility of this association at a sub-national level (including outbreaks which occurred on a local scale) and include a range of health system indicators which represent all six WHO building blocks of health systems [26].

Measles outbreaks remain a public health issue worldwide, with an increasing number of large and disruptive measles outbreaks in the last year, even 60 years after the introduction of a highly effective measles vaccine [2,3,6]. The ability to identify health systems which are particularly vulnerable to public health disasters such as measles outbreaks would greatly improve effective allocation of health resources, enhancing the accessibility and availability of care for many groups.

5. Conclusions

In this analysis, measles outbreaks were not effective at identifying under-resourced health systems in LMICs, when considering large outbreaks compared to non-large outbreaks. Further research is required to determine whether this finding holds when comparing the presence and absence of measles outbreaks. By identifying health systems with a high probability of measles outbreaks, countries and regions can implement appropriate interventions to mitigate future outbreaks.

Appendix A.1

Table A1. Sensitivity analysis results for large measles outbreak definition.

Large outbreak definition	Input variable	Adjusted odds ratio	95% confidence intervals	p-value	Brier score
$\ge$50 cases	Proportion of births delivered in a health facility (%)	0.99	(0.95, 1.04)	0.77	0.330
	Number of nurses and midwives (per 10,000 population)	1.03	(0.98, 1.08)	0.29
	Health expenditure (per capita in US$)	0.99	(0.98, 1.00)	0.18
$\ge$150 cases	Proportion of births delivered in a health facility (%)	1.00	(0.96, 1.03)	0.86	0.217
	Number of nurses and midwives (per 10,000 population)	1.02	(0.98, 1.06)	0.31
	Health expenditure (per capita in US$)	1.00	(0.99, 1.00)	0.26
$\ge$200 cases	Proportion of births delivered in a health facility (%)	1.00	(0.97, 1.04)	0.88	0.205
	Number of nurses and midwives (per 10,000 population)	1.02	(0.98, 1.06)	0.26
	Health expenditure (per capita in US$)	1.00	(0.99, 1.00)	0.31

Table A1. Sensitivity analysis results for large measles outbreak definition.

Large outbreak definition	Input variable	Adjusted odds ratio	95% confidence intervals	p-value	Brier score
$\ge$50 cases	Proportion of births delivered in a health facility (%)	0.99	(0.95, 1.04)	0.77	0.330
	Number of nurses and midwives (per 10,000 population)	1.03	(0.98, 1.08)	0.29
	Health expenditure (per capita in US$)	0.99	(0.98, 1.00)	0.18
$\ge$150 cases	Proportion of births delivered in a health facility (%)	1.00	(0.96, 1.03)	0.86	0.217
	Number of nurses and midwives (per 10,000 population)	1.02	(0.98, 1.06)	0.31
	Health expenditure (per capita in US$)	1.00	(0.99, 1.00)	0.26
$\ge$200 cases	Proportion of births delivered in a health facility (%)	1.00	(0.97, 1.04)	0.88	0.205
	Number of nurses and midwives (per 10,000 population)	1.02	(0.98, 1.06)	0.26
	Health expenditure (per capita in US$)	1.00	(0.99, 1.00)	0.31