Evaluation of Diagnostic Performance of Three Commercial Interferon-Gamma Release Assays for Mycobacterium tuberculosis

1. Introduction

Despite significant mortality of over one million deaths annually [1], Tuberculosis (TB) caused by Mycobacterium tuberculosis (MTB) can remain hidden within individuals with latent TB infection (LTBI). Identifying these individuals and prioritizing preventive treatment for those at high risk of developing active TB is essential for TB elimination [2]. Based on WHO policy, two immunological tests could be used to diagnose LTBI, namely, the tuberculin skin test (TST) and interferon-gamma release assays (IGRAs) [3]. However, both tests indirectly assess a past or ongoing immune response to MTB antigens and don’t directly confirm the presence of live bacteria [4].

The tuberculin skin test (TST) is an in vivo diagnostic tool that relies on intracutaneous injection of a mixture containing standardized purified protein derivative (PPD-S) from MTB as well as PPD from atypical Mycobacterium. A delayed-type hypersensitivity reaction at the injection site indicates prior exposure to these antigens [5]. The TST’s complex antigen mixture lacks specificity for MTB leading to false-positive reactions from prior BCG vaccination, exposure to NTMs, or past M. tuberculosis infection [6].

IGRAs, in contrast to the TST, utilize MTB-specific antigens (ESAT-6, CFP-10, and TB7.7) obtained from region of difference 1 (RD1) of the MTB genome that are absent in BCG and most NTMs, with some exceptions, and this enhances their specificity for M. tuberculosis infection [7,8].

A review of the extant literature suggests a growing body of knowledge on the performance of IGRA and its utility in the management of LTBI and we present a few pertinent findings. Some studies have suggested that IGRAs offer comparable sensitivity to TST for diagnosing latent tuberculosis infection (LTBI) while demonstrating greater specificity [9]. Additionally, IGRA results appear to correlate more directly with the degree of M. tuberculosis exposure [10]. IGRA was also reported to have a high positive detection rate (86.7%) compared with other methods, including TST (66.7%), among children with TB, and a significantly lower positive rate (6.7%) than TST (76.7%) among vaccinated children. Additionally, the specificity of IGRA was found to be better (86.7%) than TST (40%) [11]. In another study among various risk groups, researchers reported a positive agreement between IGRA and TST of 42.0%, negative agreement of 16.1% and discordance of 41.9% [12]. Exploring the potential risk factors for discordance between IGRA and TST through logistic regression, researchers found that 85.1% of cases where TST was positive and IGRA was negative was explained by BCG vaccination or migration; where TST was negative and IGRA was positive, age explained 49.1% of the discordance [13]. Moreover, other researchers concluded through a systematic review of indeterminate IGRA cases that 1 in 26 IGRA tests are indeterminate [14] which suggests that the test may still be reliable, nonetheless.

In another systematic review and meta-analysis paper, the researchers reported that some studies in children found no significant difference between IGRA and TST in identifying LTBI, whereas other studies reported that IGRA outperformed TST. The review further suggested that some studies showed that IGRA was not significantly different from TST when identifying LTBI among immunocompromised patients [15]. While inconclusive, these studies suggest that IGRA may be a useful method in identifying LTBI although further studies are needed to elucidate its performance among various groups.

For diagnosing LTBI, 2011 saw a significant development when the World Health Organization (WHO) endorsed the use of specific IGRAs, including Qiagen QuantiFERON-Gold (QFT-G), QuantiFERON-TB Gold In-Tube (QFT-GIT), and Oxford Immunotec T-SPOT.TB (T-Spot) [3]. Sine then, there have been upgrades to existing kits such as QuantiFERON-TB Gold Plus (QFT-Plus) from Qiagen and new IGRAs have been added. Some of the additions include QIAreach (Qiagen), Wantai TB-IGRA (TB-IGRA, Beijing Wantai Biological Pharmacy Enterprise), STANDARD™ F TB-Feron FIA (TBF-FIA, SD Biosensor) and the STANDARD E TB-Feron ELISA (TBF-ELISA, SD Biosensor).

In this paper, we report the results of evaluation of the diagnostic performance of TBF-FIA, TBF-ELISA, and QFT-Plus among populations that are exposed or unexposed to MTB infection. This will contribute to the growing literature by providing evidence of the performance of these assays in an endemic country like Ghana.

2. Materials and Methods

2.1. Study Design

In this prospective, cross-sectional study, we evaluated the diagnostic performance of three IGRA kits: TBF-FIA, TBF-ELISA, and QFT-Plus. We also assessed how well the first two assays agreed with the established QFT-Plus test. We recruited a cohort of individuals suspected of having latent or active tuberculosis (TB), as well as individuals without known TB exposure, at the study’s onset. Each participant was tested using all three IGRA kits, enabling a direct and simultaneous comparison of results across the assays. This approach ensures that the same patient population was used to assess each kit, providing a clear comparison of their diagnostic accuracy. By comparing the assays under the same conditions, this study design effectively minimizes potential confounding factors and biases, allowing for a reliable evaluation of the relative performance of these diagnostic tools.

2.2. Study Site/Population

The National Public Health and Reference Laboratory (NPHRL), Ghana, conducted the testing due to its expertise and accreditation to ISO 15189 in various diagnostic fields. Samples were collected from eight TB clinics within proximity to NPHRL in the Greater Accra, Eastern, and Central regions to ensure same-day delivery and optimal sample quality. According to the National TB Control Programme, the TB incidence rate in 2022 was estimated at 133 per 100,000 population representing a decline of about 17% from 2015.

2.3. Participant Recruitment

The study enrolled healthy individuals with no known TB exposure or very low likelihood of TB infection (unexposed group), and patients with clinically active TB (exposed group) as defined in the framework for the evaluation of new tests for tuberculosis infection [16]. Only patients meeting the case definition and granting informed consent were enrolled. All personnel involved in the enrolment of patients were trained on the study protocol prior to initiation of the study. Patients with conditions compromising safety or data integrity were excluded. Required data included demographics, specimen type, clinical diagnosis, test results, and collection date. Sample collection and testing commenced on 15 June 2023 and concluded on 31 March 2024.

Participants were recruited at the designated TB clinics by appointed coordinators, following the approved protocol. Demographic and clinical details were documented as per the protocol for each participant. Blood collection followed standard venipuncture procedures by trained personnel, with approximately 8ml drawn into heparinized tubes. Specimens were stored at room temperature and transported within 16 hours to the NPHRL by courier in puncture-resistant boxes, arriving on the same day of collection. Upon arrival, NPHRL staff checked the containers and paperwork for completeness and documented any discrepancies.

2.4. Measures to Minimize Bias

To minimize bias in the study, several measures were implemented. Samples were randomly collected from patients or subjects meeting the inclusion criteria to prevent selection bias, ensuring a representative and unbiased participant pool. Both positive and negative samples were tested in a random order to avoid systematic biases. The study also used a dual-group approach: one group performed the tests, while another managed the specimens, keeping the testing personnel blinded to the specimens’ status. This blinding was crucial for preventing any preconceived notions from affecting the results. These rigorous methods ensured the reliability and validity of the study’s findings.

2.5. Laboratory Testing

TBF-ELISA, TBF-FIA and QFT-Plus tests were performed in parallel following the manufacturers’ instructions under controlled conditions using calibrated equipment certified by the Ghana Standards Authority.

The STANDARD™ E TB-FERON ELISA is a blood test that detects TB infection by measuring the immune response. Whole blood is drawn and divided into three tubes: a Nil tube (baseline), a TB Antigen tube containing proteins specific to M. tuberculosis (ESAT-6, CFP-10, and TB7.7), and a Mitogen tube (positive control). If a person has been infected with TB, their T-cells will release interferon-gamma (IFN-gamma) upon encountering the TB antigens, which is then measured by the ELISA assay. The Nil tube and Mitogen tube help ensure accurate test results.

The STANDARD™ F TB-FERON FIA utilizes the same blood collection tubes, antigens (ESAT-6, CFP-10, and TB7.7), and specimen preparation as the STANDARD™ E TB-FERON ELISA. However, it employs immunofluorescence technology with the STANDARD F Analyzer to provide a qualitative measurement of interferon-gamma (IFN-γ) in the sample.

The fourth-generation QuantiFERON-TB Gold Plus (QFT-Plus) test measures cell-mediated immunity to TB by quantifying interferon-gamma (IFN-γ) released from whole blood upon exposure to specific antigens. It utilizes two main tubes: TB1 and TB2. Both contain peptide antigens simulating proteins ESAT-6 and CFP-10, targeting CD4+ T-helper lymphocytes. However, TB2 includes additional peptides designed to stimulate CD8+ cytotoxic T lymphocytes.

2.6. Data Management and Analysis

Study data were recorded in an Excel database. All data processing tasks, and statistical analysis were performed using Jupyter Notebook 6.5.2 with Python 3 (ipykernel) and RStudio version 1.1.414 (with R version 3.4.3 (2017-11-30)). Standard diagnostic performance metrics (sensitivity, specificity, positive/negative predictive values, accuracy) were estimated with associated 95% confidence interval using exact binomial methods. Cohen’s Kappa was estimated to determine agreement and evaluated against standard criteria [17].

Additional performance metrics such as Youden’s index [18], diagnostic odds ratio, positive and negative likelihood ratios were estimated with their usual meaning and interpretation implied [19].

3. Results

3.1. Participant Demographics

A total of 668 participants were recruited for the study; about 55.4% (370) were healthy individuals with no known TB exposure or very low likelihood of TB infection (unexposed group) and 44.6% (298) were patients with clinically active TB (exposed group). When analyzing sex of participants, 61.5% of participants were female, 38.3% were male, and for one participant, sex was missing. Additionally, majority of participants (42.8%) were recruited from Weija and 2.7% were recruited from Nsawam (Table 1).

Analysis of the age distribution of participants shown in Table 2 was based on 664 participants whose age was determined, 4 participants were omitted due to lack of information on age. The analysis showed that the mean age of female participants was 33.7 years (range: 0.8 – 75.0 years) and that of male participants was 40.5 years (range: 2.0 – 80.0 years). On average, the exposed group were older (39.9 years) than the unexposed group (33.4 years). The youngest participant was 0.8-year-old (9 months) female, in the unexposed group and recruited from the site in Weija. On the other hand, the oldest was 80-year-old male, in the exposed group and recruited from the site in Ashaiman. The analysis further demonstrated that age distribution of participants may generally reflect that of the target population.

3.2. Results of Assay Performance

The observed distribution of results from each kit based on the case groups of participants are presented in the stacked contingency table, Table 3, below.

From the exposed group, TBF-ELISA identified 92.3% as positive and 7.7% as negative. TBF-FIA results identified 83.2% as positive and 16.8% as negative and for QFT-Plus, 88.9% of the exposed group were identified as positive and 11.1% were negative.

When analyzing the unexposed group, TBF-ELISA identified 16.5% as positive and 83.5% as negative. TBF-FIA, on the other hand, identified 13.2% of the unexposed group as positive and 86.8% as negative; QFT-Plus results identified 11.9% of unexposed participants as positive and 88.1% as negative.

Estimated performance characteristics for each assay with associated 95% confidence interval are presented in Table 4 below.

For the study population, TBF-ELISA had the highest sensitivity estimate of 92.3% [95%CI: 88.6% - 95.0%] compared with the other two assays. Therefore, the probability of obtaining a positive test result among participants with TB infection is higher with TBF-ELISA followed by QFT-Plus (88.9% [95%CI: 84.8% - 92.3%]) and TBF-FIA (83.2% [95%CI: 78.5% - 87.3%]). Despite the differences, the overlap in the estimated confidence intervals for the three assays may suggest that the differences are not statistically significant. Alternatively, the probability of having TB infection in a participant with positive result, given as the positive predictive value, was higher for QFT-Plus 85.8% [95%CI: 81.4% - 89.5%] and lower in TBF-ELISA, 81.8% [95%CI: 77.3% - 85.8%].

The probability of a negative test result in a participant without the disease was 88.1% [95%CI: 84.4% - 91.2%] when tested with QFT-Plus, 86.8% [95%CI: 82.9% - 90.0%] when tested with TBF-FIA and 83.5% [95%CI: 79.3% - 87.1%] when tested with TBF-ELISA. Again, the overlap in confidence interval may suggest that the differences in probability may not be statistically significant. However, probability of not having a TB infection in a participant with a negative test result, given as the negative predictive value, is highest when tested with TBF-ELISA (93.1% [95%CI: 89.8% - 95.6%]) and lowest when tested with TBF-FIA (86.5% [95%CI: 82.6% - 89.8%]).

Consequently, the proportion of participants correctly classified by the assays (diagnostic accuracy) was estimated to be 88.5% [95%CI: 85.8% - 90.8%] for QFT-Plus, which is highest among the three assays and the lowest was 85.2% [95%CI: 82.3% - 87.8%] for TBF-FIA. The odds of a positive results in the exposed group relative to a positive result in the unexposed group was estimated as 60.6 [95%CI: 36.5 – 100.5] for TBF-ELISA, 59.5 [95%CI: 36.8 – 96.1] for QFT-Plus and 32.5 [95%CI: 21.2 – 49.8] for TFB-FIA. Despite the relatively higher diagnostic odds for TBF-ELISA and QFT-Plus, their confidence intervals are quite wide which may suggest less precision compared to TBF-FIA which has a relatively lower odds but narrow confidence interval.

Moreover, the probability of making an informed decision (rather than a random guess) when the assays are used in TB diagnosis, as given by the Youden index, was estimated as 0.77 [95%CI: 0.69 – 0.83] for QFT-Plus, 0.76 [95%CI: 0.68 – 0.82] for TBF-ELISA, and 0.70 [95%CI: 0.61 – 0.77] for TBF-FIA, suggesting that the utility of all three assays in decision making regarding TB infection is comparable. A related measure, the number needed to diagnose, represents the number of TB patients who need to be tested with the assays to correctly detect one person with TB infection in exposed and unexposed groups was estimated as <2 for all three assays.

Additionally, the probability of getting a positive test result in the exposed group is 5.6 times higher than in the unexposed group when using the TBF-ELISA. When using the TBF-FIA, that probability is estimated at 6.3 and for QFT-Plus, the probability is 7.5. Further, the probability of obtaining a negative result in the exposed group is 0.09 times compared with the unexposed group for TBF-ELISA, 0.19 times for TBF-FIA, and 0.13 times for QFT-Plus. These results may indicate that QFT-Plus may be relatively better when ruling-in diagnosis although its contribution to diagnosis may not be significant given that the estimated LR+ <10. In contrast, TBF-ELISA may be relatively better with ruling-out a diagnosis and given that the estimated LR- <0.1, the contribution to lowering the posterior probability that the patient has the disease may be significant.

3.3. Agreement Analysis

Comparison of the results obtained when testing all three assays is summarized in Table 5. The analysis shows that for 45.7% (305) of the tests run, all three assays were negative for the same samples. Conversely, all three assays were positive for the same samples in 39.2% (262) of the samples tested. Therefore, the proportion of samples for which all three assays reported the same results (negative/positive) was 84.9% (567).

Both TBF-ELISA and QFT-Plus only were positive in 4.8% (32) of the samples tested and only TBF-ELISA was positive in 3.6% (24) of the samples tested.

Consequently, using QFT-Plus as the reference assay, the agreement with TBF-ELISA and TBF-FIA was estimated respectively. The contingency table of observed values is presented in Table 6. TBF-ELISA and QFT-Plus were both positive in 95.1% of samples tested and negative in 88.3% of samples. On the other hand, TBF-FIA and QFT-Plus were positive in 86.4% of samples tested and negative in 91.6% of samples tested.

The overall agreement between TBF-ELISA and QFT-Plus was estimated at 91.5% with a positive agreement of 95.2% and negative agreement of 88.3%. The strength of the agreement was estimated with Cohen’s kappa as 0.829 which can be evaluated as very high equivalence (Table 7). Overall agreement between TBF-FIA and QFT-Plus was estimated as 89.2% with positive agreement of 86.4% and negative agreement of 91.6%. The two assays have a high equivalence based on Cohen’s kappa of 0.783.

4. Discussion

Although WHO has made recommendations regarding the use of IGRAs in the diagnosis of TB infection [3], some studies have attempted to elucidate its utility in various clinical settings. For example, some researchers have suggested that due to various factors including limited data, the potential benefits of its use may be limited [20]. Other researchers have also suggested low accuracy as a limitation to the use of IGRA assays [21].

In this study, we have reported that the sensitivity of the three assays lie within a combined 95% confidence interval of 78.5% to 95.0% and specificity lie within 79.3% to 91.2%. These findings are comparable to what has been reported in the literature. Confidence intervals of 62.1% to 85.9% have been reported for sensitivities and between 76.1% to 86.1% for specificities of IGRAs [22].

We also report that the sensitivity of TBF-ELISA in this study is about 3.4% higher than that of QFT-Plus and the specificity is about 4.6% lower than QFT-Plus which is comparable to findings in other studies which report that the sensitivity of TBF-ELISA is 3.7% higher than QFT and the specificity is 5.4% lower than QFT-Plus [23]. Additionally, we report a kappa statistic for agreement between the two assays as 0.83 which is comparable to 0.85 reported elsewhere [23] or 0.9176 reported in another study [24].

Being a point of care (POC) device, we report the 95% confidence interval for TBF-FIA sensitivity between 78.5% and 87.3%; the 95% confidence interval for specificity was estimated to be 82.9% to 90.0%. Although there is limited data in the literature for TBF-FIA, the findings can be viewed in relation to QIAreach, a similar POC device from Qiagen whose sensitivity and specificity have been reported to be between 87.9% to 99.6% and 88.4% to 97.6%, respectively [24]. Another study reported the sensitivity and specificity of QIAreach as 98.5% and 72.3%, respectively [25]. The absence of overlap in the confidence intervals for the sensitivity may suggest that QIAreach may have a significantly higher sensitivity than TBF-FIA however, the specificity may not be significantly different. Moreover, the agreement between TBF-FIA and QFT-Plus is high with a kappa statistic of 0.783.

Our findings suggest that the three IGRA assays evaluated in this study (TBF-ELISA, TBF-FIA and QFT-Plus) offer comparable diagnostic accuracy for TB infection. Additionally, the variability in the reported sensitivity and specificity values across different studies highlights the influence of factors such as the prevalence of TB, the population studied, and the cut-off values used for interpretation of the assays. These factors need to be considered when interpreting the results of IGRA assays in different clinical settings.

The higher sensitivity of TBF-ELISA compared to QFT-Plus observed in our study may suggest that TBF-ELISA might be a more sensitive tool for detecting TB infection. However, its lower specificity may indicate a higher likelihood of false-positive results, which could lead to unnecessary treatment and increased healthcare costs. Despite these differences, both assays generally provide consistent results.

On the other hand, the differences in sensitivity between TBF-FIA and QIAreach could be due to differences in the underlying technology or variations in the study populations. Therefore, further comparative studies are required to elucidate these differences.

Despite the promising results observed in our study, several limitations must be acknowledged. Firstly, diversity in the study population was not assessed, which may affect the generalizability of our findings. Future studies with more diverse populations are needed to confirm these results. Secondly, the analysis presented does not include indeterminate results. Such data may be invaluable in understanding the reliability and overall cost-effectiveness of the assays, as well as generating insights that can inform the development of next-generation IGRA assays with improved performance and fewer indeterminate outcomes.

5. Conclusions

This study adds to the growing body of evidence on the diagnostic accuracy of IGRA assays. While our study provides valuable insights into the performance of different IGRA assays, further research is necessary to fully understand their utility and limitations in various clinical contexts. Consequently, continued evaluation and comparison of these assays will help to optimize their use in TB diagnosis and improve patient outcomes.

All three assays evaluated demonstrated good performance, however, some variations in specificity and limited data for POC assays warrant further exploration. Future research should focus on confirming the accuracy of POC assays and evaluating their cost-effectiveness to optimize TB diagnosis strategies.