A Benchmark of In-House Homologous Recombination Repair Deficiency Testing Solutions for High-Grade Serous Ovarian Cancer Diagnosis

1. Introduction

High-grade serous ovarian cancer is the most common type of ovarian cancer, being the leading cause of death in women diagnosed with gynecological malignancies; it appears to originate from the epithelial tissue, arising primarily from the ovaries, fallopian tubes, or peritoneum [1,2,3]. Following standard treatment approaches of cytoreductive surgery and platinum-based therapy, the 5-year survival rate is approximately 30% [4,5]. Next-generation sequencing (NGS) has enabled systematic investigations of genomic and molecular alterations that drive ovarian cancer aiming to identify patients who may respond to targeted therapies [6,7]. Notably, genomic instability is a hallmark in the formation of tumors, and inefficient DNA repair is a critical driving force behind cancer establishment, progression, and evolution [8].

Homologous recombination repair (HRR) is a DNA repair pathway that acts on DNA double-strand breaks and interstrand cross-links [9,10,11,12]. Many tumor types including ovarian cancer exhibit a deficiency in the HRR pathway; such phenotype has been termed homologous recombination deficiency (HRD) [13], and it has been extensively shown that the presence of HRD can make tumors present a distinct clinical constitution involving a superior response to platinum-based therapies and poly (adenosine diphosphate [ADP]–ribose) polymerase (PARP) inhibitors (PARPi) [14,15,16]. The introduction of PARPis has transformed the management of high-grade serous ovarian cancer in both relapsed and first-line treatment settings. Developing methods to reliably determine the HRD status is of critical importance to optimize clinical benefit from these drugs. Thus, diagnostic companies have developed assays to determine HRD status and aid in treatment decisions; however, these assays may differ in what they evaluate and may lead to inconsistent results that can be complicated for prescribing physicians [17]. In general, the assays have been developed to measure the causes and/or the consequences of the HRD phenotype.

The best characterized causes of HRD in ovarian cancer are germline or somatic mutations in BRCA1 and BRCA2 [18]. Nonetheless, there is now clear evidence that HRD can arise through germline and somatic mutations or even methylation of a wider set of homologous recombination repair (HRR) related genes [19,20]. Further characterization of HRR pathway has revealed multiple protein co-factors that are necessary for functional HRR, and mutations in the genes that encode these proteins could also contribute to ovarian cancer [19,20]. Regarding testing the consequences of HRD phenotype, the assays are performed to estimate the genomic instability (genomic “scars”) or mutational signatures that measure the patterns of somatic mutations accumulating in HRD cancers irrespective of the underlying defect. Numerous research in ovarian cancer has identified mutational signatures of instability associated with an HRD phenotype including loss of heterozygosity (LOH), telomeric imbalances (TAI), and large-scale transitions (LST) [21,22,23].

Although there are some diagnostic tests commercially available to promote tumor selection for a personalized therapy based on HRD status [17], the Myriad MyChoice CDx is the most notorious FDA-approved tumor assay to assess chromosomal instability because it incorporates both the causes and consequences of HRD status [24,25], while other assays only detect potential causes of HRD status without assessing the consequences. However, this test is still vastly limited for being locally performed, highly-priced, and frequently not refunded by medical insurance. As an alternative, other commercial assays deployable in diagnostic laboratories that screen for both HRD status and genomic instability were recently launched (SOPHiA DDM^TM HRD Solution and AmoyDx HRD Focus Panel) even though the main challenge is a lack of standardized methods to define, measure, and report HRD status. Hence, conducting studies that assess the correlation of HRD status across different assays is extremely important since it can identify sources of discordance, providing an opportunity to determine the underlying causes of inconstancy and provide information for better use of these assays for cancer treatment decisions. In this study, we aimed to examine the technical viability of in-house HRD testing, particularly for ovarian cancer diagnosis, comparing the results of two commercially available NGS-based tumor tests to the reference assay Myriad MyChoice CDx.

2. Materials and Methods

2.1. Tumor Samples

The tumor DNA samples selected for this study were obtained from 85 patients with ovarian cancer referred to genetic testing, upon a medical request, in our laboratory. High-grade serous ovarian cancer samples originating from distinct primary tissues such as ovaries, fallopian tubes, or peritoneum were retrieved from the pathology department. Archival histological sections of 10µm from formalin-fixed paraffin-embedded (FFPE) tumor samples were obtained, and a hematoxylin and eosin-stained slide was reviewed by pathologists to select and mark a representative tumor area for macrodissection and DNA extraction. The clinical and pathological features of all samples used in this technical validation study are presented in Table 1 and Supplementary Table S1.

2.2. HRD Testing

In-house HRD testing assessment was carried out comparing two commercial kits: SOPHiA DDM^TM Homologous Recombination Deficiency (HRD) Solution and AmoyDx HRD Focus Panel to the reference assay Myriad MyChoice CDx. Despite the different methodologies, both commercial kits allow the simultaneous detection of BRCA status and predict a genomic instability score. While MyChoice CDx testing was performed externally, with FFPE slides being sent to a reference laboratory to perform all analytical steps, the other two methods had DNA extraction, library preparation, sequencing, data analysis, and clinical interpretation performed internally. Regions with the highest tumor cell density, previously marked by a pathologist, were scrapped from five to eight slides for each sample. DNA was extracted using Maxwell RSC DNA FFPE kit (Promega) and quantified using two methods: Qubit™ dsDNA High Sensitivity (Thermo Fisher Scientific) and/or Real-Time quantitative PCR, using the TaqMan™ Fast Advanced Master Mix (Thermo Fisher Scientific) and the TaqMan® Copy Number Assays (Thermo Fisher Scientific).

2.2.1. SOPHiA DDM^TM HRD Solution

DNA library preparation was performed using the SOPHiA DNA Library Prep kit II (SOPHiA GENETICS, USA), following the manufacturer’s recommendations. DNA inputs ranged from 3.9 to 150 ng according to the functional concentration measured by Real-Time PCR. The fragmentation time was adjusted based on the degradation level of the tumor sample. Sequencing libraries were quantified using Qubit™ dsDNA High Sensitivity Assay kit (Thermo Fisher Scientific), and capillary electrophoresis was performed to analyze the fragments size distribution using the Genomic DNA ScreenTape (Agilent Technologies). HRD testing from SOPHiA GENETICS involves a Homologous Recombination Repair (HRR) multi-gene panel and a low-pass whole genome sequencing (LP-WGS) that were carried out independently in two steps. First, the HRR panel evaluates single nucleotide variants (SNVs), small insertions and deletions (InDels), and copy number variations (CNVs) in the coding regions and splicing sites of the following genes: ATM, BARD1, BRCA1, BRCA2, BRIP1, CDK12, CHEK1, CHEK2, FANCL, PALB2, PPP2R2A, RAD51B, RAD51C, RAD51D, RAD54L, and TP53. The HRR panel was sequenced in the MiSeq System from Illumina Inc, using the MiSeq Reagent kit V3 (600 cycles in a Paired-End 2x151 cycles run). The Limit of Detection (LOD) for SNVs and InDels was 5%, and the minimal depth of coverage was 500x (96.05% sensitivity and 99.99% specificity). Importantly, for the evaluation of HRD status in ovarian cancers, it has been demonstrated that only pathogenic or likely pathogenic variants in BRCA1/2 are relevant to current literature data; therefore, despite the other genes in the HRR panel only variants on these two genes were considered in this study. Second, the LP-WGS experiments were performed either on the NextSeq 550 or NovaSeq 6000 sequencing platforms from Illumina Inc, using the NextSeq Mid Output Reagent kit (300 cycles in a Paired-End 2x151 cycles run) or the NovaSeq S1 Reagent kit v1.5 (200 cycles in a Pair-End 2x101 cycles run), respectively. The targeted mean vertical coverage during sequencing is 1-3x. The primary sequencing output was demultiplexed by bcl2fastq v2.20.0.422, and the reads were processed to trim adapters and low-quality base calls. Data analysis was carried out using the SOPHiA DDM^TM platform, in which the reads were mapped to the human genome (hg19) and sequencing depth was computed and normalized to calculate a genome-wide coverage profile. A convolutional neural network algorithm previously trained based on data from The Cancer Genome Atlas Program (TCGA) [26] takes as input a bitmap-like matrix that reflects the whole genome sequencing coverage profile and outputs a scalar value of Genomic Integrity Index (GII). Tumor samples with scores above 1.8 threshold indicate a GII positive status, reflecting an inability to repair double-strand breaks, whereas scores below that threshold indicate a GII negative status for which PARPi therapy would not be recommended. The GII score had a non-strict range of about -30 up to +30.

2.2.2. AmoyDx HRD Focus Panel

DNA library preparation was performed using the AmoyDx HRD Focus Panel kit (AmoyDx, China), following the manufacture’s recommendations. The assay was intended to determine the HRD status via the detection pathogenic variants in the BRCA gene and the determination of a genomic instability score in the tumor samples in a single workflow. The assay kit is based on the Halo-shape ANnealing and Defer-Ligation Enrichment (HANDLE) system technology, in which the probes contain an extension and ligation arms complementary to the target gene regions. DNA inputs ranging from 50 to 100 ng were pre-denatured and the probes hybridized to the target regions of BRCA1/2. Then, an extension and ligation step of the probes were performed, treated with exonuclease for the digestion of non-hybridized probes and free single and double-stranded DNAs in the solution. The linked probes were amplified by PCR, and the final product was purified. Libraries were quantified with Qubit™ before and after the purification step to verify material losses. A total of 100 ng of each library were polled together and the final concentration was measured on Qubit™. DNA libraries were sequenced on the NovaSeq 6000 platform from Illumina Inc., using the NovaSeq S1 Reagent kit v1.5 (300 cycles in a Paired-End 2x150 cycles run). In brief, the detection of pathogenic variants was carried out within the coding sequence and at the boundaries of exon-intron of BRCA gene. The estimated Genomic Scar Score (GSS) was calculated by evaluating 24,000 SNPs where GGS equal to or higher than 50 is suggestive of a positive HRD status. The GSS model from AmoyDx is built on machine learning, and it measures genomic instability by weighing different types of chromosomal copy number, which was grouped according to the combination of length, type, and site of copy number. Data analysis was carried out using the ANDAS System (AmoyDx, China) with the pipeline version 1.1.1.

2.2.3. Variant Classification

Detected variants in BRCA1 and BRCA2 were classified for their clinical impact according to the American College of Medical Genetics (ACMG) [27], Somatic Oncogenicity SOP – ClinGen (https://clinicalgenome.org) [28], and CanVIG-UK (https://www.cangene-canvaruk.org) [29] guidelines. Only variants classified as pathogenic (P), likely pathogenic (LP), and variants of unknown significance (VUS) were reported.

2.3. Data Analysis and Visualization

Comparative analysis was conducted on Python 3 with packages Pandas [30] and SciPy [31]. Results were considered statistically significant if p-value < 0.05. Data visualization was performed using the packages seaborn [32] and Matplotlib [33].

3. Results

3.1. Clinico-Pathological Features of the Tumor Samples

Our evaluation involved the comparison of BRCA1/2 status and predicted genomic instability (PGI) score with the final assigned sample status obtained from the assays. For this purpose, we utilized a dataset of 85 ovarian carcinoma samples (Table 1). The median age of the patients during the sample collection was 60 years old. It is important to note that the FFPE samples did not exhibit uniform levels of DNA input neither quantity nor quality. The estimated tumor content ranged from 20% to over 90%, with DNA input varying from 3.9 to 150 ng. Supplementary Table 1 presents all relevant information about the samples included in this study, including the pre-analytical variables assessed in our analysis.

3.2. Comparison of Predicted Genomic Instability (PGI) Score Across Different NGS-Based Methods

Although the three assays evaluated employ different underlying methods to assess genomic instability in tumor samples, all of them provide scalar values that reflect the PGI score (i.e., higher values indicate greater genomic instability, while lower values indicate lesser instability). We investigated the associations and discrepancies among the methods, and we found an overall strong correlation, with the highest pairwise correlation observed between Myriad and SOPHiA (R=0.87, p-value=3.39x10^-19; Figure 1A). It is important to note that the interval ranges and the proportion of failed samples vary between the methods. Also, not all samples could be evaluated using the three methodologies due to insufficient tumor material or DNA quantity. Still, a total of 32 samples were executed across the three assays; in which, for SOPHiA’s test, 96.9% (31 samples) passed the quality control (QC) criteria. One sample was flagged as having a “silent profile” suggesting either a genome-wide profile without copy number variations (CNVs) or low tumor content. For Amoy's test, 84.4% (27 samples) passed the QC filter, and 5 samples failed the QC criteria (Supplementary Table 2). This finding is consistent with the 84.2% success rate reported by another group who also evaluated the same commercial assay [34].

We subsequently evaluated the level of agreement in assigned sample statuses based on the pre-defined thresholds provided by the vendors, specifically 1.8 for SOPHiA and 50 for AmoyDx. In the case of SOPHiA's test, we observed categorical discrepancies in 3 samples when compared to the results obtained from Myriad’s test. Specifically, 2 samples initially assigned as negative, and 1 sample assigned as positive were predicted to have the opposite status. Notably, these samples exhibited scores near the threshold values of both tests, highlighting the need for careful consideration of their categorical classification. Contrarily, we identified 7 categorical discrepancies between Myriad and AmoyDx., in which 6 samples were initially assigned as being negative by Myriad but were predicted to be positive by AmoyDx (Figure 1B).

3.3. Correlation of BRCA1/2 Status and Predicted Genomic Instability (PGI) Score

Considering that a high degree of genomic instability is commonly observed in BRCA-mutated tumors [35,36], we investigated the association and consistency between these two markers in a set of 26 samples that we successfully executed all three methodologies and passed the quality QC criteria. All samples with a detected pathogenic variant in BRCA1/2 were positive for genomic instability, the only exception was one sample with a pathogenic variant in BRCA1 that was categorized as negative for genomic instability in the test from AmoyDx’s test. 66.7% (8 of 12) BRCA-positive samples were also positive for Myriad, 57.1% (8 of 14) for SOPHiA, and 43.8% (7 of 16) for AmoyDx. As all pathogenic and likely pathogenic variants from Myriad's results were confirmed in the other two tests, this small association deflation between BRCA1/2 status and PGI score is due to more samples having a PGI-positive score in the later assays. Noteworthy, the SOPHiA test was able to detect one somatic variant in BRCA1 with a Variant Allele Fraction (VAF) of 9.8% not previously reported by the other two tests (Figure 2).

3.4. Determination of Testing Performance Parameters

Controlling experimental variability is crucial in any clinical test. NGS-based assays, being a multi-step process, are particularly susceptible to variations that may arise from extraction, library preparation, and sequencing, as these steps involve non-deterministic processes. Even prior to these steps, the extraction of DNA from somatic specimens relies on a biopsy process that samples a heterogeneous population of cells. To evaluate the reproducibility of our experiments, we sequenced 7 unique samples as replicates within or between sequencing runs. Among them, 4 samples were sequenced across different sequencing batches, while one sample was sequenced within the same batch for SOPHiA and 3 samples within the same batch for AmoyDx (Figure 3). Remarkably, we observed a high correlation of results for all tested samples, for SOPHiA r = 0.998 (p-value = 1.17x10^-4) and for AmoyDx r = 0.995 (p-value = 0.064; Figure 3). These results highlight the consistency and reliability of the obtained data, indicating robust reproducibility within our experimental setup.

In respect to the comparison of HRD status (i.e., final assigned sample result) of the two assays to the reference Myriad’s test, the positive predictive value (PPV) and negative predictive value (NPV) were 90.9% and 96.3% for SOPHiA’s test, respectively, while AmoyDx’s test achieved 75% PPV and 100% NPV. In our cohort, we identified 44.9% and 64.9% as HRD-positive tumors using SOPHiA and AmoyDx, respectively (Table 2).

3.5. Association of Pre-Analytical Variables with Low Confidence NGS Results

Pre-analytical variables verification is an important step to ensure a reliable experiment result, and it can also assist in the prediction of samples that are likely to pass or fail during the QC step. Both in-house HRD testing protocols evaluated suggest at least 30% estimated tumor content, but have slightly different ranges for DNA input requirement, between 10 – 200 ng for SOPHiA, and 50 - 100 ng for AmoyDx. Among the four pre-analytical variables evaluated, we observed that only functional DNA qPCR and DNA input were statistically significant between pass and low-confidence experiments (p-value=3.52x10^-4; p-value=3.40x10^-4) (Figure 4A). However, we observed a negative correlation between DNA library yield and residual noise (R=-0.62, p-value=1.72x10^-9), which is computed by measuring the standard deviation of the normalized genome sequencing coverage profile with respect to the smoothed normalized genome sequencing coverage profile (Figure 4B). Indeed, it is expected that samples with low values of library yield have more fragmented DNA, resulting in potential amplification biases and less uniform coverage in the genome, making the analysis of those experiments more challenging. Figure 4C shows two examples of samples that have low and high residual noise profiles.