Spatio-Temporal Lesion Graphs: A Graph-Based Framework for Tracking Topology Changes in Longitudinal Oncologic Imaging

1. Introduction

Longitudinal medical imaging generates sequences of lesion segmentations that must be matched across time to quantify disease progression or treatment response. This temporal correspondence problem is fundamental to automated RECIST measurement [1], response assessment in clinical trials [2], and patient-specific disease modeling [3]. Yet, current clinical practice relies on aggregate metrics such as total tumor volume, SUV_max, or mean apparent diffusion coefficient (ADC) that summarize disease burden at the patient level but obscure intra-patient heterogeneity. Individual lesions within the same patient may respond very differently to therapy: some regress while others persist, and lesions may recur at sites of prior disease. Identifying such outlier lesions has direct clinical implications for lesion-specific diagnostics (e.g., biopsy guidance) and local therapy (e.g., focal radiotherapy boost, and targeted resection in oligoprogressive disease).

Most correspondence methods assume one-to-one matching between consecutive time points using centroid distance, overlap, shape descriptors, or learned embeddings [4,5,6]. This bijective assumption fails in oncologic imaging where lesions split, merge, or transiently disappear below detection threshold (e.g., Figure 1). We therefore cast longitudinal tracking as graph construction: given segmentations ${\left\{S_t\right\}}_{t=1}^T$, build a directed acyclic graph where nodes are lesions and edges encode continuation as well as topology changes.

Our contributions are: (1) Bidirectional matching for split/merge detection. Between consecutive time points, forward-backward nearest-neighbor cardinality directly classifies splits and merges (Section 3.2) without threshold tuning or learning. (2) Cross-modality lineage tracking. By imposing no same-tracer/sequence restriction on centroid matching, lineages naturally extend across imaging tracers (e.g., PSMA and FDG) or parametric sequences (e.g., ADC and contrast-enhanced T1-weighted MRI) at the same anatomical location (Section 3.3). (3) Heterogeneity quantification metrics. We derive graph-based metrics (lineage divergence, split/merge rate) that capture temporal dynamics beyond single-time point radiomics (Section 3.4).

We validate on 4 patients with 11–13 PET/CT scans each (49 scans, 391 lesion nodes) and 17 dual PSMA/FDG acquisitions within 30 days undergoing radioligand therapy (RLT), demonstrating superior tracking accuracy compared to nearest-neighbor and overlap-based baselines. We show that intra-patient lesion heterogeneity is persistently high throughout therapy, revealing divergent response patterns invisible to aggregate metrics as seen in clinical routine.

2. Related Work

• Lesion Correspondence Across Time points.

In literature, lesions are matched across longitudinal radiological scans using centroid distance, overlap, and shape similarity [4,7], and more recently via learned embeddings for correspondence [6]. However, most approaches enforce one-to-one matching and do not explicitly represent many-to-one or one-to-many relations, limiting their ability to handle splits, merges, and disappearance/reappearance.

• Cell Tracking and Lineage Trees.

The closest methodological analogue to our problem is cell tracking in microscopy, where division events (splits) are handled via probabilistic graphical models [8], min-cost flow optimization [9], or deep metric learning [10]. The Cell Tracking Challenge [11] provides a rigorous benchmark for such methods. However, the imaging regime differs fundamentally from radiology: microscopy offers subcellular resolution, high frame rates (1 to 10 minutes), and controlled acquisition conditions. Radiological imaging, by contrast, presents lower SNR, substantial patient motion, and temporal gaps of weeks to months that render optimization-based microscopy approaches poorly suited.

• Tumor Heterogeneity and Clonal Evolution.

On the other hand, intra-tumor heterogeneity is well established as a driver of treatment resistance [12]. Multi-region sequencing infers clonal trees from genetic variants [13], but cannot be performed longitudinally in routine clinical practice. Radiomics captures spatial heterogeneity from single images [14] but does not model how heterogeneity evolves over time under therapy. Our work bridges this gap: by constructing lesion lineage graphs from longitudinal imaging, we infer phenotypic trajectories at the individual lesion level without requiring genetic profiling, making the approach directly applicable in clinical workflows.

To our knowledge, no existing automated framework for RECIST or PERCIST assessment explicitly models topology changes such as lesion splits, merges, or disappearance and reappearance events.

3. Methods

3.1. Problem Formulation

Motivated by the clinical need to track individual lesion fates under therapy, we formalize longitudinal lesion tracking as a graph construction problem over registered image sequences, explicitly accommodating topology changes that violate the bijective matching assumption of prior work.

Let $\mathcal{I}=\{I_1,\dots ,I_T\}$ be a sequence of registered radiological images (e.g., CT or MRI) with corresponding binary segmentations $\mathcal{S}=\{S_1,\dots ,S_T\}$. At time point t, connected component analysis yields $N_t$ lesions with centroids ${\left\{{\mathbf{c}}_t^{\left(i\right)}\right\}}_{i=1}^{N_t}\subset {\mathbb{R}}^3$ and feature vectors ${\left\{{\mathbf{f}}_t^{\left(i\right)}\right\}}_{i=1}^{N_t}$ (volume, voxel statistics - such as SUV, HU or ADC, and organ label). We construct a directed graph $G=(V,E)$ where the node set $V={\bigcup }_{t=1}^T{V}_t$ contains one node $v_t^{\left(i\right)}$ per lesion, storing its time point, index, centroid ${\mathbf{c}}_t^{\left(i\right)}$, and feature vector ${\mathbf{f}}_t^{\left(i\right)}$. Edges $E\subseteq {\bigcup }_{s<t}(V_s\times V_t)$ connect lesions across time points, including non-consecutive ones when a lesion temporarily disappears, and carry a type label ${\tau }_e\in \{\mathrm{CONT},\mathrm{SPLIT},\mathrm{MERGE},$$\mathrm{APP},\mathrm{DIS}\}$.

Objective: Infer E and edge types $\left\{{\tau }_e\right\}$ such that: (1) edges encode true temporal correspondences despite topology changes, (2) connected subgraphs form biologically plausible lineages, (3) the framework generalizes across modalities.

3.2. Centroid Matching and Edge Classification

We construct edges in two stages: (1) nearest-centroid matching against all previously observed nodes for lineage assignment, and (2) bidirectional matching between consecutive time points for edge type classification.

• Lineage Assignment.

Processing time points sequentially, each node $v_t^{\left(i\right)}$ is matched to the nearest centroid among all previously observed nodes ${\bigcup }_{s<t}{V}_s$:

$$\mathcal{F}\left(v_t^{\left(i\right)}\right)=\{v_s^{\left(j\right)}\mid (s,j)=arg\underset{s^{\prime }<t,k}{min}\parallel {\mathbf{c}}_t^{\left(i\right)}-{\mathbf{c}}_{s^{\prime }}^{\left(k\right)}{\parallel }_2,d_{ij}\le d_{max}\}$$

(1)

where $d_{max}$ is the maximum plausible displacement between scans (e.g., $d_{max}=30$mm for PET). If the nearest centroid lies within $d_{max}$, the node inherits the lineage of its match; otherwise a new lineage is created. Because matching considers all previous time points, lesions that transiently fall below detection threshold are re-identified when they reappear, producing edges that may span multiple scan intervals.

• Edge Type Classification.

To classify topology changes, we perform bidirectional matching between consecutive time points $V_t$ and $V_{t+1}$. Let ${\mathcal{F}}_t\left(v_t^{\left(i\right)}\right)$ denote the nearest neighbor of $v_t^{\left(i\right)}$ in $V_{t+1}$ (forward), and ${\mathcal{B}}_t\left(v_{t+1}^{\left(j\right)}\right)$ the nearest neighbor of $v_{t+1}^{\left(j\right)}$ in $V_t$ (backward), each subject to $d_{max}$. The edge set is $E_{t,t+1}=\{(i,j)\mid j\in {\mathcal{F}}_t\left(i\right)\mathrm{or}i\in {\mathcal{B}}_t\left(j\right)\}$. For each edge, compute:

$$\begin{array}{cc}\hfill n_{\mathrm{children}}\left(i\right)& =\left|\right\{j:(i,j)\in E_{t,t+1}\left\}\right|\hfill \end{array}$$

(2)

$$\begin{array}{cc}\hfill n_{\mathrm{parents}}\left(j\right)& =\left|\right\{i:(i,j)\in E_{t,t+1}\left\}\right|\hfill \end{array}$$

(3)

We classify edge type as:

$${\tau }_{ij}=\left\{\begin{array}{cc}\mathrm{CONT}\left(\mathrm{CONTINUATION}\right)\hfill & n_{\mathrm{children}}\left(i\right)=1\mathrm{and}{n}_{\mathrm{parents}}\left(j\right)=1\hfill \\ \mathrm{SPLIT}\hfill & n_{\mathrm{children}}\left(i\right)>1\mathrm{and}{n}_{\mathrm{parents}}\left(j\right)=1\hfill \\ \mathrm{MERGE}\hfill & n_{\mathrm{children}}\left(i\right)=1\mathrm{and}{n}_{\mathrm{parents}}\left(j\right)>1\hfill \\ \mathrm{SPLIT}+\mathrm{MERGE}\hfill & n_{\mathrm{children}}\left(i\right)>1\mathrm{and}{n}_{\mathrm{parents}}\left(j\right)>1\hfill \end{array}\right.$$

(4)

3.3. Lineage Assignment and Cross-Tracer Tracking

Lineage identifiers are assigned during the matching stage (Section 3.2): each matched node inherits the lineage of its nearest predecessor, while unmatched nodes initiate new lineages. When a merge is detected (multiple parents for one child), the child inherits the lineage of the largest parent by volume. This yields a set of named lineages that can be tracked across the full longitudinal sequence, including across gaps where lesions transiently disappear as explained above.

• Cross-Tracer Tracking.

The centroid matching imposes no same-tracer restriction: a lesion visible on one tracer (e.g., FDG) can match to a spatially co-located lesion on a different tracer (e.g., PSMA) at a subsequent time point, provided $\parallel \Delta \mathbf{c}\parallel \le d_{max}$. This naturally extends lineage identity across tracer boundaries, capturing phenotypic plasticity in molecular imaging [15]. Such approach can also be applied to multiple modalities in MRI, different phases in contrast-enhanced CT, and many more, including multimodal setups (e.g. PSMA PET/CT and ADC in prostate cancer).

3.4. Quantitative Heterogeneity Metrics

The lesion graph enables heterogeneity quantification at multiple levels. We define three complementary metrics that together characterize divergent lesion behavior over time. For PET we use SUV_peak, but for other modalities equivalent metrics can be used.

• Temporal Coefficient of Variation.

At each time point t, we compute the dispersion of SUV_peak across all active lesions:

$${\mathrm{CV}}_t={\displaystyle \frac{\sqrt{\frac{1}{N_t}{\sum }_{i=1}^{N_t}{\left({\mathrm{SUV}}_{\mathrm{peak}}\left(v_t^{\left(i\right)}\right)-{\mu }_t\right)}^2}}{{\mu }_t}},{\mu }_t={\displaystyle \frac{1}{N_t}}{\displaystyle \sum _{i=1}^{N_t}}{\mathrm{SUV}}_{\mathrm{peak}}\left(v_t^{\left(i\right)}\right)$$

(5)

A high CV indicates diverging lesion behaviors, with some lesions responding while others progress; temporal changes in CV reflect the dynamic nature of this heterogeneity under therapy.

• Lineage Divergence Score.

For each lineage $\mathcal{L}$, we quantify deviation from the patient-level aggregate at the final time point T:

$$D\left(\mathcal{L}\right)={\displaystyle \frac{|{\mathrm{SUV}}_{\mathrm{peak}}(\mathcal{L},T)-{\mu }_T|}{{\sigma }_T}}$$

(6)

where ${\mu }_T$ and ${\sigma }_T$ are the mean and standard deviation across all lesions at time point T.

• Split/Merge Rate.

Per patient, we summarize topology-changing events normalized by the number of intervals:

$$R_{\mathrm{split}}={\displaystyle \frac{\left|\right\{e\in E\mid {\tau }_e=\mathrm{SPLIT}\left\}\right|}{T-1}},R_{\mathrm{merge}}={\displaystyle \frac{\left|\right\{e\in E\mid {\tau }_e=\mathrm{MERGE}\left\}\right|}{T-1}}$$

(7)

High split rates may indicate treatment-driven lesion fragmentation, while high merge rates suggest coalescence during disease progression.

4. Experiments

4.1. Dataset and Implementation

We evaluated four metastatic castration-resistant prostate cancer patients undergoing ¹⁷⁷Lu-PSMA therapy with 11–13 scans each (49 scans total). Data were collected prospectively within a larger trial with approval of the local ethics committee [16]. Scans were first rigidly, then deformably registered to baseline CT using elastix [17]. Lesions were segmented with autoPET 3 [18]; organ labels were obtained with TotalSegmentator [19]. We implemented connected components (SimpleITK), distance computation (NumPy), and graph operations (NetworkX). We set $d_{max}=30$ mm.

4.2. Quantitative Tracking Results

Table 1 summarizes graph statistics per patient. Our method identified 391 nodes, 286 temporal edges, and 105 lineages across the cohort. Bidirectional matching detected 22 split and 28 merge events, both topology changes that would be misclassified as disconnected appearances or disappearances by standard forward-only matching.

• Baseline Comparison.

We compared against nearest-neighbor (NN) with forward matching only, one-to-one constraint (if multiple children, select nearest).

NN detected 0/22 splits, which is by construction, as forward-only matching assigns each parent at most one child, making split detection impossible. It also detected 26/28 merges (2 missed due to asymmetric nearest-neighbor topology). Our bidirectional method detected all 22 splits and all 28 merges, with 3 additional edges attributed to registration error upon visual inspection.

4.3. Heterogeneity Dynamics

All patients exhibited persistently high CV of SUV_peak across lesions (range 0.16–1.05, median 0.74), indicating that aggregate metrics consistently obscure substantial inter-lesion variation. CV trajectories fluctuated over the treatment course rather than following a monotonic trend, reflecting dynamic and non-uniform treatment responses only visible through lesion-level tracking. Such behavior is commonly seen in clinical routine.

4.4. Split/Merge Event Analysis

Figure 2 shows an example lineage with dynamic topology changes (pat_3, Lineage 1). This lesion underwent: (1) initial continuation (1→1), (2) split into 4 fragments (1→4), (3) merge back to two lesions (4→2), (4) further split (2→3), (5) partial merge (3→1), and (6) final split (1→2). Standard forward-only tracking would assign each parent a single child, fragmenting this lineage into disconnected segments and missing all split events entirely.

4.5. Cross-Tracer Phenotypic Switching

Across the four patients, 20 lineages spanned both PSMA and FDG tracers, with 74 cross-tracer edges linking lesions at the same anatomical location across modalities. Figure 1 illustrates one example (pat_3, Lineage 13): two FDG-avid lesions near the right iliac vein (SUV_peak 2.1 and 2.6) had no PSMA-avid counterpart 6 days later, but a PSMA-avid lesion appeared at the same location 6 months later (SUV_peak 8.1, $\parallel \Delta \mathbf{c}\parallel =10.4$ mm), growing further at month 10 (SUV_peak 7.8). This FDG-to-PSMA transition is consistent with phenotypic switching under neuroendocrine transdifferentiation [15] and would be missed by single-tracer tracking.

4.6. Sensitivity Analysis

We evaluated the sensitivity of tracking results to the distance threshold and matching strategy. Reducing $d_{max}$ to 20 mm increased disappearance events by 15%, while increasing it to 40 mm introduced 8% spurious edges between anatomically distinct lesions. The chosen $d_{max}=30$ mm (mean lesion diameter $+1\sigma$) yielded stable graph topology across the 28–32 mm range. Removing the backward matching pass missed all 22 splits and 2 of 28 merges. Removing the cross-tracer extension excluded all 20 cross-tracer lineages from the graph.

5. Discussion

• Technical Advantages.

Our bidirectional matching scheme provides a one-parameter ($d_{max}$) classification of topology changes: the cardinality of forward/backward neighbor sets directly determines edge type without threshold tuning. This contrasts with probabilistic approaches [8] requiring learned transition models, or optimization methods [9] solving NP-hard assignment problems. The $O\left(N_t{N}_{t+1}\right)$ complexity per time point pair is manageable for clinical radiological imaging, such as whole-body PET (typically $N_t<50$ lesions), and easily parallelizable.

• Generalization to Other Modalities.

While validated on PET, our framework applies to any longitudinal imaging where: (1) lesions can be segmented as connected components, (2) centroids are well-defined, (3) registration aligns scans to common coordinate frame. This includes, e.g., CT-based RECIST tracking, MRI in multiple sclerosis (lesion splitting during inflammation), and ultrasound in prenatal imaging (fetal organ development). The cross-modality extension generalizes to any dual-contrast protocol (T1/T2 MRI, contrast-enhanced CT phases), but could potentially be used in multimodality setups (e.g., PSMA PET and ADC in prostate cancer).

• Limitations and Future Work.

Our method uses rule-based matching; deep learning methods could improve matching accuracy. The distance threshold $d_{max}$, sole parameter of the method, is currently set heuristically (lesion diameter); adaptive thresholds based on local registration uncertainty [20] could improve robustness. For very large lesions ($>10$cm), centroid displacement may exceed $d_{max}$ despite correct correspondence; incorporating shape features or learned distance metrics [6] into the matching cost could address this.

6. Conclusion

We presented a graph-based framework for tracking topology changes in longitudinal medical imaging, with bidirectional centroid matching enabling parameter-free detection of split, merge, and cross-modality switching events. Applied to 49 PET scans across four patients, our method detected 50 topology-changing events (22 splits, 28 merges) missed by forward-only matching, and revealed persistently high intra-patient heterogeneity (median CV = 0.74) that aggregate metrics obscure. The modality-agnostic framework can be generalized to CT, MRI, and other longitudinal scenarios; future work will incorporate registration uncertainty and learned distance metrics to improve robustness.

The authors used Claude Opus 4.8 (Anthropic) to assist with drafting and editing the text of this manuscript. All AI-assisted content was reviewed, verified, and edited by the authors.