Deep Looking in Digital Pathology: An Explainable Federated Learning Framework for Lung Cancer Classification

1. Introduction

Lung cancer is the leading cause of cancer-related death worldwide, accounting for approximately 1.8 million deaths each year [1,2]. Histopathological examination of hematoxylin and eosin (H&E)-stained tissue is the reference standard for confirming malignancy and for distinguishing lung adenocarcinoma (ACA) from lung squamous cell carcinoma (SCC), a distinction that determines molecular testing and therapy. Yet this workflow suffers from inter-observer variability, cognitive fatigue under growing caseloads, and an uneven distribution of expert pathologists. Whole-slide imaging (WSI) and convolutional neural networks (CNNs) have created computational pathology, in which deep learning classifies tumor subtypes and even predicts mutations from morphology [3,4]. In Deep Medicine, Topol argues that the promise of artificial intelligence (AI) is not to replace clinicians but to absorb repetitive cognitive work so that physicians can return to “deep looking”—careful observation, empathy, and communication [5].

Two barriers stand between this vision and practice. The first is the data silo problem. Robust models need diverse data from many institutions, scanners, and staining protocols, but medical images are protected by the European General Data Protection Regulation (GDPR) [6], the U.S. Health Insurance Portability and Accountability Act (HIPAA), and Taiwan’s Personal Data Protection Act (PDPA) [7], and models trained at a single center often degrade elsewhere because of domain shift. Federated learning (FL) moves the model rather than the data: each institution trains locally, and only model parameters are aggregated [8,9,10]. Real hospitals, however, hold non-independent and identically distributed (non-IID) data, which causes “client drift” and can destabilize the standard Federated Averaging (FedAvg) algorithm [11,12]. The second barrier is the trust gap: a model that outputs “adenocarcinoma, 98%” without showing what it looked at offers an unverifiable answer [13]. Gradient-weighted Class Activation Mapping (Grad-CAM) highlights the image regions that drive a prediction [14,15], and medical large language models (LLMs) such as MedGemma 1.5 [16] can turn model outputs into clinical language—but sending patient data to cloud-hosted LLMs would reintroduce the privacy problem that FL solves.

We therefore propose the Lung Cancer Analysis System (LCAS), which integrates FL, explainable AI (XAI), and an on-premises medical LLM. A DenseNet121 backbone [17] is trained collaboratively by three simulated hospitals with FedAvg and FedProx, and its decisions are explained by Grad-CAM heatmaps and by reports generated locally by MedGemma 1.5. Our contributions are as follows:

• A three-layer system architecture (federated training, validation, and clinical deployment) in which raw images never leave the institution, evaluated on 18,000 H&E images from three heterogeneous sources acting as hospitals with source-induced non-IID data.
• A comparison of high-capacity (DenseNet121), compound-scaled (EfficientNet-B3), and lightweight (MobileNetV3-Small) backbones under local-only, centralized, and federated training, including their communication cost.
• An analysis of FedProx under non-IID data, with an ablation of the proximal coefficient μ, external blind testing, blur robustness, and a cross-modal stress test.
• A “three-dimensional explanation” that couples numerical (probability), visual (Grad-CAM), and textual (LLM report) evidence in a bilingual web platform that runs fully offline.

Related Work

CNNs have been applied extensively to lung histopathology, most often on the LC25000 dataset [18]. Garg and Garg compared eight ImageNet-pretrained CNNs, including DenseNet169 and MobileNet, and visualized them with Grad-CAM and SmoothGrad (96–100% accuracy) [19]; Moin et al. compared DenseNet121/169/201 and other backbones with seven class-activation XAI methods (97–99%) [20]; Tummala et al. combined compound-scaled EfficientNetV2 with Grad-CAM (99.97%, five classes) [21], and Anjum et al. reported about 97% with multiresolution EfficientNets [22]. Hasan et al. benchmarked DenseNet, MobileNetV3, a vision transformer, and YOLO11 on LC25000 combined with LungHist700 (best 98.38%) [23]. Other single-dataset studies report 99–100% [24,25]. All of these train and test on pooled data. In FL, Adnan et al. trained DenseNet-feature multiple-instance models on TCGA LUAD/LUSC slides from several hospitals with FedAvg and differential privacy, reaching 82.4% versus 84.8% for centralized training [26]; FedAvg has also been applied to gigapixel WSIs [27] and to LC25000 [28,29,30], and Ajowho combined FL with explainable heatmaps [31]. A systematic review by Touhami et al. found FedAvg to be the dominant strategy, reported accuracies of 69.37–99.72%, and highlighted non-IID data as a key challenge [32]. FedProx [12] and SCAFFOLD [33] were designed to counter client drift. Most prior FL studies split one dataset artificially, and few combine FL with both visual and on-premise textual explanation. Table 1 compares the closest studies; this work is the only one that combines federated training on heterogeneous sources, visual and textual explanation, and an independent external test.

2. Materials and Methods

2.1. System Architecture

Figure 1 shows LCAS’s system architecture as a layered pipeline. H&E patches from three hospitals pass through data partitioning (one source per node, 80:10:10 split, non-IID by source) and pre-processing (resizing, Macenko stain normalization, conservative augmentation, and blur simulation) into an ImageNet-pretrained backbone. The model is trained along two paths—centralized training on pooled data as the performance ceiling and federated training in which images stay in each hospital and only weights are exchanged (FedProx local updates and FedAvg aggregation)—and then classifies each patch with a softmax over three classes. Each prediction is explained along a visual path (Grad-CAM) and a textual path (a structured report generated locally by MedGemma 1.5). The model is evaluated on internal, external, blur, and cross-modal test data and deployed on-premises in the LCAS web platform, where the pathologist reviews the output—class, heatmap, and report.

2.2. Datasets and Simulated Hospital Nodes

Three public H&E datasets from different sources were assigned to three hospital nodes (Table 2), each providing lung adenocarcinoma (ACA), squamous cell carcinoma (SCC), and benign tissue (N): Hospital 1, LC25000 (768 × 768 JPEG images derived from HIPAA-compliant validated samples and expanded with the Augmentor library) [18]; Hospital 2, HistoSet-5×14, a balanced 14-class multi-organ collection augmented with geometric transformations [34]; and Hospital 3, the Comprehensive Lung Cancer Imaging Dataset (CLID), de-identified patches extracted by pathologists from clinical WSIs [35]. Each class at each node was randomly down-sampled to 2000 images (6000 per hospital; 18,000 in total) and split 80:10:10 into training, validation, and test subsets. Although class counts are balanced, the nodes differ in staining, preparation, and acquisition, so the partition is non-IID by source (feature-distribution skew). An IID reference setting was created by pooling and shuffling all images before dividing them evenly among the nodes.

External blind test set. The test split of the Roboflow “Lung Cancer” dataset (version 1; 9537 images at 640 × 640) [36], containing 958 images (325 ACA, 304 SCC, 329 benign), was never used for training or model selection. Hospital 4 stress test. De-identified retrospective imaging data from Chung Shan Medical University (CSMU)—49,824 DICOM files from 174 patients, organized by patient, study (PET/CT, thoracic CT, chest X-ray), series, and instance—served as a fourth hospital. Because these grayscale radiological images differ fundamentally from H&E histology, we used them only as an exploratory robustness test under severe domain shift, not as a diagnostic benchmark.

2.3. Pre-Processing and Data Augmentation

Images were resized to 224 × 224 pixels with bilinear interpolation (300 × 300 for EfficientNet-B3) and normalized with ImageNet channel statistics. In federated training, each node applied Macenko stain normalization [37] locally, estimating H&E stain vectors via singular value decomposition of optical density and mapping stain concentrations to a common reference, reducing color differences across sources without sharing images. Augmentation was conservative because H&E color carries biological meaning: random horizontal/vertical flips, rotations (±15–30°), resized cropping (80–100% of the field), mild zoom (0.9–1.1×), and slight brightness changes, without strong color jitter. To simulate scanner defocus, Gaussian blur (5 × 5 kernel, σ ∈ [0.1, 2.0]) was applied during training and, for the robustness test, at inference.

2.4. Backbone Architectures

DenseNet121 [17] was the core backbone. In a dense block, each layer receives the feature maps of all preceding layers,

x_𝓁 = H_𝓁([x₀,x₁,…,x_𝓁−1])

(1)

where [·] denotes channel concatenation and H_ℓ comprises batch normalization, ReLU, and 1 × 1 and 3 × 3 convolutions. Dense connectivity promotes feature reuse and keeps the model small, which suits fine nuclear texture and FL bandwidth limits. The ImageNet classifier was replaced by dropout (0.5), a 1024→512 fully connected layer with ReLU, dropout (0.3), and a 3-unit output layer (Figure 2). As capacity ablations, MobileNetV3-Small [38] (depthwise separable convolutions for constrained bandwidth) and EfficientNet-B3 [39] (compound scaling of depth, width, and resolution) received analogous heads. All backbones were initialized with ImageNet weights [40] and fine-tuned end-to-end; Table 3 lists their size and communication cost.

2.5. Federated Learning Protocol

In each communication round t, the server broadcasts the global weights w^t to the K = 3 hospitals; each hospital k trains a local copy on its private data for E = 1 epoch and uploads w_k^t+1; and the server aggregates the uploads. FedAvg [10] uses a sample-size-weighted average:

$$w^{t+1}={\displaystyle \sum _{k=1}^K}{\displaystyle \frac{n_k}{n}}\hspace{0.17em}{w}_k^{t+1}$$

(2)

where n_k is the number of training samples at hospital k and n = Σ_k n_k (equal weights in our balanced setting). To counter client drift, FedProx [12] adds a proximal term to the local objective:

$$\underset{w}{\min }\hspace{0.33em}{h}_k\left(w;w^t\right)=F_k\left(w\right)+{\displaystyle \frac{\mu }{2}}{∥w-w^t∥}^2$$

(3)

where F_k(w) is the local cross-entropy loss and μ ≥ 0 controls how far local models may drift; μ = 0 recovers FedAvg. We evaluated μ ∈ {0, 0.01, 0.1, 1.0} and used μ = 0.01 in the main experiments. The traffic of one round and of the whole training is

C_round = 2K |w|,  C_total = T ⋅ C_round

(4)

where |w| is the serialized model size. Algorithm 1 summarizes the procedure; local optimization used Adam [41] with a learning rate of 1 × 10⁻⁴ and a batch size of 32, and one round took approximately 6–11 min depending on the hardware.

Algorithm 1. Federated training with FedProx (FedAvg when μ = 0)

Input: K = 3 hospitals with private data Dk; T = 20 rounds; E = 1 local epoch; Adam, η = 10−4; batch size 32; μ = 0.01

1: Initialize w0 from the ImageNet-pretrained backbone with a new 3-class head

2: for t = 0, …, T − 1 do: server broadcasts wt to all hospitals

3: for each hospital k in parallel: apply Macenko normalization and augmentation locally; set wk ← wt

4: for E epochs over mini-batches b ⊂ Dk: wk ← wk − η∇[ℓCE(b; wk) + (μ/2)‖wk − wt‖²]

5: upload wkt+1 ← wk (no images or labels leave hospital k)

6: server aggregates wt+1 ← Σk (nk/n) wkt+1

Output: global model wT, deployed to LCAS after external validation

Baselines. Centralized training pooled all 18,000 images on one server as the performance ceiling (Adam, learning rate 1 × 10⁻⁴, batch size 64 for DenseNet121 and 32 for the other backbones, up to 50 epochs with early stopping on validation loss); a baseline configuration without augmentation and an enhanced configuration with dynamic augmentation were run. Local-only training used data from a single hospital, representing an institution that does not collaborate. Results of repeated runs are reported as mean ± SD.

2.6. Explainability and Report Generation

Figure 2 shows the LCAS AI diagram. For Grad-CAM [14], let A^k denote the k-th 7 × 7 feature map of the last dense block and y^c the pre-softmax score of class c. Channel weights are the spatially pooled gradients, and the localization map is their rectified weighted sum:

$${\alpha }_k^c={\displaystyle \frac{1}{Z}}\sum _i\sum _j{\displaystyle \frac{\partial y^c}{\partial A_{\mathit{ij}}^k}}$$

(5)

$$L_{\mathrm{Grad}-\mathrm{CAM}}^c=\mathrm{ReLU}\left(\sum _k{\alpha }_k^c{A}^k\right)$$

(6)

where Z = 49 is the number of spatial positions. The map is upsampled to 224 × 224, rendered with a jet color scale, and overlaid on the input. To test whether federated training removes institutional bias, the 1024-dimensional pooled features of the global model were projected with t-SNE [42] and colored by class and by hospital.

Reports are produced by MedGemma 1.5 [16] served through the local OpenAI-compatible API of LM Studio [43], so no data leave the hospital network. A prompt builder maps the softmax probability to a wording strength (e.g., ≈1.00 → “highly consistent with”; ≈0.90 → “suspicious for”) and converts the location of high Grad-CAM activation into a morphological descriptor (e.g., activation on tumor cell nests for SCC). A fixed system prompt then instructs the model: “You are a clinical pathologist specializing in digital pathology. Based on the AI analysis data below (predicted class, confidence score, and visual-feature description), write a structured report—FINDINGS: describe the morphology highlighted by Grad-CAM; IMPRESSION: state the model’s diagnosis and its confidence; RECOMMENDATION: suggest follow-up examinations. Use only the data provided and do not invent patient information.” Together, the probability, heatmap, and report form a three-dimensional explanation.

2.7. Platform, Metrics, and Implementation

LCAS is a bilingual (Traditional Chinese/English) web application that accepts JPEG, PNG, or TIFF images, shows a three-stage progress indicator (classification, heatmap, report), and exports reports as PDF or text with a disclaimer that they must be reviewed by a physician. Performance was measured with accuracy, precision, recall, F1-score (per class and macro/weighted averages), and confusion matrices:

$$\mathrm{Accuracy}={\displaystyle \frac{\mathit{TP}+\mathit{TN}}{\mathit{TP}+\mathit{TN}+\mathit{FP}+\mathit{FN}}}$$

(7)

$$\mathrm{Precision}={\displaystyle \frac{\mathit{TP}}{\mathit{TP}+\mathit{FP}}}, \mathrm{Recall}={\displaystyle \frac{\mathit{TP}}{\mathit{TP}+\mathit{FN}}}$$

(8)

$$F_1={\displaystyle \frac{2\cdot \mathrm{Precision}\cdot \mathrm{Recall}}{\mathrm{Precision}+\mathrm{Recall}}}$$

(9)

All models were implemented in PyTorch with Python 3.12 and trained on Kaggle (dual NVIDIA T4 GPUs, CUDA 12.8), a workstation with an NVIDIA RTX 4050 Laptop GPU (Intel Core i5-12500H, 32 GB RAM, CUDA 12.9), and an Apple M5 system (32 GB) with the Metal Performance Shaders backend; LM Studio with MedGemma 1.5 ran on the workstation.

3. Results

3.1. Centralized Baseline

DenseNet121 reached a validation accuracy of 0.9933 after one epoch and 1.0000 by epoch 3; early stopping ended training at epoch 13 (validation loss 0.0001), with brief fluctuations at epochs 7 and 9 (loss ≤ 0.0050) and no divergence between training and validation curves. On the held-out test data, DenseNet121 achieved a precision, recall, and F1-score of 1.00 for all classes, while MobileNetV3-Small and EfficientNet-B3 achieved macro averages of 0.99, their only errors being a few adenocarcinomas (recall 0.97). Over repeated runs, centralized accuracy was 99.50% ± 0.21% (DenseNet121), 99.64% ± 0.18% (EfficientNet-B3), and 97.20% ± 0.55% (MobileNetV3-Small) (Table 4). On the external Roboflow set, centralized DenseNet121 classified all 958 images correctly, and MobileNetV3-Small misclassified one adenocarcinoma as benign (Figure 3a,b).

3.2. Federated Performance Across Scenarios

Table 4 compares the training scenarios. Local-only models scored highly on their own test data (95.12–99.45%) but are trained on a single source. In the IID federated setting, FedAvg exceeded 98% global accuracy within five rounds and approached 100% for DenseNet121 and EfficientNet-B3, matching the centralized ceiling without exchanging images. In the realistic non-IID setting, the FedProx global models reached, on the external blind test set, 96.97% ± 0.62% accuracy (F1 96.96%) for DenseNet121, 97.49% ± 0.52% (F1 97.49%) for EfficientNet-B3, and 91.23% ± 0.95% (F1 91.21%) for MobileNetV3-Small—that is, about 2.5, 2.2, and 6.0 percentage points below centralized training, with low run-to-run variation (SD ≤ 0.95%).

3.3. Convergence and the Proximal Coefficient

Figure 4 shows the training dynamics over 20 FedProx rounds. For DenseNet121, mean local accuracy rose from 0.880 (round 1) to 0.972 (round 2) and 0.991 (round 4), while the loss fell from 0.350 to 0.046. Small setbacks followed at rounds 5, 9, 12, and 15 (e.g., loss 0.031 → 0.037 between rounds 11 and 12), reflecting conflicting updates from heterogeneous hospitals; weighted aggregation and the proximal constraint absorbed them, and the curves stabilized after about round 15. Final training losses were 0.028 (DenseNet121), 0.014 (EfficientNet-B3), and 0.008 (MobileNetV3-Small). MobileNetV3-Small had the lowest training loss but the lowest external accuracy, suggesting that its 2.04 M parameters fit each source without reconciling them into transferable features.

Table 5 reports the μ ablation for DenseNet121. FedAvg (μ = 0) converged fastest (14 rounds) but had the lowest accuracy (98.42%). A small proximal term (μ = 0.01) raised accuracy to 99.00%, μ = 0.1 performed best (99.11%; F1 99.13%), and a strong constraint (μ = 1.0) reduced accuracy to 98.75% by limiting local learning. A small-to-moderate μ thus balances local fitting and global stability; μ = 0.01 was adopted because it converged faster than μ = 0.1 with nearly identical accuracy.

3.4. External Validation, Robustness, and Cross-Modal Stress Test

Table 6 and Figure 5 give the per-class results of the federated models on the 958 external images. Benign tissue was recognized almost perfectly (F1 0.980–0.997). For DenseNet121, the main error was predicting ACA as SCC (20 of 325), yielding high ACA precision (0.993) but lower recall (0.926); EfficientNet-B3 showed the same pattern, with perfect SCC recall. MobileNetV3-Small confused the carcinoma subtypes in both directions (23 ACA→SCC; 48 SCC→ACA). Malignant tissue was rarely called benign by the stronger models (7 of 629 carcinomas for DenseNet121, none for EfficientNet-B3), so the dominant errors were subtype confusions between malignancies rather than missed cancers.

Under Gaussian blur (σ = 0.1–2.0) on the internal test images (n = 1800), accuracy remained 0.9978, with only four errors (one ACA as benign, one ACA as SCC, two SCC as ACA) and no errors on benign tissue (Figure 3c). On the Hospital 4 DICOM data (Table 4), MobileNetV3-Small (97.76% ± 1.84%) and EfficientNet-B3 (92.00% ± 6.91%) were relatively stable, whereas DenseNet121 varied widely (77.22% ± 22.82%). Because these are radiological images rather than H&E patches, the numbers are not diagnostic accuracy; they show that a model specialized for histological texture can become unreliable outside its modality, so out-of-distribution detection should precede use on unfamiliar data.

3.5. Feature Space and Grad-CAM Explanations

In the t-SNE embedding of the global model (Figure 6), samples formed three compact, well-separated clusters by tissue class, whereas samples from Hospitals 1–3 were thoroughly interleaved inside each cluster, with no hospital-specific sub-cluster. The federated model thus organizes its features by biology rather than by institution.

Figure 7 shows Grad-CAM results for the centralized DenseNet121 (all predictions were correct with 1.000 confidence). For ACA, activation concentrated on densely packed atypical nuclei around a gland-forming structure; for SCC, on a cohesive cluster of large hyperchromatic tumor cells, consistent with nested growth; and for benign tissue, on a broad region spanning thin alveolar septa rather than a sharply delimited cell cluster. The federated model produced similar patterns (Figure 8): SCC and benign examples were predicted with a confidence of 1.000 and highlighted tumor nests and alveolar or bronchial epithelium, whereas one ACA example with loosely arranged cells received a confidence of 0.918 and a more diffuse heatmap—exactly the kind of case in which visual evidence helps the pathologist judge how much to trust the model. Both training regimes therefore base their decisions on diagnostically meaningful structures, and federated training preserves explanation quality.

3.6. LCAS Platform and Generated Report

Figure 9a shows LCAS analyzing an SCC image (probability 100.0%). The MedGemma report described polygonal tumor cells with distinct borders, keratin pearls, and coarse chromatin with Grad-CAM activation in the keratinizing region (FINDINGS); stated that the DenseNet121 prediction was lung SCC with 100.0% confidence and that the morphology met diagnostic criteria (IMPRESSION); and suggested immunohistochemistry (p40, p63, CK5/6), PD-L1 testing, and staging with multidisciplinary discussion (RECOMMENDATION), followed by a disclaimer. All computations ran locally; Figure 9b shows the federated analytics dashboard.

4. Discussion

This study shows that federated training can classify lung histopathology across heterogeneous institutions with accuracy close to centralized training while keeping images on site. Under IID conditions, FedAvg matched the centralized ceiling, and under realistic source-induced non-IID conditions, the FedProx models still reached 96.97% (DenseNet121) and 97.49% (EfficientNet-B3) on an independent external test set—about 2–2.5 points below pooling all data. For institutions that cannot share slides, this small cost buys access to colleagues’ data, and t-SNE shows that the model generalizes beyond any single hospital.

Against heterogeneity [11,32], local Macenko normalization reduced color discrepancies, and the FedProx proximal term kept local updates near the global model [12]. Model capacity mattered as much as aggregation: MobileNetV3-Small needs one-sixth of DenseNet121’s traffic (Table 3) but lost 6 points under FL, whereas DenseNet121 balanced accuracy, stability, and traffic (183 MB per round).

Accuracy alone does not earn trust [13]. Grad-CAM maps from both centralized and federated models focused on glandular structures, atypical nuclei, and tumor nests, and the low-confidence ACA case’s diffuse map visibly signaled uncertainty. The LLM layer adds a textual dimension grounded in the model’s actual outputs, and running MedGemma 1.5 locally extends the privacy guarantee of FL from training to reporting. Unlike most prior work (Table 1), our federated results were measured on a separate external dataset after training on three independently collected sources.

This study has limitations. First, near-perfect internal accuracy suggests that augmented near-duplicates may appear in both training and test splits under patch-level splitting, so internal results are upper bounds. Second, the federation was simulated. Third, FL does not eliminate privacy risk, because shared weights can leak information through gradient inversion or membership inference [44,45], and no differential privacy or secure aggregation was used. Fourth, Hospital 4 images were radiological, not histological. Fifth, we assessed Grad-CAM maps and LLM reports qualitatively, rather than having pathologists rate them. Finally, the task is patch-level three-class classification rather than slide-level diagnosis. Future work will add differential privacy [46] and secure aggregation, real multi-hospital deployment with patient-level splits, a pathologist reader study, and out-of-distribution detection.

5. Conclusions

We presented LCAS, a privacy-preserving and explainable framework for lung cancer histopathology that integrates federated learning, Grad-CAM, and a locally deployed medical LLM. Trained across three heterogeneous sources without exchanging images, the FedProx DenseNet121 model reached 96.97% accuracy on an independent external test set, close to the centralized benchmark, and remained robust to blur. The platform presents each decision through numerical, visual, and textual evidence entirely on-premises. In the spirit of Deep Medicine, the aim is to relieve pathologists of repetitive work, not to replace them.