Speaker-Attributed Transcription Toward Automated Scoring of NACC Uniform Data Set Assessments: A Whisper–Pyannote Pipeline and Pilot Evaluation

Standardized cognitive and functional assessments, such as those in the National Alzheimer’s Coordinating Center (NACC) Uniform Data Set (UDS), are central to detecting and monitoring Alzheimer’s disease, yet their administration and scoring depend on trained examiners who record and score responses by hand. We present a modular speech-processing pipeline that captures and structures spoken responses from UDS assessments administered by a clinician or by the Zeus2Q humanoid robot. The pipeline converts recordings to 16-kHz mono audio, segments speakers with the pyannote.audio 3.1 diarization pipeline, transcribes each speaker turn with Whisper medium.en, optionally personalized by participant-level fine-tuning, and emits a speaker-attributed, time-stamped JSON record aligned with UDS items for downstream scoring. The pipeline runs end to end on Google Colab and on an NVIDIA H200 GPU cluster. In a pilot on volunteer recordings of UDS-4 Form B7, the NACC Functional Assessment Scale, a fine-tuned Whisper model achieved a normalized word error rate (WER) of 25.7% and a character error rate of 18.1% on an evaluation recording. Error analysis showed that a single deleted response would produce at least two incorrect item codes if responses were assigned to items in order of occurrence, so WER alone does not capture clinical utility. Continued training and a single-example run drove the training loss to near zero within 40 steps, which reflects memorization rather than generalization and highlights the overfitting risk of few-sample personalization. We specify the evaluation protocol for a planned 200-participant study: diarization error rate, held-out WER against a zero-shot baseline, and item-level agreement with clinician scoring.