Preprints.org logo
Preprint
Article

This version is not peer-reviewed.

Speaker-Attributed Transcription Toward Automated Scoring of NACC Uniform Data Set Assessments: A Whisper–Pyannote Pipeline and Pilot Evaluation

Submitted:

11 October 2026

Posted:

12 October 2026

You are already at the latest version

Abstract
Standardized cognitive and functional assessments, such as those in the National Alzheimer’s Coordinating Center (NACC) Uniform Data Set (UDS), are central to detecting and monitoring Alzheimer’s disease, yet their administration and scoring depend on trained examiners who record and score responses by hand. We present a modular speech-processing pipeline that captures and structures spoken responses from UDS assessments administered by a clinician or by the Zeus2Q humanoid robot. The pipeline converts recordings to 16-kHz mono audio, segments speakers with the pyannote.audio 3.1 diarization pipeline, transcribes each speaker turn with Whisper medium.en, optionally personalized by participant-level fine-tuning, and emits a speaker-attributed, time-stamped JSON record aligned with UDS items for downstream scoring. The pipeline runs end to end on Google Colab and on an NVIDIA H200 GPU cluster. In a pilot on volunteer recordings of UDS-4 Form B7, the NACC Functional Assessment Scale, a fine-tuned Whisper model achieved a normalized word error rate (WER) of 25.7% and a character error rate of 18.1% on an evaluation recording. Error analysis showed that a single deleted response would produce at least two incorrect item codes if responses were assigned to items in order of occurrence, so WER alone does not capture clinical utility. Continued training and a single-example run drove the training loss to near zero within 40 steps, which reflects memorization rather than generalization and highlights the overfitting risk of few-sample personalization. We specify the evaluation protocol for a planned 200-participant study: diarization error rate, held-out WER against a zero-shot baseline, and item-level agreement with clinician scoring.
Keywords: 
Alzheimer’s disease
;  
automatic speech recognition
;  
cognitive assessment
;  
human–robot interaction
;  
speaker diarization
;  
Uniform Data Set
;  
Whisper
Subject: 
Computer Science and Mathematics  -   Computer Science

I. Introduction

Alzheimer’s disease (AD) is the most common cause of dementia, and its early detection relies on standardized assessment of cognition and everyday function. Across the Alzheimer’s Disease Research Centers funded by the U.S. National Institute on Aging, such data are collected with the National Alzheimer’s Coordinating Center (NACC) Uniform Data Set (UDS) [1,2], which combines clinician ratings, co-participant (informant) questionnaires, and a neuropsychological test battery [3]. Administering these instruments requires trained examiners, and spoken responses are typically recorded and scored by hand, a time-consuming process that leaves room for transcription and scoring inconsistencies.
Speech is a promising substrate for automation. Spoken responses carry the content needed for scoring as well as acoustic and linguistic markers associated with cognitive decline [4,5,6], and markers derived from other routinely collected data, such as survey response behaviors [7] and unstructured clinicians’ notes [8], are being explored for the early detection and staging of dementia, alongside digital tools for patients and caregivers [9]. Advances in automatic speech recognition (ASR), notably Whisper, which was trained on 680,000 hours of weakly supervised audio [10], have made accurate transcription of diverse and noisy speech practical. ASR-based cognitive tests have been validated against manual scoring [11,12], and ASR-based assessment has been validated in other clinical populations [13].
Most speech-based AD research, however, addresses diagnostic classification from spontaneous speech such as picture descriptions [5,14]. Automating the standardized assessment workflow itself imposes different requirements: the system must determine who is speaking at each moment, accurately transcribe short and often single-word responses from older adults, and attach each response to the correct form item so that established scoring rules can be applied. These requirements become critical when a socially assistive robot administers the test [15], because no human examiner is present to record the answers.
This paper describes a pipeline built for this purpose and reports its first pilot results. Our contributions are threefold:
1)
A modular pipeline, comprising audio preprocessing, speaker diarization with pyannote.audio 3.1, turn-level transcription with Whisper medium.en, and speaker-attributed JSON output aligned with UDS items, that runs on both Google Colab and a GPU cluster.
2)
A participant-level fine-tuning procedure for adapting Whisper to individual speakers, with an analysis of its overfitting behavior in the few-sample regime.
3)
A pilot evaluation on UDS-4 Form B7 recordings, including an error analysis that traces transcription errors to their consequences for item-level scoring and motivates clinically grounded evaluation metrics.
Section II reviews related work, Section III describes the data and methods, Section IV reports the results, Section V discusses implications and limitations, and Section VI concludes.

III. Materials and Methods

A. System Overview

Figure 1 summarizes the pipeline. A UDS assessment is administered by a clinical researcher or by the Zeus2Q humanoid robot, and the session audio is recorded. Recordings are preprocessed, segmented by speaker, transcribed turn by turn, and merged into a speaker-attributed, time-stamped JSON record in which each segment carries a UDS task label. This record is the interface to automated scoring and to clinical data systems. Stages 1–5 are implemented; item scoring (stage 6) is planned.

B. Assessment Instruments

The pipeline targets UDS version 4 (UDS-4). The pilot uses Form B7, the NACC Functional Assessment Scale (FAS) [26], which is adapted from the Functional Activities Questionnaire [27]. Form B7 rates ten instrumental activities of daily living, such as paying bills, preparing a balanced meal, and remembering appointments, on a closed set of responses: not applicable, normal, has difficulty but does by self, requires assistance, dependent, and unknown. Because this response vocabulary is closed, a correct transcript maps unambiguously to an item code, which makes B7 a convenient first test of transcription-to-score fidelity. The form is completed from information provided by a co-participant (study partner) rather than from the participant’s own test performance, so the respondent in a B7 session is normally the informant. The task-mapping schema also defines labels for neuropsychological battery tasks recorded on Form C2, namely verbal fluency, story recall, and number span [3], whose open-ended responses must be transcribed verbatim and scored with task-specific rules; these tasks are the next target. Diarization was additionally exercised on two-speaker administrations of MMSE and MoCA items.

C. Participants and Recordings

The pilot dataset comprises recordings collected from 15 volunteer participants, together with self-recorded samples from members of the research team. All participants were volunteers rather than patients, and informed consent was obtained from each individual prior to data collection. The recordings consist of spoken responses to tasks derived from UDS-4 Form B7 and were used to evaluate the diarization and automatic speech recognition components of the proposed pipeline. In total, the dataset contains [insert total number of recordings] recordings, representing approximately [insert total duration] minutes of speech. Reference transcripts, where available, are stored in the dataset manifest described in Section III-D. The reference transcripts were [insert who produced the transcripts, e.g., manually prepared by the research team] and were subsequently [insert verification procedure, e.g., reviewed against the original audio by a second researcher for accuracy] before being used for ASR evaluation and model adaptation.
Planned study. Following the planned Patient-Centered Outcomes Research Institute (PCORI) study protocol, approximately 200 participants will be recruited at Lamar University (LU) and Indiana University (IU) over three years. Eligible participants are aged 65 years or older, fluent in English, and able to provide informed consent; individuals with significant hearing or vision impairment are excluded because such impairments would compromise test administration. Assessments will follow the UDS protocol and will be administered by a clinical researcher or by the Zeus2Q robot.
Ethics:The pilot dataset consists of speech recordings collected from 15 adult volunteers. Informed consent was obtained from each participant prior to participation and audio recording. The institutional review status and permissible use of the pilot recordings for publication are being confirmed with the Lamar University Institutional Review Board.

D. Preprocessing

Recordings captured as .m4a files are converted with FFmpeg to single-channel, 16-kHz WAV files, the input format expected by both Whisper and pyannote.audio. Raw and converted files are stored in separate directories (wavs_raw and wavs16k). A manifest file (refs.csv) maps each file identifier to its reference transcript, where available, and to a training, validation, or test split. Python scripts verify that every audio file has a manifest entry and a matching reference transcript, and they rebuild the manifest when entries are missing.

E. Speaker Diarization

Diarization uses the pretrained pyannote/speaker-diarization-3.1 pipeline, which applies a powerset segmentation model over sliding windows, extracts an embedding for each locally detected speaker, and clusters the embeddings into global speakers [21,22]. The output is a set of time-stamped speaker turns in RTTM format. The anonymous speaker labels are then mapped to the administrator and respondent roles The anonymous diarization labels were mapped to conversational roles according to speaking order. In each administration, the speaker producing the initial assessment prompt was labeled as the administrator, while the other detected speaker was labeled as the respondent. The role assignment was then checked against the transcript content to ensure that prompts and responses were attributed consistently. Diarization accuracy is measured with the diarization error rate (DER),
DER = (TFA + TMISS + TCONF) / TREF
where TFA, TMISS, and TCONF are the durations of false-alarm speech, missed speech, and speaker confusion, and TREF is the total duration of reference speech [19]. DER will be computed with pyannote.metrics [28].

F. Speech Recognition

Each diarized turn is transcribed with Whisper medium.en, the 769-million-parameter English-only Whisper model [10]. Whisper encodes an 80-channel log-Mel spectrogram of each 30-s input window (3,000 frames) and decodes text with segment-level timestamps. Because turns are transcribed individually, each transcript inherits its time span and speaker role from the diarization output.

G. Participant-Level Fine-Tuning

To adapt Whisper to individual accents, speaking rates, and voice characteristics, the model is fine-tuned on short recordings from each participant with the Seq2SeqTrainer of the Hugging Face Transformers library [29]. Table I lists the pilot runs. Run A fine-tuned the model on six recordings for eight epochs; Run B continued optimization for 100 steps; and Run C trained for 100 steps on a single example (one 30-s input window with a 55-token target sequence) to verify the data pipeline and training loop. Runs B and C used a learning rate of 5 × 10−5. Training was tracked with Weights & Biases.
For participant s with N_s training utterances, speaker-adaptive Whisper fine-tuning estimates a personalized parameter set by minimizing the average negative log-likelihood of the participant-specific reference transcriptions:
θs* = arg minθ [ − 1 / Ns Σi=1Ns log pθ(yi | xi) ]
Here x_i denotes the i-th audio segment, y_i is its reference transcription, and pθ(y_i | x_i) is the sequence probability produced by Whisper. The formulation makes the adaptation explicitly participant-specific; generalization must therefore be evaluated on utterances excluded from that participant’s fine-tuning set.
Table I. Participant-Level Fine-Tuning Runs of Whisper medium.en.
Table I. Participant-Level Fine-Tuning Runs of Whisper medium.en.
Run Training data Schedule Training loss (first → last logged)
A 6 recordings 8 epochs 2.580 (epoch 1) → 0.512 (epoch 8)
B Continuation of Run A 100 steps 0.0219 (step 10) → 0.0000 (steps 40–100)
C 1 example (one 30-s window; 55 target tokens) 100 steps 0.2820 (step 10) → 0.0000 (steps 50–90)

H. JSON Integration and Task Mapping

Diarization and ASR outputs are merged into a JSON record in which each segment carries its start and end times, speaker role, transcribed text, and UDS task and item labels (Listing 1). Segments are assigned to UDS-4 tasks programmatically, producing a structured record that can be passed to scoring rules and exported to clinical data systems.
Listing 1. Illustrative JSON record for one Form B7 item (values are examples).
Preprints 237842 i001

I. Implementation

The pipeline is implemented in Python 3 with PyTorch, Hugging Face Transformers, and pyannote.audio. It was validated on Google Colab and on Lamar University’s high-performance computing cluster with an NVIDIA H200 GPU, confirming CUDA compatibility and consistent execution in both environments.

J. Evaluation Metrics

Transcription accuracy is measured with the word error rate,
WER = (S + D + I) / N
where S, D, and I are the numbers of substitutions, deletions, and insertions in a minimum-edit alignment of the hypothesis to the reference, and N is the number of reference words. The character error rate (CER) is defined analogously over characters. Both metrics are computed after lowercasing and removing punctuation, so that formatting differences are not counted as recognition errors. Because the clinical target is the score rather than the transcript, the planned evaluation will also measure item-level agreement between automatically derived and clinician-assigned codes, using percent agreement and Cohen’s κ [30].

IV. Results

A. End-to-End Operation

The pipeline processed pilot recordings end to end, from raw .m4a files to speaker-attributed JSON records, on both computing platforms. On two-speaker dialogues, including administrations of MMSE and MoCA items, inspection of the diarization output indicated that administrator and respondent turns were separated with little overlap. This observation is qualitative; DER has not yet been computed because manually annotated reference turns are not yet available.

B. Fine-Tuning Behavior

Figure 2 plots the training loss. In Run A, the loss decreased monotonically from 2.580 after the first epoch to 0.512 after the eighth. In Runs B and C, it fell below 10−3 within 20 and 40 steps, respectively, and it was logged as 0.0000 from step 40 (Run B) and step 50 (Run C) onward. Near-zero training loss on one or a few examples shows that a model of this size can memorize its training targets. This confirms that the data pipeline and training loop work, but it is not evidence of generalization, which can be established only on held-out recordings.

C. Transcription Accuracy

Table II reports transcription accuracy on a held-out evaluation recording using the final checkpoint from the 100-step fine-tuning run. The evaluation recording was excluded from the model's training data and was used only for post-training assessment. After normalization by lowercasing the transcription and removing punctuation, the fine-tuned model achieved a WER of 25.7%, consisting of two substitutions, six deletions, and one insertion against 35 reference words, and a CER of 18.1%. Our initial scoring lowercased the text but retained punctuation, which produced a WER of 31.4% and a CER of 19.0% because punctuation mismatches, such as “self,” versus “self.”, were counted as recognition errors. The normalized results therefore provide a more appropriate estimate of lexical transcription accuracy. Because a WER of 25.7% on short, closed-vocabulary responses remains substantially higher than the single-digit error rates reported for Whisper on clean read English [10], comparison with the zero-shot Whisper baseline on the same recording is necessary to determine whether participant-level fine-tuning improved recognition performance.
Table II. Transcription Accuracy on the Evaluation Recording.
Table II. Transcription Accuracy on the Evaluation Recording.
Model WER (%) CER (%)
Whisper medium.en, zero-shot [zero-shot WER] [zero-shot CER]
Whisper medium.en, fine-tuned 25.7 18.1
Both rows must use the same recording and the same normalization (lowercasing, punctuation removed).

D. Error Analysis

Table III aligns the eleven reference responses with the model output. Although UDS-4 Form B7 contains ten scored items, the pilot recording contains eleven responses because one item was repeated during the administration, resulting in an additional response that was retained in the reference transcript for transcription-error analysis. Ten of the eleven responses were recovered with their response category intact, including a morphological variant (“difficulties” for “difficulty”) and a truncated form (“Require assistance” for “Required assistance”) that category-level matching would absorb. One of four consecutive identical “Has difficulty but does by self” responses was deleted, and a spurious fragment, “Requires,” was inserted before the final “Require assistance.”
Table III. Response-level alignment of reference and asr output.
Table III. Response-level alignment of reference and asr output.
Pos. Reference ASR output Outcome
1 Not applicable Not applicable Correct
2 Normal Normal Correct
3 Not applicable Not applicable Correct
4 Has difficulty but does by self Has difficulty but does by self Correct
5 Has difficulty but does by self Has difficulties but does by self Variant; category kept
6 Has difficulty but does by self Has difficulty but does by self Correct
7 Has difficulty but does by self — Deleted
8 Dependent Dependent Correct
9 Required assistance Required assistance Correct
— — Requires Inserted fragment
10 Required assistance Require assistance Variant; category kept
11 Normal Normal Correct
Case and punctuation ignored. Positions 4–7 are identical, so the deletion cannot be attributed to a specific one of them.
These two structural errors matter more than their word counts suggest. If responses were assigned to form items in order of occurrence, the deletion would shift the subsequent responses by one position: “Dependent” would be assigned to an item whose reference answer is “Has difficulty but does by self,” and “Required assistance” to the item whose reference answer is “Dependent.” The result is at least two incorrect item codes before the inserted fragment happens to realign the sequence. WER counts the deletion as six word errors, exactly as it would count six errors that leave every score unchanged, and so cannot reveal which errors alter the clinical record. The deleted response was also one of a run of identical responses. If this sensitivity to repetition generalizes, it matters for verbal fluency tasks, in which repeated words are counted separately as repetition errors and must therefore be transcribed verbatim.

V. Discussion

The pilot shows that the components needed to automate UDS response capture, namely preprocessing, diarization, turn-level transcription, and structured output, can be integrated into a single pipeline that runs reproducibly on cloud and cluster hardware. It also yields three lessons for the next phase.
Transcript accuracy is not the end point. A single WER value mixes errors that are harmless for scoring with errors that change item codes. For formfs with closed response vocabularies such as B7, mapping each response to the nearest category label would absorb variants such as “difficulties,” and supplying the category vocabulary to Whisper as a decoding prompt may prevent such errors in the first place. Item assignment should be anchored to the administrator’s preceding prompt, which diarization makes available, rather than to the order of responses. Evaluation should report item-level agreement with clinician scoring alongside WER, following the comparison of automatic and manual scores used to validate CogMo [11].
Few-sample personalization needs safeguards against overfitting. Full fine-tuning of a 769-million-parameter model on a handful of recordings drives the training loss rapidly toward zero. Safer practice is to hold out each participant’s evaluation audio, freeze the encoder or use parameter-efficient methods such as low-rank adaptation (LoRA) [31], lower the learning rate, apply data augmentation such as SpecAugment [32], and stop training early on validation WER. Every fine-tuned result should be reported alongside the zero-shot model evaluated on the same held-out audio.
Robot administration simplifies part of the problem. When the Zeus2Q robot administers the test, the intervals of its own speech can be logged by its dialogue manager, so diarization reduces to detecting and attributing non-robot speech: the participant’s and, for informant-based forms such as B7, the co-participant’s. The logged prompt times also provide natural anchors for item assignment.
Limitations. The results rest on a single evaluation recording, a six-recording fine-tuning set, and volunteer speakers rather than older adults with cognitive impairment, whose speech may include long pauses, word-finding difficulty, and reduced intelligibility. ASR accuracy is known to degrade for older voices [24], and Whisper hallucinates more often in speech with long pauses [25]. Diarization has been assessed only qualitatively, B7 is an informant-based rating rather than a performance test, and no comparison with clinician scoring has yet been made. The present results should therefore be read as a functional validation of the pipeline, not as an estimate of its clinical accuracy.
Future work. The next phase will (i) annotate reference speaker turns and report DER; (ii) report normalized WER and CER on held-out recordings for zero-shot, fully fine-tuned, and parameter-efficiently fine-tuned models; (iii) implement category mapping and prompt-anchored item assignment and measure item-level agreement with clinician scoring; (iv) extend the pipeline to the Form C2 tasks of verbal fluency, story recall, and number span, with verbatim transcription and task-specific scoring; (v) explore a multimodal fusion layer that combines acoustic, linguistic, and timing features into a unified representation of performance; and (vi) move from offline batch processing to real-time operation on the Zeus2Q robot during the planned 200-participant study at LU and IU.

VI. Conclusion

We have presented a modular pipeline that turns recordings of UDS assessments into speaker-attributed, time-stamped, task-aligned JSON records using pyannote.audio diarization and Whisper transcription, with optional participant-level fine-tuning. The pipeline runs end to end on cloud and cluster platforms. A pilot on UDS-4 Form B7 recordings yielded a normalized WER of 25.7% and showed that one deleted response can corrupt more than one item score, while the training-loss dynamics showed how quickly few-sample fine-tuning memorizes its data. These findings define the evaluation protocol for the planned multi-site study: quantitative DER, held-out WER against a zero-shot baseline, and, above all, item-level agreement with clinician scoring.

Acknowledgments

The authors thank the volunteers who contributed recordings.

References

  1. D. L. Beekly et al., “The National Alzheimer’s Coordinating Center (NACC) database: The Uniform Data Set,” Alzheimer Dis. Assoc. Disord., vol. 21, no. 3, pp. 249–258, 2007.
  2. L. Besser et al., “Version 3 of the National Alzheimer’s Coordinating Center’s Uniform Data Set,” Alzheimer Dis. Assoc. Disord., vol. 32, no. 4, pp. 351–358, 2018.
  3. S. Weintraub et al., “Version 3 of the Alzheimer Disease Centers’ neuropsychological test battery in the Uniform Data Set (UDS),” Alzheimer Dis. Assoc. Disord., vol. 32, no. 1, pp. 10–17, 2018.
  4. K. Ding, M. Chetty, A. Noori Hoshyar, T. Bhattacharya, and B. Klein, “Speech based detection of Alzheimer’s disease: A survey of AI techniques, datasets and challenges,” Artif. Intell. Rev., vol. 57, no. 12, Art. no. 325, 2024. [CrossRef]
  5. K. C. Fraser, J. A. Meltzer, and F. Rudzicz, “Linguistic features identify Alzheimer’s disease in narrative speech,” J. Alzheimer’s Dis., vol. 49, no. 2, pp. 407–422, 2016.
  6. König et al., “Automatic speech analysis for the assessment of patients with predementia and Alzheimer’s disease,” Alzheimer’s Dement. Diagn. Assess. Dis. Monit., vol. 1, no. 1, pp. 112–124, 2015.
  7. H. Jin, D. Junghaenel, B. Orriens, P. Lee, and S. Schneider, “Developing early markers of cognitive decline and dementia derived from survey response behaviors: Protocol for analyses of preexisting large-scale longitudinal data,” JMIR Res. Protoc., vol. 12, Art. no. e44627, 2023. [CrossRef]
  8. R. Prakash, M. Dupre, T. Østbye, and H. Xu, “Extracting critical information from unstructured clinicians’ notes data to identify dementia severity using a rule-based approach: Feasibility study,” JMIR Aging, vol. 7, Art. no. e57926, 2024. [CrossRef]
  9. B. Hattink, R. Droes, S. Sikkes, E. Oostra, and A. Lemstra, “Evaluation of the Digital Alzheimer Center: Testing usability and usefulness of an online portal for patients with dementia and their carers,” JMIR Res. Protoc., vol. 5, no. 3, Art. no. e144, 2016. [CrossRef]
  10. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “Robust speech recognition via large-scale weak supervision,” in Proc. 40th Int. Conf. Mach. Learn. (ICML), PMLR, vol. 202, 2023, pp. 28492–28518.
  11. H.-H. Kong et al., “Development and validation of a self-administered computerized cognitive assessment based on automatic speech recognition,” PLoS ONE, vol. 19, no. 12, Art. no. e0315745, 2024. [CrossRef]
  12. D. ter Huurne et al., “Validation of an automated speech analysis of cognitive tasks within a semiautomated phone assessment,” Digit. Biomark., vol. 7, no. 1, pp. 115–123, 2023.
  13. D. Kim et al., “Usefulness of automatic speech recognition assessment of children with speech sound disorders: Validation study,” J. Med. Internet Res., vol. 27, Art. no. e60520, 2025. [CrossRef]
  14. S. Luz, F. Haider, S. de la Fuente, D. Fromm, and B. MacWhinney, “Alzheimer’s dementia recognition through spontaneous speech: The ADReSS challenge,” in Proc. Interspeech, 2020, pp. 2172–2176.
  15. Di Nuovo, S. Varrasi, A. Lucas, D. Conti, J. McNamara, and A. Soranzo, “Assessment of cognitive skills via human–robot interaction and cloud computing,” J. Bionic Eng., vol. 16, no. 3, pp. 526–539, 2019.
  16. J. T. Becker, F. Boller, O. L. Lopez, J. Saxton, and K. L. McGonigle, “The natural history of Alzheimer’s disease: Description of study cohort and accuracy of diagnosis,” Arch. Neurol., vol. 51, no. 6, pp. 585–594, 1994.
  17. M. F. Folstein, S. E. Folstein, and P. R. McHugh, “‘Mini-mental state’: A practical method for grading the cognitive state of patients for the clinician,” J. Psychiatr. Res., vol. 12, no. 3, pp. 189–198, 1975.
  18. Z. S. Nasreddine et al., “The Montreal Cognitive Assessment, MoCA: A brief screening tool for mild cognitive impairment,” J. Am. Geriatr. Soc., vol. 53, no. 4, pp. 695–699, 2005.
  19. T. J. Park, N. Kanda, D. Dimitriadis, K. J. Han, S. Watanabe, and S. Narayanan, “A review of speaker diarization: Recent advances with deep learning,” Comput. Speech Lang., vol. 72, Art. no. 101317, 2022.
  20. H. Bredin et al., “pyannote.audio: Neural building blocks for speaker diarization,” in Proc. IEEE Int. Conf. Acoust., Speech, Signal Process. (ICASSP), 2020, pp. 7124–7128.
  21. Plaquet and H. Bredin, “Powerset multi-class cross entropy loss for neural speaker diarization,” in Proc. Interspeech, 2023, pp. 3222–3226.
  22. H. Bredin, “pyannote.audio 2.1 speaker diarization pipeline: Principle, benchmark, and recipe,” in Proc. Interspeech, 2023, pp. 1983–1987.
  23. M. Bain, J. Huh, T. Han, and A. Zisserman, “WhisperX: Time-accurate speech transcription of long-form audio,” in Proc. Interspeech, 2023, pp. 4489–4493.
  24. R. Vipperla, S. Renals, and J. Frankel, “Ageing voices: The effect of changes in voice parameters on ASR performance,” EURASIP J. Audio, Speech, Music Process., vol. 2010, Art. no. 525783, 2010.
  25. Koenecke, A. S. G. Choi, K. X. Mei, H. Schellmann, and M. Sloane, “Careless Whisper: Speech-to-text hallucination harms,” in Proc. ACM Conf. Fairness, Accountability, Transparency (FAccT), 2024, pp. 1672–1681.
  26. National Alzheimer’s Coordinating Center, “UDS version 4.0, Initial Visit Packet, Form B7: Functional Assessment—NACC Functional Assessment Scale (FAS),” Jan. 2025. [Online]. Available: https://files.alz.washington.edu/documentation/uds4-ivp-b7.pdf.
  27. R. I. Pfeffer, T. T. Kurosaki, C. H. Harrah, J. M. Chance, and S. Filos, “Measurement of functional activities in older adults in the community,” J. Gerontol., vol. 37, no. 3, pp. 323–329, 1982.
  28. H. Bredin, “pyannote.metrics: A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems,” in Proc. Interspeech, 2017, pp. 3587–3591.
  29. T. Wolf et al., “Transformers: State-of-the-art natural language processing,” in Proc. Conf. Empirical Methods Natural Lang. Process.: Syst. Demonstrations (EMNLP), 2020, pp. 38–45.
  30. J. Cohen, “A coefficient of agreement for nominal scales,” Educ. Psychol. Meas., vol. 20, no. 1, pp. 37–46, 1960.
  31. E. J. Hu et al., “LoRA: Low-rank adaptation of large language models,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2022.
  32. D. S. Park et al., “SpecAugment: A simple data augmentation method for automatic speech recognition,” in Proc. Interspeech, 2019, pp. 2613–2617.
Preprints 237842 g001
Figure 1. Pipeline overview. Stages 1–5 are implemented; stage 6 (dashed outline) is planned. In stage 1, the assessment is administered by a clinical researcher or by the Zeus2Q robot.
Figure 1. Pipeline overview. Stages 1–5 are implemented; stage 6 (dashed outline) is planned. In stage 1, the assessment is administered by a clinical researcher or by the Zeus2Q robot.
Preprints 237842 g001
Preprints 237842 g002
Figure 2. Training loss of Whisper medium.en during participant-level fine-tuning, from the training logs. (a) Run A (six recordings), per epoch. (b) Runs B and C, per logged step; the collapse to zero indicates memorization of the training targets.
Figure 2. Training loss of Whisper medium.en during participant-level fine-tuning, from the training logs. (a) Run A (six recordings), per epoch. (b) Runs B and C, per logged step; the collapse to zero indicates memorization of the training targets.
Preprints 237842 g002
Disclaimer/Publisher’s Note: The statements, opinions and data contained in all publications are solely those of the individual author(s) and contributor(s) and not of MDPI and/or the editor(s). MDPI and/or the editor(s) disclaim responsibility for any injury to people or property resulting from any ideas, methods, instructions or products referred to in the content.
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.