Preprint
Article

This version is not peer-reviewed.

Recall Under Paraphrase for Trust-Locked Patient Memory: A Blind Benchmark, Hybrid Lexical–Embedding Matching and Training-Free Hand-Set Heads

Submitted:

18 November 2026

Posted:

19 November 2026

You are already at the latest version

Abstract
A simulated psychiatric patient must identify what a learner is asking about before it can decide whether to disclose a case detail. We evaluated this matching step on 512 AI-generated questions across four fictional psychiatric cases. The benchmark writer saw descriptions of the case memories but had no access to the matching rules. We reserved 248 questions for development and 264 for testing. On the test set, keyword matching recalled 35.1% of expected risk memories. Adding a small sentence encoder raised recall to 72.7%, while the share of questions about non-risk memories incorrectly matched to a risk memory rose from 6.1% to 15.2%. The hybrid matcher took a median of 2.1 ms per question on a laptop CPU. Adding hand-set heads raised recall to 77.9% and the false-hit rate to 21.2%. All methods fell short of the 95% recall target. The hybrid matcher scored 90.7% on the development set, where we also used missed questions to write probes. A separate attempt to maximize recall reached 100% on development data at a 36.9% false-hit rate. Tested in scripted conversations, that configuration sometimes prompted unasked disclosures. The results show why evaluation should report both recall and false hits on held-out questions.
Keywords: 
;  ;  ;  ;  ;  
Copyright: This open access article is published under a Creative Commons CC BY 4.0 license, which permit the free download, distribution, and reuse, provided that the author and preprint are cited in any reuse.