Standardizing Physician Notes Improves Accuracy of Clinical Concept Extraction Without Information Loss

1. Introduction

Electronic Health Records (EHRs) have transformed healthcare documentation by improving the legibility and availability of patient data [1]. They solved the “availability problem”—where paper charts were often inaccessible—and the “legibility problem” of handwritten notes [2,3,4]. Yet, physician dissatisfaction with EHRs remains high, driven by poor interface design, excessive documentation burden, clerical overload, and limited perceived benefit for patient care [5,6,7,8,9,10,11,12].

Since the release of ChatGPT in late 2022, interest in large language models (LLMs) has surged in healthcare [13,14,15,16,17]. LLMs are now being explored for a range of clinical applications: decision support, diagnosis explanation, summarization, concept extraction, ontology mapping, and transforming documents into interoperable formats [18,19,20,21,22,23]. Among these, LLM-based clinical note standardization holds special promise for improving note quality and downstream reusability. Some recent surveys show increasing physician satisfaction with EHRs due to improvements referable to AI [24]/

We define standardization as the process of improving the structural and linguistic integrity of a clinical note without altering its clinical meaning. This includes correcting spelling and grammar, replacing colloquialisms with standardized terms (e.g., upgoing toe →Babinski sign), expanding abbreviations (e.g., OU→ both eyes), and enforcing consistent formatting and section headers.

Clinical notes are typically written through direct entry, dictation, copy-paste, or auto-insertion of structured fields. As a result, they often suffer from inconsistent structure, misspellings, excessive abbreviations, slang, and ambiguous terminology [25,26,27,28,29,30]. Up to 20% of note content may be abbreviations or acronyms [31,32,33,34,35,36,37,38]. Moreover, many notes lack machine-readable codes (e.g., ICD-11, SNOMED CT, LOINC, RxNorm) and are poorly formatted or excessively verbose [39,40,41,42].

LLMs can help address these limitations by:

• Expanding abbreviations based on context (e.g., MS” → multiple sclerosis’’ or mental status”).
• Correcting typographical and grammatical errors to improve clarity [43].
• Replacing colloquialisms and dictation artifacts with standardized terminology (e.g., heart attack’’ → “myocardial infarction’’) [44].
• Structuring notes into canonical sections (e.g., History, Examination, Impression, Plan) to enhance navigability.

These improvements (Figure 1) can:

• Enhance the readability and clinical utility of notes.
• Improve the accuracy of pipelines that extract and normalize medical concepts to standard ontologies.
• Facilitate conversion of clinical notes into structured formats for data exchange such as Fast Healthcare Interoperability Resources (FHIR) [45,46,47,48].

2. Methods

This study evaluates the effectiveness of GPT-4 in standardizing 1,618 de-identified clinical notes from a Neurology Clinic. We hypothesize that standardization will improve structural consistency, linguistic clarity, and the extractability of clinical concepts, thereby enhancing downstream applications such as ontology-based normalization, natural language processing (NLP), and data interoperability.

The evaluation framework (Figure 2) compares concept extraction accuracy under three conditions: from raw notes, from GPT-4-standardized notes, and from ground truth phrases. We define concept extraction as identifying relevant clinical terms or phrases in free text. Concept normalizationrefers to mapping each extracted term to a standardized ontology concept with its corresponding identifier. Notably, correct identification of a term does not guarantee accurate normalization, as models may capture semantic meaning without retrieving the correct machine-readable code [49,50,51,52,53,54,55].

Data Acquisition: Institutional Review Board (IRB) approval was obtained from the University of Illinois to use de-identified clinical notes from the Neurology Clinic. All notes were de-identified using REDCap [56]. We selected notes containing neuroimmunological diagnoses, including multiple sclerosis, myasthenia gravis, neuromyelitis optica, and Guillain–Barré syndrome. From an initial pool of 21,028 notes (2016–2022), a subset meeting the following inclusion criteria was selected: (1) outpatient visits, (2) authored in the Neurology Clinic, (3) written by physicians (residents or attendings), and (4) minimum note length of 2,000 characters. The final dataset comprised 1,618 typed notes exported as ASCII text and stored in a JSON-compatible format (Figure 3).

Note Standardization: GPT-4o (Open AI) was prompted to standardize neurology notes (Appendix A) into a standard format (Appendix B). GPT-4 produced a detailed list of all modifications for each note (Appendix C). Character count, word count, sentence count, spelling and grammar corrections, abbreviation expansion, and word substitutions were calculated for each note (Figure 4). To evaluate fidelity, 100 randomly selected note pairs were manually reviewed to ensure no clinically significant information was omitted and to screen for any spurious additions, such as hallucinations or confabulations.

Concept Extraction Using doc2hpo: We used the open-source Python implementation of doc2hpo [57], executed locally with the Aho-Corasick string-matching algorithm and a custom negation detection module (`NegationDetector`) that applies a windowed keyword search within sentence boundaries. We used the HPO release dated 2025-03-03. No modifications were made to the doc2hpo source code. doc2hpo was applied under three conditions:

• Raw notes (unmodified exports from REDCap),
• Standardized notes (post-GPT-4 processing),
• Ground truth phrases (GPT-4 extracted candidate HPO terms).

For the ground truth, GPT-4 was prompted to identify all candidate HPO-eligible phrases in each note (Appendix D). These phrases were combined into a single synthetic sentence for doc2hpo input (e.g., “The patient has ataxia, weakness, aphasia, and hyperreflexia.”). doc2hpo returned a list of matched terms and corresponding HPO IDs. For each of the 1,618 notes, we compiled three term sets:

• Ground truth terms: GPT-4 extracted phrases successfully matched by doc2hpo.
• Raw-note terms: doc2hpo output from raw notes.
• Standardized-note terms: doc2hpo output from standardized notes.

Metric Computation: For each extracted term, we classified its match status as follows:

• True Positive (TP): Term present in both the extracted set and the ground truth set.
• False Negative (FN): Ground truth term not captured in extraction set.
• False Positive (FP): Term extracted but not part of ground truth set.
• True Negative (TN): Term present in the note but not successfully mapped to HPO by doc2hpo.

Performance metrics (precision, recall, accuracy, and F1 score) were calculated by the method of Powers [58].

3. Results and Discussion

3.1. Note Standardization Improves Structure and Readability

We used GPT-4 to standardize 1,618 de-identified clinical notes from a neurology clinic. Each note was converted from plaintext to a structured JSON format with high-level section headers (e.g., History, Examination, Impression, Plan). GPT-4 was prompted to standardize each note by correcting spelling and grammatical errors, expanding acronyms and abbreviations, and replacing non-standard phrasing (Appendix E). GPT-4 corrected and average of 4.9 ± 1.8 grammatical errors, 3.3 ± 5.2 spelling errors, and expanded 12.6 ± 8.1 abbreviations or acronyms per note. Additionally, it substituted standardized clinical terminology for 3.1 ± 3.0 instances of jargon or non-standard language. A manual review of 100 randomly selected notes confirmed the preservation of clinical content, with improvements in clarity, structure, and overall readability. No hallucinations or spurious additions were identified during human expert review.

3.2. Standardization Improves HPO Term Extraction by doc2hpo

To evaluate whether GPT-4-based standardization improves phenotype term extraction, we compared three parallel data flows:

• Raw Notes → doc2hpo → Raw Terms
• Standardized Notes → doc2hpo → Standardized Terms
• GPT-4 Extracted Phrases → doc2hpo → Ground Truth Terms

All extracted terms were mapped to the HPO using doc2hpo. The ground truth was defined as HPO terms extracted by doc2hpo from GPT-4-supplied candidate phrases. For each note, term sets from the raw and standardized notes were evaluated against the corresponding ground truth set using standard classification metrics: precision, recall, accuracy, and F1 score.

GPT-4-standardized notes yielded higher precision and accuracy compared to raw notes. Recall also improved, rising from 0.61 to 0.68. Although modest in absolute terms, this gain was statistically significant due to the large sample size (n = 1,618) and contributed to a measurable improvement in F1 score (Figure 3). The combined improvements suggest that GPT-4 preprocessing enhances overall extraction performance without sacrificing sensitivity. The simultaneous increase in both precision and recall indicates that GPT-4 did not introduce spurious terms or omit clinically relevant information. These results support the hypothesis that physician note standardization by LLMs improves the performance of NLP pipelines performing ontology-term normalization.

3.3. Downstream Use Cases After Note Standardization

Once physician notes are standardized by a large language model, several downstream applications become feasible (Figure 1):

• Standardized notes can be integrated in real time into the EHR to enhance note quality, readability, and clinical interpretability.
• Extracted terms can be passed into NLP pipelines that map these terms to ontology concepts, making clinical data computable for exploratory analysis, machine learning, population health, and quality improvement.
• Normalized concepts can be restructured as FHIR resources to support efficient data exchange [45,46,47,48,59].

Major medical centers are already exploring the use of LLMs to analyze free text in the EHR, driving predictive analytics [60] and improving documentation quality among house officers [61].

3.4. Addressing Systemic Challenges in EHR Documentation

While large language models improve note structure and readability, systemic challenges in EHR documentation remain. Copy-and-paste practices, unverified carry-forward data, and documentation that omits or exaggerates care are beyond the scope of current AI tools [62,63,64]. Standardization alone does not reduce documentation burden, which is more fundamentally tied to EHR design, clinician workflows, and organizational practices [10,65,66,67]. Promising technologies like ambient AI may help, but broader reforms are needed to address who documents, what is documented, and how. Encouragingly, recent surveys suggest that the introduction of AI tools is already improving physician satisfaction with the EHR [24].

3.5. Limitations

This study analyzed 1,618 de-identified neurology notes, primarily from patients with multiple sclerosis. Broader validation across diverse diagnoses, note types, and clinical settings is needed. Ground truth medical concepts were identified using GPT-4 with human expert review. While this approach enabled scalable evaluation, future work should incorporate fully manual annotation for comparison. We did not assess the processing time or computational costs of standardization. Although deploying LLMs in production may incur costs, many institutions are actively exploring this path [61,68]. De-identification ensured HIPAA compliance, but explicit consent from clinicians and patients was not obtained. In active clinical environments, reconfiguration of notes may require additional safeguards or institutional approval. Nevertheless, our results support the growing interest in note reconfiguration, template engineering, and documentation quality. Once digitized, clinical text can be adapted to multiple formats, enhancing interoperability and downstream utility [61,69,70,71,72].

4. Conclusions

Standardizing physician notes with large language models improves their readability, consistency, and interoperability without loss of clinical content. In this study, GPT-4 enhanced note structure, corrected errors, and expanded abbreviations, facilitating more accurate extraction and normalization of medical concepts by doc2hpo. These findings suggest that LLMs can augment both clinical and research workflows by making unstructured text more usable for downstream NLP and ontology-based analysis. As adoption grows, early evidence indicates that such tools may also contribute to improved physician satisfaction with electronic health records [24].