Predictive Inpatient Screening Tool Improves Outcomes for Children at Risk for Infection-Related Decompensation

1. Introduction

Sepsis continues to be a major cause of death in children worldwide, with survivors often suffering increased rates of readmissions and diminished long-term quality of life[1,2,3,4,5]. While international consensus definitions of pediatric sepsis have continued to evolve[6,7,8,9,10,11], these advances largely serve to define improved targets for the development of early screening tools rather than as metrics to be incorporated directly as diagnostic indicators[12]. Although a number of sepsis screening tools have been designed and are in widespread use, a large majority of these are focused on the screening of patients in the emergency department (ED)[13], as the progression of sepsis cases to organ dysfunction and physiological decompensation often occurs within 24 hours of hospital arrival. However, due to the relative lack of clinical data available at the time of ED arrival and to the inherently low incidence of Phoenix criteria sepsis[9,10,11] (previously called severe sepsis), any such tool that is programmed into the electronic health record (EHR) and employed to screen all ED patients will invariably have a low positive predictive value (PPV)[14], often leading to trigger fatigue. Organizations have ameliorated this shortcoming by implementing tiered alert systems whereby a lower-level automated alert triggers an additional manual screening and possible huddle, leading to higher stage alerts that have improved sensitivity, specificity, and PPV[13,14,15,16].

Sepsis screening tools designed for use in general, non-ICU inpatient pediatric units are less common[13,17,18,19] but are likely to show much better performance relative to ED-based alerts due to the availability of more detailed clinical inputs, collected and documented over a longer period of observation. Following our earlier implementation of an electronic screening alert in the ED setting[20,21], we have now created an improved electronic tool for use in non-ICU, inpatient units. The development of this tool was based on an approach that began with a broader definition of “infection-related decompensation,” which focuses on identifying those cases likely to progress to an advanced level of decompensation severity and so would benefit from early recognition and treatment regardless of whether they would meet the pediatric Sequential Organ Failure Assessment (pSOFA) or Phoenix criteria, while also exhibiting high sensitivity in predicting Phoenix- or pSOFA-defined sepsis and high overall severity-based predictive value. Additionally, we present the alert tool’s validation, EHR refinements to enhance clinician awareness and communication of alerts, and improvements in clinical outcomes following implementation.

2. Methods

2.1. Study Setting

The electronic assessment tool described here, called the Inpatient Children at High Risk Alert Tool (iCAHR-AT), was developed and implemented over the period 2018-2025 at the Children’s Hospital of The King’s Daughters Health System (CHKDHS), a 266-bed freestanding children’s hospital located in Norfolk, Virginia, USA with ~9,000 inpatient (IP) and observation (Obs) admissions per year. Input data for the creation and operation of the assessment tool were derived from historical and real-time data collected during the patient stay as recorded in the hospital’s Cerner Millennium® (Cerner Corporation [now Oracle Health®], Kansas City, MO, USA) EHR.

2.2. Retrospective Gold Standard Case Identification for Tool Calibration

“Gold standard” (GS) cases of infection-related decompensation (IRD) were identified from a study population of 10,992 IP and Obs admissions to the hospital admitted and discharged during 01/2018 – 12/2019. Based on the tool’s design for use in general care inpatient units and on the consensus of the hospital’s subspecialty physician groups, we excluded patients primarily assigned to one of the following medical services or locations: Cardiology, Cardiac Surgery, Hematology/Oncology, Inpatient Psychiatry, and Neonatal Intensive Care Unit (NICU).

We selected retrospective GS IRD cases from this 2018-19 IP/Obs study population based on the following criteria: (1) Evidence within the previous 7 days of a confirmed or suspected infection having potential for a systemic response (listed in Table 1); (2) Evidence of one or more organ dysfunctions (OD’s) as defined by the pSOFA scoring method developed by Matics and Sanchez-Pinto[7], which we modified (see supplement S1) for use in general care units outside the ICU[22,23]; (3) Elimination of low severity cases as denoted by an outcome of “Minor” Severity of Illness (SOI) as assigned by the 3M™ Corporation’s All Patient Refined-Diagnosis Related Groups (APR-DRG, v36) algorithm[24] at the time of coding (post-discharge); (4) Chart review by clinicians of cases meeting the above criteria to eliminate chronic OD cases, other conditions that may mimic sepsis (such as ingestions and autoimmune or metabolic conditions), and cases where decompensation appeared unrelated to infection (such as those following and directly related to major surgery or trauma).

For each identified case of IRD, reviewers also determined the initial time of acknowledged decompensation, defined as the earliest time of: Rapid Response Team (RRT) called, unplanned transfer to the ICU, or a pSOFA determination of an acute OD.

2.3. Tool Creation

A flow diagram illustrating the steps involved in creating the iCAHR-AT algorithm is presented in Figure 1.

Data elements (listed in Supplement S2) were collected for the study population from the Cerner Millennium EHR using SAP® Web Intelligence Rich Client software. Preliminary analysis and tool development were conducted using SAS® v9.4 software.

In addition to the exclusion of patients having care primarily assigned to the specialty services listed above, we also excluded data elements collected during periods when patients were physically located in any of the following units: Sedation, Surgery Operating Room (OR), Post-Anesthesia Care Unit (PACU), and Pediatric Intensive Care Unit (PICU). An exception was made for the collection and analysis of microcultures or viral panels, which were accepted from all units or services.

Candidates for predictors of IRD were selected based on literature sources such as the Sepsis-2[6] or the pSOFA[7] definitions, our own earlier screening tools[20,25], preliminary univariate analysis, and clinician consensus. For purposes of screening tool creation, we considered most predictive factors as dichotomous variables, with thresholds of abnormality taken from the above sources or from age-specific 5^th or 95^th centiles.

Based on our previous work [26], we utilized de-identified Cerner Health Facts® (now Oracle Health Data Intelligence®) vital signs data from >200 hospitals for children aged 0-17 years collected during 2014-2017 to derive age-specific centiles and z-scores (normalized standard centiles) for heart rate (HR), respiratory rate (RR), SBP shock index (“SBP-SI” = HR ÷ systolic blood pressure), MAP shock index (“MAP-SI” = HR ÷ mean arterial pressure) and temperatures (oral [oTMP], rectal [rTMP], and temporal artery [taTMP]). We developed these metrics from distributions generated using Generalized Additive Models for Location, Scale, and Shape (GAMLSS) methodology[27,28] and software written in the R programming language. The age-specific modeled centiles for HR, RR, SBP-SI and MAP-SI are shown in supplemental tables S3-S6, respectively, and modeled z-scores for HR, RR, oTMP, rTMP, and taTMP are shown in supplemental tables S7-S11.

Preliminary development of the iCAHR-AT algorithm modeled the association of the identified GS IRD cases with abnormal values of the factors listed in Supplement S2 using a LASSO penalized logistic (binary) regression. After identifying a group of preliminary significant factors, we added all first-order interactions among these factors (e.g., the co-occurrence of abnormal heart rate and prolonged capillary refill time, or of abnormal values of temperature and white blood count (WBC), etc.). In concordance with our previous screening tools[20,25], we considered abnormal values for vital signs (HR, RR. SBP[29], MAP, TMP), pulse oximetry, neurological tests, and respiratory measures (e.g., bronchiolitis score), as well as maximum vital sign z-scores, as valid for 24 hours; lab results for 48 hours; and positive indicators of infection (coded diagnoses, cultures, viral panels, or chest x-ray results) for 7 days.

2.4. Tool Refinement Process

To create a more predictive algorithm that would preferentially select “early” rather than “late” predictors of IRD, and cases that would reach a higher level of physiological decompensation (as defined by the APR-DRG SOI metric[24]) we performed improvement cycles as follows.

First, for all patients in the study, we calculated the time of “maximum abnormal factors,” defined as the initial time at which the maximum number of factors identified in the preliminary analysis above exhibited abnormal values (as defined in Supplement S2).

Next, we created a SAS® model that simulated the real time clinical events during each patient encounter included in the study sample. The simulation duration for a given encounter varied according to the initial time of maximum abnormal factors (MAF) for that encounter. If the MAF time occurred less than 24 hours following IP/Obs admission, we ended the simulation at the MAF time. If the initial MAF occurred between 24 and 48 hours following admission, we ended the simulation at 6 hours prior to MAF; and for initial MAF greater than 48 hours post-admission, we ended the simulation 12 hours prior. This new “early factor” model was then re-analyzed using the LASSO penalized logistic regression technique to identify a new set of significant factors and weights for earlier prediction of the GS IRD cases.

Further improvement cycles were performed, with emphasis on predicting high severity cases, i.e., those progressing to more extreme levels of physiological decompensation. Although more recent definitions of pediatric sepsis no longer require the presence of systemic inflammatory response syndrome (SIRS), our previous work in creating sepsis screening tools [20,25] suggests that SIRS is associated with higher severity outcomes. Therefore, we performed an additional LASSO regression analysis using the “early factor” methodology described above to predict a new set of GS IRD cases identified as those in the original set that also manifested SIRS (defined in Supplement S2) before the initial time of MAF. These cases, referred to as “GS SIRS IRD,” were used to create and train the final model.

2.5. Final Model

After creating a preliminary optimized real-time model (with associated Receiver Operating Characteristic [ROC] curve) using the processes described above, we selected an optimum firing threshold for our hospital based on consideration of several factors: ROC factors (sensitivity, specificity); PPV for cases of “Major” or “Extreme” SOI; and time between alert firing and maximum physiologic decompensation (i.e., the “treatment window”).

To maintain consistency with our current ED version of the Children at High Risk Alert Tool (eCAHR-AT), which has been operational in our hospital’s ED since August 2017, the weights of the final, optimized algorithm were standardized to set the firing threshold of 5.0, matching that of our ED algorithm.

2.6. Validation and Testing

2.6.1. Internal Validation

We performed a split-sample validation of the final model derived from our 2018-19 study sample by comparing its performance across patient admissions during 2018 with those during 2019. Using logistic regression to generate ROC curves representing iCAHR-AT alert firing as the predictor of GS SIRS IRD cases, we compared the area under the curve (AUC) estimates for the two subsamples and evaluated the difference using an unpaired t-test[30].

2.6.2. External Validation

Beginning in May 2023, we implemented a prototype of the iCAHR-AT model in a “silent” mode, whereby we collected information relevant to the scoring and firing of the iCAHR-AT for all IP/Obs patients currently being treated by services and in units where the alert tool would become active and visible in the EHR to clinicians following full implementation later in January 2024.

Using the identical criteria employed for our original 2018-19 study population, both for inclusion in the study sample and for identification as GS infection-related decompensation cases, we identified a patient sample of 4,050 IP/Obs encounters admitted between 05/14/2023 and 01/22/2024. Identification of GS SIRS IRD cases in the external validation dataset was performed without knowledge of the iCAHR-AT’s scoring or firing. As was the case for encounters in the 2018-19 sample, data collected for the external validation sample also included information on the initial time of maximum abnormal factors (MAF) and, for GS SIRS IRD cases, whether patients manifested SIRS before the time of MAF.

We compared the ROC curves of the original 2018-19 tool calibration and the external validation datasets using the same methodology employed in the internal validation described above, using an unpaired t-test to compare the AUC estimates of the two study samples. For the external validation sample, the ROC curve represents iCAHR-AT firing as the predictor of the new external validation GS SIRS IRD cases.

2.7. Association of iCAHR-AT Score with Severity of Illness

For each encounter in the original 2018-19 study sample and the 05/2023-01/2024 external validation sample, we determined the maximum iCAHR-AT score attained during the patient stay and examined the association between maximum score and the occurrence of high-severity outcomes, defined as a final APR-DRG v36 SOI of “Major” or “Extreme”.

2.8. Live Implementation of iCAHR-AT

2.8.1. Initial Implementation

The iCAHR-AT was entered into live clinical use at CHKD in February 2024. Its initial appearance in the Cerner EHR “Powerchart” consisted of the “Results Review” display of iCAHR-AT scores and alert firings in real time, as well as a “Summary View” detailing the abnormalities that contributed to the current score. These are shown as an overview in Figure 2, with a more detailed description of the Summary View given in supplement S12.

2.8.2. Communication and Acknowledgment Refinements

In addition to the above notifications in the Cerner EHR, we introduced further improvements to enhance communication and clinician response to iCAHR-AT firings. These include: 1) automated paging and phone notification of unit charge nurse upon alert firing, whose task was to notify the patient’s current provider (often a resident physician) on call; 2) mandatory notification of the attending physician or advanced practice provider by the alerted provider; 3) automated opening of a Cerner “Powerform” (also detailed in supplement S12) upon the initial provider’s opening of the alerted patient chart, requiring the provider’s written assessment and acknowledgement.

The effectiveness of our efforts to increase provider awareness and response through the above refinement steps was evaluated, and compliance was measured with each refinement over the course of the study.

2.8.3. Resident Training

Because our institution is a pediatric teaching hospital with new residents rotating through each month, we also implemented a new resident training system that included verbal instruction, a training handout (supplement S12), and an introductory video.

2.8.4. Pre-Alert

Given the iCAHR-AT’s dependence on the availability of patient data such as vital signs, labs, cultures, and radiology results, we were concerned that a patient showing significant abnormalities might not trigger an alert simply due to the sparsity of available data. We therefore introduced a “Pre-alert” that would appear in the Powerchart Summary View when a patient reached an iCAHR-AT score of ‘4’, too low to trigger an alert but nevertheless of concern. The Pre-alert, detailed in supplement S13, would list those additional metrics used by the iCAHR-AT algorithm to perform a complete evaluation but had not been collected within the past 24 hours (for neurological and vital sign related results), 72 hours (for labs), or 7 days (for cultures, viral panels, or x-rays). As the Pre-alert serves only as an optional aid in suggesting additional metrics that might aid the iCAHR-AT in its evaluation, it does not trigger any paging or phone call, nor mandate a physician response.

2.9. Clinical Outcomes

2.9.1. Acuity and Severity

We hypothesized that the iCAHR-AT score correlates with both the current acuity and overall severity of the patient’s condition with respect to infection-related decompensation and that a decrease in the score over time reflects an improvement of the patient’s condition. We similarly hypothesized that the highest iCAHR-AT score attained during a patient’s stay correlates with the overall severity of illness and degree of physiological decompensation for that encounter. Therefore, we created a series of graphs, including boxplots and a time-series X-bar control chart, comparing outcomes for patients who fired the tool during the “silent” (pre-live) period with those who fired it after “live” implementation of the iCAHR-AT. All plots were created using Minitab® Statistical Software v.22 (State College, PA).

2.9.2. Boxplots

Two series of boxplots were created to compare outcomes between the Silent and Live epochs, with differences evaluated using two-sample t-tests. One shows the average 48-hour trend in iCAHR-AT score from the time of initial firing until 48 hours post-firing for all patients who fired the alert and remained in a general care, non-ICU unit for the entire 48-hour post-firing period. We calculated individual patient trends from a regression line for that encounter over the 48-hour period.

A second series of boxplots shows the average of the maximum patient scores attained at 48 hours or greater post-admission following the initial alert firing. As a balancing measure, we added boxplots for each of the two epochs showing the mean iCAHR-AT score at the initial time of firing.

2.9.3. Control Chart

To show the relationship of patient improvement as measured by 48-hour iCAHR-AT score trend with the refinements in communication and firing acknowledgement described above, we created a staged graph that shows the effect of each of the refinements, summarized above and in supplement S10, on the corresponding mean iCAHR-AT score change, by month and stage, with stages corresponding to the introduction of each new refinement.

2.9.4. Identification of Infectious Agents

To investigate whether the implementation of iCAHR-AT facilitated the correct identification of pathogens present in IRD patients, we examined all cases where the alert tool fired and microbiological cultures or polymerase chain reaction (PCR) panels were collected during the period when the tool was active. We compared the proportion of these cases that yielded a positive finding of an infectious agent between the Silent and Live epochs.

2.9.5. Transfers from General Care Units to the ICU

To explore whether implementing iCAHR-AT might aid in the management of infection-related decompensation on general care units and reduce the need for unplanned transfers of decompensating patients to the ICU, we compared the overall rate of post-firing floor-to-ICU transfers between the Silent and Live epochs.

2.9.6. Length of Stay

We assessed whether an improvement (reduction) in overall hospital length of stay (LOS) for patients firing the iCAHR-AT occurred following live implementation. To adjust for differences in severity of illness among cases, we obtained Medicare Severity-Diagnosis Related Group (MS-DRG) Average Length of Stay (ALOS) and Geometric Mean Length of Stay (GM-LOS) data for each patient who alerted during the Silent and Live epochs. These metrics are used by the U.S. Centers for Medicare and Medicaid Services (CMS) to determine resource and hospital stay reimbursement for Medicare and Medicaid patients and thus serve as the “expected” LOS metrics. The MS-DRG ALOS represents the arithmetic mean LOS for patients in a given MS-DRG severity group, and the GMLOS represents the corresponding geometric mean, which is less influenced by outliers. Actual LOS / MS-DRG ALOS and Actual LOS / MS-DRG GMLOS ratios were calculated separately for each patient, and overall differences in these actual-to-expected metrics between the Silent and Live periods were evaluated using a Wilcoxon Rank-Sum nonparametric test.

3. Results

3.1. Characteristics of Patients in the Study Samples

Characteristics of patients in our study are summarized in Table 2 as three groups: 1) Patients in the original 2018-19 tool calibration sample used in iCAHR-AT model development; 2) Patients in the 05/2023 - 01/2024 sample, used as subjects for the external validation sample and as the patient population for the “silent” epoch of tool implementation; 3) Patients in the 02/2024 – 05/2025 sample, comprising the patient population for the “live” epoch of tool implementation. The table shows that, while “sepsis” as a final ICD-10 diagnosis is rare (0.6% – 1.7%) among all of the study cohorts, a final diagnosis of an infection having the potential for a systemic response (as listed in Table 1) is much more common (23% - 28%). The existence of a large patient population with potential for infection-related decompensation, combined with the high percentage of iCAHR-AT alert firings with high severity outcomes (37% – 59% in the pre-live epochs), supports our goal of creating a tool with broader ability to predict infection-related decompensation compared with other existing sepsis screening alerts.

3.2. Characteristics of the iCAHR-AT Final Model

Table 3 lists the factors and their corresponding weights selected as significant predictors of infection-related decompensation in the final model. All factors except z-scores are dichotomous, with values expressed as the listed weights if abnormal according to the definitions given in Supplement S2. For factors representing interactions, the additional weight listed is added if both interacting factors were abnormal during the periods when each was valid. For z-scores, the final weight is calculated as the listed weight factor multiplied by the square of the abnormal z-score, with abnormality defined as ≥2 SD above normal for RR or HR, and ≥2 SD above or below normal for TMP.

3.3. Internal Validation

Our split-sample internal validation of the iCAHR-AT model evaluated the null hypothesis of no difference in AUC estimates between the 2018 and 2019 cohorts of our two-year calibration study sample. The results were, for the 2018 time period: N=5,479, AUC=0.9114, standard error (SE)=0.0242; for the 2019 time period: N=5,513, AUC=0.8795, SE=0.0324. The AUC difference (0.0319) is not statistically significant (p=0.43) and does not allow rejection of the null hypothesis.

3.4. External Validation

The external validation of the model also evaluated the null hypothesis of no difference in AUC estimates between the original calibration sample and the external validation sample. The results for the original 2018-19 calibration sample were: N (total)=10,992, N (GS SIRS IRD)=83, AUC=0.8972, SE=0.0195; for the external validation 05/23-01/24 sample: N(Total) = 4,050, N (GS SIRS IRD)=50, AUC=0.8843, SE=0.0263. The AUC difference (0.0129) is not significant (p=0.69), supporting the null hypothesis and allowing us to pool the ROC results from the two samples.

3.5. Performance characteristics of iCAHR-AT for the pooled samples

The ROC and other performance characteristics for the iCAHR-AT based on the 15,042 patients in the pooled original calibration and external validation cohorts are summarized in Table 4. In addition to showing high sensitivity and specificity (85.0% and 93.6%) for the GS SIRS IRD sepsis cases, the alert tool also identified children who progressed to “major” or “extreme” severity outcomes with a high PPV and NPV (43.6% and 83.7%). For cases not reaching maximum decompensation within the first 24 hours following arrival, the tool alerted at an average of 40 hours in advance of maximum decompensation, affording a meaningful treatment window for improving patient outcomes.

3.6. Association of Maximum iCAHR-AT Score with Severity of Illness Outcomes

Figure 3 shows the association between the maximum iCAHR-AT score and the occurrence of “Major” or “Extreme” SOI outcomes among patient encounters. As was similarly observed in the relationship between maximum alert scores and SOI outcomes for our earlier ED-based IRD alert tool[20], patients with maximum scores below the alert’s firing threshold had a low probability of a high SOI outcome, which gradually rose as the score increased. When the firing threshold of ‘5’ was reached, further increases in the maximum score correlated with a much steeper rise in the likelihood of such an outcome, reaching over 90% for maximum scores of ‘9’ or greater. This supports our hypothesis that the iCAHR-AT score correlates with the severity of a patient’s condition.

3.7. Clinical Outcomes

3.7.1. Acuity and Severity

As shown in Figure 4, patients firing the iCAHR-AT following live implementation exhibited a significantly (p<.001) greater average decrease in iCAHR-AT score over the 48 hours following initial firing compared with those who fired during the silent epoch. Given the strong association between iCAHR-AT score and physiological decompensation as measured by the APR-DRG SOI metric, this suggests that implementing the tool facilitated a greater patient improvement during this post-firing period.

Figure 5B shows that patients hospitalized at least 48 hours who fired the alert during the live epoch exhibited a significantly (p<.001) lower average post 48-hour maximum iCAHR-AT score compared with those firing during the silent epoch. This suggests that, on average, implementation of the iCAHR-AT also contributed to a lower patient morbidity by 48 hours post-admission. As a balancing measure, Figure 5A shows that the average iCAHR-AT scores at the time of initial alert firing are similar (5.77 for the silent epoch vs. 5.64 for the live), suggesting that the lower maximum scores for the live epoch are not due merely to lower initial severity.

The control chart shown in Figure 6 illustrates the temporal correlation of the patient improvements summarized in Figure 4 with each of the communication and acknowledgment refinements that were introduced following the live implementation of the alert. As the figure shows, the introduction of each enhancement (notification of charge nurse and initial provider, mandatory call to attending physician, and required written acknowledgement and patient assessment) resulted in a corresponding improvement as measured by a greater lowering of iCAHR-AT score during the initial 48-hour post-firing period.

3.7.2. Identification of Infectious Agents

Among patients with iCAHR-AT firings and microbiological cultures or PCR panels collected following inpatient admission, the percentage of cases with a positive identification of a bacterial, viral, or fungal organism increased significantly (p=.002, exact chi-square) from silent=26.5% (41/155) to live=40.9% (144/352).

3.7.3. Post-Firing Transfers from General Care Units to ICU

The percentage of patients transferred from the floor unit to the ICU at any time following initial alert firing showed a near significant (p=.086, exact chi-square) decrease from the silent epoch =11.6% (38/328) to the live epoch =8.2% (61/746).

3.7.4. Length of Stay

The median LOS for patients firing the iCAHR-AT decreased from 3.70 days during the silent epoch to 2.90 days during the live epoch. When adjusted for severity by conversion to Actual LOS÷ MS-DRG ALOS or Actual LOS÷ MS-DRG ALOS as described earlier, this decrease is highly significant (p=.003 using MSDRG ALOS, and p=.004 using MS-DRG GMLOS ).

4. Discussion

Early detection, preferably before the patient meets the clinical definition of “sepsis,” is important in preventing the worst outcomes for children with infection-related illness. Definitions have changed and improved, but sepsis as currently defined by pSOFA or Phoenix criteria remains a rare disease. To be of value to clinicians and less prone to overfiring and trigger fatigue, screening tools may need to function more broadly and with emphasis on predicting future deterioration.

Our iCAHR-AT, while defining its benchmark “gold standard” target cases using the pSOFA sepsis criteria, was constructed using a methodology that emphasized the early identification of cases that would progress to a high severity of illness. As such, the alert tool demonstrated a PPV of 43% for infection-related illness that progressed to “major” or “extreme” severity as defined by the APR-DRG SOI criteria. This represents an alternative to the two-tiered systems of other pediatric hospitals described earlier, which are likely to require a greater resource commitment for the enhanced monitoring of patients identified as “positive” by the initial screen.

As summarized earlier, iCAHR-AT implementation proceeded through a series of communication and mandatory response refinements. Resident training initiatives were implemented spanning several months. After demonstrating the strong association between a patient’s maximum iCAHR-AT score and the degree of physiological decompensation reached during the encounter as measured by the SOI metric, we then used the iCAHR-AT score as a proxy measure of illness severity. This was utilized to compare post-firing outcomes among patients having clinically silent firings during the pre-implementation period with those who fired following live implementation of the alert. We consider our significant findings of a greater average decrease in iCAHR-AT scores during the initial 48-hour post-firing period and of a lower average maximum iCAHR score at 48 hours or greater post-admission for patients who alerted during the live epoch as prima facie evidence that the iCAHR-AT has facilitated a more rapid clinical improvement among patients who activated the alert. This finding is also supported by discovering a significant decrease in severity adjusted length of stay among patients who activated the alert following its live implementation.

Our finding of a significant increase in the proportion of positive microbiological culture or panel results following live tool activation among patients meeting iCAHR-AT alert criteria and having samples collected suggests that the tool may also be helpful in triggering the identification of pathogens present in cases of infection-related decompensation. Additionally, both this apparent improvement in pathogen identification and the above mentioned improvements in post-firing iCAHR-AT scores and severity adjusted LOS may have contributed to the near-significant decrease in unplanned transfers from general care units to the ICU following alert implementation. This further suggests value for the iCAHR-AT in improving management of IRD cases on general care units, reducing the costs associated with escalation of care and improving hospital resource utilization.

5. Study Limitations and Future Directions

Future steps in the development of iCAHR-AT involve the introduction of natural language processing (NLP) tools to improve the contextual accuracy of data capture from unstructured free-text fields such as detailed history and reason-for-visit (RFV) information, summary results from transferring hospitals, history and physical (H&P) reports, radiologic imaging impressions, pathology reports, and specialty consults[31]. These methods may also lead to the identification of additional predictive factors or interactions. However, even in the current age of artificial intelligence (AI) ascendancy, predictive tools such as iCAHR-AT still rely completely on the availability of clinical information. While such information is usually most lacking for ED patients at the time of initial triage, it may also be lacking in the inpatient setting, as in cases where a child was initially admitted for a (presumed) non-infectious condition requiring minimal testing beyond regular vital signs collection.

While some existing pediatric sepsis screening tools are designed for widespread use at children’s hospitals, we consider it unlikely that a single predictive algorithm would be optimal for all patient populations and all hospitals. Rather, different local or regional patient populations may also show different disease patterns and susceptibilities, requiring a screening tool that is better honed to these differences. In addition, hospitals and their associated providers may show different preferences as to the target cases of the alert, for example, bacterial infections only versus inclusion of viral illnesses such as RSV pneumonia, encephalitis, and meningitis. Depending on available resources and goals, individual hospitals may also select different points along a screening tool’s ROC curve, choosing to optimize specificity over sensitivity, or vice versa.

6. Conclusions

We designed and implemented a pediatric inpatient assessment tool that identifies or predicts the development of physiological decompensation due to sepsis or other infection-related conditions that would otherwise result in major or extreme severity of illness in about 43% of alerted cases, with activation averaging 40 hours in advance for those where decompensation occurred at greater than 24 hours post-admission. The implementation of the tool was accompanied by a series of communication and resident training refinements, including mandatory notification of the attending physician and completion of an electronic acknowledgement form following alert firing. Most importantly, we have shown the introduction of the iCAHR-AT to be associated with more rapid clinical improvements (decreased severity) among children following firing, as well as an overall decrease in severity adjusted hospital length of stay, compared with those meeting firing criteria prior to this time.