Teams, Tools, Processes and Resources to Manage Oncologic Clinical Decision Support: Lessons Learned from City of Hope’s Multistate, Academic, and Community Oncology Enterprise

1. Introduction

Clinical decision support systems (CDSS) in oncology are now a foundational expectation for enterprise support and oversight to deliver high-quality cancer care [1,2]. While the main components of CDSS for cancer therapies are broadly agreed to include oncology clinical pathways (OCP) and Computerized Physician Order Entry (CPOE), the framework to understand, resource, and budget for the teams, tools, and processes to implement, maintain, optimize, and learn from these systems has not been standardized. City of Hope uses the Epic Beacon cancer treatment orders (called protocols by Epic) for CPOE and Elsevier’s ClinicalPath cancer pathways for medical oncology, hematology, and radiation oncology decision support.

The expansion of City of Hope’s care from an initial academic site in Duarte, CA, to multiple CA community network sites led to the adoption of clinical decision support (CDS) with ClinicalPath Pathways (formerly VIA Oncology) for medical oncology and hematology in 2017, and for radiation oncology in 2018. The City of Hope (COH) converted all sites to the Epic EHR, encompassing the Willow Pharmacy application for medication management and the Beacon Oncology application for Computerized Physician Order Entry (CPOE) for cancer therapies and other EHR functionalities in December 2017. This was followed by stepwise integrations of staging and Beacon treatment orders in Epic with the ClinicalPath navigation system for both standard-of-care therapies and, in May of 2023, for COH clinical trials. The opening of a second academic campus in Irvine, CA, in August 2022 and the acquisition of former Cancer Treatment Centers of America sites in Chicago, Atlanta, and Phoenix (COH-CAP Sites) in February of 2022, followed by their transition to COH’s Epic EHR in October 2023 identified a need for more formal enterprise programs and a request from leadership to understand the resources needed to support the clinical pathways and Beacon orders.

Prior to the development of this formal program, modifications and additions to Beacon orders were made by submitting requests to the IT team via a trackable ticketing system. A clinical pharmacist and medical oncologist collaborated with disease teams to assess the ClinicalPath pathways updates and utilization analytics and prioritize and finalize new treatment builds. These two individuals worked alongside COH quality and business intelligence teams to develop analytics for pathway utilization and understand the Beacon builds for various cancers. They also coordinated two enterprise-wide projects to improve COH’s Epic Beacon protocols using existing and some additional budgeted resources from the precision oncology team, which helped inform the current project.

The first Epic Beacon protocol project was updating our antiemetic regimen standards in 2020 based on the updated National Comprehensive Cancer Network (NCCN) guidelines. These included adding an option for olanzapine to highly emetogenic chemotherapy (HEC) regimens, reclassifying antiemetic levels to highly emetogenic chemotherapy (HEC) from moderate emetogenic chemotherapy (MEC) for carboplatin regimens with AUC >4, and changing to more optional use of lorazepam with dosing guidelines for older patients to reduce fall risks. The value-based MD lead and informatics pharmacist identified and oversaw this project.

In 2020, a second project was identified after recognizing that oral chemotherapies needed to be built as Beacon protocols in Epic, like therapies with IV, IM, and their combinations. The need to standardize the growing number of oral chemotherapy treatment orders has also been identified from studies as a vital component in reducing chemotherapy order errors. (Weingart, 2018) For COH, having all medical oncology and hematology treatments built in the Beacon protocols meant that the clinical and financial outcomes could be tracked by patient, disease, stage, biomarker, and line of therapy for quality and value insights. The medical director of value-based care (a medical oncologist) and the informatics pharmacist developed and led an oral chemotherapy build project after identifying 106 oral chemotherapy treatment orders needing to be built into Epic Beacon protocols to coordinate with ClinicalPath pathway therapies for care and comprehensive reporting. The leads worked with the precision oncology team, which funded the additional staff to support oral Beacon protocol builds as most of the oral regimens included drugs that targeted genomic mutations or biomarkers, which is a focus of the precision medicine program. The project was developed to build 140 Beacon protocols: 106 oral regimens and 34 other regimens requested by clinicians. An initial 5-month project from March 2022 through August 2022 was later approved for extension through September 2022. The disease leads and community oncologists volunteered to provide the clinical content. The disease leads, their PharmD, and nursing disease specialists also worked with the two project leads to edit and validate the final Beacon protocol builds before sending them to the P&T committee for approval and activation in Epic. Over the six months, $150,000.00 was used to fund a project manager and two beacon analysts who worked with the two project leads and disease teams. Those two analysts spent 1248 hours building and editing 133 new Beacon protocols: 84 oral protocols were completed and moved into production ( breast (25), lung (6), myeloma (17), skin cancer (2), adrenocortical cancer (1), chronic graft vs host disease, (cGVHD) (2), chronic lymphocytic leukemia (CLL) (12), chronic myeloid leukemia (CML) (5), colorectal (1), idiopathic thrombocytopenic purpura (ITP) (4), melanoma (3), myeloproliferative diseases (2), myelofibrosis (2) and prostate cancer (2)), 25 other protocols were moved into production, and 13 (10 oral) were built but awaiting P&T approval over the six months of the project. Thus, analysts spent an average of 9.38 hours per finalized protocol. Of the original 106 oral regimens identified, ten were unnecessary, and ten were awaiting clinical content when the project ended.

Since then, the City of Hope has undergone significant expansion; adding a second academic center, incorporating COH-CAP sites in three other states, and expanding CA network sites. This transition necessitated the establishment of a more formal program to oversee the thousands of Beacon treatment orders, periodic updates to components such as antiemetic regimens, proper incorporation of all required steps for treatment, building of new therapies, the management of clinical trial integrations and utilization, as well as updates, and analytics for the ClinicalPath oncology pathway program. Beacon ordering is integrated into the ClinicalPath pathways system with COH’s clinical trials for medical oncology and hematology. ClinicalPath also provides pathway recommendations for radiation oncology, but the treatment orders are placed separately in the ARIA radiation therapy system. The COH quality team built robust new analytic tools in Tableau to track each Beacon protocol order and ARIA radiation treatment order to ClinicalPath pathway navigations within 14 days of the order. Dashboard reports are sent weekly to clinicians and administrators to show navigations and identify patients with missing information for completion. Regional and enterprise-wide tracking of pathway navigations is also available in the new Tableau reports. The literature emphasizes the importance of CPOE as part of Clinical Decision Support Systems (CDSS) in oncology to minimize potentially severe or life-threatening toxicities and to optimize outcomes for evidence-based cancer therapies personalized to each patient [3]. While the literature discussed the importance of CDSS and Dr. Weese and colleagues shared their work across their multiple sites to incorporate Beacon protocols and clinical trials with the Aurora Cancer Center ClinicalPath pathways in their Epic system in preparing for Value Based Care, [4] a framework to understand the resources and organizational processes for this work is needed by oncology leaders.

Recognition of the need to standardize complex cancer therapies and tailor them to individual patients dates to the Institute of Medicine’s 1999 report, “Ensuring Quality Cancer Care.” The report emphasized the importance of prescribing treatments based on patients’ specific tumors and overall health [5]. In their subsequent 2013 report, “Delivering High-Quality Cancer Care: Charting a New Course for a System in Crisis,” the Institute outlined ten goals to support the main components of their framework for delivering high-quality care. Although not explicitly mentioned, Clinical Decision Support Systems (CDSS) with clinical pathways and COPE are fundamental components for addressing their six pillars of high-quality care: shared decision-making, workforce support, evidence-based care, learning health systems, measuring and improving quality care, and expanding access and affordability [6]. More recently, in October 2023, the National Cancer Policy Forum and the American Society of Clinical Oncology (ASCO) co-hosted a workshop to evaluate progress in the ten years since the 2013 IOM report. Sadly, the workshop concluded that while progress has been made, most of the original challenges persist. Specific to CDSS, they call out the need to “Redesign the EHR to provide more useful information to support patient care; improve clinical workflow and clinician well-being; and generate evidence to support a learning health system” and the need to “Leverage digital innovation to support all aspect of cancer care.” [7].

The 96% adoption rate of Electronic Health Records (EHRs) by hospital systems in the United States can be attributed to the 2009 Health Information Technology for Economic and Clinical Health (HITECH) Act. This legislation funded health information technology and established use standards for hospitals and practitioners through Medicare’s Meaningful Use program from 2011 to 2015 [8]. As a result of this initiative, EHR adoption increased significantly, including in oncology. During this period, there was also a substantial shift in oncology services, with most cancer care transitioning from community-based practices to larger healthcare systems. This trend led to the widespread adoption of electronic health records (EHRs), with Epic emerging as the dominant EHR in US healthcare, having been rated as the top overall software suite for the past 14 years in KLAS’s surveys. As of January 2024, Epic is utilized in 40% of all US hospitals, followed by Cerner at 22% and Meditech at 13% [9]. As of August 2024, Epic was used in 3,000 hospitals and 67,000 clinics, including 20 of the 20 top hospitals rated by US News and World Report. Epic is the EHR used by 580,000 physicians, serving 325 million patients worldwide and 280 million in the US [10]. Epic’s comprehensive support ranges from hospital and outpatient care to telehealth, billing, collections, and integration of outside medical records. Epic offers specialized modules, developer support, and dedicated oncology steering boards to drive further advancements in oncology care [11,12,13,14,15]. What has been missing, however, is data to understand the time and staffing needed to manage, build, update and oversee Beacon protocols and order updates for a health care system, as well as their integration with oncology pathways and clinical trials, engagement of staff and faculty along with outcome analytics.

CPOE has been documented to significantly decrease near-miss incidents, as well as severe and life-threatening toxicities, particularly for cancer patients [16,17]. Chemotherapy medication errors can occur at any stage of the process, including premedication, hydration, incorrect dosing, adjustments for toxicities, drug sequencing in combination treatments, infusion timing, drug dilution, scheduling, dose modification or delay criteria, and emergency medications. American Society of Clinical Oncology (ASCO) experts have established eight principles for safe EHR use in chemotherapy orders. These principles focus on ensuring accuracy, standardizing orders, automating calculations, integrating orders into workflows, providing decision support through alerts and dose maximums, allowing flexible changes as needed, and supporting efficiency, reliability, and usability. They have also stressed the importance of prioritizing safety over convenience due to the high risks of chemotherapy-related errors for vulnerable cancer patients [18,19].

The Joint Commission has identified eliminating medication errors as a primary focus in improving US health care. They supported CPOE and the Healthcare Failure Mode and Effect Analysis (HFMEA) methodology in 2000 to reduce medication errors. This approach applies engineering principles to promote reliability, safety, and quality to health care processes. A comprehensive assessment for a cancer center to use the HFMEA methodology to evaluate the risks and vulnerabilities in chemotherapy processes. After identifying 11 potential causes for four failure modes in outpatient care, CPOE for chemotherapy orders was recognized as the top priority project for risk reduction. The resulting 2008 CPOE project led to a significant decrease in chemotherapy prescription errors from 3.34% to 0.40%. This progress was made possible by reducing errors in ordering, preparing, compounding, dispensing, and administering chemotherapy. This is consistent with findings from other studies, which have shown error rates of 13% for handwritten chemotherapy orders and 5-15% error rates for paper-based chemotherapy orders are significantly reduced through standardized Computerized Physician Order Entry (CPOE) [17,20,21].

Reducing medication errors remains a significant focus of quality improvement, as described in the ongoing reviews and work by The Joint Commission and the Leapfrog Group [22,23,24]. Studies of the prevalence, factors, and interventions to reduce these errors are summarized in studies and systematic reviews. While organizations need to prioritize the building of the most used orders, it is also essential to ensure that even rarely used drugs and regimens are built to the same comprehensive organizational standards as when a drug or regimen is used infrequently, recall and experience to catch near miss errors is compromised and the risk of errors is increased [25]. This is even more important for chemotherapy orders as the therapeutic window is narrow, and the toxicities can be severe and life-threatening [26].

Also important is the standardization and regular review of built order sets. The Institute for Safe Medication Practices (ISMP) and the Centers for Medicare & Medicaid Services (CMS) have published guidelines regarding CPOE. The 2010 ISMP guidelines include standard order components and formatting, content development guidelines, and approval and maintenance recommendations. The ISMP recommends a biannual review of order sets to ensure the contents are up-to-date and relevant [27]. The Center for Medicare/Medicaid Services (CMS), through the office of the National Coordinator for Health Information Technology, has issued Safety Assurance Factors for EHR Resilience (SAFER) guidelines [28]. Each guide has recommended best practices designed to help an organization reduce EHR-related patient safety risk, optimize the safe use of EHRs, and create a ‘culture of safety’. The CPOE guide has three domains (Safe Health IT, Using Health IT Safely, and Monitoring Safety) addressing 29 best practices. The two most relevant to the work discussed here are: Recommended Practice 1.2, ‘Evidence-based order sets are available in the EHR for common tasks and conditions and are updated regularly’ and Recommended Practice 2.2, ‘Clinicians are engaged in implementing, reviewing and updating CDS.’ The Pathways and Protocols program that was developed at COH will provide disease teams and a governance committee to oversee the biannual review of Beacon orders to meet recommendation 1.2 and their review of Beacon orders and ClinicalPath navigations meets recommendation 2.2.

The Beacon module by Epic offers CPOE for complex oncology therapies for medical and hematologic oncology treatments. However, each organization is responsible for building its oncology therapy orders according to its standards, including naming, order components, formulary, coding, insurers, and other factors. NCCN provides cancer treatment regimen templates with drug dosing and scheduling based on their guidelines and published studies that can be used as a starting point for NCCN members. Epic is working to develop ways for Epic practices to share standard order templates, starting with cooperative group clinical trial orders. Currently, however, each EPIC using heaelth care system has to create their Beacon orders from clinical content provided by their multidisciplinary disease experts or consultants, with order components decided on by the organization. Standardizing these order components is an opportunity to improve safety and quality of care but a set of order components has not been established.

In a 2024 Grand Rounds presentation, clinical pharmacists reported on the initial conversion of their EHR to Epic with Beacon cancer treatment orders for Thomas Jefferson University’s Kimmel Cancer Center. They identified the need for a formal process to review their existing treatment plans and standardize the components for their Epic Beacon order builds for anticancer infusion orders. They excluded oral chemotherapy orders and research orders without standard-of-care treatments. For this project, they identified 15 cancer types and procedures (BMT, CAR-T, Breast, CNS, GI, GU, GYN, Head and Neck, Thyroid, Leukemia, Lung, Lymphoma, Melanoma, Multiple Myeloma, and Sarcoma) which had 1036 orders to be reviewed, standardized, and built. They specified components for their new Beacon orders, including the drugs, supportive care medications, required laboratory orders, and other monitoring recommendations to ensure safe and effective care based on the FDA package inserts. Of the first 375 orders reviewed for five cancer types, 311 orders (83%) needed updating, with an average of 1.7-3.07 interventions per order [29]. The US Oncology group also identified the need to standardize their treatment order sets after reviewing the 511 cancer treatment orders in their iKnowMed EHR. They found that 51 treatments could be removed or consolidated and that edits were needed in the remaining 460 treatment orders. The identified needed edits needed to standardize, update, or correct drug dosing, drug cycles, treatment references, and naming information [30].

As a significant component of CDSS, oncology clinical pathways have also been shown to improve efficiencies, value, and quality of care by guiding busy clinicians from multidisciplinary experts and synthesizing complex evidence-based science [31,32,33,34]. Oncology clinical pathways are available at the City of Hope through Elsevier’s ClinicalPath oncology treatment pathways for medical oncology, hematology, and radiation oncology. These pathways are overseen by disease committees that meet regularly to review evidence and update the pathways. National academic and community disease experts co-lead the committees with support from experienced Elsevier staff and oncology pharmacists. All member organization physicians can participate in committee meetings after disclosing conflicts of interest. City of Hope faculty serve on most disease committees and as co-chairs for several. City of Hope clinical trials are integrated from our OnCore system into the specific ClinicalPath pathways, so COH clinicians see our trials first, along with their principal investigator and the sites where they are open across the enterprise. A click to select a trial notifies the clinical trial team. If a trial is not selected, the reason is noted and mineable, and the standard of care therapies are then presented.

City of Hope’s ClinicalPath pathways are the foundation of a four-tiered pyramid of decision support that includes the Clinical Path pathways with clinical trials, formal enterprise-wide tumor boards, expert faculty consultations, and complex oncology case conferences, as previously described [35]. The ClinicalPath pathways guide initial and subsequent lines of therapy for 61 cancer types. The pathways cover 37 solid tumor types within nine categories: Breast, CNS, GI, GU, GYN, Head and Neck, Thyroid, Respiratory & Thoracic, Skin Cancers, and Sarcomas, and 24 hematologic disease types within seven categories: AML and APL, CML, CLL, ITP, Lymphomas, MDS, Multiple Myeloma and Plasma Cell Dyscrasias. Pathways for cervical cancer and ALL are expected in 2025, and expanded breast pathways now include inflammatory breast cancer so that the ClinicalPath system will provide pathway guidance for over 99% of all cancer patient diagnoses. Figure 1 shows the solid tumor and hematologic diseases with and without ClinicalPath pathways as of September 2024.

Our Epic Beacon system currently has 1100 Beacon treatment orders (called Beacon protocols by Epic), and 612 of these are linked to one or more of the 61 cancer subtype pathways. The addition of AML pathways in September 2024 added 72 more protocols to be mapped to our Beacon system. COH’s remaining orders are associated with COH’s other standard-of-care therapies and procedures that are not covered or detailed in the pathway system (primarily BMT, Car-T, TIL, cervical, ALL, and Myeloproliferative diseases). Figure 2 below shows the decision support roadmap for diseases and their therapies, whether they have an associated ClinicalPath pathway or not. The process of navigating and documenting the pathway or non-pathway disease choices from our Epic system is shown in Figure 2.

While oncology clinical pathways are accepted as fundamental to efficient, safe, standardized, reportable, and high-quality cancer care, the programming and Epic Beacon treatment orders for medical oncology and hematology, their integration with ClinicalPath pathways from COH’s Epic Beacon orders streamlines the steps to improve clinician efficiencies and patient safety while empowering informative analytics. The ClinicalPath pathways are developed with COH faculty participation and leadership working with academic and community cancer clinicians across the US and Canada through the ClinicalPath disease committees. This brings a critical review of the evidence together with experienced disease experts to add depth and breadth to guidance and detailed recommendations for optimal treatment options for each disease subtype and line of therapy. The ClinicalPath oncology treatment pathways provide patient-specific treatment guidance consistent with City of Hope disease leads and broader evidence-based guidelines from groups like the National Comprehensive Cancer Network (NCCN), American Society of Clinical Oncology (ASCO), Multinational Association of Supportive Care in Cancer (MAASC), and other specialty societies. The pathways and guidance notes help busy clinicians rapidly review patient-specific treatment options and COH clinical trials based on the patient’s tumor factors, health, co-morbidities, preferences, and sequence of care.

City of Hope now provides outpatient cancer care at two California-based academic centers, 9 CAP inpatient, 7 CAP outpatient, and 32 CA network sites across four states. Thirty-one outpatient sites provide outpatient cancer therapies for medical oncology and hematology. Given this growth, COH’s COO (VT) oversees care delivery quality and safety, focusing on CDSS. A medical director (LB) was appointed to develop and lead a formal Pathways and Protocols Program. In discussing the optimization of COH’s ClinicalPath system, it became clear that a deeper understanding of the current state of the pathway program and our Beacon protocols was needed to improve clinician satisfaction and ensure care quality, safety, and multi-stakeholder efficiencies.

Thus, work to build a Pathways and Protocols Program was initiated to provide enterprise-wide support. An experienced clinical oncology pharmacist (YHL) was hired to oversee the Beacon protocols as our CPOE. A program manager (BM) was provided to help organize a core team that met weekly to develop the Pathways and Protocols Program. They worked to understand the staffing and funding needed to identify and catch up on missing Beacon protocols and maintain them going forward. Gaps in current Beacon protocol builds were identified by linking current ClinicalPath protocols with our Beacon protocol list and reviewing tickets submitted by clinicians requesting modifications or building of protocols in Beacon. The list of protocols not in our Beacon system was then reviewed with disease team leaders appointed by the medical oncology and hematology chairs. Working with the clinical oncology pharmacist lead, the disease leaders prioritized building orders based on the frequency of use and the potential impact on patient care. They identified the current 1100 beacon treatment orders and an additional 400 oncology treatment orders not in our Beacon system that need to be built.

The team then evaluated the resources needed to maintain the Beacon protocols. They reviewed the 50-60 new oncology drugs and indications being approved annually that need associated treatment protocols built or updated. With current resources, 80 Beacon protocols per year have been built or updated in the system over the past two years, resulting in a backlog of 326 protocols. Based on the continued development of new oncology therapies, we estimate the ongoing need to build about 220 new protocols per year or about 15-18 protocols monthly, which will take more resources than we currently have.

In close collaboration with IT leadership and Epic Beacon experts, the core team held weekly meetings to determine the resources and budgets to catch up with the missing protocols and maintain the Beacon system. This inclusive approach ensures that all stakeholders are part of the decision-making process, which includes building the expected 15-18 new protocols monthly, performing the biannual regulatory Beacon protocol reviews, and providing the antiemetic build and implementation updates needed for all protocols. Surveys were done to understand the resources used by other organizations and the impact of doctors needing to build treatment orders ad hoc when they are not built in our system. The components of this work, including the significant collaboration with IT staff and Beacon experts, are described in the methods section.

2. Materials and Methods

The Enterprise Pathways and Protocol Program Development used project management principles to develop the program goals and governance. COH’s Chief Clinical Officer is the executive sponsor of the Pathways and Protocols Program. He appointed a medical director and worked with the Pharmacy leads to engage an experienced oncology pharmacist to co-lead the program. A project manager, crucial in guiding the work, was assigned to this role. This group formed a core team that met weekly, developed a program charter and a governance committee, and plans to coordinate the work of the disease teams, pathway, and Beacon protocol staff. They collaborated closely with the analytics, informatics, Epic Beacon, ClinicalPath, pharmacy, and other staff to analyze the current state and develop plans to optimize the functionality and efficiencies of the ClinicalPath pathways and Beacon protocols.

Our ClinicalPath medical oncology and hematology pathways’ current state with their associated treatment protocol list was defined. The clinical pharmacy lead collected the names and number of Beacon protocols built in our EHR for diseases with and without clinical pathways. The treatment protocol list from the ClinicalPath pathways was compared to our built protocols to develop a list of beacon protocols to be prioritized for building in our EHR. A list of requested Beacon protocols builds or edits for diseases in and not in the ClinicalPath pathways was developed from requests submitted to IT from clinicians and pharmacists. Working with Beacon and IT staff, the resources currently available to manage and build Beacon protocols were determined, along with their established capacity to build about 80 protocols per year based on output over the previous two years. Given the 50-60 new oncology drugs or indications being developed each year, we project ongoing needs to build, modify, or update about 220 beacon protocols annually or about 15-20 per month.

An additional need to update our enterprise antiemetic protocols was identified after the CAP sites joined the enterprise. These antiemetic protocols are included in each Beacon treatment order. The need to update them to the current NCCN standards and the opportunity to standardize them across our entire enterprise led the pharmacy experts to propose new antiemetic protocols for high, moderate, low, and minimal emetogenic treatments. Once built, the antiemetic protocols will be updated in all current Beacon treatments and added to new protocols. Clinicians can choose an oral-only antiemetic protocol or a combination of oral pre- and post-meds with IV antiemetic treatment on the day of therapy. The antiemetic regimens are further adjusted for the four current chemotherapies requiring coordination of steroids to treat hypersensitivity and emetogenicity, adjusted for multi-day regimens and regimens where steroids must be held or minimized. Thus, 24 antiemetic regimens were identified to be built for both oral only or combined oral and IV medications. The clinical pharmacy leadership developed the specific drugs and dosing for each regimen, which was reviewed and approved by the governance committee after review and approval by the medical oncology, hematology, and nursing departments.

After the Beacon protocol gap analysis identified 326 Beacon protocols missing from our system, the disease team and clinical pharmacy leads were asked to prioritize the protocols by frequency of use. This resulted in 200 protocols being identified as high priority for being built and 126 that were not currently or rarely used.

The EHR was queried to understand how many Beacon orders clinicians and pharmacists had to build or modify on the fly because our built protocols were incomplete. Over the last two years, the number of Beacon protocols ordered annually and modified or built annually was tallied. Three surveys were then done to gather the staff’s time to build or modify Beacon protocols daily, weekly, and monthly. The hours were summarized for each primary enterprise site. Assuming every 4 hours of clinician time saved would allow at least one new patient to be seen, estimates for additional new patient capacity were calculated for time that can be freed up by building order sets for the enterprise. The COH IT, Beacon, and pharmacy leads met with the protocols and pathways core team to review the resources and time it takes to build, modify, and validate Beacon protocols. An informal benchmarking survey was also conducted to understand the number of protocols and resources used to build and maintain Beacon oncology protocols in two other multi-site, multistate cancer programs. The benchmarking study supported the COH team’s recommendations for resources to catch up with unbuilt Beacon protocols and maintain new builds and regulatory reviews.

This work was then used to proactively develop a resource and program plan to catch up on building the needed Beacon orders and maintain them going forward for our enterprise. This work also identified the need for a standardized set of components for Beacon protocols, so a model set of components was developed.

3. Results

 

3.1.1. Enterprise Pathways and Protocols Program Development:

• Program Leadership: Our executive sponsor is our Chief Clinical Officer. He appointed a medical director who co-leads the program with the program’s clinical pharmacist. They work with the program manager as a core team. They actively interact with the clinical pharmacy, IT, ClinicalPath, Beacon leadership, department chairs, faculty, and nursing leadership.
• Program charter codified the membership, meetings, objectives, reporting structure, scope of work, and deliverables. The core team developed the charter, which was reviewed and approved by the medical oncology, hematology, and radiation oncology departments and the governance committee.
• A Governance Committee was formed with multidisciplinary stakeholders across the enterprise. They meet quarterly and oversee pathway and Beacon protocol policies, standardize Beacon order components, update antiemetic guidelines, ensure regulatory compliance, support resources to meet the program goals, and review analytics to document our high-quality care.
• Disease Teams were formed for each disease or disease subtype at the preference of the disease lead and department chairs. The teams are led by an academic faculty member appointed by the department chair. They consist of a clinical pharmacy and nursing disease expert, as well as any interested faculty across the enterprise who wants to attend. Attendance from physicians, APP, nursing, and pharmacy representatives from the CA community, CAP sites, and the Duarte and Irvine academic sites is welcomed. Epic Beacon and IT staff are encouraged to attend as well. COH has disease teams for pathway and non-pathway diseases as follows:

  ▪

  Solid Tumors:

  • Breast, GU, GI by subtype, Gynecology, Head and Neck, Lung, Melanoma and Skin, Brain, Sarcoma and Soft tissue, Thyroid

  ▪

  Hematology:

  • Lymphoma by subtype, ITP, Myelodysplasia/myeloproliferative, Myeloma and plasma cell diseases, CML, AML, ALL, CAR-T, BMT.

Although BMT and CAR-T therapies are recommended as appropriate in the ClinicalPath pathways, we have separate disease committees at COH to oversee the Beacon protocol builds and other components of those therapies. The disease teams have a specific agenda for the pathways and protocols program meetings. They meet monthly unless less frequent meetings can address the needs of their disease. The monthly disease team meetings address the following:

• Overseeing Beacon protocol builds, identifying and prioritizing builds
• Providing content for regimen builds, validating builds when done, and sending to COH’s P&T committee for final approval before going live in Epic
• Reviewing ClinicalPath updates and any pathway issues
• Reviewing COH clinical trials in the ClinicalPath system
• Regulatory review of Beacon orders biannually
• Reviewing any analytics of interest on pathway choices and clinical trial accruals

3.2.1. Establishing Standard Components for Beacon Protocols for the City of Hope Enterprise

Standardizing the components of our Beacon protocols ensures the order sets address the many functions needed for safe and effective patient care. A set of Beacon order components was identified, then reviewed and edited by the governance team and the departments of medical oncology, hematology, clinical pharmacy, and nursing before being finalized. They are to be considered and added as appropriate to every enterprise Beacon protocol. The components are shown in Table 1.

In addition to Beacon protocol order components, protocol formation should be standardized. The protocol should have a standardized naming convention, including commonly known treatment acronyms, individual drug names, and disease sites. The protocol description should include regimen details and emetogenicity level. The protocol should also include reference hyperlinks for the landmark trials of the treatment regimen, as shown in Table 2.

3.3.1. Current State Gap Analyses and Time Impact Surveys

• Gap Analysis and prioritization for missing protocols was performed to determine the number of Beacon orders missing from our Epic system. City of Hope’s Epic system has 1100 Beacon protocols built. Built protocols were matched against the 612 on pathway regimens in the ClinicalPath system as of April 2024. This matching identified 326 regimens missing from the current Beacon protocol builds. These 326 regimens were divided into their respective disease pathways. The disease leads and pharmacists were then surveyed to determine the usage frequency to prioritize getting them built. The prioritization criteria were divided into four categories based on order frequencies: priority 1: once every 1-4 weeks; priority 2: 1-3 months; priority 3: 4-12 months; or priority 4, rare or not currently used. This resulted in 36 priority 1, 75 priority 2, 89 priority 3 beacon protocols, and 126 noted to be used rarely or not needed. Results are shown in Table 3.
  Table 3. Survey Results of Prioritization of ClinicalPath associated Beacon protocol builds.

  Priority by usage frequency	Number of regimens (n=326)
  1-Use once every 1-4 weeks	36
  2-Use once every 1-3 months	75
  3-Use once every 4-12 months	89
  4-Not currently or rarely used	126

  Table 3. Survey Results of Prioritization of ClinicalPath associated Beacon protocol builds.

  Priority by usage frequency	Number of regimens (n=326)
  1-Use once every 1-4 weeks	36
  2-Use once every 1-3 months	75
  3-Use once every 4-12 months	89
  4-Not currently or rarely used	126

• Gap Analysis of Beacon orders built on the fly over two years was performed to understand the impacts of unbuilt protocols. The Epic system showed over 35,500 protocols were ordered annually, with 7500 being built or modified by clinicians or pharmacists. 5400 Beacon protocols were created, while 2100 were made by modifying existing protocols to create the needed treatment. The number and staff who built the missing protocols are shown in Figure 3. The 7500 protocols built or modified represent 21% of all Beacon protocols. Many protocols were built several times by different staff who needed to get their patients treated. We heard from one pharmacist who built a the same regimen ten times over a few months for different patients.

• Three studies were conducted to estimate the time spent by staff in building Beacon protocols ad hoc. While the findings varied, all three studies revealed a significant daily time commitment from clinicians, clinical pharmacists, advanced practice providers (APPs), nurses (RNs), and others. The potential time savings, as reported, by having Beacon orders built in Epic could be redirected towards seeing a significantly higher number of new or follow-up patients and providing more comprehensive care to those patients. For instance, the CA and CAP community site study revealed that the 250 hours per month were being used by physicians to build Beacon orders. This could translate into 63 more new patients being seen monthly across those sites, assuming that for every 4 hours saved, at least one new patient could be seen. This revelation carries significant implications for accelerating access to high-quality cancer care, a top priority for our enterprise.

  ○

  The first study estimated the physician’s time to build the Beacon orders. It is based on the Epic order study discussed above. Given that 7500 annual ad hoc Beacon protocols were built and based on the time experienced builders take to create or modify Beacon orders, estimates of the time used were made. Treatment plan modifications included the time to change orders in Beacon and to review and ensure the accuracy of the changes. Experienced Beacon order builders estimate the time needed to be 5 to 40 minutes, depending on the complexity of the modifications or new builds. When the combined MD, APP, RN, and PharmD times were evaluated, it was estimated that at least 128 hours per month were being wasted on these tasks that could be centralized. This data was primarily from the California network and academic sites, containing only 6 months of CAP site data as they transitioned to Epic in October 2023. Table 4 shows how the estimated wasted time was calculated.

  Table 4. Estimated monthly staff time to build or modify unbuilt Beacon orders based on Epic study.

  Role	% Time Editing Beacon Orders	Estimated hours/month Editing Beacon Orders	Estimated hours/year Editing Beacon Orders
  MD	32%	30.8-50.8	369-610
  PharmD	37%	35.6-58.8	427-705
  APP	19%	18.3-30.2	219-362
  RN	3%	2.9-4.8	35-57
  Others	9%	8.7-14.3	104-172
  Total Hours Used:		96-159	1154-1906

  Table 4. Estimated monthly staff time to build or modify unbuilt Beacon orders based on Epic study.

  Role	% Time Editing Beacon Orders	Estimated hours/month Editing Beacon Orders	Estimated hours/year Editing Beacon Orders
  MD	32%	30.8-50.8	369-610
  PharmD	37%	35.6-58.8	427-705
  APP	19%	18.3-30.2	219-362
  RN	3%	2.9-4.8	35-57
  Others	9%	8.7-14.3	104-172
  Total Hours Used:		96-159	1154-1906

Hours based on our Epic study of 576 providers who modified or built new Epic orders from April 2022 through April 2024. Data is primarily from California sites, as CAP sites were only on Epic for 6 months of the study. (October 2023 through April 2024). Based on 7500 studies annually needing to be built new or from modifying an order. Time ranges from 5-40 minutes based on complexity.

  ○

The second study of medical oncologists’ and hematologists’ time estimates to build Beacon orders was based on emails and discussions with the regional site leads for the California community network and CAP sites. They were asked to survey their doctors about the time their doctors were using each clinic day to build or modify orders that were not available. Based on the number of doctors and time spent at each site, monthly totals were calculated for wasted time per MD. Making the conservative assumption that for every 4 hours freed from creating orders, a doctor could see at least one new patient, the new patient potential for all the doctors at each community site was calculated. This conservative estimate shows that at least 63 more new patients could be seen monthly at the network sites across four states to better serve patients. Results are shown in Table 5.

Time per regional community site to build Beacon orders using 5 days of clinic per week per doctor.

Total time per month is calculated by the time per doctor per month times the number of doctors at each site. New patient potential is calculated from doctors being able to see at least one new patient for every 4 hours freed from order development.

  ○

The third study was done to report the time clinical pharmacists spent building or editing Beacon orders that were not in our system. The clinical pharmacist program lead (YL) spoke to the clinical pharmacists at each of the sites with clinical pharmacists to gather their hours per day spent editing unbuilt Beacon orders. These pharmacists were those working at Duarte and Lennear, Irvine academic outpatient clinics, and the three CAP sites. This showed significant daily, weekly, and monthly use of our pharmacist’s time, conservatively estimated at 40 hours per day of clinical pharmacist time across the enterprise which could be used to better serve patients and the organization. Results are shown in Table 6.

The results of these three studies each show the significant amounts of clinician and clinical pharmacist time being used to build or modify Beacon protocols which are not in our Epic system. In addition, it was noted that many of these missing orders were being built multiple times by different people at different sites as Beacon orders built for individual patients are not available for others to use. Orders built on the fly have also been found to focus on the treatment details and many do not include all of the standard order components that have been agreed to for efficiency and safety. Each survey, whether from the numbers of regimens built per year ad hoc or the physician or clinical pharmacists surveys, supports the argument that building the backlog of Beacon orders and keeping new Beacon orders up to date would free time across the enterprise to provide more care to current and new patients. Although the estimated additional revenue per new patient will differ for every organization, the costs of catching up and maintaining Beacon orders can be expected to be more than offset by additional patient revenues leading to a positive return on investment.

3.4.1. Evaluating the Time and Staffing Required by Staff to Build Standardized Beacon Protocols

• Resource analysis has considered personnel needed for program management, leadership, clinical coordination, and technical support to maintain the EHR workload, catch up on current needs and plan for future growth. Program leadership includes an MD medical director leading the pathway and protocol program and an informal pharmacist with clinical oncology knowledge coordinating interdisciplinary teamwork among specialties to ensure program development fits COH’s practice and standards. To recover the current deficit and maintain program growth, sufficient technical support is required to build BEACON protocols for on- and off-pathway regimens.
• The core team worked with the IT and Epic Beacon staff to understand the time it takes to build beacon protocols with the standardized components for our Epic system. They reviewed data from the 2020 project to build 106 oral regimens over 6 months to meet QOPI and other oral CPOE standards as has been discussed. The medical director and an informatics pharmacist collected the clinical content from faculty disease leads, community oncologists, and clinical pharmacists, then worked with the budgeted beacon analyst builders. They were able to complete 94 protocol builds (84 moved into production and 10 pending Pharmacy and Therapeutics (P&T) approval) plus 25 other protocols using an average of 9.38 hours of Beacon analyst time per protocol.
• From the time impact survey of physicians and clinical pharmacists discussed previously, expert clinical pharmacists noted that even modifying Beacon orders and reviewing their accuracy can take 5-40 minutes, depending on the complexity of the protocol. For new protocol builds, the time to build consists of the time to gather the clinical content for each applicable component of the order, the time to build the protocol in the Epic system, the time for MD, PharmD, and RN review and validation of the Epic build, then time for any edits and re-review for validation before a the final protocol order is sent to COH’s monthly Pharmacy and Therapeutics (P&T) committee for final approval. After that, an approved protocol can be activated across all of our Epic sites. An alternative methodology is for an experienced oncology pharmacist to be trained and certified as an Epic builder. They can then gather the clinical content and build the initial Epic order with all its components to avoid any back-and-forth between the clinical content provider and the Beacon analyst builder. Oncology pharmacists are the most experienced in the many nuances and details needed to build a complete oncology order protocol, but few have Epic Beacon builder certification. Thus, most organizations have clinical pharmacists or clinicians provide the content and then work with an Epic Beacon analyst builder until they are satisfied with the Beacon protocol. A disease team of an MD, RN, and PharmD then reviews the final order for any edits and their approval so it can be sent to the P&T committee for final approval and movement into the Epic system.
• The core team estimated the additional staffing needed to catch up and maintain our Beacon protocols. This was calculated to catch up 400 unbuilt protocols (326 prioritized from the ClinicalPath pathways and 71 requests for updates from clinicians to IT) and to provide the biannual protocol review for the current 1100 Beacon protocols. New protocol builds are estimated to require an average of 4 hours of PharmD time, ranging from 2 to 10 hours. Updates to current protocols are estimated to take 1.5 hours of PharmD time, and protocol review and standardization, including the updated new antiemetic regimens, is estimated to take 2 hours per protocol. Thus, the catch-up work would require 3907 hours of PharmD time: (400 protocols x4 hours)+ (71 protocols x 1.5 hours) + (1100 protocols x 2 hours), which would require two full-time PharmDs. For the Maintenance of Beacon protocols, we estimated 220 new protocols annually take an average of 4 hours of PharmD time (with a range of 2-10 hours), 200 clinician requests for updates or modifications annually take an average of 1.5 hours of PharmD time and the ongoing biannual review of half of what will be 1500 Beacon protocols means 750 protocols need to be reviewed annually at 2 hours of PharmD time each. Thus, for the Maintenance of Beacon Protocols, it will take 2750 hours of PharmD time: (225x4) + (200 x 1.5) + 750 x2), which would require 1.5 full-time PharmDs. The results are shown in Table 7.

• Going through a similar process for the Beacon analyst builders and Willow analyst builders along with the project manager, an overall recommendation was made for the added resources needed to catch up and maintain our Beacon protocols. These recommendations are shown in Table 8. The total budget for the catch-up work was estimated to be $1.9 million dollars over 14 months. The total budget for the annual maintenance work was estimated to be $1.1 million dollars.

• A benchmarking study was conducted on staffing to build and maintain Beacon protocols at two large cancer organizations, and it was compared to COH’s current and proposed staffing. Two COH staff who had recently transitioned from two large, multi-state, multi-site Epic Beacon using cancer programs gathered information about the number of Beacon protocols and the resources used to build and maintain them. Results, comparing the current and proposed additional resources for Beacon protocols at City of Hope, are shown in Table 9. The benchmarking study supports the need for additional staff to catch up and maintain COH’s Beacon orders.

3.5.1. Recommendations for Updated Enterprise Antiemetic Standards for Oral and IV Dominant Regimens:

• The clinical pharmacy team recommended that the following medications be standardized for the antiemetic regimens to meet HEC, MEC, LEC, and MIN needs in our Beacon protocols consistent with the latest NCCN guidelines, site and disease team needs. Table 10 shows the recommended drugs for the oral and IV dominant antiemetic protocols by emetogenic risk category.

• The antiemetic doses are then further adjusted when NK1 RA medications need to be deleted, or steroids adjusted or when partnered with one of the four chemotherapy regimens where steroid dosing needs to be coordinated with hypersensitivity and emetogenic prevention. Table 11 shows the different modifications of the antiemetic drug dosing to be built.

• These combinations resulted in 24 standardized regimens for antiemetics that can be added to Beacon protocols. These 24 regimens will be built in two ways: as oral dominant or oral plus day of therapy IV regimens to meet different payer and care needs. These regimens were developed with enterprise pharmacy input and then reviewed and approved by medical oncology, hematology, and nursing departments. Building these pre- and post-medication protocols will be prioritized for addition to all new Beacon protocols. They will also be added to the current Beacon protocols as they are reviewed biannually or earlier, pending resource availability.

4. Discussion

Benchmarking and reporting on high-quality cancer care delivery is increasingly important to a healthcare organization’s reputation and success. The increasing complexity of cancer therapies tied to disease subtypes, staging, molecular and other biomarkers, clinical features, patient needs, and sequencing has led to the widespread adoption of EHRs for documentation and provision of CDSS with CPOE and oncology clinical pathways. For oncology organizations, CPOE focuses on complex cancer treatment orders. For Epic EHR organizations, oncology CPOE is done through the Beacon treatment protocols.

Clinical pathways can be provided as a component of CDSS using custom or third-party clinical pathway tools. These tools are ideally integrated with Epic Beacon treatment orders to prompt and guide clinical trials and standard-of-care treatment choices. Once a treatment is chosen in the pathway system, it can be linked to the Beacon protocol for efficient ordering.

The ClinicalPath Pathways for medical oncology, hematology, and radiation oncology meet ASCO’s established high-quality cancer care pathway criteria, as do the other nationally available McKesson Value Pathways powered by NCCN [36]. Those pathway systems have been shown to optimize costs and outcomes by the Pittsburg Cancer program, the US Oncology group, and the City of Hope using Beacon treatment orders and the ClinicalPath pathway program [35,37,38,39,40,41]. Pathway tools facilitate oncology enterprises’ prompting, delivery, and measurement of high-quality care delivery personalized for each patient based on evidence-based guidelines and their faculty’s expert oversight. The costs of such systems are proprietary, and analysis of the costs and benefits of participating in national pathway programs that leverage central management and national faculty expertise compared to organizational pathways have not been published. Thus, benchmarking data is not publicly available for pathway programs as a component of CDSS. A framework to understand the resouces to build and maintain oncology clinical pathways with national groups vs inidividual enterprises would be to oncology organizations working to optimize value-based care.

For CPOE, however, we present our work to benchmark the costs and resources to update, maintain, and oversee the building and regulation of Epic Beacon treatment orders. We present our work to understand the number of standard-of-care protocols to support medical oncology and hematology treatments for diseases with and without clinical pathways. We present the types of resources needed to create and build Beacon treatment orders for Epic, the roles of a governance group, and disease teams to oversee Beacon treatment orders and oncology clinical pathways. Given that Elsevier’s ClinicalPath oncology pathways, like McKesson’s Level 1 Pathways by NCCN, are proprietary, we can only present the governance and disease team oversight roles we have identified to oversee the use of these pathways by clinicians and their integration into our Epic EHR. The actual number of Beacon treatment protocols can vary pending individual organization’s decisions to build sequential therapies, such as the common AC-T regimen for breast cancer as an Adriamycin/Cytoxan, “AC” treatment order to be followed by the paclitaxel “T” treatment order which would be 2 Beacon treatment protocols, or to build it as one, AC-T Beacon treatment order. City of Hope initially built components of treatment orders separately but found that it was more efficient for planning, eduacation, scheduling, preauthorization, and analytics to build standard sequential therapies as one Beacon treatment protocol. Oncology enterprises and programs face many challenges and demands regarding time, staffing, and resources to deliver high-quality cancer care. While oncology pathway programs have been shown to improve outcomes and the value of cancer care and are part of most alternative payment incentive programs by payers, the same is not true for CPOE. Despite extensive evidence showing that computerized provider order entry significantly reduces errors and prevents minor to life-threatening toxicities for vulnerable cancer patients, there are currently no quality metrics by the major value-based payers, ASCO, or other regulatory organizations that specify keeping current with building complex cancer orders in the EHR nor metrics for integrating such orders or patient-specific features into pathway systems to provide more efficient and comprehensive clinical decision support with reportable data. Each organization has been left to implement EHRs, including Epic, focused on the metrics of the state and national regulators, any incentive-based payer contracts and the interests, expertise and resourcing of their staff.

Epic, the most extensively used health system EHR in the United States, shares best practices through its Epic Earth and other sites, biannual meetings, and monthly oncology steering board. The oncology steering board’s 12 nationally elected experts advise Epic builders on projects that can better support oncology care. Oncology, however, as only one specialty, and while having had improvements, has yet even to have all the Epic staging forms updated with discrete data entry for standards called out in a December 2017 AJCC editorial and unanimously voted as a priority by the steering board in 2020, which one of the authors, (LB) is a 4th term member [42].

The work to integrate available discrete staging elements from Epic into the ClinicalPath pathways has helped remove clicks and time for entering staging data before navigating pathway options for diseases like breast cancer, where the Epic staging form is robust, and the discrete data pulls into the ClinicalPath system. A limited number of other diseases are fully mapped into ClinicalPath, while some are partially, and most await additional mapping from Epic into the ClinicalPath pathways which will reduce clicks and duplicative data entry to improve efficiencies and pathway navigations.

Linking our clinical trial management system, OnCore, into the ClinicalPath system allows COH’s 800 clinical trials to be loaded by Elsevier into our ClinicalPath disease pathway flow nodes daily. The trial placements are specific to the cancers, biomarkers, and lines of therapy. While challenges remain to remove as many clicks as possible, the ability to show COH sites across the US the sites of each clinical trial, where that clinical trial is active, who the primary investigatoris, and a button that alerts the specific clinical trial team to evaluate a patient for a trial improves the value of pathway navigation for clinicians. Additional clinical trial information is available from the pathway tool with a link to clinicaltrials.gov. Further improvements are expected as Dr. Kim at COH has developed a clinical trial tool that can be used as a standalone and as a potential link within our ClinicalPath pathway tool to rapidly let clinicians review and download the eligibility, inclusion/exclusion criteria and the COH consent form as well as other COH specific trial information.

5. Conclusions

Oncology program leaders need a framework and resourcing information to build, update, maintain and regulate Beacon treatment orders in Epic. Beacon orders are essential CPOE which is a key component of Clinical Decision Support Systems along with oncology clinical pathways. By sharing our City of Hope enterprise framework and experiences, we hope others can program and staff their organizations to keep Epic Beacon treatments up to date, linked to comprehensive oncology clinical pathways with clinical trials, and improve safety and efficiency for oncology patients and staff.