Multi-Timescale Scheduling Optimization of Hospital Integrated Energy Systems for Intelligent Energy Management

1. Introduction

With the growing demand for medical services and the advancement of green hospital initiatives, the Hospital Integrated Energy System (HIES) has emerged as a critical solution for achieving energy conservation, emission reduction, and high-quality healthcare delivery. As energy-intensive facilities, hospitals operate around the clock and exhibit distinct characteristics, including diversified energy demands and hierarchical power supply requirements. The energy systems in modern hospitals encompass various forms such as electricity, heating, cooling, and compressed air systems, which collectively enhance the overall stability and sustainability of hospital energy infrastructure[1,2].

Although technologies such as distributed photovoltaics, diesel generators, and energy storage systems are increasingly deployed in hospitals, maintaining energy balance within the Hospital Integrated Energy Systems (HIES) remains highly challenging due to the complexity of medical loads, uncertainty in patient flows, and the stringent reliability requirements of medical equipment. In particular, coordinating scheduling strategies across multiple timescales under complex dispatch cycles remains an urgent and unresolved issue[3,4,5].

The scheduling strategies of HIES typically operate on two levels: long-term and short-term. Long-term scheduling spans time horizons such as annual or quarterly periods and focuses on equipment maintenance planning and energy contract management to ensure energy supply security over extended durations[6,7,8]. In contrast, short-term scheduling addresses intra-day, hourly, or minute-level energy management, aiming to respond rapidly to fluctuations in medical demand and the uncertainty of distributed energy generation[9,10].

Conventional hospital energy management approaches often concentrate on optimizing a single timescale while neglecting the interdependencies between long- and short-term scheduling. Long-term strategies frequently impose rigid operational boundaries that restrict the flexibility of short-term decisions, making it difficult to respond to sudden surges in demand during surgery peaks or patient influxes. Meanwhile, the dynamic behavior of short-term scheduling can critically influence the effectiveness of long-term planning. The absence of an effective coordination mechanism between timescales can thus result in significant energy waste and increased risk of power supply interruptions in medical areas.

Recent studies have largely focused on isolated optimization within a single scheduling level. For instance, research on short-term energy management has explored optimization strategies for battery storage systems and thermal storage to enhance adaptability to short-term fluctuations[11,12]. Other studies have employed scenario analysis or robust optimization to handle uncertainties in hospital energy scheduling. However, these methods often lack the capacity for effective cross-timescale coordination[13,14,15]. Moreover, traditional model-driven approaches face computational bottlenecks when addressing HIES with complex load profiles and multi-source uncertainties, where the high dimensionality of decision variables renders solution processes computationally intensive.

To address these challenges, this paper proposes a multi-timescale coordinated scheduling method for HIES based on Deep Reinforcement Learning (DRL). Specifically, the long-term scheduling problem is formulated as a Markov Decision Process (MDP), with optimal short-term energy supply plans embedded in each MDP step to ensure seamless integration between timescales. Furthermore, a hybrid data- and model-driven framework is developed by integrating DRL with Mixed Integer Linear Programming (MILP), which reduces computational complexity while enhancing system flexibility. This framework effectively mitigates energy waste and improves energy efficiency in hospital operations.

2. Problem Description

In the scheduling of Hospital Integrated Energy Systems (HIES), the dynamic characteristics of different energy components and load demands often correspond to highly heterogeneous scheduling timescales [24]. To ensure optimal and safe system operation, it is essential to account for the complex interactions across multiple time scales—namely, the coordination and interdependence between long-term planning (e.g., equipment capacity investment, annual maintenance scheduling) and short-term operations (e.g., intra-day charging/discharging, standby unit dispatch) [25]. Particularly in wind-solar-storage-diesel hybrid HIES architectures, scheduling spans a wide range of timescales, from long-term activities such as annual or monthly capacity expansion and maintenance strategy formulation, to short-term processes such as photovoltaic/wind power tracking, real-time energy storage dispatch, and rapid diesel generator response within intra-day, hourly, or even minute-level intervals.

These components must operate in tandem under stringent medical power reliability requirements—such as uninterrupted power supply for surgical rooms and electromagnetic compatibility for sensitive devices—resulting in a highly complex multi-timescale optimization framework. A typical HIES architecture is illustrated in Figure 1. As shown, the long-term decision layer (e.g., yearly equipment expansion and maintenance planning) and the short-term execution layer (e.g., intra-day energy dispatch and emergency response) exhibit different decision objectives and cycle characteristics, yet their interaction is critical.

For instance, in Figure 1, Region ② (e.g., ICU and surgical room clusters) is pre-configured with significantly higher guaranteed power capacity than Region ① (general wards and administrative areas). This implies that limited energy storage and fast-response generation resources must be strategically reserved for Region ②. Consequently, compared to Region ①, Region ② faces a substantially lower risk of power shortage during emergencies, such as overlapping urgent surgeries and equipment startups, thereby achieving higher reliability.

This illustrates how the flexibility of short-term scheduling—such as the discharge speed of energy storage systems or cold-start latency of diesel generators—is constrained by the resource allocation framework established in long-term planning. Conversely, long-term planning decisions must be evaluated against insights derived from short-term simulation results, including the risks of critical load shortfalls and the costs of service interruption under extreme scenarios. Therefore, there is a pressing need for tightly coordinated optimization between long-term planning and short-term scheduling in HIES.

As a data-driven approach, Deep Reinforcement Learning (DRL) can effectively circumvent the complexities of directly modeling intricate systems by learning from historical data. However, in Hospital Integrated Energy Systems (HIES), which are characterized by stringent medical safety constraints and a large number of decision variables, searching for optimal solutions within a vast solution space often results in constraint violations, thereby disrupting the normal operation of the Markov Decision Process (MDP). As illustrated in Figure2, MDP training frequently fails during the initial stages due to constraint conflicts, making it difficult to generate complete decision trajectories and hindering effective learning and optimization. While DRL has shown promising performance in problems with relaxed constraints and small solution spaces, its effectiveness is significantly limited when applied to complex medical systems like HIES, which feature multi-timescale dynamics, strict operational constraints, and high-dimensional decision variables.

3. Multi-Timescale Scheduling Method Based on Deep Reinforcement Learning

To address the multifaceted challenges in HIES scheduling—including multi-timescale operation, complex medical load connectivity, and system uncertainties—a hybrid strategy that integrates both model-driven and data-driven approaches is proposed. In the proposed framework, Deep Reinforcement Learning (DRL) is employed to optimize long-term scheduling decisions, while Mixed Integer Linear Programming (MILP) is utilized to handle short-term dispatch tasks. This design ensures both computational efficiency and strict adherence to medical operational constraints. The overall architecture of the proposed method is illustrated in Figure 3, highlighting the interaction mechanisms between long-term and short-term scheduling processes.

3.1. Coupling Mechanism Between Long-Term and Short-Term Scheduling

The Hospital Integrated Energy System (HIES) comprises components operating on different scheduling time scales, necessitating coordination between long-term and short-term operations. Long-term scheduling primarily focuses on managing annual equipment maintenance plans and optimizing the monthly operational states of various devices. The main decision variable is the change in equipment capacity for each month, denoted as $\mathsf{\Delta }{C}_{i,d}$, where i represents the equipment type and $d$ indicates the month$\left(d=\mathrm{1,2},\dots ,12\right)$.

The variation in equipment capacity determines the control requirements for equipment operational states at the beginning and end of each month, and is expressed as:

$$C_{i,d,0}=C_{i,d-1,T}+\mathsf{\Delta }{C}_{i,d}$$

where $C_{i,d,0}$ denotes the available capacity of equipment $i$ at the beginning of month $d$, and $C_{i,d-1,T}$ represents its available capacity at the end of the previous month. These capacity constraints serve as operational boundaries for short-term scheduling. $T$ denotes the total number of hours in a day (T = 24).

In contrast, short-term scheduling addresses the intraday response to medical loads, operating on a daily time scale with an hourly resolution. The decision variables include the output power of various energy components, namely diesel generators $P_{w,t}^{DG}$, energy storage systems$P_{w,t}^{ES}$, distributed photovoltaics $P_{w,t}^{PV}$, grid-purchased electricity $P_{w,t}^G$, and medical load power $P_{w,t}^{ML}$, where w denotes the typical day scenario and $t$ is the time index.

3.2. Model-Driven Approach for Short-Term Scheduling

The short-term scheduling process optimizes the intraday operation of generation resources within the Hospital Integrated Energy System (HIES), subject to capacity variation constraints imposed by the annual equipment maintenance plan. Accordingly, it can be formulated as a Mixed-Integer Linear Programming (MILP) model and embedded into the long-term scheduling framework as an environment model within a Markov Decision Process (MDP). The objective of the HIES is to minimize the total operational cost over the course of a day, while accounting for the associated costs of energy storage systems, diesel generators, and medical load management.

$$minC=D_d\sum _{w=1}^W p_{w,d}\sum _{t=1}^T \left(c_{ES}\left|P_{w,t}^{ES}\right|+c_{DG}\left|P_{w,t}^{DG}\right|+c_G\sum _{i=1}^N P_{i,w,t}^G\right)\mathsf{\Delta }t$$

Here, $C$ denotes the operational cost, and $D_d$ represents the total number of days in month $d$. $W$refers to the number of typical days used to represent monthly variability. $p_{w,d}$ indicates the occurrence probability of scenario $w$ in month $d$. $c_{ES}$denotes the unit cost for charging/discharging the energy storage system, $c_{DG}$ represents the unit operating cost of the diesel generator, and $c_G$ is the unit cost of grid-purchased electricity. $N$ denotes the total number of medical load zones.

Meanwhile, the MILP model is subject to the following constraints:

$$\mathrm{s}.\mathrm{t}.C_{i,w+\mathrm{1,0}}=C_{i,w,0}+D_d{p}_w\left(C_{i,w,T}-C_{i,w,0}\right)$$

$$SO{C}_{w,t+1}=SO{C}_{w,t}-{\displaystyle \frac{P_{w,t}^{ES}\mathsf{\Delta }t{\eta }_{cha}}{E}},P_{w,t}^{ES}\le 0$$

$$SO{C}_{w,t+1}=SO{C}_{w,t}-{\displaystyle \frac{P_{w,t}^{ES}\mathsf{\Delta }t}{\left({\eta }_{dis}E\right)}},P_{w,t}^{ES}\ge 0$$

$$SO{C}_{w,T}=SO{C}_0$$

$$SO{C}_{min}\le SO{C}_{w,t}\le SO{C}_{max}$$

$$\left|SO{C}_{w,t+1}-SO{C}_{w,t}\right|\le \mathsf{\Delta }SO{C}_{max}$$

$$-P_{max}\le P_{w,t}^{ES}\le P_{max}$$

$$\sum _{i=1}^N P_{i,w,t}^{ML}+P_{w,t}^{ES}+P_{w,t}^{DG}+P_{w,t}^{PV}=P_{w,t}^G+P_{w,t}^L$$

These constraints respectively represent the state of charge (SoC) of the energy storage system, the power demand of medical loads, and the energy supply–demand balance, among others.

3.3. Data-Driven Approach for Long-Term Scheduling

The long-term scheduling aims to develop an annual capacity allocation strategy for critical medical equipment, which requires dynamic planning of monthly capacity deployment. This task inherently involves sequential decision-making [30], and can therefore be formulated as a Markov Decision Process (MDP). The MDP is defined by a quintuple $(S,A,P,R,\gamma )$, where:

$S$: State space, representing the equipment health status and load risk;

$A$: Action space, corresponding to monthly capacity adjustment decisions;

$P$: State transition probabilities, incorporating the stochastic degradation of equipment;

$R$: Reward function, embedding the cost of medical service interruptions;

$\gamma$: Discount factor, balancing short-term and long-term benefits.

3.3.1. State Space

The environmental state encompasses the required equipment capacity, medical load demand, distributed generation output, environmental parameters, and the level of medical services. It is defined as:

$$S_d=\left\{d,S_{C,d},S_{ML,d},S_{PV,d},S_{ENV,d},S_{MS,d}\right\}$$

where each state component is defined as $S_d$ denotes the current state vector; $S_{C,d}$ represents the equipment capacity state vector; $S_{ML,d}$ denotes the medical load state vector; $S_{PV,d}$ represents the photovoltaic output state vector; $S_{ENV,d}$ corresponds to the environmental parameter state vector; and $S_{MS,d}$ indicates the medical service level state vector.

3.3.2. Action Space

In the multi-timescale scheduling of medical systems, the agent’s action is defined as the monthly capacity adjustment strategy for critical equipment, denoted as:$a_d=\left\{\mathsf{\Delta }{C}_{i,d}\right\}$where $\mathsf{\Delta }{C}_{i,d}$ represents the capacity adjustment of equipment $i$ in month d, and $i\in \{\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{g}\mathrm{y}\mathrm{s}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{a}\mathrm{g}\mathrm{e},\mathrm{p}\mathrm{h}\mathrm{o}\mathrm{t}\mathrm{o}\mathrm{v}\mathrm{o}\mathrm{l}\mathrm{t}\mathrm{a}\mathrm{i}\mathrm{c}\mathrm{s},\mathrm{d}\mathrm{i}\mathrm{e}\mathrm{s}\mathrm{e}\mathrm{l}\mathrm{g}\mathrm{e}\mathrm{n}\mathrm{e}\mathrm{r}\mathrm{a}\mathrm{t}\mathrm{o}\mathrm{r}\mathrm{s}\}$.

To enhance the agent’s ability to explore under uncertain medical environments, Ornstein–Uhlenbeck (OU) noise is introduced as follows:

$$x_d=x_{d-1}+\phi \left(\tau -x_{d-1}\right)+\kappa \omega$$

where $x_d$ is the current noise value, which fluctuates with seasonal load variations; $x_{d-1}$ is the previous noise value reflecting historical decision bias; $\phi$ is the mean reversion speed; $\tau$ is the long-term mean of the noise; $\kappa$ is the volatility amplitude; and $\omega$ is a Gaussian random sample.

By applying this mechanism, the agent can maintain policy stability while improving exploration diversity. Therefore, the actual executed action, denoted as ${\widehat{a}}_d$, is the sum of the deterministic action $a_d$ and the noise $x_d$.

3.3.3. Reward Function

The reward function serves as the core mechanism guiding the agent to learn optimal medical scheduling strategies, and its design must balance both economic efficiency and medical safety. The improved medical reward function $r_d$ is defined as:

$$r_d={\alpha }_{1}F-{\beta }_{1}C-{\beta }_2{G}_1-{\beta }_3{G}_2$$

where $F$ is the indicator of medical service continuity, $G_1$ denotes the penalty term for capacity constraint violations, and $G_2$ represents the penalty term for violating medical safety requirements. The coefficients ${\alpha }_1,{\beta }_1,{\beta }_2,$and ${\beta }_3$ are weighting factors used to adjust the relative importance of rewards and penalties for different objectives.

3.3.4. Policy Learning

As the core component of the coordinated scheduling approach for Hospital Integrated Energy Systems (HIES), policy learning is implemented using the Twin Delayed Deep Deterministic Policy Gradient (TD3) framework, which integrates both long-term and short-term scheduling outcomes. To enhance training stability in medical scenarios, TD3 employs a dual Q-network architecture. The parameters of the Q-networks are updated following a target network smoothing mechanism:

$${\theta }^{\mathrm{\text{'}}}\leftarrow \lambda \theta +\left(1-\lambda \right){\theta }^{\mathrm{\text{'}}}$$

The parameters $\theta$ and $\theta \mathrm{\text{'}}$ denote the weights of the main network and the target network, respectively, and the overall network architecture is illustrated in Figure 4. The smoothing factor $\lambda \in \left(\mathrm{0,1}\right)$ (set to $\lambda =0.995$ in this study) is applied to stabilize the target network updates. The action-value function $Q^{\mathrm{*}}\left(s_d,{\widehat{a}}_d\right)$, â_d) is updated based on the Bellman equation as follows:

$$Q^{\mathrm{*}}\left(s_d,a_d\right)\leftarrow Q\left(s_d,a_d\right)+\alpha \left[r_d+\gamma Q\left(s_{d+1},a_{d+1}\right)-Q\left(s_d,a_d\right)\right]$$

where $\alpha$ is the learning rate. The objective of policy learning is to maximize the expected return. Through iterative optimization, the learning process integrates the global perspective provided by long-term data-driven scheduling and the local precision enabled by short-term model-driven decisions. This results in a multi-timescale optimized scheduling strategy that significantly enhances the overall efficiency of the system.

4. Case Study Analysis

4.1. Simulation Environment and Parameter Configuration​​

To validate the effectiveness of the coordinated scheduling method for the Hospital Integrated Energy System (HIES), simulations were conducted based on a real-world energy system from a tertiary hospital. The experiments were implemented on a computing platform equipped with an AMD Ryzen 7 5800 8-core processor.

The HIES consists of the following core components: a 2000 kW rooftop photovoltaic (PV) system, a 1500 kW diesel generator, and a 3600 kWh hybrid energy storage system (comprising lithium-ion batteries and flywheels). The hospital comprises 800 beds and includes several critical load zones such as the surgical complex, intensive care unit (ICU), emergency department, and general wards.

Surgical Complex: Dual-loop power supply with reliability ≥ 99.99%.

ICU: Uninterruptible power supply (UPS) with seamless switching (response time ≤ 0.1 s).

General Wards: Tiered power supply protection according to priority.

Table 1. Core Components, Technical Parameters, and Medical Constraints of the HIES.

Component	Parameter	Medical Constraint	Value
Photovoltaic System	Peak Power	Total Harmonic Distortion ≤ 3%	2000 kW
Diesel Generator	Cold Start Time	ICU Power Supply Protection	≤ 90 s
Energy Storage System	Total Capacity	Emergency SoC Reserve ≥ 30%	3600 kWh
Flywheel Storage	Response Time	Transient Support for Operating Rooms	≤ 200 ms
Critical Load (OR)	Peak Load (Operating Rooms)	Voltage Fluctuation within ±5%	850 kW

Table 1. Core Components, Technical Parameters, and Medical Constraints of the HIES.

Component	Parameter	Medical Constraint	Value
Photovoltaic System	Peak Power	Total Harmonic Distortion ≤ 3%	2000 kW
Diesel Generator	Cold Start Time	ICU Power Supply Protection	≤ 90 s
Energy Storage System	Total Capacity	Emergency SoC Reserve ≥ 30%	3600 kWh
Flywheel Storage	Response Time	Transient Support for Operating Rooms	≤ 200 ms
Critical Load (OR)	Peak Load (Operating Rooms)	Voltage Fluctuation within ±5%	850 kW

4.2. Experimental Results and Analysis

To validate the superiority of the TD3 algorithm in multi-timescale scheduling for medical applications, this study compares the convergence characteristics and scheduling performance of three deep reinforcement learning (DRL) algorithms. As shown in Figure5 , TD3, DDPG, and SAC exhibit significant differences in handling the HIES scheduling task:

The convergence performance of the three deep reinforcement learning (DRL) algorithms—Deep Deterministic Policy Gradient (DDPG), Soft Actor-Critic (SAC), and Twin Delayed Deep Deterministic Policy Gradient (TD3)—was evaluated in the context of HIES multi-timescale scheduling. Among them, TD3 demonstrated superior stability and accuracy. DDPG achieved faster training speed but exhibited relatively lower stability, while SAC performed poorly in this specific problem setting. Therefore, TD3 is identified as a suitable and effective algorithm for addressing complex system optimization tasks in HIES.

4.2.1. Overall Performance Analysis

To evaluate the effectiveness of the proposed method, a comprehensive performance comparison was conducted against two traditional approaches: Scenario Reduction (SR) and Robust Optimization (RO), as summarized in Table 2. The results demonstrate that the proposed method significantly improves distributed energy utilization, achieving 96.72%, compared to 82.45% for SR and 88.31% for RO. In contrast, the power outage rate for critical medical loads was reduced markedly from 2.8% (SR) and 1.5% (RO) to only 0.15%, thereby substantially enhancing the reliability of medical power supply.

4.2.2. Short-Term Scheduling Analysis

To comprehensively evaluate the short-term energy scheduling performance of the proposed method in hospital scenarios, representative days from critical seasons in 2021 were selected. Specifically, typical days in spring (moderate load), summer (air-conditioning peak), autumn (transitional season), and winter (surgical peak and influenza season) were chosen for analysis. The results are summarized in Table 3.

Firstly, regarding photovoltaic (PV) utilization, the Scenario Reduction (SR) method relies on predefined strategies and exhibits limited responsiveness to the actual rooftop PV output. In winter, its daily PV utilization rate drops to 76.4%. The Robust Optimization (RO) approach tends to retain excessive backup capacity to guarantee worst-case power balance, resulting in a suboptimal summer utilization rate of 88.9%, which is significantly lower than the 95.7% achieved by the proposed method. By leveraging deep reinforcement learning (DRL) to dynamically model the probabilistic distribution of rooftop PV generation and incorporating Mixed-Integer Linear Programming (MILP) for short-term optimization, the proposed method maintains high PV utilization levels across all seasons.

Secondly, for energy storage system (ESS) scheduling, the SR method fails to adequately account for lithium battery longevity and the high operational standards of the medical sector (e.g., national standard: ≥4000 cycles). In winter, the average daily cycling rate reaches 2.0 cycles/day. Considering an approximate degradation rate of 0.025% per cycle, this could lead to a significant reduction in system lifespan. RO shows even higher cycling at 2.4 cycles/day, prioritizing extreme reliability at the expense of economic efficiency. In contrast, the proposed method incorporates DRL-based charge/discharge thresholds and MILP-based state-of-charge (SoC) constraints, achieving a 25%–37% reduction in daily cycling frequency, thereby extending system lifespan and reducing operational costs.

In terms of critical load assurance, SR fails to fully address unexpected load surges. For instance, during winter surgical peaks, the power supply guarantee rate for operating rooms is only 94.3%, falling short of rigid hospital requirements—≥99.99% for operating rooms and ≥97% for general wards. RO achieves guarantee rates above 98% throughout the year but sacrifices PV utilization and ESS longevity. The proposed method adopts a multi-objective trade-off strategy. Although the winter guarantee rate for surgical and critical departments is slightly lower at 96.8%, it effectively balances economic efficiency and reliability through optimized coordination of diesel generators and ESS dispatching.

Lastly, regarding diesel generator peak-shaving contribution, SR, relying on static rules, achieves only 37.2% contribution in winter. RO shows marginal improvement but remains limited due to conservative capacity reservation. The proposed method dynamically optimizes diesel generator output and ESS synergy via DRL, increasing the peak-shaving contribution to 42.7%. Considering a backup diesel generator capacity of 1.5 MW, this corresponds to a winter peak-shaving capability of approximately 0.64 MW, significantly enhancing the hospital’s resilience to load fluctuations and sudden high-demand events.

4.2.3. Long-Term Scheduling Analysis

At the long-term operational level, the policy network based on deep reinforcement learning (DRL) significantly enhances the coordination and adaptability of the hospital energy system by learning the dynamic relationships among rooftop photovoltaic (PV) generation, energy storage capacity, and average monthly energy demand. The results are summarized in Table 4.

Firstly, in terms of energy storage capacity control, the DRL-based strategy demonstrates superior performance. Under the traditional Scenario Reduction (SR) method, the capacity regulation error reaches as high as 15.2%, whereas the proposed method reduces the error to 3.9%, achieving a 74.3% improvement. This indicates that the DRL-based scheduling strategy can more accurately align the state of charge (SoC) of battery storage with the hospital’s actual energy demands, thereby minimizing capacity degradation caused by overcharging or deep discharging, and ensuring continuous and stable power supply for critical load areas such as operating rooms and intensive care units (ICUs).

Secondly, the coordination mechanism between DRL and MILP rolling optimization plays a crucial role in compensating for PV generation forecast deviations. By employing DRL to predict the probabilistic distribution of rooftop PV output and integrating it with a monthly rolling MILP model, the system dynamically adjusts reserve capacity and energy storage dispatch strategies. Compared to the SR method, the proposed approach improves the forecast deviation compensation rate by 38.6%, effectively smoothing out the fluctuations in monthly renewable output and enhancing both PV utilization and the hospital’s overall energy self-sufficiency.

Lastly, in terms of real-time power tracking accuracy, the MILP model performs hourly rolling optimization of the combined output from diesel generators, PV systems, and energy storage, enabling highly efficient load response. Through this fine-grained multi-source dispatching, the real-time scheduling deviation is strictly controlled within 1.5%, with the average monthly power tracking accuracy improved by 52.9%. These results clearly demonstrate the proposed method’s advantage in ensuring real-time responsiveness and power stability for high-sensitivity medical loads—especially operating rooms and mission-critical equipment—thus providing nearly uninterrupted and highly reliable energy support for hospital operations.

4.2.4. Case-Based Scheduling Analysis

To validate the practical applicability of the proposed method within a hospital-integrated energy system, a case study was conducted using the energy infrastructure of a newly established campus of a tertiary hospital. The system includes a 2000 kW rooftop photovoltaic array, a 1500 kW diesel generator, and a 3600 kWh hybrid energy storage system composed of lithium-ion batteries and flywheels. The key load zones cover the surgical suite, the intensive care unit (ICU), and the general wards.

To validate the practical applicability of the proposed method within a hospital-integrated energy system, a case study was conducted using the energy infrastructure of a newly established campus of a tertiary hospital. The system includes a 2000 kW rooftop photovoltaic array, a 1500 kW diesel generator, and a 3600 kWh hybrid energy storage system composed of lithium-ion batteries and flywheels. The key load zones cover the surgical suite, the intensive care unit (ICU), and the general wards.

Figure 6 illustrates the reward evolution trend during the training process of the deep reinforcement learning agent. In the initial phase (0–150 episodes), the reward exhibits significant fluctuations as the agent continuously explores and gradually learns the seasonal load patterns and responses to extreme events. After 240 episodes, the reward begins to show a clear convergence trend. By episode 400, the average reward fluctuation narrows to within ±1.17, indicating that the agent has effectively learned an optimal scheduling policy.

Notably, at episode 150, the agent successfully identifies a sharp increase in winter heating demand, resulting in a 23% boost in reward. Furthermore, by episode 320, the agent significantly improves the storage reserve strategy during surgical peak hours, leading to a stabilization of the reward curve.

Figure 7 illustrates the dynamic annual capacity adjustments of the diesel generator and energy storage system. For the diesel generator, capacity is increased by 18% during the winter months (December to February) to meet the elevated heating demand typical of the cold season.For the energy storage system, capacity is expanded by 15% during flu season (February to March and October to November) to address emergency backup requirements and patient surges.The total annual capacity variation amounts to only 0.37 GWh, which represents just 3.2% of the system’s designed capacity. This performance is significantly better than that of traditional empirical methods, which exhibit a variation of up to 5.6%.

Figure 8 illustrates the power balance results of the Hospital Integrated Energy System (HIES) on representative days across each month. It is evident from the figure that the hospital’s load structure exhibits significant seasonal variations, and the energy contribution from different sources varies accordingly. The analysis is detailed as follows:

First, grid-purchased electricity serves as the base load provider throughout the year. Its output increases notably in the winter and early spring months (January–March and December), reflecting the hospital's reliance on highly reliable energy sources during heating seasons and periods of unexpected high demand. For instance, in January, February, and December, the share of grid electricity reaches its annual peak, ensuring uninterrupted power supply for critical zones such as operating rooms and intensive care units (ICUs).

Second, photovoltaic (PV) generation demonstrates pronounced seasonal variability. From late spring to early autumn (April–September), PV output increases steadily, peaking in July and August to meet the surge in air-conditioning demand. During this period, PV contributes more than 25% of the total load, significantly improving renewable energy utilization, while reducing both the cost of grid electricity and carbon emissions.

The energy storage system (ESS) plays a key role in peak shaving, valley filling, and emergency backup across all months. During periods of sharp load fluctuations in summer and winter, the ESS effectively smooths intraday power supply–demand dynamics through flexible charge–discharge strategies. It also provides millisecond-level backup response in extreme scenarios, such as sudden emergency room surges. Experimental results indicate that the ESS’s discharge contribution increases significantly in July and December, ensuring continuous and secure hospital power supply.

Third, the diesel generator acts as a crucial supplementary source to balance the system, especially when distributed energy and ESS are insufficient to meet the demand. Slight increases in the share of grid electricity during summer and winter reflect the system’s compensatory mechanism under extreme climate conditions. Notably, under the coordinated scheduling of PV and ESS, the overall grid power consumption is significantly reduced compared to traditional management approaches, improving both the self-sufficiency and economic efficiency of the hospital’s energy system.

In addition, other renewable energy sources (Other RE) introduce greater flexibility and sustainability to the system. Their contribution increases in wind-rich months (e.g., spring and autumn), providing extra redundancy for system dispatching.

Finally, the total load (black curve) fluctuates throughout the year, with higher levels in summer and winter. The proposed multi-energy coordinated scheduling strategy ensures the rational allocation and efficient utilization of various energy sources under different seasonal and representative day scenarios. This supports secure, economical, and low-carbon operation of the HIES. Overall, the proposed dispatching framework effectively accommodates highly variable hospital loads and frequent extreme events, guaranteeing the continuity of critical medical services and the overall efficiency of system operation.

4.2.5. Generalization and Applicability Validation​​

To further verify the generalizability and practical applicability of the proposed scheduling framework, three types of medical institutions were selected for systematic comparative analysis: a newly established tertiary hospital, a community hospital, and a specialized outpatient center.

The results of Table 5 demonstrate the following:

In the new tertiary hospital, the annual duration of surgical interruptions was significantly reduced to just 0.3 hours/year using the proposed method, compared to 4.2 hours/year under traditional approaches.

In the community hospital scenario, the PV utilization rate reached 95.1%, significantly outperforming the traditional method's 86.7%.

For the specialized outpatient center, the equipment expansion cost was reduced by 18% compared to the baseline solution.

These results clearly indicate that the proposed scheduling framework can enhance system-wide economic performance by 21%–28%, while strictly adhering to hard medical safety constraints (e.g., ensuring ≥99.99% power reliability for operating rooms).

In terms of economic benefits, the proposed framework can save approximately CNY 2.86 million annually in equipment expansion costs and reduce direct economic losses due to power outages by around CNY 9.2 million. The results are summarized in Table 6.

From the perspective of technical scalability, the system supports 5G edge computing deployment with a decision-making latency of less than 50 milliseconds, and has been certified for electromagnetic compatibility (EMC) by national medical equipment standards. These features demonstrate strong engineering feasibility and broad potential for large-scale application.

4.2.6. Sensitivity Analysis

To investigate the impact of energy storage capacity and algorithm parameters on system performance, a sensitivity analysis was conducted. The results show that when the storage capacity is increased from 1800 kWh to 3600 kWh, both distributed energy utilization and critical load interruption rate are significantly improved. This confirms the essential role of high-capacity storage in ensuring the reliability and efficiency of hospital energy systems.

Table 7. Sensitivity Analysis Results.

Variable	Variation Range	Distributed Energy Utilization (%)	Critical Load Interruption Rate (%)
Storage Capacity (kWh)	1800→3600	92.31→96.72	0.28→0.15
DRL Batch Size	64→256	94.15→96.72	0.19→0.15

Table 7. Sensitivity Analysis Results.

Variable	Variation Range	Distributed Energy Utilization (%)	Critical Load Interruption Rate (%)
Storage Capacity (kWh)	1800→3600	92.31→96.72	0.28→0.15
DRL Batch Size	64→256	94.15→96.72	0.19→0.15

5. Conclusion

This paper proposes a multi-timescale coordinated scheduling method for Hospital Integrated Energy Systems (HIES), based on Deep Reinforcement Learning (DRL) and Mixed-Integer Linear Programming (MILP). Through comprehensive system testing and analysis, the following conclusions are drawn:

The proposed method successfully decomposes the complex HIES scheduling problem into a Markov Decision Process and a Mixed-Integer Linear Programming problem, reducing computational complexity by approximately 35.6% compared to traditional methods and greatly improving computational efficiency.

Compared with traditional scheduling approaches, the proposed framework more accurately captures the complex variability of medical loads, resulting in a substantial reduction in equipment capacity control errors and improved accuracy in long-term scheduling.

The multi-timescale scheduling framework effectively coordinates multiple energy resources—including diesel generators, energy storage systems, and distributed photovoltaics—thereby mitigating fluctuations in medical loads and enhancing the adaptability of the hospital energy system to varying medical demands.

The proposed method offers robust technical support for the intelligent and low-carbon development of hospitals, demonstrating considerable practical value and promising potential for large-scale deployment.

Future work will focus on exploring the agent's environmental adaptability under expanded system scenarios and investigating the applicability of the method in more complex medical environments, thereby offering robust technical support for the large-scale implementation of high-efficiency HIES.