The Portuguese Public Hospitals Performance and Sustainability Evolution Before and During the SARS-CoV-2 Pandemic (2017-2022)

1. Introduction

Covid-19 is a respiratory disease caused by a SARS-CoV-2 virus belonging to the coronavirus family, a major threat to public health worldwide [1]. Symptoms of the infection in humans by this type of virus range from upper respiratory tract infections (as in a common cold) to severe acute respiratory syndrome, pneumonia, and bronchitis [2]. Elders, children, immunosuppressed people, and those with weak immunologic systems usually express a more severe form of the disease when they get infected, which may result in death [3]. The virus was first identified in Wuhan city, China, in January 2020 [4]. It soon spread all over the globe, primarily because of most governments' inefficacy to control their own countries' borders [5]. By August 2022, i.e., 32 months after its discovery, Covid-19 had infected over 586 million people worldwide, of whom 6.42 million have passed away (death rate of about 1.1%). By the same time, about 62.6% of people worldwide were fully vaccinated since the end of December 2020, having been administered 12.4 billion doses.1 The development of Covid-19 vaccines has accelerated since the World Health Organization declared a pandemic in March 2020 [6,7]. After conducting a literature review and meta-analysis, Zheng et al. [8] concluded that these vaccines, especially Pfizer-BioNTech and Moderna vaccines, are highly protective against SARS-CoV-2 virus-related diseases in real-world settings. However, there are still about one million new infections and over three thousand deaths daily (August 2022),2 a worrying figure resulting perhaps from the relatively low rate of people fully vaccinated.

The first patient infected with SARS-CoV-2 in Portugal was detected in March 2020. Since then, over half of the country's population has been infected: in August 2022, there were 5.37 million Portuguese citizens infected, with 24,707 Covid-19-related deaths.2 It corresponds to an average death rate of 0.46%, far below the world average. One can attribute part of this success to saving lives or avoiding Covid-19-related mortality (compared to the peers) to some sequential strategies implemented, including lockdowns [9], limitation of access to essential services like hospital care, and massive testing and vaccination, despite a low adherence/high hesitance at the beginning [10]. Compared to the global average, Portugal has performed well regarding vaccination. 86.4% of Portuguese citizens are fully vaccinated against SARS-CoV-2, having been administered 24.8 million doses so far (August 2022). Perhaps because of such a high rate, most stringency measures taken to hold the outbreaks were lifted in early 2022. Still, Portugal is amongst the countries with the highest infection rates, which does not seem to worry the Portuguese Ministry of Health, given the achieved high full vaccination rate.

Although beneficial to contain the virus from spreading, blocking access to health care services was not without adverse effects on citizens' quality of life. Medical appointments, surgeries, exams, and other treatments in primary and secondary health care levels were postponed, and admission to emergency rooms and other hospital services was limited to those severely ill. However, denying or delaying care to those searching for it will eventually cause the exacerbation of their illnesses, which become more complex to treat and, ultimately, costlier. This fact jeopardizes the Portuguese publicly available National Health Service's (already weakened) sustainability. Besides, the barriers created for entering the health system could have contributed to the increase in excess mortality, not directly related to Covid-19. For instance, the non-Covid-19 excess mortality in Portugal fluctuated between 51% and 92% in 2020 [11].

Any health care provider, hospitals included, aims at improving patients' quality of life while being economically, socially, and environmentally sustainable [12]. In Portugal, given the universal and general nature of its National Health Service (NHS), these providers must be capable of treating all patients efficiently without denying them timely, appropriate, evidence-based, and safe care services, equitably and regardless of the patients' purchasing power [13]. Excellence or quality in health care provision is achieved when all these aspects are met, i.e., when providers are efficient and effective in treating their patients, and there are no barriers to access. Meanwhile, managers should be held responsible for the poor or good health care results regarding both resource consumption and response to society [14].

It should be evident that the effects of a pandemic on hospital performance may not be the entire responsibility of its managers and staff. Part of the results during such a period result from policy measures imposed at the central, regional/federal, or local levels, but also from other factors non-discretionary to hospital managers. Nevertheless, one must assess hospital quality results to search for best practices, i.e., the resilient entities that kept or even improved their performance during the difficult times of the most recent known pandemic. By doing so, hospitals with poor outcomes may adopt and adapt those practices to improve their own, hopefully becoming more resilient in future outbreaks [15]. Also, the design and implementation of good contingency plans could contribute to a quicker response by these players, avoiding events like the overload and saturation of health systems' resources in the early stages of a pandemic [16]. With this study, we do not aim at demerit hospitals with poor performance during pandemic times.

Several studies have been published on this topic, evaluating the performance of hospitals or health care systems. Concerning the former scenario, Nepomuceno et al. [17] used a Data Envelopment Analysis (DEA) model for hospital bed vacancy and reallocation during the Covid-19 pandemic. That way, the authors observed the existence of 3,772 beds feasible to be evacuated by 64% of health units. Kamel and Mousa [18] estimated the operational efficiency of isolation hospitals in Egypt. They found that less than half of the sample achieved efficiency, and the number of nurses and beds explained more the results than the number of medical doctors. Through a value-based DEA model, Henriques and Gouveia [19] assessed the outbreak's impact on the efficiency of Portuguese state-owned enterprise hospitals. They reported that the lack of resources does not explain inefficiency levels.

There are many more studies comparing countries or states within one country. Aydin and Yurdakul [20] integrated machine learning (ML) with a stochastic imprecise DEA model to assess the performance of 142 countries against the Covid-19 outbreak. In short, the machine learning techniques included clustering analyses with k-means and hierarchical clustering methods. The authors found that gross domestic product, smoking rates, and the rate of diabetic patients do not explain the effectiveness level of countries. Taherinezhad and Alinezhad [21] also integrated ML with DEA to evaluate nations' performance during the outbreak. Ibrahim et al. [22] utilized DEA for country efficiency analysis and concluded that nearly 90% of studied countries were inefficient in pandemic control, and 80% were inefficient during the treatment of positive cases. Using an inverted DEA model, Ferraz et al. [23] constructed a Covid-19 index for Brazil's microregions; unsurprisingly, the poorest regions are the most vulnerable. Differently, Mariano et al. [24] used the Network-DEA (NDEA) to analyze the regional discrepancies within the same country, having identified the state of Amazonas as the least efficient. Similarly, Hamzah et al. [25] also used an NDEA model to investigate the relative efficiency level of managing Covid-19 in Malaysia. The authors verified a good overall performance of the Malaysian health system, reflecting the robust preparedness and rapid reallocation of resources during the outbreak. Also resorting to the DEA model potential, Klumpp et al. [26] compared the performance of countries within the OECD and found that systems oriented to primary care were more efficient than the others. In opposition, highly dependent on public funding and heavily regulated health care systems were less efficient.

Besides the fact that most studies we found are focused on comparing countries or states (leaving a considerable gap in the literature), researchers seem more interested in the technical efficiency of resource utilization rather than the effective results in health care. Most results achieved concern the first year of the pandemic, while more than twenty months have gone since. Given the growth rates of infected cases and mortality because of Covid-19, it is difficult to understand whether the results are already outdated. In our perspective, evaluating hospital performance requires efficiency and effectiveness analyses in a time range broader than one year to check if low-quality hospitals remain as such during the pandemic or if high-quality hospitals become less resilient, worsening their health results. No study before this has tried to understand if changes in hospital performance could be affected by Covid-19-specific spread features.

Additionally, one must account for the fact that hospitals do not operate all in the same environment, and some are impaired because of the hazardous conditions in which they operate. Allegedly, the earlier studies seem to have overlooked this important aspect. Researchers on hospital performance tend to consider population demographics and epidemiology variables to characterize that operational environment [27]. Pecoraro et al. [28] analyzed and compared the efficiency of ordinary hospital bed management in France, Germany, Italy, and Spain. The authors concluded that structural components of these countries justify part of the pressure in hospital systems during pandemics. These structural components included the heterogeneity of Covid-19 case distribution and the availability of beds. Concerning the first component, it is worth mentioning that the distribution of cases is linked to some population characteristics, particularly the education level, related to compliance with the implemented sanitary measures. For instance, Carlucci et al. [29] concluded that women, most educated people, middle-aged individuals, and health workers are more likely to adhere to lockdowns and other imposed policies. Jabbari et al. [30] and Oyeyemi et al. [31] reached similar conclusions, indicating that people's education (health literacy) plays a vital role in containing outbreaks. Furthermore, the success of hospitals largely depends on the complexity of cases handled therein [32]. Concerning Covid-19, evidence has suggested that aging, gender, chronic underlying disease (like hypertension and diabetes mellitus), mental status impairment, length of hospital stay, and high risk of acute deterioration are risk factors associated with unfavorable outcomes [33,34]. It should be evident that hospitals in areas with higher rates of chronic diseases or older/less educated populations are under worsened conditions to treat their patients. Thus, there should exist an adjustment mechanism for hospital benchmarking.

The objectives of this study are fourfold: first, to estimate the hospital quality levels (performance) individually and momentaneously, with a correction for the operational environment; second, to compare performance results and evaluate a trend; third, to investigate whether the hospital quality worsened because of the outbreak of SARS-CoV-2; forth, to test if the disease spreading features (e.g., reproduction rate, hospital admissions due to Covid-19 per million inhabitants) help to explain the estimated performance evolution.

Based on these objectives, we formulate three main research questions and two hypotheses per question. In line with the paper's primary goal, these research questions are linked to hospital performance before and during the Covid-19 pandemic. We consider the period before (January 2017 to February 2020) as a baseline to which we can compare the results of the next time interval (March 2020 to May 2022).2 By the time we wrote this paper (December 2022/January 2023), the pandemic was not considered extinct. Thus, comparing the hospital performance before, during, and after the pandemic is impossible. As commonly made in statistics, we state our null hypotheses considering that there is a chance that hospitals may have worsened their performance during the outbreaks, but there is also a chance that they have improved it, at least in some performance dimensions. In other words, we assume the absence of performance change over time by the null hypothesis and should search for evidence disproving it. It is because there are no solid theoretical suggestions concerning this topic. That is, the research questions, R_i, and corresponding null, H₀, and alternative, H₁, hypotheses are as follows:3

(R₁) How did the static performance of Portuguese public hospitals evolve from January 2017 to October 2021?

H_0(R1): The static performance of Portuguese public hospitals did not change in the considered period.

H_1(R1): Portuguese public hospitals' static performance changed in that period.

(R₂) How did the dynamic performance of Portuguese public hospitals grow from January 2017 to October 2021?

H_0(R2): The dynamic performance of Portuguese public hospitals did not change in the considered period.

H_1(R2): Portuguese public hospitals' dynamic performance changed in the period.

(R₃) Can SARS-CoV-2-related indicators justify the performance evolution in Portuguese public hospitals?

H_0(R3): SARS-CoV-2 does not influence the performance evolution of Portuguese public hospitals.

H_1(R3): SARS-CoV-2 influenced the performance evolution of Portuguese public hospitals.

2. Materials and Methods

2.1. Quality in Health (Hospital) Care

2.1.1. Definition of Quality in Health (Hospital) Care

Measuring the performance of hospitals is not an easy task. Indeed, it is necessary to consider all possible dimensions of hospital care and how they interact to reach the hospital's primary goal: to improve the patient's quality of life while operations remain financial and environmentally sustainable [35]. Meanwhile, quality of life is another quasi-transcendent concept resulting from health and non-health-related dimensions [36]. Concerning the capacity of hospitals (or any other health care entity) to improve patients' quality of life, those dimensions are primarily associated with the results (e.g., the patient lives or dies after surgery) but also with the process of care (e.g., the patient suffered a septicemia event in the nursery) and hospital attributes (e.g., facilities are clean and technology-updated); see Ferreira, Marques, Nunes & Figueira [37]. Overall, such a capacity is inherently linked to the quality of care [38,39,40], ultimately leading to patient (dis)satisfaction [41] and quality of life. The (North American) Institute of Medicine defines the quality of care using six domains: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity in access [42,43,44]:

• Safety is the capacity to avoid harm to patients;
• Effectiveness is related to "making the right things," which in health care corresponds to using scientific knowledge to treat patients in the best possible way;
• Patient-centeredness corresponds to human and social skills, necessary in any health care treatment as patients' needs, beliefs, values, preferences, gender, sexual orientation, and ethnicity must be respected;
• Timeliness is the capacity to provide care whenever the patient needs it, without potentially harmful delays;
• Unlikely effectiveness, efficiency (and productivity) is related to "making things right" without wasting resources; and
• Equity regards the fairness of resource distribution, as two patients in the same condition should receive equal treatment.

Table 1 provides the correspondence between the main quality domains identified above in different literature sources. Although the Institute of Medicine divides quality into six items, they are somehow linked to the three quality categories according to Donabedian (attributes, process of care, and results). The attributes are associated with the infrastructure and facilities, thus with the availability of resources, which in turn condition the access (and timeliness) to the service and its efficiency. Indeed, given the patient level in hospital service, the more resources, the better the access but the lower the efficiency. Thus, a trade-off between these two concepts is expected to exist. The care process comprehends all the events once the patients enter the hospital infrastructure until they leave the system. These events can be adverse (like in-hospital infections) or not, but result primarily from the received care, especially the patient safety (or absence of it) and the patient-centeredness. The latter is heavily linked to care appropriateness. Finally, the care results include improved quality of life and patient satisfaction [45].

2.1.2. Quality Variables

Building on the previous discussion and the definition given by the Portuguese Ministry of Health itself, we consider the following set of criteria: (1) efficiency and productivity; (2) access; (3) safety; and (4) care appropriateness. Each criterion is operationalized by several indicators that have been extensively used in the literature concerning Portuguese public hospitals; see e.g., Amado et al. [46], Ferreira, Marques & Nunes [47], Ferreira & Marques [48,49], Pereira et al. [50], Ferreira, Nunes & Marques [51,52], and Pederneiras et al. [12], to name a few. The main reason behind these indicators is that they are monitored by the Ministry of Health and figured out in yearly contracts between the hospitals' management and the ministerial tutelage. For that reason, among many other possible indicators, these appear to be most relevant for the primary stakeholder [53]. However, since safety and care appropriateness may be interlinked and the boundary between them is not clear, we decided to gather their related indicators in the same group:

• Efficiency and productivity:

  a)

  Occupancy rate. This variable stands for the average rate of beds occupied by an inpatient each day (e.g., on an average day, 75 beds out of 150 were occupied; thus, the occupancy rate was 75/150 = 0.5 or 50%). The optimal occupancy rate ranges from 80 to 90%, being 85% frequently deemed as the optimal occupancy rate [54]. Less than 80% indicates underutilization of beds or excessive resources (low efficiency). In comparison, an occupancy rate above 90% suggests overutilization of beds and a lack of this resource for peak events, like during SARS-CoV-2 pandemic outbreaks. Furthermore, high occupancy rates tend to directly influence the incidence of hospital-acquired infections [55].

  b)

  Standard patients per Full-Time Equivalent (FTE) doctor. This indicator reflects the productivity of hospitals by relating the standardized number of patients seen and a resource (doctors in this case). More patients per FTE doctor means larger hospital productivity (monetization of an asset).

  c)

  Standard patients per FTE nurse is another productivity indicator with a similar interpretation.

• Access:

  a)

  Rate of first medical appointments within the legally fixed period. There are two main ways of getting a medical appointment in a Portuguese public hospital: either through the emergency room or via health care centers (primary care). For the latter scenario, Portuguese legislation defines the maximum time between the request and the first appointment. This indicator measures how many patients have seen their access to secondary care denied or delayed. The larger the indicator, the better the access to care and, consequently, the hospital performance. During the Covid-19 pandemic, many non-urgent medical appointments were canceled, decreasing this indicator (and the access). Accordingly, the patients' health status may have worsened, reducing their quality of life and increasing the costs of future health services (as the severity of illness is positively associated with health expenditures; see Thuong et al. [56]).

  b)

  Rate of enrolled patients on the waiting list for surgery within the legally prescribed time. As before, there is a maximum legal time for patients to enroll on the waiting list for surgery, either major (requiring hospitalization) or minor. A low rate means that patients face difficulties accessing the service they need, i.e., a barrier that may result from administrative processes, bureaucracy, or lack of resources. During the Covid-19 pandemic, most non-urgent surgeries were canceled, meaning that this indicator (and the access) decreased. For instance, Ciarleglio et al. [57] and the CovidSurg Collaborative [58] reported the harmful effects of Covid-19 and lockdown on emergency and elective surgery due to delayed access.

  c)

  Average time before surgery. This indicator measures the average number of days the patients stay in the hospital ward after admission until they are surgically operated in the operating room. More significant average times mean that patients unnecessarily occupy a bed (and other resources) that another patient elsewhere could use.

  d)

  Rate of hip surgeries within the first 48h. This indicator quantifies the percentage of geriatric hip surgery within the first 48h after fracture (in the total hip surgeries). Hip fracture has long been reported as an essential predictor of in-hospital mortality in patients aged 65 years or older [59]. Two days (48h) of patient presentation is the limit of time recommended by the American Academy of Orthopedic Surgeons for hip surgery [60] to prevent complications. Thus, this indicator is a good proxy for hospital timeliness. Interestingly, Brent et al. [61] observed a 15% reduction in admissions for hip surgery as well as a reduction in compliance with many surgery standards following the Covid-19 pandemic in Ireland.

• Safety and care appropriateness:

  a)

  Bedsore rate. Bedsores or pressure ulcers are skin or underlying tissue injuries commonly found in low-mobility patients' heels, ankles, and hips, spending most of the in-hospital time lying on their beds. High rates indicate a considerable probability of bedridden patients developing skin wounds, thus jeopardizing their safety. Challoner et al. [62] mentioned that prone positioning has been employed to treat severe hypoxia in Covid-19 patients, which may constitute a risk of developing pressure ulcers on the head, neck, and genitalia. Sleiwah et al. [63] report similar findings regarding the perioral pressure ulcers resulting from using devices to secure endotracheal tubes in Covid-19 patients admitted to the intensive care units. These results thus suggest that patients' safety in terms of bedsores may have been compromised during the pandemic.

  b)

  Rate of in-hospital developed septicemia (postoperative). The indicator is the percentage of septicemia cases developed in-hospital divided by the total inpatients. Septicemia or nosocomial infection is caused by bacteria, viruses, and fungi and is acquired during hospital ward stays. If developed within the hospital (often in the postoperative period), this event results from the lack of patient safety, primarily poor cleanliness of materials. Some authors have reported an increase in nosocomially acquired infections during the Covid-19 pandemic, mostly because of ventilator-associated pneumonia and bacteremia [64,65]. Therefore, the literature suggests that this indicator has probably increased, implying that patients' clinical safety is worsening.

  c)

  Rate of catheter-related bloodstream infection events. Catheter-related bloodstream infections result from bacteremia in intravenous, not adequately sterilized catheters, being a significant cause of nosocomial bacteremia. These costly events and complications may cause high morbidity and mortality [66]. Recently, Pérez-Granda et al. [67] noticed an increase in the frequency of catheter-related bloodstream infections during the Covid-19 pandemic, claiming the need to reinforce classic and new preventive measures to avoid these events. The authors associate the increase of infections with the harsh circumstances (increased workload and use of staff with a sub-optimal degree of training with intensive care patients). However, other authors reached the opposite conclusions, e.g., Heidempergher et al. [68]. That being said, the literature is not clear about the effect of the pandemic on this indicator and, consequently, the hospital's performance regarding the patients' safety.

  d)

  Rate of postoperative pulmonary embolism events and thromboembolisms. Thromboembolisms occur when blood clots form in deep veins and break loose, traveling through the bloodstream, often to the lungs (pulmonary embolism). This event is more likely to occur after major surgeries or injuries. The consequences include blood flow and oxygen restrictions, damaging organs and tissues, and ultimately causing death. Narayan et al. [69] mentioned that hospital-acquired venous thromboembolism is a leading cause of morbidity and mortality, and about one in ten cases is preventable. Meanwhile, this value increases for critically ill patients due to elevated risk of thrombosis, like coma or paralysis [70,71]. Schulman [72] mentions that the best estimates indicate that about half a million Americans each year suffer from pulmonary embolisms. At least a tenth of a million deaths may be directly or indirectly related to these diseases, which are too many as this in-hospital death cause is highly preventable. Additionally, Covid-19 can lead to systemic coagulation activation and thrombotic complications [73], resulting in pulmonary embolism events and thromboembolisms. However, a systematic review and meta-analysis conducted by Porfidia et al. [74] show that the incidence of this disease in Covid-19 patients is unclear.

  e)

  Rate of performed minor surgeries in the potential minor surgeries. Minor surgeries, like dental restorations and cataract surgeries, are minimally invasive procedures that do not require an operating theatre or an inpatient service admission. In opposition, major surgeries like cesarean sections, organ or joint replacements, total hysterectomies, and heart or bariatric surgeries usually involve opening the body and, consequently, major tissue trauma and a more significant risk of infection (worsening the patients' safety). Recoveries in these cases are more extended than minor surgeries. In many cases, however, one can solve the same problem through minor or major surgery. The best alternative depends on each case, but the benefits are frequently similar. Therefore, the infection risks and recovery period must not be overlooked nor outweighed. Medical guidelines argue that should the patient's clinical issue be appropriately solved through minor surgery, one should adopt it instead of a major procedure to reduce the risk of infection and recovery. Thus, the performance indicator is such that the closer to 100%, the better the care appropriateness. Baboudjian et al. [75] concluded that minor surgery is still safe in the Covid-19 era if all appropriate protective measures are implemented. This result suggests that there was no significant decrease in the indicator. Although many surgeries were canceled or postponed, the ones that were not could have been minor procedures (whenever appropriate) to limit the patient exposition to SARS-CoV-2, hopefully increasing this indicator.

  f)

  Rate of readmissions within 30 days after discharging. Readmitting patients after releasing them for the same reasons of the first admission results, in many cases, from poor care appropriateness. For instance, the patient was not totally healed and was incorrectly discharged, searching for health care for a while. However, it is usual that the clinical condition has worsened, making the patient's illness more severe and complex. The literature suggests that the 60-day readmission of Covid-19 survivors is less likely than that of pneumonia or heart failure survivors, but the opposite conclusion concerning the 10-day readmissions was also reached [76]. However, recent studies are more concerned with the readmission of Covid-19 patients than other patients readmitted in the Covid-19 era. Therefore, there is no clear evidence that total readmissions have increased or decreased in this period.

  g)

  Rate of inpatients staying hospitalized for more than 30 days. Staying in the hospital ward for more time than required dramatically increases the risk of acquiring severe nosocomial infections [77], developing other comorbidities, or even dying. With the pandemic's development, the risks associated with lengthy stays in the hospital may have increased, jeopardizing the patients' safety. For instance, the longer the patient stays in the inpatient service, the higher the probability of being infected by SARS-CoV-2; thus, the higher the risk of developing an often-fatal bacteria-related hospital-acquired pneumonia [78].

Although neither of these variables is strictly linked to SARS-CoV-2-related hospital admissions, the truth is that the pandemic may have played an important role in delaying and worsening health care for other patients that represent most of the hospital production. In addition, Covid-19-related hospital admissions were low compared to the total inpatients in Portugal; thus, it seems pointless to include specific dimensions of the Covid-19 pandemic when measuring a general hospital's performance. These, measured typically at the national or municipal level, can instead be used as independent variables, potentially explaining the hospital's performance.

The pandemic may have worsened the patients' health status before and during internment in the hospital ward. In some cases, the decrease in hospital performance may not have resulted from poor safety or lack of care appropriateness. Instead, the increase in adverse events may have resulted from the increased severity and complexity of the illness. Therefore, there must exist an adjustment for these situations. The case-mix index (CMI) is widely spread across the healthcare management literature [79]. The higher (lower) the index is, the more (less) complex the patients treated in the hospital are. A unitary CMI corresponds to a year's national standard. Meanwhile, CMI = 1 + k suggests that, on average, the hospital treats patients whose complexity is larger than the national standard, thus consuming k% more resources than an average hospital (k can be either positive or negative). However, Ferreira and Marques [80] verified that the CMI is ineffective when evaluating performance through time as the baseline (national average expenditures) is not steady. Additionally, the authors concluded that once appropriate demography and epidemiology-based have been included in the model to help explain efficiency, the CMI becomes useless. Later, to study the impact of patient safety, care appropriateness, and access on hospital efficiency in a single moment, the same authors used the CMI to adjust the performance indicators [35]. Their strategy encompassed the classification of these indicators into desirable and undesirable. The former corresponds to those indicators contributing positively to the performance (whenever increasing), like the rate of hip surgeries within the first 48h. Conversely, undesirable indicators contribute negatively to performance when increasing, such as the rate of readmissions within 30 days after discharge.

Once classified, the indicators were adjusted with the CMI as follows: undesirable indicators were divided by the CMI while the desirable ones were multiplied by it. That way, one expects the impact of patients' high complexity in performance indicators to be mitigated, and hospitals may become comparable. Provided that the baseline used to compute the CMI every year tends to change, we opted to follow Herr [81] and adopt the average delay as a proxy of complexity. It is a valid assumption as the more complex the patients are, the more days they must stay in the hospital ward. Let $d_j^t$ be the annual number of hospital days in hospital j (j=1,…,n) and year t (t=0,…,T); likewise, let $p_j^t$ be the total inpatients admitted to that hospital, j, in the same year, t. The average delay is simply $A{D}_j^t=d_j^t/p_j^t$. The national average delay in the period [0, T] (i.e., the baseline) is $NAD={\displaystyle \sum }_t{\displaystyle \sum }_j{d}_j^t/{\displaystyle \sum }_t{\displaystyle \sum }_j{p}_j^t$. Therefore, the proxy of complexity based on average delay is simply $C_j^t=A{D}_j^t/NAD$. Should $C_j^t>1$, the more days (on average) the patients were hospitalized in hospital j and year t, compared to the baseline, thus expectedly more complex and consuming more resources. Thus, the indexes $C_j^t$ and CMI have the same interpretation.

2.2. Performance Assessment Methods

2.2.1. The Benefit-of-Doubt Approach

One easy and straightforward way of estimating a hospital's performance based on several indicators is the so-called Benefit-of-Doubt (BoD), derived from the well-known Data Envelopment Analysis (DEA). Let $l_{ij}^t$ denote the observation associated with the ith indicator (i=1,…,m), the jth hospital (j=1,…,n), and the moment t. These indicators assume non-negative values. It is believed that they contribute positively to the performance of the hospital. Since it is not always the case, we follow Cherchye et al. [82,83] and scale the indicators as follows:

$${\ell }_{ij}^t=\{\begin{array}{c}\frac{\underset{j,t}{\max }{l}_i-l_{ij}^t}{\underset{j,t}{\max }{l}_i-\underset{j,t}{\min }{l}_i}, \mathrm{if} \mathrm{the} \mathrm{indicator} \mathrm{is} \mathrm{undesirable}\\ \frac{l_{ij}^t-\underset{j,t}{\min }{l}_i}{\underset{j,t}{\max }{l}_i-\underset{j,t}{\min }{l}_i}, \mathrm{if} \mathrm{the} \mathrm{indicator} \mathrm{is} \mathrm{desirable}\\ 1-\left|\frac{o_i-l_{ij}^t}{\underset{j,t}{\max }{l}_i-\underset{j,t}{\min }{l}_i}\right|, \mathrm{otherwise}\end{array}$$

(1)

In Eq. (1), $o_i$ stands for a hypothesized optimal value associated with the indicators that are neither desirable nor undesirable. In some cases, indicators should not be maximized nor minimized, but there is a value (or a range) in which the optimal indicator should lie. For instance, the occupancy rate should range between 80 and 90% so that we can fix $o_i$ as the middle point of the interval, i.e., (90+80)/2 = 85%. After Eq. (1), the resulting indicators range between 0 and 1, and it becomes clear that the higher, the better.

There should exist a set of m non-negative weights, $w_{1j}^t,\dots ,w_{ij}^t,\dots ,w_{mj}^t$, that maximize the overall composite indicator, CI, for the hospital j in instant t, from now on denoted by $C{I}_j^t$ [84]. Let us assume that such a CI can be assessed through an additive model: $C{I}_j^t={\displaystyle \sum }_{i=1}^m{w}_{ij}^t{\ell }_{ij}^t$. Because the weights are such that the CI is maximal, no other weights set should result in a higher CI. These weights (also named multipliers) are hospital and time-dependent. Although some authors have criticized the fact that weights are not equal for all entities under evaluation [85], in the words of Grego et al. [86] and Decancq and Lugo [87], this is the method’s beauty as common-weights "decrease the desirability of this method – that of favorable weights in the eyes of policymakers – based on which this approach gained such momentum in the first place."

Provided that all indicators should be considered for the construction of the CI, weights must not be zero but above a threshold, $\zeta >0$, denoting the minimum acceptable value for a weight: $w_{ij}^t\ge \zeta , i=1,\dots ,m, j=1,\dots ,n$. Let us assume $\zeta =0.05$. Moreover, it should be possible to impose that an indicator (compared to the others) is not underrepresented; therefore, we add the following constraint: $w_{ij}^t/{\displaystyle \sum }_r{w}_{rj}^t\ge \xi \iff \xi {\displaystyle \sum }_r{w}_{rj}^t-w_{ij}^t\le 0, \forall i=1,\dots ,m$, for a given level $\xi$, say $\xi =0.05$ as well [88]. This constraint reduces the heterogeneity of weights' distributions. The resulting linear-programming BoD model is as follows [89]:

$$\begin{array}{l}C{I}_h^t=\underset{w}{\max }{\displaystyle {\displaystyle \sum }_{i=1}^m}{w}_{ih}^t{\ell }_{ih}^t\\ \mathrm{s}.\mathrm{t}.\\ {\displaystyle {\displaystyle \sum }_{i=1}^m}{w}_{ij}^t{\ell }_{ij}^t\le 1,j=1,\dots ,n,\\ \xi {\displaystyle {\displaystyle \sum }_r}{w}_{rj}^t-w_{ij}^t\le 0, i=1,\dots ,n, j=1,\dots ,n,\\ w_{ij}^t\ge \zeta , i=1,\dots ,m, j=1,\dots ,n.\end{array}$$

(2)

As obtained from Eq. (2), the CI is bounded upwardly by one and downwardly by zero. If CI = 1, the hospital h is performing well at t and can be considered a best practice among its peers (of the same period). Otherwise, its performance is not outstanding, and the managers should revise processes (following a search for best practices) to improve it.

As detailed before, we have three distinct criteria describing hospital care quality. In that sense, we can apply Eq. (2) for the set of indicators corresponding to each criterion and construct four partial performance indicators: efficiency composite indicator (ECI), access composite indicator (ACI), and safety/care-appropriateness composite indicator (SCACI). Finally, and following Matos et al. [90], we can aggregate these three into an overall CI using Eq. (2) but with ECI, ACI, and SCACI instead of the 16 standardized indicators $\ell$. That way, we reduce the impact of many dimensions in results resolution, commonly known as the curse of dimensionality. Simultaneously, the influence of each initial indicator on constructing the partial performance indicators is maintained. That is, since $w_{i_{1}j}^t$ is the weight of indicator i to evaluate hospital h in instant t in terms of ECI, and $W_{1j}^t$ is the weight of ECI in the overall composite indicator, the contribution of the initial indicator is given by $w_{i_{1}j}^t\cdot W_{1j}^t$.

2.2.2. Performance Evolution

As determined using Eq. (2), the CI provides a static performance measure. However, the performance also encompasses a dynamic parcel that needs to be evaluated to investigate if hospitals have approached the best practices and improved their performance during the evaluation time. Indeed, we can build on Ferreira and Marques [91] and propose the following Total Factor Productivity (TFP) associated with the hospital j and two instants, t and t+1:

$$TF{P}_j\left(t,t+1\right)={\left({\displaystyle \prod }_{i=1}^m{\left[\frac{{\ell }_{ij}^{t+1}}{{\ell }_{ij}^t}\right]}^{{\omega }_{ij}} \right)}^{1/{\displaystyle \sum }_i{\omega }_{ij}},$$

(3)

where ${\omega }_{ij}$ is a non-negative weight of indicator i and hospital j that resulted from averaging the weights $w_{ij}^t$ obtained for that indicator and that hospital all over the period. Thus, if N is the number of evaluated time instants, ${\omega }_{ij}=\frac{1}{N}{\displaystyle \sum }_{t=1}^N{w}_{ij}^t$. TFP > 1 means that hospital j watched a productivity improvement between t and t+1, resulting from enhancements in (most of) the indicators.

Building upon Portela & Thanassoulis [92,93], we can formulate the weighted Geometric Distance Function (GDF) that relates to the CI as computed by (2). Let us use a star * to denote the optimal level of an indicator given an instant t. These targets can be easily obtained by constructing the dual version of the model (2). The GDF of hospital j in moment t is:

$$GD{F}_j^t={\left({\displaystyle \prod }_{i=1}^m{\left[\frac{{\ell }_{ij}^t}{{\left({\ell }_{ij}^t\right)}^{*}}\right]}^{{\omega }_{ij}}\right)}^{1/{\displaystyle \sum }_i{\omega }_{ij}},$$

(4)

and the closer to one, the higher the hospital's performance at that moment. Note that some hospitals may have (scaled) indicators equal to zero, which would mean a GDF equal to zero (while they could exhibit good performance in other indicators). We replace those zeros with minimal positive quantities to avoid this problem.

The static performance evolution between those moments can be defined as the ratio between two GDF measures:

$$\begin{gathered} P_j\left(t,t+1\right)=\frac{GD{F}_j^{t+1}}{GD{F}_j^t}={\left({\displaystyle \prod }_{i=1}^m{\left[\frac{{\left({\ell }_{ij}^t\right)}^{*}}{{\left({\ell }_{ij}^{t+1}\right)}^{*}}\frac{{\ell }_{ij}^{t+1}}{{\ell }_{ij}^t}\right]}^{{\omega }_{ij}}\right)}^{1/{\displaystyle \sum }_i{\omega }_{ij}}= \\ \hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}\hspace{1em}=TF{P}_j\left(t,t+1\right){\left({\displaystyle \prod }_{i=1}^m{\left[\frac{{\left({\ell }_{ij}^t\right)}^{*}}{{\left({\ell }_{ij}^{t+1}\right)}^{*}}\right]}^{{\omega }_{ij}}\right)}^{1/{\displaystyle \sum }_i{\omega }_{ij}}, \end{gathered}$$

(5)

that is, the performance change between two moments, $P_j\left(t,t+1\right)$, is related to productivity change in the same period, as measured by the TFP. If P > 1, hospital j improved its (static) performance between t and t+1. According to Eq. (5), the static performance evolution is proportional to the productivity growth, and the remaining parcel is the reciprocal of the technology shift, T, between t and t+1:

$$TF{P}_j\left(t,t+1\right)=P_j\left(t,t+1\right)\stackrel{T_j\left(t,t+1\right)}{\overbrace{{\left({\displaystyle \prod }_{i=1}^m{\left[\frac{{\left({\ell }_{ij}^{t+1}\right)}^{*}}{{\left({\ell }_{ij}^t\right)}^{*}}\right]}^{{\omega }_{ij}}\right)}^{1/{\displaystyle \sum }_i{\omega }_{ij}}}}=P_j\left(t,t+1\right)T_j\left(t,t+1\right),$$

(6)

i.e., the technology shift T is the relationship between targets in both moments. T > 1 indicates that benchmarks in t+1 exhibit better performance than those in t. In other words, there was an improvement in the dynamic performance of hospital j.

This simple decomposition of TFP into two indices (P and T) presupposes that the hospitals face constant returns to scale, which is a reliable assumption since the indicators $\ell$ are ratios, meaning that two hospitals distinct in terms of size become comparable. In other words, the notion of size vanishes.

We individually evaluate each performance growth indices, TFP, P, and T, per quality criterion (efficiency and productivity, access, safety, and care appropriateness) and their aggregation (overall composite indicator). The evaluation was made monthly, starting January 2017 and ending May 2022 (i.e., 59-time intervals: Jan 2017-Feb 2017; to Apr 2022-May 2022).

2.2.3. A Relational Model

To answer the third research question, (R₃), we use the multiple linear regression model; see Eq. (7):

$$E\left[y\right]={\alpha }_0+{\alpha }_1{x}_1+{\alpha }_2{x}_2+\cdots +{\alpha }_n{x}_n={\alpha }_0+{\displaystyle \sum }_{p=1}^q{\alpha }_p{x}_p,$$

(7)

with the stepwise forward method as a heuristic to obtain the most representative model with only the statistically relevant independent variables, $x_p, p=1,\dots ,q$, and no multicollinearity problems [36]. We consider three dependent variables (TFP, P, and T), y, resulting in three distinct models to check if the pandemic’s evolution may help explain the growth or decay of any of these indices. The selected independent variables, x, reflect the national average monthly change of SARS-CoV-2 outbreaks in Portugal since March 2020, when the virus was first introduced within the country. Therefore, the following seven (independent, explanatory) variables are measured as rates of growth between t and t+1 (for t > February 2020), i.e., measured as percentages, $\raisebox{1ex}{$x_p\left(t+1\right)-x_p\left(t\right)$}\!\left/ \!\raisebox{-1ex}{$x_p\left(t\right)$}\right., p=1,\dots ,7$:

x1. 

Infected people per million inhabitants. The number of positive cases of SARS-CoV-2 multiplied by 1,000,000 and divided by the number of Portuguese inhabitants.

x2. 

Covid-19-related deaths per million inhabitants. The number of deaths because of SARS-CoV-2 was multiplied by 1,000,000 and divided by the number of Portuguese inhabitants.

x3. 

Reproduction rate. Usually represented by R, this rate is a standard transmissibility parameter that measures how many people can be infected by a positive case. For instance, R=2 means that one infected person can infect two people with the Covid-19 virus. Therefore, the reproduction rate must be below one to curb the spread of a pathogen.

x4. 

Intensive care unit admissions (because of Covid-19) per million inhabitants. The number of Covid-19-related hospital entries requiring intensive care, multiplied by 1,000,000 and divided by the number of Portuguese inhabitants. The search for intensive care because of SARS-CoV-2 resulted from severe consequences of the disease like pneumonia, acute respiratory distress syndrome, multi-organ failure, septic shock, and, in many cases, death. Such a demand may have compromised the access to the same level of care by other patients as beds, and other resources (such as ventilators) are of limited availability.

x5. 

Hospital admissions (because of Covid-19) per million inhabitants. The number of Covid-19-related hospital ward admissions not requiring intensive care, multiplied by 1,000,000 and divided by the number of Portuguese inhabitants. Although with less severe complications than those admitted to the intensive care unit, these inpatients also demand specialized nursery care. As in the previous case, this demand may constitute a barrier to access by other patients.

x6. 

Vaccination (complete) rate. This indicator measures the percentage associated with the fully vaccinated population (BioNTech/Pfizer, Moderna, Novavax, AstraZeneca, Johnson & Johnson). For instance, a citizen with a single shot of BioNTech/Pfizer is not considered as the minimum number of doses required is two. Evidence suggests that complete vaccination diminishes the severity of illness provoked by Covid-19, thus the need for hospitalizations and the burden on hospitals. This variable was zero until December 2020, when the first citizens got the shots.

x7. 

Stringency index. According to the Oxford Coronavirus Government Response Tracker, the stringency index is a composite indicator based on nine response metrics related to the restrictions imposed by governments: school closures; workplace closures; cancellation of public events; restrictions on public gatherings; closures of public transport; stay-at-home requirements; public information campaigns; restrictions on internal movements; and international travel controls [94]. The index ranges from 0 to 100, the highest level associated with the strictest response. One of the major goals underlying the imposition of these restrictions is the reduction of infected people and, by extension, the hospital burden with patients requiring (often intensive) medical care.

2.3. The Case Study

2.3.1. The Portuguese National Health Service

The Portuguese health care system is featured by a complex network of providers, rulers, and regulators. Providers can be either public or private or even belong to the social sector. The public health care provision service is known as National Health Service (NHS), which follows a Beveridge model, being thus publicly funded with money collected from taxes. The NHS is universal, general, and tendentiously free, with only a few moderating fees when patients access care services [95].

Health care in Portugal has three main "layers": primary (e.g., primary health care centers, clinics), secondary (hospitals), and tertiary (continuing care, palliative care) [96,97]. Hospitals are the entities consuming more resources within the national health system. According to Portugal Statistics, hospitals spent roughly €9,110,868 thousand in 2020, representing 43% of total health care expenditures.4 Also, these entities more than doubled the consumed resources in twenty years (€4,212,793 thousand in 2000), an increase frequently associated with the aging population and technological and pharmaceutical developments [98]. The NHS represented a total expenditure of €11,679.8 million in 2020, i.e., 55% of the national health care expenditures. That year, each citizen signified an average burden of €1,186.8 to the NHS, while that value was €609.6 twenty years ago.

There are currently 102 hospital facilities belonging to the NHS, of which 25 are specialized hospitals (e.g., oncology centers, psychiatric hospitals, and maternities) and 77 are general hospitals. Together, these entities have 22,226 beds, of which 91.6% are in general hospitals. There were 21,297 medical doctors and 39,913 nurses working in hospitals of the NHS in 2020. However, that year there were 57,198 registered medical doctors and 77,984 registered nurses, most working in the private or social sectors. There has been an exodus of qualified health staff to these sectors or even abroad, looking for higher wages and working conditions better than in the NHS [99].

Despite the number of physical facilities, there are currently 45 hospital entities in the NHS, resulting from successive structural reforms in the public sector (mainly after the introduction of the New Public Management [100]. These reforms included the vertical and horizontal amalgamation of health care infrastructures nearby to form entities with integrated management, expectedly to enjoy economies of scale and scope [101]. A vertical margining corresponds to the integration of different levels of care, e.g., primary and secondary levels (one hospital and the primary health care centers located in the former's vicinity). The new entities are the Local Health Units (LHU), and there are currently eight of these, primarily located in the countryside, where access to health care is limited. In opposition, the horizontal merging results from integrating two or more close hospitals under the umbrella of the same administrative council, creating the so-called Hospital Centers (HC) [102]. There are 21 HC in Portugal. The remaining 16 public hospital entities are distributed as follows: five specialized hospitals and eleven singletons, one of which is a public-private partnership (PPP). Although the PPP management is not under the direct control of the Ministry of Health, the provider must follow general guidelines from the Ministry and the Directorate-General for Health. Since this entity also belongs to the NHS, it should be considered within our sample.

Past studies concerning the Portuguese NHS have disregarded specialized hospitals (because of their unique production technology), PPPs (because of missing data, especially the financial ones), and LHUs (because collected data regard both the hospital and the primary health care centers). See Matos et al. [90] and Amado et al. [46] for details. Although we agree with removing specialized hospitals from the sample, when we look at the set of selected variables (vide subsection 2.2.1), all concern the hospital activity, and there are no financial data (nor data gaps). Therefore, we should dismiss neither LHUs nor the PPP. That being said, our sample comprises 40 entities observed in a 65-month time frame.

It is also worth mentioning that the Central Administration of Health System in Portugal classifies hospitals in groups, B to F, depending on features like size, scope of activities carried out, differentiation, and operational conditions (epidemiology and demographics). Such a classification resulted from an empirical analysis through k-means and a comprehensive set of variables characterizing hospitals. That way, entities within the same group become comparable as they share size, scope, and environment, an important issue when benchmarking is to be considered. In the words of Cook et al. [103], hospitals must be "assumed to be homogeneous (that is, comparable in terms of the indicators selected for the analysis), while those within the groups share objectives, policies, etc., which may differ across groups." Creating those groups or clusters ensures the desired comparability among hospitals being compared. Therefore, each hospital cluster has its own performance analysis. Cluster B contains the smallest and less differentiated hospitals in a total of nine institutions. Cluster C is the largest one, with 17 health care entities. They receive and treat patients more complex than the hospitals in cluster B. Cluster D comprehends eight central hospitals that are more differentiated than the ones in clusters B and C. Cluster E has the biggest secondary care facilities within the whole NHS, i.e., those treating the most complex patients. Hospitals of cluster E are in Lisbon (the capital), Oporto (the second largest city in Portugal), and Coimbra but receive patients from the entire country. Together, the metropolitan areas of Lisbon and Oporto comprise 45% of the population on Portugal's mainland. Also located in these cities, there are the oncology centers (Instituto Português de Oncologia, in Portuguese words) that form cluster F; for the reasons above, we will not consider this cluster hereinafter.

We gathered data about KPIs from the official database, maintained by the (Portuguese) Central Administration of Health System and publicly available.5 Meanwhile, we collected data on explanatory variables from the Our World in Data platform, a project of the Global Change Data Lab, under the supervision of the University of Oxford.6

2.3.2. Covid-19 in Portugal: Some Figures

Like many other countries, Portugal has been severely hit by the Covid-19 disease. According to the latest official figures, the country recorded almost 5.5 million infected and 25,000 deaths by August 2022. Figure 1(a) shows the Covid-19 daily number of new cases registered in Portugal since March 2020. There were three prominent peaks of infection: January 2021, January 2022, and May 2022. Peaks in January were most likely because of Christmas and New Year's Eve. Interestingly, the last two main infection peaks occurred when most Portuguese citizens were already fully vaccinated. By January 2022, about 83% of the population had completed the vaccination plan. However, more infected people do not mean higher mortality, as shown in Figure 1(b). Indeed, the peak of infections in January 2021 led to higher mortality than the peaks in 2022, and the reason can be vaccination, which seems to reduce mortality among Covid-19 patients. Nonetheless, two peaks in mortality by February and June 2022 were observed, which coincide with the peaks of infected people in the same year (with a certain delay).

Concerning hospital and intensive care unit admissions due to Covid-19 (see Figure 2), there were more than six thousand hospitalizations at the beginning of 2021, when the first peak of infections was observed, and more people died because of this disease. By that time, the admissions to intensive care also reached the maximum in the entire period. Once people get fully vaccinated, hospital admissions decreased to values close to the pandemic's beginning. Interestingly, when infections reached their records in 2022, hospital admissions did not increase as much as in 2021, most likely because of vaccination. It is worth mentioning that the gap between hospital admissions and intensive care unit admissions became larger because the disease severity decreased. Overall, there is a strong correlation (R=0.86, p<0.001) between hospital/intensive care unit admissions and mortality due to Covid-19.

3. Results

3.1. Efficiency and Productivity

Hospital efficiency and productivity were assessed using the occupancy rate and the availability of clinical human resources per treated patient (after adjustment by complexity and severity of illness). Figure 3 exhibits the evolution of hospital average efficiency and productivity in terms of static performance (Figure 3(a)) and dynamic performance (Figure 3(b)). Additionally, Table 2 presents the summary of these dimensions and the TFP per group of hospitals and period: pre-pandemic period (January 2017–February 2020); and pandemic period (March 2020–May 2022). For a better analysis, we split the pandemic period into two: first year, pre-vaccination (March 2020–February 2021); and after vaccination (March 2021–May 2022).7 While the static performance measures how hospitals get closer to the frontier, thus becoming more or less efficient, the dynamic performance measures the frontier shift between two consecutive years. TFP is an aggregated measure of static and dynamic performance. Therefore, should a hospital approach the frontier in one year and the benchmarks in that frontier are more productive than before, the hospital has improved its TFP (overall performance).

When analyzing the static performance of hospitals' efficiency and productivity, there was a decay between January 2017 and February 2020, with an average ranging from 0.9554 (group E) to 0.9995 (group B). In other words, hospitals' efficiency was estimated to have decreased up to 4.46% in that period. Note that the observed decay in group B was not considered statistically significant as per the Kruskal-Wallis nonparametric statistical test. In this period, hospitals were moving away from the frontier. However, with the pandemic's start, there was a tiny efficiency and productivity improvement of 0.33-3.96%, depending on the hospital group, from March 2020 until May 2022. If we split this period in two, we verify a global improvement in both, although they are not different in the light of statistical evidence. Overall, considering the entire period, hospitals are more inefficient in 2022 than in 2017, except for the ones in group B (the smallest ones). In other words, hospitals in May 2022 were further away from the frontier than in January 2017. Regarding the dynamic performance evolution, we observe a worsening of efficiency in the whole period, which means that the frontier has progressively shifted towards a more inefficient region. Overall, the benchmarks in May 2022 are less efficient than the ones in January 2017 by about 2-4%. Overall, hospitals nowadays seem to be in a worse position than five years ago concerning efficiency and productivity.

Table 3 presents the beta coefficients for the three different multiple regression analyses. Malmquist indices P, T, and TFP were the dependent variables of each analysis, to be explained by seven Covid-19-related variables (as independent ones). Beta coefficients are the linear coefficients (resulting from the ordinary least squares method) divided by the variables' standard deviation. Therefore, beta coefficients allow us to determine the weight or importance of each explanatory variable in explaining the dependent variable. Additionally, the same table contains information about the adjustment quality (coefficient of determination, usually represented by R²) and appropriateness. The latter is assessed through three main conditions over the regression residuals: normality, homoskedasticity, and independence. Normality was tested using the nonparametric Kolmogorov-Smirnov statistical test for a null hypothesis H₀: the sample follows a Gaussian distribution with a zero average and constant standard deviation resulting from data. We tested the residual's homoskedasticity by plotting residuals as a function of fitted or predicted values of the dependent variable. If the resulting fitting line has a slope and intercept both equal to zero, then residuals are homoscedastic. Finally, we assessed the residual's independence using the Durbin-Watson test, for which a value of 2 suggests the absence of autocorrelation.

A look into the results displayed in Table 3 allows concluding that neither of the considered potentially explanatory variables help justify the evolution of the static performance of hospitals, P, in terms of efficiency and productivity, i.e., whether they approach or move away from the frontier, becoming more or less efficient. That is, hospitals did not improve their resource usage because of the Government's stringency or the rates of infection, mortality, or vaccination. Even the admissions to the hospital wards or the intensive care units could not explain this performance measure. For this reason, the coefficient of determination was only 0.039, i.e., the adopted linear model with the stepwise forward method and Covid-19-related dimensions only explains about 4% of the hospitals' static performance change regarding efficiency and productivity. However, the same cannot be said about their dynamic performance change, T. Some variables tend to explain why the frontier shifts with time, so benchmarks in one year can outperform or be outperformed by the benchmarks in another moment. In the case of the efficiency and productivity group of variables, it seems that benchmarks can get more efficient and productive with time because of variables like the infection and reproduction rates, the admission rates to hospital wards and intensive care units, and the stringency index. There is a positive technology (frontier) shift promoted by more infected people and higher reproduction rates associated with a higher need for hospital care. More admitted patients (regardless of being infected by SARS-CoV-2 or not) means a higher monetization of assets. Also, the average complexity of patients increased in this period because only the more severe cases were attended in hospitals, raising, even more, the output, although virtually. Finally, more significant stringency indices tend to be associated with a negative frontier shift, i.e., a loss of productivity, because many treatments in hospitals were denied or postponed, thus decreasing the hospital output. This linear model has reached an acceptable adjustment quality (explaining about 56% of the dynamic performance change) and appropriateness (no test over residuals failed). There is, thus, evidence to reject the null hypothesis H_0(R3) that concerns the third research question, (R₃), and the variables group of efficiency and productivity.

3.2. Access

As shown in Figure 4 and Table 4, hospitals of groups B and E improved their performance regarding access between 2017 and the beginning of 2020, right before the pandemic started. In opposition, groups C and D hospitals worsened their performance levels in the same period. Similar results were observed for both static and dynamic performances. For instance, considering the hospitals within group D, the access to their services was reduced by 8.36% on average, while the frontier watched a negative shift (worsening the performance) of 9.31%. Overall, the average performance of hospital providers in terms of access reduced by nearly 17% in just three years. The pandemic exacerbated the barriers to health care in Portugal in all groups, especially concerning the first year (March 2020 to February 2021), when lockdowns, postponement of medical appointments and surgeries, and other pandemics containment policies were in force. Concerning the entire period (2017-2022), there was an aggravation of barriers to proper secondary health care. Although potentiated by the pandemic and the measures taken to prevent the virus from widespread, such walls result primarily from structural problems such as the highly reported lack of medical doctors and nurses in public services. Most of the clinical staff have opted to work in the private sector, where the working conditions and remunerations are better than in the public sector, which then limits the availability of the already scarce resources, delaying, even more, the already delayed medical and nursing care, and jeopardizing the sustainability and mission of the NHS in Portugal.

In the case of access to hospital care, the explanatory multiple linear models seem to reproduce similar results, regardless of whether the dependent variable is P, T, or TFP. According to the evidence in Table 5, all models reached acceptable adjustment quality (explaining at least 59% of the performance change) and appropriateness (as no test over residuals was rejected). Concerning access, only three variables seem to help explain performance change: admissions to intensive care units or hospital wards because of Covid-19 and stringency index. All the significant variables exhibited a negative influence on the performance change. The reasons behind this result seem apparent. For the case of the Government's stringency, there was a substantial limitation on the access to health care services, decreasing the number of appointments and enrollments on surgery waiting lists within the legally defined time as well as the surgeries themselves, and increasing simultaneously the average time before surgery. Concerning the admissions to hospital care, one must recognize that hospital resources (mainly beds and clinical staff) are rivalrous, i.e., there is competition for the same resource. Suppose an inpatient infected with SARS-CoV-2 occupies a bed or a ventilator. In that case, the same resource will not be available for another user for a while, denying or delaying the appropriate care for that person. We expected these results as Covid-19 and the policies implemented to contain it constituted a massive barrier to hospital care. There is, thus, evidence to reject the null hypothesis H_0(R3) that concerns the third research question, (R₃), and the variables group of access.

3.3. Safety and Care Appropriateness

Unlike what happened to the access to hospital care, primarily resulting from administrative, political, and contingency barriers, safety and care appropriateness in hospitals improved during the entire period (see Figure 5 and Table 6). Although some ups and downs were observed in both static and dynamic performance estimates, hospitals are currently in a better position than in 2017, delivering better care without as many undesirable events as before. The pandemic does not seem to have influenced this set of dimensions, as hospitals and their clinical staff seem to have adopted good empirically-based practices without jeopardizing the patients' safety. Overall, not only hospitals approached their own year-based frontier, as this frontier has moved towards a region featured by better care outcomes. As shown in Table 7, no Covid-19-related variable seems to explain the hospital performance change concerning safety and appropriateness of care. The linear models were also unsuitable, exhibiting very small coefficients of determination, and two criteria over residuals failed (homoskedasticity and independence). Thus, there is no evidence to reject the null hypothesis H_0(R3) that concerns the third research question, (R₃), and the variables group of safety and care appropriateness. These results suggest that the safety and appropriateness of care in Portuguese public hospitals have nothing to do with the evolution of Covid-19. Hospitals seem to deliver safe and appropriate care regardless of the widespread pandemic, suggesting that all best practices and international and national guidelines have been adopted to treat all patients equally well. Hospital care does not seem to have been affected by the pandemic nor the harsh conditions in which the clinical staff had to work and the burnout to which they were subject.

4. Conclusions

This work has dealt with three important research questions that have not been answered in the literature. Evidence suggests that hospital performance has been heavily conditioned by the Ministry of Health's external impositions, especially concerning the barriers created to contain pandemics dissemination. The Covid-19 pandemic’s features have justified part of the hospitals’ performance after March 2020, especially the access and efficiency dimensions. Although hospitals have adopted safe and appropriate clinical practices internally, some resource availability problems need to be addressed urgently. The lack of medical doctors and nurses in public health care provider services is not a new problem and was exacerbated by the harsh conditions in the workplace created by the pandemic. Introducing policies that conditioned access to those services raised the existing barrier to the NHS. Postponement or cancellation of medical appointments and surgeries and the constraints on the emergency departments only result in the worsening of the patient's quality of life and the rise of costs in the future because the clinical status becomes more complex and severe to treat. It becomes evident that these policies put the NHS's sustainability and mission at stake. If future pandemics occur, it is paramount to assess the past best practices within the Portuguese NHS and internationally. But this benchmarking exercise does not end with identifying those best practices: one needs to adapt and then adopt them in the NHS, such that the health care providers’ resilience and capacity to answer for the demanded care are not at risk. In the current times, featured by an ongoing pandemic, the exodus of clinical staff to the private sector or abroad, and political instability, it is challenging to implement the required structural reforms that allow the NHS to fulfill its mission. Nonetheless, they are undoubtedly necessary and should be focused on improving clinical staff working conditions, hiring more staff, fostering meritocracy, reducing bureaucracy, and guaranteeing equitable access to quality health care for the entire population (in particular, the people living in the countryside).