The sustainable management of public hospitals is usually threatened by long-period operating deficit, especially during the pandemic of COVID-19. This study aims at quantitatively decomposing the historical changes in the annual operating costs of public hospitals in Japan to identify the main driving forces responsible for a worsening imbalance between operating costs and income over the past two decades. A dataset of the annual operating costs of public hospitals in Japan is compiled, in which influencing factors are redefined to make the data amenable to the application of a decomposition method referred to as the Logarithmic Mean Divisia Index (LMDI). Using the LMDI method, the contribution of each influencing factor to the changes in public hospital operating costs was quantitatively determined. The results indicate an overall positive effect arising out of the national reform of public hospitals that aimed to reduce the operating costs of these institutions, but the rapid increase in the prices of medicines in recent years made a major contribution to the increasing operating costs. The pandemic reveals a damage on public hospitals’ financial balance but the city-wide lockdown offset part of the increased medical expense. Both long-term and short-term pricing policies are required for the sustainable management of public hospitals.