Shaping Adaptive and Sustainable Smart Urban Ecosystems

1. Introduction

The modern urban environment is conceptualized as a dynamic, evolving, increasingly complex urban ecosystem where the convergence of physical and computational elements is essential to keep high the quality of life [1,2]. In this way, municipalities face significant open issues and challenges, often conflicting such as infrastructure degradation, inadequate service provision, growing demand, cost, and environmental impact. These challenges are central to the 11th United Nations Sustainable Development Goal (SDG 11), which mandates the creation of sustainable cities and communities that are inclusive, safe, and resilient, as illustrated in Figure 1 [19].

Despite this potential, even the sustainability of the smart city Information and Communication Technology (ICT) infrastructure is a critical concern. Current models often suffer from a lack of systematic capacity planning, resilience, and long-term sustainability strategies. Many existing infrastructures follow siloed patterns and a static deployment bias, where sensors, devices and networks are specifically designed, deployed and configured for a given vertical application, with fixed parameters regardless of the fluctuating urban context. This absence of resource reuse and modularity leads to low adaptivity, rendering such systems unsustainable in the face of rapid environmental or social shifts.

A viable solution to these limitations is the integration can be Artificial Intelligence (AI), moving towards intelligent cities [48]. This evolution represents a fundamental paradigm shift toward holistic urban ICT sustainability, where AI serves as the neural blueprint of the urban entity [3,4]. Such adaptive urbanism allows the city to function like a living organism, mutating its behavior to optimize its response to bio-social demands while minimizing its environmental footprint [10].

Related work identifies a distinction between classical computational methods, which lack the real-time adaptivity required for complex urban scenarios, and more advanced machine learning approaches. Although supervised learning is highly effective for predictive tasks, it remains limited by its reliance on extensive labeled datasets, that are rarely available for dynamic urban control [20,21,22,23]. Consequently, Reinforcement Learning (RL) has emerged as a dominant methodology, enabling autonomous agents to derive optimal strategies through direct interaction with the urban environment [24,27]. Furthermore, breakthroughs in Multi-Agent Reinforcement Learning (MARL) [25,31] and Large Language Models (LLM) [26,28,29] have introduced advanced reasoning capabilities, although balancing the computational overhead of AI with the energy constraints of edge devices remains a persistent open problem in the design of sustainable systems [30].

The main contribution of this work is the proposal of a holistic approach to multi-dimensional (environmental, economic, and social) urban ICT infrastructure sustainability, characterized by its three main pillars: environmental ($P_e$), economic ($P_c$), and social ($P_s$). This framework moves beyond traditional siloed patterns by prioritizing resource reuse, proper capacity planning, and multi-objective optimization. The proposed methodology is framed within the Cyber-Physical Systems (CPS) paradigm, where smart-intelligent cities are modeled as interconnected urban CPS. A real-time, adaptive AI-driven model based on Deep Reinforcement Learning (DRL) and Deep Q-Networks (DQN) [32,34,35,36] is adopted to optimize and trade-off among environmental, economic, and social sustainability metrics.

The proposed framework represents a strategic solution to address the multi-faceted nature of SDG 11. By leveraging the DQN-based CPS, the integration of the three pillars ($P_e,P_c,P_s$), and the related metrics, allows for a direct contribution to the operational targets of SDG 11, as summarized in Table 1. The effectiveness of this solution is demonstrated through a case study on Low-Power Wide-Area Network (LPWAN) [16] technologies, specifically the LoRaWAN specification [17].

The remainder of this work is organized as follows: Section 2 discusses the background and related work; Section 3 presents the proposed approach and sustainability metrics; Section 4 details the experimental case study; Section 5 analyzes the results of our investigation; and Section 6 concludes the work with future perspectives.

2. Background and Related Work

The smart governance of the city in the Society 5.0 era, put humans at the center of decision processes, supported by CPS [11] as digital bridge between people (policymakers, citizens) and digital services, thus creating an adaptive urban environment. The smart governance must be supported by adequate infrastructure, institutional quality, and inclusive policies [9]. In such a context, social sustainability ($P_s$) is crucial to realizing livable communities that are focused on achieving human well-being and enhanced quality of life [12,13]. Our work addresses this social/democracy challenge [38] in the governance of future cities, characterized by a deep integration among the environment, technology, and planet inhabitants.

Moreover, designing and maintaining sustainable CPS to effectively control fleets of smart devices, is a challenge mainly asking for a policy-technology balance. For example, while scientists and technical officers optimize for latency and matching efficiency, policymakers (i.e., municipalities) focus on safety, equity, and environmental protection (i.e., the SDG). Because service demand is inherently unpredictable and undergoes stochastic changes, dynamic fleet control represents significant operational hurdles in high-dynamic (i.e., high-frequent) services. The consequence is the priority to identify the best-fit and context-aware approach capable of managing the non-linear trade-offs between environmental, economic, and social metrics within the urban context.

Traditional smart city models often suffer from a static deployment bias, where the underlying ICT infrastructure is deployed on purpose and configured with fixed parameters regardless of the fluctuating urban context. The evolution of urban infrastructures has recently moved towards the integration of cognitive CPS and Artificial Intelligence of Things (AIoT) [10,15].

Recent literature suggests that the next frontier is the transformation of these networks into fully autonomous CPS [6]. Scheduling is the most effective solution to address production-friendly hybrids that combine AI with heuristics. A fundamental contribution to the evolution of smart city management is provided by [10], which explores the synergy between AIoT and Digital Twin (DT) as Smart City Brain. This work emphasizes how predictive planning and real-time management can pioneer environmental synergies, moving beyond simple data collection towards a cognitive urban management system. In particular, the integration of AI within the urban context allows for predictive planning, which is essential to achieve long-term sustainability goals. DT serves as the “virtual mirror” of the physical city, enabling simulations that optimize resource allocation before their physical implementation. However, while [10] focuses on high-level environmental synergies and the overarching architecture of the city brain, there is a recognized need to bridge the gap between macro-scale planning and micro-scale operational efficiency at the device level.

The literature emphasizes how high-dimensional datasets [14] are essential for training robust autonomous agents, particularly when dealing with the stochastic nature of modern Internet of Things (IoT) deployments. Without sufficient data granularity, the convergence of optimization algorithms can be compromised, leading to bias projections (i.e., inaccurate estimations). Furthermore, large datasets facilitate the validation of comparative studies across different communication protocols and low-power technologies, providing a statistical basis to evaluate the long-term sustainability of large-scale IoT deployments [16]. As a consequence, transition from theoretical models to viable urban CPS calls for a data-centric perspective to minimize uncertainty in operational costs and maximize the efficiency of shared resource management. To such a purpose, a long-term testbed collecting data from sensors deployed at different locations across a city, after conducting intelligent radio coverage planning [17], has been implemented. This systematic approach led to the generation of structured datasets tailored to this paper investigations.

Economic sustainability ($P_c$) is crucial in cities suffering from poor infrastructure, inadequate service provision, and environmental degradation. In such a context, for example, the concept of smart cities has recently gained traction as a potential solution to the challenges facing African cities [39]. In this light, city governance is truly smart if, beyond updating and enhancing the accessibility and usability of provided services, it actively preserves the environment. This implies safeguarding the natural and landscape features that define the environmental sustainability ($P_e$) of the city and its technological and infrastructural components.

In [40], the authors discuss this challenge, and propose a deep learning multivariate architecture to predict wind speed and global horizontal irradiance. Although they use only timestamp-based features, enabling reliable prediction of renewable energy without sensing infrastructure and achieving strong predictive accuracy, they assert their proposed framework is validated using a single dataset and may exhibit reduced performance under highly variable cloudy conditions. We adopt a traditional IoT-based approach for the monitoring, by collecting data about weather conditions and quality of the air, thereby incorporating the seasonal patterns of the data.

De-carbonization is part of the environmental sustainability challenge, as demonstrated by several studies in literature [41,42]. However, often these studies suffer from unavailable real testbeds and are limited to simulations.

A study oriented to the quantitative assessment of urban ecosystems is represented by the work of [43], in which the authors developed a comprehensive integrated model to evaluate the environmental and economic sustainability of smart cities. Their approach is characterized by the use of distinct indicators to calculate an overall Sustainability Index, providing a mathematical framework to balance carbon emissions and energy efficiency with economic viability. However, the challenge remains in the dynamic nature of these indicators: while their model provides a diagnostic tool for urban planning, it primarily operates on aggregated data. Our research extends this integrated vision by embedding such sustainability metrics directly into a CPS architecture. Using the real-time sensing capabilities discussed in the AIoT literature [10], we transform the integrated model of [43] into a dynamic optimization problem. Specifically, we adopt their logic of multi-dimensional indicators from a “top-down” planning model to a “bottom-up” adaptive control loop. However, although the authors in [10] provide the architectural blueprint for environmental synergies, our work focuses on the trade-offs between the three pillars of sustainability, which are often in conflict. Therefore, a trade-off analysis is crucial to determine the optimal configurations of CPS in urban ecosystems.

In this context, various computational approaches have been proposed to address the inherent volatility of smart city environments, offering distinct methodological perspectives to model and control the complexity of urban dynamics. A classical distinction in literature is among three primary paradigms: Classical Methods (CM), comprising Linear Programming and Heuristics, Supervised Learning (SL), and Reinforcement Learning (RL). The $CM$ group is traditionally used for static resource allocation; although these methods are computationally efficient, they lack the adaptability required to respond to contextual shifts in real-time. Furthermore, they often require a precise closed-form mathematical model of the entire environment, which is notoriously difficult to obtain in complex smart city scenarios.

The $SL$ methods [20,21,22,23] are highly effective for predictive tasks, such as forecasting traffic flow or energy demand; however, they require extensive labeled datasets representing optimal decisions, which are rarely available for dynamic urban control. In the context of this framework, the optimal balance between the pillars of sustainability is not pre-defined but must be discovered dynamically through exploration.

Unlike the aforementioned approaches, $RL$ algorithms have emerged as a dominant methodology, allowing agents to derive optimal strategies through direct interaction with the urban environment. While foundational theories established the mechanics of agent-environment interaction [24], modern implementations now leverage Deep RL to discover state-of-the-art algorithms that outperform human-engineered heuristics [27]. In the context of smart cities, the challenge of resource allocation is increasingly addressed through MARL, providing a robust framework for optimizing distributed infrastructures [25,31].

Recent breakthroughs in LLM and RL from human feedback have further introduced advanced reasoning capabilities [26,28,29]. Although these models are often associated with natural language processing, their underlying logic of “learning to reason” and “scaling through feedback” is directly applicable to CPS. By deciphering the nuances of feedback loops and long-term rewards [30], it is possible to design urban controllers that address sustainable trade-offs between energy saving and service availability.

A comparison between the computational approaches $CM$, $SL$, and $RL$ is reported in Table 2, evaluating them based on heterogeneity of the data, portability, environment interaction, and integration of sustainability.

Data heterogeneity refers to ingest and normalize different data streams. While $CM$ requires rigid rules that struggle with high-dimensional IoT data, and $SL$ focuses on finding patterns in historical datasets, $RL$ is goal-oriented: it processes heterogeneous inputs to optimize a sustainability score, making it more resilient to sensor variability.

Portability assesses the feasibility of running the engine on resource-constrained hardware. $CM$ methods are computationally light but inflexible; $SL$ models can be memory-intensive; RL enables an asymmetrical workload: intensive training is performed offline, while the resulting policy is a lightweight inference engine runnable at-the-edge.

Environment interaction represents the feedback loop. $CM$ is essentially an open-loop paradigm; $SL$ remains a passive observer of historical data. In contrast, $RL$ is a closed-loop system. This is crucial for sustainability: the agent observes how a power-saving action at time t affects reliability at $t+1$, adapting its strategy autonomously based on the observed outcome.

Sustainability integration evaluates how the model incorporates environmental and economic pillars. Sustainability is not a fixed threshold (as in $CM$) nor a historical replication (as in $SL$), but a context-aware balance. While $CM$ uses static weights that fail when the urban context changes, and $SL$ requires non-existent “perfectly labeled” trade-off datasets, $RL$ provides dynamic Pareto balancing, shifting prioritization based on real-time urgency.

Based on the comparative assessment above, $RL$, and in particular $DRL$, was identified as the most suitable technique [32]. Raw data are directly fed into a multi-layered deep neural network that progressively extracts higher-level features [33]. As part of the DRL family, $DQN$ has emerged as a pivotal technology in smart city research [34,35,36], primarily due to its ability to dynamically adapt to fluctuating urban conditions and orchestrate complex data from interconnected devices [37]. The rationale for this choice is mainly due to the ability of DQN to handle the “curse of dimensionality” inherent in the multi-dimensional context-aware approach. In the smart city context, using Deep Neural Networks (DNN), a DQN agent adapts its behavior to rapid weather-induced hardware stress and fluctuating traffic density, without relying on pre-defined or historical “perfect” datasets.

Recent advances have further integrated DNN and federated learning [5,7] to anticipate future demand patterns, thereby enhancing system-wide sustainability [8]. This technological convergence promises to shape the future of adaptive urban ecosystems.

3. Context-Aware Optimization

To evaluate the systemic impact of the ICT infrastructure in urban environments, this study defines a multi-dimensional framework taking into account environmental, economic, and social sustainability metrics. They are quantified through recognized scientific metrics, providing a standardized approach to evaluate how a Smart City infrastructure interacts with the urban ecosystem and its governance.

As illustrated in Figure 2, our approach introduces a conceptual framework integrating multiple CPS into a cohesive urban ecosystem. It is characterized by the AIoT as a “cognitive” engine that transforms the old concept of static infrastructure in an adaptive urban ecosystem. AIoT acts as a stochastic controller that processes real-time telemetry to optimize system responses.

The architecture of the proposed adaptive urban ecosystem is illustrated in Figure 3. The framework conceptualizes the transition from a static sustainability model to a dynamic, context-aware system. As shown in this diagram, the system is driven by two primary inputs: the Sustainability, which defines the theoretical SDG11, and the Urban Context, providing real-time environmental and operational data. These inputs converge on the Preliminary Setup Layer (PSL), highlighted by the dashed red boundary. Within this layer, the sustainability pillars $P_e\left(t\right)$, $P_c\left(t\right)$, and $P_s\left(t\right)$ are not treated as static metrics but as dynamic functions of the urban context.

The interaction among these pillars is governed by a Multi-Objective Optimization engine. This component processes the input parameters (i.e., from the PSL) to resolve inherent trade-offs, also ensuring that the infrastructure remains efficient and reliable under varying urban conditions. Finally, an Adaptive Feedback loop returns the optimized state to the Urban Context, enabling a continuous self-healing and self-configuring cycle.

3.1. Smart Urban Context

The core of the proposed framework lies in the formalization of the smart urban context as a dynamic CPS ecosystem governed by a state-space representation. Thereby, a dynamic urban context $C\left(t\right)$ can be defined as the n-tuple of Equation (1)

$$C\left(t\right)=\left(c_1\left(t\right),c_2\left(t\right),\dots ,c_n\left(t\right)\right)\in \mathcal{C}\subseteq {\mathbb{R}}^n$$

(1)

where each component $c_i\left(t\right)$ represents a specific environmental or systemic metric, such as traffic congestion levels, network availability, or sudden urban emergencies, and where $\mathcal{C}$ is the context state space.

To tune the model to urban-wide areas, usually organized into geographical districts (e.g., urban districts, industrial zones, or residential quarters), a context matrix $S\left(t\right)$ is formally defined by eq. (2):

$$S\left(t\right)=\left(\begin{array}{ccc}{C}_{1,1}\left(t\right)& \dots & C_{1,q}\left(t\right)\\ ⋮& \ddots & ⋮\\ C_{p,1}\left(t\right)& \dots & C_{p,q}\left(t\right)\end{array}\right)\in {\mathcal{C}}^{p,q}\subseteq {\mathbb{R}}^{n,p,q}$$

(2)

mapping $p\ast q>0$ districts of the urban area into the corresponding $C_{i,j}\left(t\right)$ context representing the $i,j$-th geographical district at time t. This formulation ensures that the context matrix $S\left(t\right)$ captures the dynamics of the city and its districts, preserving geographic and topological distance through the matrix adjacency. This allows the municipality to identify spatial correlations and proximity-based dependencies, which are essential for risk prevention (e.g., hydro-geological monitoring) and urban optimization (e.g., commercial promotion in high-footfall areas). Therefore, it is possible to identify complex spatial-temporal correlations that would be lost in a lower-dimensional or aggregated representation, finally identifying optimal trade-offs among the three pillars of sustainability. This representation enables a comprehensive overview of the city dynamics by aggregating localized district-level data and identifying potential spatial correlations between adjacent districts. Using this structured information, the framework can detect criticalities and vulnerabilities associated with specific districts based on the interplay of their context metrics, such as traffic congestion, weather patterns, and crowd density.

The practical implications of such granular monitoring are manifold: high-resolution telemetry is crucial for hydro-geological risk prevention and for optimizing urban traffic planning through proactive interventions; the ability to analyze real-time visitor and tourist flows enables more effective promotion of events and commercial activities in areas with higher crowding; the high-dimensional observation space is transformed into an actionable decision-support tool, allowing the municipality to identify both systemic risks and strategic opportunities by capturing the inter-dependencies between neighboring districts. This representation does not merely describe the environment but actively shapes the weighting of the sustainability pillars within the proposed multi-objective optimization process, ensuring that the resulting urban trade-offs are both resilient and context-aware. The brain of the optimization process determines the criticality of the current situation. For example, in high-critical scenarios, the controller may prioritize immediate social and network performance by extending the duration of high-power operational states. Conversely, in routine contexts, the optimization logic shifts toward energy saving and cost reduction. As a result of this analysis, the context impacts the sustainability of the smart urban ecosystem and the selected main metrics that characterize its pillars, as described below. This holistic representation emphasizes that the transition toward a smart city is not merely technological but is intrinsically linked to measurable sustainability goals, where CPS bridge the gap between physical infrastructure and socio-economic outcomes.

3.2. Sustainability Pillars and Metrics

Environmental sustainability $P_e$ is traditionally quantified by the ecological cost of the digital infrastructure, focusing primarily on metrics such as energy consumption and carbon footprint. However, in the context of a Smart City, this perspective is inherently limited. True environmental sustainability must be redefined through the proactive role of CPS, acting as responsive sentinels. Beyond mere device-level energy savings, these systems monitor both local and neighboring contexts to dynamically manage urban dynamics. By anticipating traffic peaks, high-density events, and congestion, the DQN policy prioritizes the systemic prevention of pollution surges, using the infrastructure as an active tool for ecological mitigation rather than a passive energy consumer. In such a scenario, energy consumption is not a static value but a baseline strictly coupled with the operational context $S\left(t\right)$. It serves as a direct measure of the CPS impact on environmental and economic costs, as the power consumption scales dynamically across different modes, from low-power stand-by or idle states to high-performance active monitoring.

Let $d_{i,j}$ be the number of devices deployed across the $i,j$-th urban district, the total energy consumptionE over an observation interval $[t_0,t]$ is formally defined by Equation (3):

$$E(s,t)={\int }_{t_0}^{t}P(S\left(\tau \right),\tau ),d\tau \left[\mathrm{kWh}\right].$$

(3)

where the lowercase s represents the specific realization of the state at the current time t, i.e. $s=S\left(t\right)$. The total power P is the result of the sum of the instantaneous power measured at time t for each device in the district, formally defined by Equation (4):

$$P(s,t)=\sum _{i=1}^p\sum _{j=1}^q\sum _{k=1}^{d_{i,j}}{P}_{i,j,k}(C_{i,j}\left(t\right),t)\left[\mathrm{W}\right].$$

(4)

The carbon footprint ($CF$) represents the Global Warming Potential (GWP) [44] associated with the system existence and operation in the observation interval $[t_0,t]$. It accounts for the embodied carbon of each device, provided as a manufacturer specification, and the emissions derived from its contextual energy consumption. Leveraging the energy formulation defined in Equation (3), the total $CF$ for the infrastructure across the n districts is formalized by Equation (5):

$$CF(s,t)=\sum _{i=1}^p\sum _{j=1}^q\sum _{k=1}^{d_{i,j}}GW{P}_{i,j,k}+{ϵ}_{geo}·E(s,t)\left[{\mathrm{kgCO}}_2\mathrm{eq}\right]$$

(5)

where $GW{P}_{i,j,k}$ is the aggregate cradle-to-gate carbon footprint of the k-th device located in the $i,j$-th urban district (including its enclosure, electronics, and battery where applicable). This term represents the embodied impact linked to the physical existence of the device. The term ${ϵ}_{geo}·E(s,t)$ represents the operational impact, where ${ϵ}_{geo}$ denotes the grid emission intensity factor [45] and $E(s,t)$ is the context-dependent energy consumption derived from the instantaneous power $P(s,t)$ as defined by Equation (3). This formulation allows the framework to evaluate how the selection of specific hardware models and their operational duty cycles, driven by the urban dynamics captured in $S\left(t\right)$, jointly influence the total environmental sustainability of the infrastructure.

The economic sustainability ($P_c$) ensures the financial viability and scalability of the infrastructure within municipal budget constraints. It is modeled through the Total Cost of Ownership (TCO), which balances the initial investment with long-term maintenance. The TCO represents the comprehensive financial metric for evaluating the ($P_c$) of the CPS infrastructure over its entire operational life. It is defined by Equation (6) as the sum of the initial investment and the accumulated operating costs:

$$TCO(s,t)=CAPEX\left(t_0\right)+OPEX(s,t))\left[\mathrm{Currency}\right]$$

(6)

The term $CAPEX\left(t_0\right)$ represents the Capital Expenditure incurred at the initial time $t_0$, acting as a static and context-independent value. It includes the procurement of devices for all the districts, including hardware costs, licenses, and physical installation. By defining it at $t_0$, the model treats the initial investment as a fixed boundary condition for the optimization problem, as formally defined by Equation (7):

$$CAPEX\left(t_0\right)=\sum _{i=1}^p\sum _{j=1}^q\sum _{k=1}^{d_{i,j}}\left(H_{i,j,k}+I_{i,j,k}+L_{i,j,k}\right){|}_{t_0}\left[\mathrm{Currency}\right]$$

(7)

where $H_{i,j,k}$, $I_{i,j,k}$, and $L_{i,j,k}$ denote the hardware, installation, and licensing costs for the k-th device in the $i,j$-th geographical district, respectively.

Conversely, $OPEX(s,t)$ represents the dynamic and context-aware Operational Expenditure. It is formally defined by Equation (8):

$$OPEX(s,t)={\displaystyle \frac{(t-t_0)}{T}}·K_T+OPE{X}_V(s,t)\left[\mathrm{Currency}\right]$$

(8)

where the $K_T$ term represents the time-invariant operational costs (e.g., cloud subscription fees or fixed infrastructure leasing) in the T period. The variable operational expenditure $OPE{X}_V$ during the observation interval $[t_0,t]$ is formally defined by Equation (9) as a direct function of the geographical urban context, capturing how urban dynamics drive resource allocation and energy demand.

$$OPE{X}_V(s,t)={\int }_{t_0}^t\left(\sum _{i=1}^p\sum _{j=1}^q\sum _{k=1}^{d_{i,j}}\left[V_{i,j,k}\left(\tau \right)·P_{i,j,k}(C_{i,j}\left(\tau \right),\tau )+\left(M_{i,j,k}\left(\tau \right)+{\delta }_k·B_{i,j,k}\left(\tau \right)\right)·{\mathcal{O}}_{pred}(C_{i,j}\left(\tau \right),\tau )\right]\right)d\tau$$

(9)

In this formulation, the first term of the sum captures the monetary cost of the energy, where $V_{i,j,k}$ is the unit price (currency-per-kW), and $P_{i,j,k}$ is the power of Equation (4). The integral term also captures the localized maintenance and hardware preservation costs across the districts over the observation interval $[t_0,t]$. Here, $M_{i,j,k}$ denotes the routine maintenance for the k-th device in the $i,j$-th district, encompassing inspections and standard operational overhead. The second part of the sum addresses the critical management of power-source longevity. The binary parameter ${\delta }_k$ indicates the presence of a battery for the k-th device model, as specified by Equation (10).

$${\delta }_k=\left\{\begin{array}{cc}1\hfill & \mathrm{if}\mathrm{the}k\text{-}\mathrm{th}\mathrm{device}\mathrm{is}\mathrm{battery}\text{-}\mathrm{powered}\hfill \\ 0\hfill & \mathrm{if}\mathrm{the}k\text{-}\mathrm{th}\mathrm{device}\mathrm{is}\mathrm{mains}\text{-}\mathrm{powered}.\hfill \end{array}\right.$$

(10)

The term $B_{i,j,k}\left(t\right)$ represents the replacement cost for the battery (including procurement and field logistics). These factors are modulated by the predictive operator ${\mathcal{O}}_{pred}$, which estimates the degradation rate and the “state-of-the-health” of the device. By incorporating this predictive logic, the model acknowledges that operational stress directly accelerates battery chemical aging. The above formulation is useful for long-term planning, as battery-operated devices (${\delta }_k=1$) can incur periodic replacement costs and labor, whereas mains-powered devices (${\delta }_k=0$) contribute more significantly to the energy cost.

The social sustainability ($P_s$) is evaluated through the system ability to provide a reliable and timely service to the community. Information systems and IT services are increasingly performing a wide variety of organizational functions and personal activities. Therefore, high-quality information systems and IT services are essential to provide value and avoid possible negative consequences for their stakeholders. According to the ISO / IEC System Quality and Software Quality Requirements and Evaluation (SQuaRE) family of International Standards [47], the social value of a CPS is quantified through objective quality of service (QoS) parameters. In this work, we redefine these metrics to be context-aware, ensuring they respond dynamically to the urban context. A QoS metric is the Packet Delivery Ratio (PDR) measuring the communication integrity and the reliability of the data flow from the devices to the application. It is formally defined by Equation (11):

$$PDR(s,t)={\displaystyle \frac{{\int }_{t_0}^t{\sum }_{i=1}^p{\sum }_{j=1}^q{\sum }_{k=1}^{d_{i,j}}{\mathrm{\Psi }}_{RX,I,j,k}(S\left(\tau \right),\tau )d\tau }{{\int }_{t_0}^t{\sum }_{i=1}^p{\sum }_{j=1}^q{\sum }_{k=1}^{d_{i,j}}{\mathrm{\Psi }}_{TX,I,j,k}(S\left(\tau \right),\tau )d\tau }},t-t_0\gg {\tau }_{max}$$

(11)

where the $PDR$ is defined as the ratio of cumulative successfully received packets (${\mathrm{\Psi }}_{RX}$) to total transmitted packets (${\mathrm{\Psi }}_{TX}$) across the urban scenario. By modeling these variables as time-varying processes, the integral structure captures the cumulative throughput over the observation window $[t_0,t]$, effectively filtering out instantaneous stochastic noise. To ensure statistical consistency, the interval $[t_0,t]$ is assumed to be significantly larger than the maximum network propagation delay. Furthermore, the explicit dependence of both integrals on the urban context $S\left(t\right)$ formalizes how urban stressors, such as localized fading or congestion, modulate this ratio.

The ${\tau }_{max}$ is defined by Equation (12):

$${\tau }_{max}=\underset{i,j,k}{max}{\tau }_{Prop,i,j,k}\left(t\right)$$

(12)

where the term ${\tau }_{Prop,i,j,k}\left(t\right)$ denotes the instantaneous latency experienced by a packet transmitted by the k-th device within the $i,j$-th district. This delay accounts for the physical distance between nodes, the signal propagation speed in the urban medium, and potential context-driven retransmissions or multi-hop overheads.

The Service Timeliness, denoted as $\sigma (s,t)$, quantifies the average end-to-end latency over the observation window $[t_0,t]$, ensuring the system meets specific real-time requirements. It is formally defined by Equation (13) as the mean temporal gap between data generation at the sensing layer and its final reception at the application level:

$$\sigma (s,t)=\mathbb{E}{\left[t_{D,i,j,k}-t_{S,i,j,k}\right]}_{(t-t_0)}\left[\mathrm{s}\right]$$

(13)

where $t_S$ and $t_D$ represent the timestamps of data sensing and delivery for each packet, respectively. To keep the operational integrity of safety-critical services, we impose a strict latency constraint:

$$\sigma (s,t)\le {\tau }_{req}$$

(14)

where ${\tau }_{req}$ represents the application-specific deadline. While ${\tau }_{max}$ of Equation (12) defines the physical upper bound of the network propagation delay, the timeliness $\tau$ accounts for the total system latency, including processing times and queuing delays. The condition $\sigma \le {\tau }_{req}$ ensures that the response of the system to the urban environment $S\left(t\right)$ is based on timely rather than outdated information to preserve the effectiveness of the decision-making process under dynamic conditions.

The Service Availability (${\mathcal{A}}_s$) [47] represents the fraction of time the infrastructure is fully operational and capable of fulfilling its functional requirements. Although the service availability at an individual device level is sensitive to localized stressors, the ${\mathcal{A}}_s$ is a critical system-level Key Performance Indicator (KPI). The proposed model leverages the aggregation of individual availability metrics to derive a systemic indicator that reflects the capacity of the smart city infrastructure to provide continuous services, despite possible failures of single nodes.

Defining the availability of a device $a_k$ by Equation (15):

$$a_k\left(t\right)={\displaystyle \frac{Uptim{e}_k(t,t_0)}{Downtim{e}_k(t,t_0)+Uptim{e}_k(t,t_0)}}$$

(15)

where $Uptim{e}_k(t,t0)$ denotes the cumulative duration within the time interval $(t_0,t)$ in which the k-th device is fully operational and the service is active. Conversely, $Downtim{e}_k(t_0,t)$ accounts for periods of inactivity, and the service is inactive. Thereby, the system-level availability is formally specified by Equation (16):

$${\mathcal{A}}_s(s,t)=\sum _{i=1}^p\sum _{j=1}^q{\displaystyle \frac{1}{d_{i,j}}}\sum _{k=1}^{d_{i,j}}{a}_{i,j,k}\left(t\right)$$

(16)

where $d_{i,j}$ denotes the number of devices deployed across the $i,j$-th urban district. Unlike static reliability models, this formulation accounts for environmental and operational stressors across the full urban context $S\left(t\right)$. A high ${\mathcal{A}}_s$ value is mandatory for mission-critical urban services, ensuring 24/7 availability and resilience against localized disruptions.

3.3. DQN-Based Optimization

As discussed in Section 2, a DQN-based optimization strategy has been adopted to achieve context-aware governance of urban resources, thereby facilitating the transition toward more adaptive and sustainable smart city ecosystems. A general schema of the proposed method is depicted in Figure 4. This architectural framework transitions from raw environmental data to a high-level decision-making state through a three-level hierarchical abstraction. At the most granular level (System Metrics), the system captures systemic metrics represented as $c_n\left(t\right)$. Such metrics are geographically aggregated into $p·q>0$ district contexts ($C_{i,j}\left(t\right)=(c_1\left(t\right),c_2\left(t\right),\dots ,c_n\left(t\right))$) encapsulating the n multi-dimensional status of the $i,j$-th district at time t.

The integration of this state representation into the DQN pipeline follows a specific information flow. The Environment block, acting as the CPS, outputs the context matrix $S\left(t\right)$. Therefore, the matrix $S\left(t\right)$ is passed to the DQN agent, which processes the data based on its internal structure. It uses convolutional layers to keep the grid shape intact. This allows the agent to recognize spatial patterns, such as how a problem in one district might affect its neighbors. By preserving this grid structure, the agent better understands the layout of the city, leading to more effective urban management decisions.

More specifically, at each time step t, the agent observes the global state of the city through the context matrix $S\left(t\right)\in {\mathcal{C}}^{p,q}$. This representation captures the status of all functional districts $(i,j)$, characterized by systemic metrics whose raw data, collected from IoT devices, are normalized before being processed by the DNN to ensure numerical stability and effective feature extraction. Once the agent detects the optimal policy for the devices of each district $(i,j)$, it transforms this policy in an operational action $a_{i,j}\in A^{p,q}$ to set devices to a specific operational condition. By discretizing the operational conditions into a finite set of $h\in \mathbb{N}$ operational modes as defined by Equation (17):

$$O=\{{\mathbf{o}}_1,...,{\mathbf{o}}_h\},{\mathbf{o}}_h\in \mathbb{N}$$

(17)

the $a_{i,j}$ action can be defined as a pair of eq (18):

$$a_{i,j}=(o_s,o_f)\in A^{p,q}\subseteq O^2$$

(18)

with $o_s,o_f\in O$ as starting and final state of an action. It is important to remark that, in general, the operational conditions are usually orthogonal to the sustainability pillars, and $o_s$ may be the same as $o_f$, i.e., no changes are enforced by $o_s=o_f$ actions.

The Sustainability Gain $G^{\pi }(s,a)$, is formally defined by Equation (19), for a specific state-action pair $(s,a)\in SxA$. It is the expected cumulative discounted return, which the DQN agent aims to maximize to identify the optimal policy ($\pi$). Specifically, $G^{\pi }(s,a)$ corresponds to the action-value function, satisfying the Bellman optimality criterion [49].

$$G^{\pi }(s,a)=\mathbb{E}\left[\sum _{y=0}^{\infty }{\gamma }^y\sum _{i=1}^p\sum _{j=1}^q{\mathcal{R}}_{i,j}\left(S(t+y),A(t+y)\right)\right],\gamma \in (0,1)$$

(19)

This formulation encapsulates the multi-level complexity of urban infrastructure through several key components. The expectation operator $\mathbb{E}[·]$ accounts for the intrinsic stochasticity of the urban environment. State transitions and the resulting rewards are influenced by unpredictable events, such as traffic spikes or sudden emergencies, which characterize the described dynamics. The temporal index t denotes the current decision epoch, while $y\in \mathbb{N}$ represents the discrete look-ahead horizon. The term $t+y$ indicates that the agent current return is not merely a function of immediate rewards, but a discounted accumulation of expected future states $S(t+y)$ and actions $A(t+y)$. The integration of an infinite horizon ($y\to \infty$), moderated by the discount factor $\gamma \in (0,1)$, ensures the strategic sustainability of the policy: for $\gamma \to 0^{+}$, the objective function collapses into a single-step optimization, where the agent considers only the immediate reward. Conversely, as $\gamma \to 1^{-}$, the agent equally weights present and future rewards. However, in this case, the agent loses the ability to rank different strategies, as any policy providing a positive reward would result in the same mathematical value (∞). By keeping $\gamma <1$, the series converges to a finite number, allowing the DQN to mathematically determine which policy is superior by comparing finite values.

The sustainability gain $G^{\pi }(s,a)$ aggregates the local rewards ${\mathcal{R}}_{i,j}$ at the $i,j$-th district level to ensure computational scalability within large-scale smart city deployments. As the number of urban IoT devices may grow into the thousands, a device-level reward structure would introduce high-frequency noise and a dimensional explosion in the feedback signal, severely impacting the DQN convergence. By a district-level reward ${\mathcal{R}}_{i,j}$, the system considers each geographical area as a functional domain. This ensures that the learning complexity remains tied to the resolution of the urban ($p\times q$) grid rather than the fluctuating density of the deployed hardware. It is formally defined by Equation (20), aggregating the environmental, economic, and social sustainability metrics of all devices within that area:

$$\begin{array}{c}\hfill {\mathcal{R}}_{i,j}(s,a)=1.0-\left[{\lambda }_1{\mathcal{L}}_{\mathrm{e}}({\eta }_{i,j},a_{i,j})+{\lambda }_2{\mathcal{L}}_{\mathrm{c}}({\eta }_{i,j},a_{i,j})+{\lambda }_3{\mathcal{L}}_{\mathrm{s}}({\eta }_{i,j},a_{i,j})\right]\end{array}$$

(20)

where ${\lambda }_k\in [0,1]\subset \mathbb{R}$ are the sustainability factors balancing the impact of normalized losses $\mathcal{L}$ on the system, such that ${\sum }_{k=1}^3{\lambda }_k=1$. The reward function ${\mathcal{R}}_{i,j}$ formalizes the operational efficiency of the $(i,j)$-th district by coupling the local action $a_{i,j}$ with the expanded neighborhood [50] state $\left({\eta }_{i,j}\right)$. This formulation adopts a penalty-from-unity approach: starting from an ideal value of $1.0$, the reward is decreased by a weighted sum of these components. As a consequence, maximization of the expected return $G^{\pi }(s,a)$ is mathematically equivalent to minimizing the long-term cumulative loss. Thereby, a policy $\pi$ is a parametric configuration tha can be identified by the 3-tuple of Equation (21):

$$\pi =({\lambda }_1,{\lambda }_2,{\lambda }_3).$$

(21)

The spatial dependency is modeled upon the Moore neighborhood convention, which is widely adopted in networked RL to capture local inter-dependencies and interference patterns in distributed sensing infrastructures [50]. It is formally defined by Equation (22):

$$\eta (i,j)=\{C_{u,v}\in \mathcal{S}:|u-i|\le r,|v-j|\le r\}$$

(22)

where $\eta (i,j)$ is composed of the context subset (sub-matrix) of the global state space $\mathcal{S}$ centered on the district $(i,j)$ with radius r.

By conditioning environmental (${\mathcal{L}}_{\mathrm{e}}$), economic (${\mathcal{L}}_{\mathrm{c}}$), and social (${\mathcal{L}}_{\mathrm{s}}$) losses on the context of adjacent districts, the system prevents the emergence of selfish policies that might optimize local parameters at the expense of neighboring areas. Each loss component ${\mathcal{L}}_x$ of Equation (20) is modeled as a weighted aggregation of a specific subset of normalized metrics as specified by Equation (23)

$${\mathcal{L}}_x=\sum _{k=1}^n{\omega }_k·{\widehat{c}}_k;{\widehat{c}}_k={\displaystyle \frac{c_k}{c_k^{max}}}$$

(23)

where ${\omega }_k$ represents the relative weight of each normalized metric ${\widehat{c}}_k$ within that specific loss, such that ${\sum }_{k=1}^n{\omega }_k=1$, and $c_k^{max}$ is the maximum value threshold allowed to preserve the operation of the equipment. The hierarchical weighting structure allows the DQN agent to internalize a complex multi-dimensional state while maintaining a clear balance between high-level sustainability goals and specific physical constraints.

4. Case Study

To validate the proposed approach, a real-world urban deployment (i.e. the historical center of a Sicilian city) consisting of five neighboring districts has been investigated. The effectiveness of this solution is demonstrated through a case study on LoRaWAN technologies [17] as the smart city ICT infrastructure. LoRaWAN is a LPWAN specification designed to wirelessly connect battery operated things to the internet in regional, national or global networks, and targets key IoT requirements such as bi-directional communication, end-to-end security, mobility and localization services [18].

As shown in Figure 5, a 3x3 map defines the urban domain where districts A, B, C, D, and E represent the active operational area covered by a central LoRaWAN gateway located in district C (0,0). Non-covered peripheral areas are masked to focus the DQN agent policy on the core interconnected districts. Each active district is equipped with one installation point (pole) including several IoT devices to capture the multi-faceted dynamics of the district, as reported in Table 3.

Each cell corresponds to a surface area of $0.04k{m}^2$, due to the urban morphology of the historic center, and its impact on the LoRaWAN communication. To ensure the replicability of the proposed approach, the cell size can be appropriately scaled based on the topology and geo-morphological features of the new monitored environment.

More specifically, Pole A acts as an integrated sensing cluster hosting one environmental station, one traffic sensor, and one parking node, while Pole B provides a dual-source setup for environmental and traffic monitoring. A similar configuration is replicated on Pole C, which hosts an additional environmental station, a traffic sensor, a parking node, and a standalone LoRaWAN gateway providing the backbone for the LoRaWAN communication layer. Mobility dynamics are further captured by Pole D, which serves as a high-density hub with one traffic sensor and two parking nodes, and by Pole E, conceived for localized parking monitoring with a single sensing unit.

Thereby, the experimental testbed comprises 12 heterogeneous edge devices and a central gateway $GW$, leveraging hardware and firmware technology provided by the innovative company SmartMe.io1. The gateway acts as the primary orchestrator, ensuring seamless connectivity between the edge nodes and a dedicated cloud-based smart mobility platform for real-time data processing and analytics. This technological infrastructure is categorized into three functional typologies of devices, as shown in Table 4.

All poles are permanently mains-powered, and devices operate in a continuous DC active state. As a consequence, the $OPE{X}_V$ formulation (Equation (9)) is simplified (${\delta }_k=0$). Table 5 reports the baseline energy metrics under standard operating conditions, assuming a static duty-cycle without any dynamic policy intervention. These values represent the reference consumption against which the agent energy-saving capabilities are subsequently measured.

This baseline has been established during the start-up phase of the project, where measurements have been calibrated and validated using certified instrumentation. Therefore, this baseline is designed to be periodically updated and refined, provided that such updates remain compatible with existing network constraints and infrastructure overhead (e.g., the 1% duty-cycle in LoRaWAN communication).

During runtime, the system collects a set of heterogeneous physical measurements from the urban environment, formally defined as a real-valued 10-dimensional tuple by Equation (24):

$$C\left(t\right)=\left(c_1\left(t\right),c_2\left(t\right),\dots ,c_{10}\left(t\right)]\right)\in \mathcal{C}\subseteq {\mathbb{R}}^{10}$$

(24)

in compliance with Equation (1).

Table 6 reports the metrics considered for the examined scenario, including features and related impacts on the city dynamics. The radio parameters ($c_9,c_{10}$) and urban demand indicators ($c_4,c_5,c_6$) specify the transmission power and frequency. Physical stressors ($c_1,c_2$) derived from the weather dataset, alongside the Ultraviolet (UV) index ($c_3$), influence the probability of hardware fatigue and maintenance frequency. In our $TCO$ model, these environmental factors directly impact the amortization $OPEX$ and $CAPEX$ by modulating the expected life-cycle of the mains-powered devices. Furthermore, the performance metrics of the LoRaWAN network ($c_7,c_8$), combined with the activity density ($c_6$), define the overall $QoS$.

The Table 6 metrics, selected from weather, traffic, parking and LoRaWAN datasets, serve as the primary inputs for the optimization process. To prevent bias during the DQN training phase, these raw observations are normalized into a normalized demand vector $\widehat{C}$, as defined by Equation (23). Through the normalization process, the DQN agent executes the operational phases of its architecture to maximize the cumulative reward $\mathcal{R}$ defined by Equation (20). The training process integrates experience replay and target network synchronization to ensure the stability of the Q-value estimation. To effectively navigate the trade-off between investigating new strategies, a trade-off mechanism allows the agent to explore the state-action space before converging toward the optimal policy $\pi$ maximizing the sustainability gain $G^{\pi }(s,a)$.

A distinctive feature of this implementation is the spatial modeling of the state, which extends beyond localized observations. The logic implemented integrates a context with the dynamics detected in adjacent districts $(A,B,D,E)$, defined as the radius $r=1$ Moore neighborhood set $\eta (0,0)$ of Equation (22). Therefore, it is not just a quick reaction to local changes, but it is possible to understand how different districts affect each other. For example, how traffic moves or how environmental changes spread from the outer districts to the center. The agent can anticipate incoming data traffic from neighboring areas and adjust settings in advance to stay within the constraints.

The experimental setup implements three ($h=3$) discrete operational conditions for the edge infrastructure, as defined by Equation (25) specializing Equation (17):

$$O=\{{\mathbf{o}}_{\mathrm{ECO}},{\mathbf{o}}_{\mathrm{STD}},{\mathbf{o}}_{\mathrm{HPM}}\}$$

(25)

where each mode $o_{ECO}$, $o_{STD}$, and $o_{HPM}$ defines a specific power profile. The $o_{ECO}$ mode identifies the lower operational bound for the edge device, characterized by a power consumption of $\approx 2$ W. From a hardware perspective, this configuration corresponds to an idle state or a low-power duty cycle where GPU-intensive tasks are suspended and sensing frequencies are minimized. In the $o_{STD}$ mode, the device operates at its nominal power profile of $\approx 4$ W. This setup ensures compliance with standard regulatory limits while providing sufficient data granularity for routine urban monitoring. The $o_{HPM}$ mode is characterized by maximum computational and transmission effort to ensure high system responsiveness during urban emergencies or peak demand. In this configuration, the device enables full real-time edge-inference, resulting in a power surge of $\approx 7$ W. This value aligns with the 10 W Thermal Design Power (TDP) profile of the Jetson Nano, accounting for sustained GPU utilization during complex neural inference.

In applying DRL techniques to urban scenarios, overfitting is a critical challenge: it occurs when the neural network accurately learns the specific noise or outliers of the training dataset but fails to generalize its policy to unseen conditions. In our context, an over-fitted DQN agent might trigger wrong actions in response to sensor anomalies.

5. Results

To validate the robustness of the proposed approach, a comprehensive analysis has been conducted on the urban infrastructures described in Table 3. The C district has been selected as the primary $(0,0)$ reference due to its full-fledged configuration, as shown in Table 4. Such a multi-modal data environment provides a challenging and representative scenario. Three policies, characterizing the agent behavioral attitude toward urban dynamics, emerge as trade-offs from the multi-objective optimization. We define these policies as lazy, balanced, and responsive.

The lazy policy ${\pi }_l=(0.6,0.3,0.1)$ identifies an energy-saving and cost-saving oriented profile. In this configuration, the DQN agent keeps the devices in a low-power state to minimize energy consumption and reduce the associated economic expenditure. It is simultaneously focused on the device lifespan: by operating primarily in less demanding and less stressful conditions, it preserves the hardware electronic integrity. Thereby, the system ensures long-term operational resilience by avoiding the thermal and computational stress typical of higher-performance modes. Although this policy safeguards hardware longevity and related costs, it introduces a significant operational risk to the urban scenario: by failing to capture micro-scale traffic fluctuations and peak congestion events, the system underestimates the actual urban stress. As a consequence, it can represent a systemic risk for the community, as decision-makers are provided with insufficient data that masks pollution hotspots and traffic bottlenecks. In summary, the lazy policy achieves device-level resilience impacting on the city observability and consequently on its sustainability.

The responsive policy ${\pi }_r=(0.1,0.3,0.6)$ identifies a social-oriented profile. This configuration is highly sensitive to fluctuations in urban traffic demand; while it accepts a higher energy and economic cost at the device level, it minimizes the information latency. By operating at peak performance, the system avoids underestimating mobility dynamics, which is crucial to prevent flawed or weak decision-making. In this case study, responsiveness translates in substantial environmental and social benefits: by providing high-resolution monitoring of traffic congestion and parking occupancy, the system enables more effective urban flow management and a reduction in traffic congestion. Consequently, the ability to detect and react to micro-scale mobility fluctuations allows for the targeted mitigation of pollution hotspots. In this configuration, local energy E consumption and capital $CAPEX$ expenditure could be a strategic investment to achieve a systemic reduction of the overall environmental footprint.

The balanced policy ${\pi }_b=(0.3,0.4,0.3)$ identifies a nominal baseline profile designed to ensure that urban monitoring remains reliable and inclusive without reaching the energy peaks of the responsive mode while avoiding the information latency of the lazy approach.

To account for seasonal variations in traffic-parking patterns and environmental conditions, four representative months have been selected to sample the annual operational cycle from August 2025 to March 2026: August (summer), October (autumn), January (winter), and March (spring). This seasonal sampling ensures that the DQN policy is not over-fitted to specific temporal conditions but remains effective and generalized across different climatic and social contexts. The following analysis details the behavioral patterns learned by the DQN agent across the four representative months, illustrating how the lazy, balanced, and responsive policies map the normalized traffic demand to specific operational conditions $O=\{o_{ECO},o_{STD},o_{HPM}\}$.

In the August scenario depicted in Figure 6, the agent faces distinct summer traffic peaks where the policies exhibit highly differentiated behaviors. The lazy policy is in the $o_{ECO}$ state for the majority of the day, showing high tolerance for traffic increases and transitioning to $o_{STD}$ only during the mid-day and evening peaks. The balanced policy serves as a stable baseline, remaining in $o_{STD}$ for almost the entire 24-hour cycle to ensure constant monitoring. Finally, the responsive policy acts as a vigilant sentinel, proactively switching to $o_{HPM}$ during the two main traffic surges.

In the October scenario depicted in Figure 7, the lazy policy switches from the $o_{ECO}$ mode to the $o_{STD}$ mode only during the two highest peaks of the day, maintaining its low-power profile like in August. The balanced policy remains in $o_{STD}$ throughout the active city hours (07:00–21:00) and reverting to $o_{ECO}$ only during deep night. The responsive policy switches into $o_{HPM}$ at each traffic peak, specifically targeting the early morning rush, the mid-day plateau, and the evening return, thus maximizing the permanence in the high-performance $o_{HPM}$ condition.

By observing the January scenario depicted in Figure 8, the analysis reveals how the agent adapts to winter traffic demand. The lazy policy confirms the extreme cost-oriented attitude observed in August and in October. Devices remain in $o_{ECO}$ even during significant traffic demand, with only a brief transition to $o_{STD}$ during the late morning. Although the balanced policy is not high-responsive to the midday peaks (0.5–0.7) in traffic demand, it maintains devices in $o_{STD}$ during the (07:00–21:00) daily period when the city center is expected to be persistently affected by vehicular traffic flows. Devices are switched to $o_{ECO}$ during the early morning hours (normalized traffic demand < 0.1), and around the 22:00. The responsive policy, instead, promptly escalates to $o_{HPM}$ to cover the broad mid-day traffic plateau and a spike around the 16:00.

Observing the March scenario depicted in Figure 9, the analysis confirms the characteristic behavior of the three policies. The lazy policy maintains devices in the $o_{ECO}$, activating the $o_{STD}$ condition only during the peaks of demand (around hours 11:00 and 17:00). The balanced policy maintains its cost-oriented objective. Finally, the responsive policy confirms its high sensitivity.

The cross-seasonal analysis confirms that the DQN agent has successfully synthesized three distinct optimization policies. The policy convergence observed across the four representative months suggests a high degree of robustness: regardless of the seasonal baseline, the lazy policy consistently acts as a lower bound for energy expenditure, while the responsive policy serves as a high-fidelity upper bound, with a direct benefit in terms of social sustainability.

The balanced policy exhibits an intermediate and stabilizing behavior. It effectively filters out minor traffic fluctuations to maintain devices in a steady $o_{STD}$ monitoring state, proving to be a compromise profile between social, economic, and environmental sustainability, without incurring the economic penalties of the responsive mode. However, the balanced policy is weaker than responsive to address environmental sustainability.

The economic sustainability of the examined CPS is evaluated through a $TCO$ analysis over a 10-year operational horizon. The initial $CAPEX$ is established as the baseline, and comprises the acquisition of the equipment shown in Table 4, alongside a centralized gateway, software orchestration platforms, and professional installation costs. The growth in $TCO$ is driven by the $OPEX$, and it is primarily conditioned by energy consumption costs, assuming a baseline rate of $0.20\text{€}/\mathrm{kWh}$, and system maintenance costs.

The results of Figure 10 allow an investigation of the policy sensitivity: the responsive policy exhibits higher growth, reaching an OPEX-to-CAPEX ratio of approximately $5.6$% after 10 years. The DQN agent consistently prioritizes the $o_{HPM}$ mode to ensure near-zero latency and high information fidelity during peak traffic demand, which inherently maximizes the power draw across the NVIDIA Jetson Nano rails.

The lazy policy achieves the highest economic sustainability, limiting the 10-year $OPEX$ increase to less than 4%. All policies show a subtle transition to a linear trend after the initial five-year phase, reflecting their stabilization and leading to a predictable marginal cost per year.

Finally, the balanced policy shows a mid-range trade-off. These savings are achieved with 95% confidence intervals, ensuring the reliability of the economic projections.

The training process of the DQN agent has been monitored through three key metrics, as illustrated in Figure 11. The Epsilon curve represents the exploration-exploitation trade-off. It follows a planned decay from $1.0$ to a minimum threshold of $0.1$ around episode 700. This transition ensures that the agent sufficiently explores the state-action space in the early stages before shifting toward the exploitation of the learned optimal policy in the final phases of training. The high stability of the reward and loss trends in the last 200 episodes indicates that the agent has reached a reliable and converged operational state. The Mean Squared Bellman Error (MSBE) represents a fundamental metric [49] for evaluating approximation accuracy, formulating learning objectives, and investigating the theoretical properties of RL algorithms. The $MSBE$ trend shows a consistent downward trajectory, starting from approximately $0.55$ and approaching zero as the training progresses. This reduction indicates that the model has accurately captured the underlying dynamics of the context. The Cumulative Reward trend exhibits a strong upward slope, stabilizing at a high plateau (approximately at $0.9$) after 800 episodes. This trend confirms that the agent is successfully learning a control policy that maximizes the objective function, balancing energy efficiency and service quality according to the defined reward structure.

Figure 12 shows a radar of the multi-objective optimization policies resulting from our analysis, mapped on a set of UN SDG 11 targets. This map is obtained considering the set of heterogeneous metrics of the dynamic urban context $C\left(t\right)$ defined in Equation (24).

The responsive policy demonstrates a clear prioritization of Target 11.2 (Sustainable Transport) and Target 11.5 (Disaster Resilience and Safety). By proactively escalating to the high-performance state $o_{HPM}$ during traffic peaks, the agent ensures the information fidelity required to mitigate congestion and reduce emergency response times. This “citizen-centric” behavior acknowledges that a localized increase in energy consumption $E(s,t)$ is a strategic investment to achieve systemic environmental benefits and public safety, directly supporting Target 11.6 (reducing the environmental impact of cities).

Conversely, the lazy policy aligns primarily with Target 11.b (Resource Efficiency). By maintaining the infrastructure in a low-power $o_{ECO}$ state for the majority of the operational cycle, it minimizes the carbon footprint $CF(s,t)$ and the operational expenditure $OPEX$. While this approach maximizes the physical and financial longevity of the CPS, it results in lower performance regarding real-time mobility management.

Finally, the balanced policy identifies an equilibrium point for Target 11.7 (Inclusive and Reliable Monitoring). By acting as a stable baseline in $o_{STD}$, it provides continuous and reliable data flows without the energy surges of the responsive mode or the data scarcity of the lazy profile.

The analysis confirms that the DQN-based framework does not merely optimize a technical trade-off but offers an optimal set of strategies. This allows municipal decision-makers to dynamically tune the infrastructure behavior to meet specific sustainability priorities, from strict resource conservation to high-fidelity urban resilience.

6. Conclusions and Future Directions

The proposed multi-dimensional decision-making framework successfully transitions urban management from static, siloed ICT deployments to an integrated, adaptive urban Cyber-Physical System. By leveraging Deep Reinforcement Learning, specifically Deep Q-Networks, the model effectively navigates the non-linear trade-offs inherent in the environmental, economic, and social pillars of sustainability. The empirical results from the LoRaWAN case study demonstrate that reward-shaping mechanisms allow for the precise modulation of node duty-cycles, balancing energy conservation with service availability. This approach mitigates the inefficiencies of traditional high-performance configurations while preserving the resilience required for modern urban infrastructures. Moreover, this methodology provides a scalable blueprint for achieving carbon-neutral, cost-effective, and socially equitable smart-intelligent cities. This research confirms that the intersection of autonomous AI agents and granular urban telemetry is essential for fulfilling the complex requirements of UN Sustainable Development Goal 11.

Future iterations of this work will explore the integration of decentralized federated learning to further enhance data privacy and system-wide scalability in heterogeneous urban environments. To address long-term challenges, aging-/reliability-aware framework are envisioned to be integrated in the DQN controller. A feasible way can be by introducing a cumulative stress counter as a state variable, the agent could learn to proactively mitigate hardware aging.