IoHT and Edge Computing Aided Pandemic-Compliant, Resilient and Perceptive Platform for Smart-City Human Habitat

1. Introduction

Globally, the COVID-19 pandemic situation has improved dramatically, with infection rates declining but not completely disappearing. Despite widespread immunization efforts, the emergence of new variants and continued infections indicate that COVID-19 may persist alongside other influenza-like illnesses as an ongoing challenge for humanity [1,2]. Experts emphasize the importance of curtailing virus transmission through human-to-human contact, including aerosol transmission, which is typical of many influenza-like diseases. Guidelines from the World Health Organization (WHO) recommend measures such as maintaining a minimum distance of six feet or two meters between individuals in crowded areas and wearing masks to prevent virus spread [1,61]. Given these recommendations, there is a need for the development of monitoring tools to reinforce mask-wearing standards in public places and uphold prescribed bio-safety measures [2]. Contact tracing remains a crucial component in preventing virus spread, with governments and agencies advocating for the use of privacy-preserving contact-tracing applications to collect relevant data while safeguarding user privacy [3]. Research indicates that contact tracing and associated policies are effective in preventing the spread of COVID-19 and similar contagious diseases [4]. By enabling infected individuals to voluntarily or otherwise quarantine themselves, these measures help mitigate the impact of community contamination, reducing overall suffering within colonies of human habitats and nations [5,6].

The combination of maintaining social distance, wearing masks, and implementing contact tracing have been recognized as crucial in preventing widespread infection of influenza-like viruses, including COVID-19 [7]. It is equally important to implement measures for monitoring adherence to social distancing and mask-wearing guidelines [8]. Similarly, robust contact tracking and database maintenance systems that allow frequent updates and accessibility are required to help reduce the transmission of viruses linked with influenza outbreaks. These elements are essential in designing pandemic-resilient infrastructure for human habitation.

Among the array of technological solutions, the Internet of Things (IoT) stands out, leveraging sensor packs, wireless communication networks, and AI-driven automated decision-making [9,10]. Furthermore, advancements in edge-computing [11], deep learning (DL) [12], and deep transfer learning (DTL) [13] have empowered IoT to play a decisive role in healthcare, particularly in combating outbreaks of influenza-like viruses such as COVID-19. The application of IoT in healthcare has given rise to Medical IoT (MIoT) or Internet of Healthcare Things (IoHT), offering significant benefits to stakeholders. These technologies facilitate effective tracking of agents (patients, medical professionals, and resources), automatic data sensing, authentication, and diagnostic decision support within health management systems [14].

Many published studies have concentrated on inventing and implementing technology for observing social distancing, mask-wearing, and contact tracing, and these activities are frequently carried out individually. Despite the extensive use of AI tools in these endeavors, there exists an opportunity to integrate IoT, edge, and cloud computing along with AI to create comprehensive frameworks for effectively monitoring adherence to protocols related to influenza viruses, including COVID-19 outbreaks. In this work, we present the design of a pandemic-compliant mechanism for monitoring adherence to influenza virus protocols, including social distancing norms and mask-wearing, as well as contact tracing. We utilize the Grove AI HAT and Raspberry Pi 4 combination, connecting multiple sensors to form edge nodes that collaborate with a cloud server hosting various ML/DL tools within a residential premise. In particular, we use DL models that have already been trained, like RESNET-50, MobileNetV2, and SocialdistancingNet-19, in a framework called Hybrid Multi-Head Attention-Aided Multi-Tasking Deep Network with Diffusion Stability (HMAAHDNDS), which is based on attention and DTL. This framework utilizes samples from open-source databases to monitor protocol adherence and conduct contact tracing. Additionally, it generates images from text descriptions of violators and is equipped to detect cyber-attacks when connected to public networks. We benchmark our approach using various AI models, including feed-forward (FF) Artificial Neural Networks (ANN), Convolutional Neural Networks (CNN), VGG-16, Random Forest (RF), Decision Tree (DT), ResNet-50, and MobileNetV2. The results obtained from this integrated system, referred to as the Internet of Intelligent Infrastructure Things (IoIIT), demonstrate the effectiveness of our proposed approach.

The specific contributions of the work are:

• Design of a pandemic-compliant mechanism for effective monitoring of the adherence to bio-safety norms notified for influenza-type virus infections. The mechanism is configured to perform facemask-wearing detection, social-distance norm adherence, contact tracing etc.
• Edge nodes designed using Grove AI Hat and Raspberry Pi4 trained, tested and synchronized with cloud resident DL tools are deployed as part of a residential complex.
• Most importantly, several pre-trained DL models are configured to formulate a hybrid platform named HMAADNDS which effectively performs multiple tasks including the detection of cyber-attacks.

2. Proposed Pandemic Compliant Perceptive Infrastructure Design

Extensive analysis of existing literature highlights the existence of a significant opportunity for devising solutions that seamlessly integrate various aspects such as face mask detection, mask classification, selection of appropriate mask types, adherence to social distancing guidelines, and contact tracing in the context of influenza viruses, including COVID-19. An AI-driven approach holds the potential to automate these processes, manage accumulated data effectively, generate insightful analytics, ensure higher reliability, and establish a connection with confirmed cases of infection. These are essential features that should be incorporated in the design of a pandemic-resilient intelligent infrastructure as part of a human residential complex. Figure 1 illustrates an integrated methodology that combines adherence to social distancing norms, recognition of face masks, and contact tracing about influenza viruses, including COVID-19. Notably, our focus extends to developing a comprehensive strategy for monitoring protocol adherence during pandemics, encompassing social distancing norms and correct mask usage. This involves leveraging the capabilities of the Grove AI HAT and Raspberry Pi 4 combination, which connects multiple sensors to form edge nodes operating in tandem with a cloud server. These components are integral to a residential setting and employ a suite of machine learning (ML) and DL tools. To elucidate the operational dynamics of the proposed approach, a concise overview of the relevant background is provided below.

2.1. Setting Up IoT, Edge, and Cloud Computing Framework for Protocol Monitoring and Contact Tracing

The primary focus of our proposed solution centers around the development of an IoT and edge computing setup (depicted in Figure 2) tailored for monitoring compliance with mask-wearing and social distancing guidelines, alongside enabling efficient contact tracing, text-to-scene generation, and cyber-threat detection. The edge computing infrastructure comprises a Grove AI HAT [64], a Raspberry Pi 4 (model B), and an assortment of sensors. Each entry point in a residential premise (with four entrances) is equipped with an IoT pack containing a camera, an IR thermometer, an ultrasonic sensor, and a proximity sensor. The Grove AI HAT interfaces with the camera via a 24-pin serial communication connector, which is crucial for monitoring adherence to physical distancing norms. In case of violations, a loudspeaker alerts individuals, prompting necessary action. This forms the basis of the IoIIT. Similarly, an infrared thermometer measures individuals’ temperatures non-invasively as they enter the premises. The ultrasonic sensor identifies unauthorized entry through the gates. The edge node has sufficient AI-aided decision-making capability. The Raspberry Pi handles data recording, and processing, and maintains a tally of individuals passing through. Additionally, there is a provision for integrating a display to showcase temperature readings, social distancing metrics, compliance status, etc.

The ultrasonic and infrared temperature sensors, along with the Raspberry Pi, are interconnected, ultimately linking the Grove AI HAT and a cloud server via Wi-Fi. Remote control of the setup is facilitated through VNC Connect software [64]. Utilizing edge computing, calculations are performed during the processing stage for each sensor’s data about an individual scan. Subsequently, in a multitasking scenario, the edge computing platform selects a sensor node based on its location and updates the cloud server’s records accordingly, guided by the sensor data analysis. The sensor locations are fixed and are marked in terms of port connections in the Raspberry Pi and the subsequent feed to the cloud server.

The combination of IoT, AI, and Wi-Fi access completes the process of data collection, processing, local decision-making, and communication. Typically, both wired and wireless networks are utilized by various applications to directly interface with the edge node and the cloud server computing platform. Additionally, both the cloud servers and edge nodes record all node parameters, enabling them to calculate results based on a repository of comprehensive information. In our experiments, we deployed four such edge-computing nodes at the entrance of a residential complex, as illustrated in Figure 3. As individuals enter, each sensor takes readings and transmits them to the edge-computing node. Initial decisions regarding actions to be taken, including monitoring compliance with mask-wearing and social distancing norms, and executing contact tracing, are made at the edge node level. This information is shared with the cloud server and stored in a database. For each of these actions, the edge node provides decision support, aided by dedicated Deep Neural Networks (DNNs) deployed on the cloud server. These DNNs undergo end-to-end learning and are trained to handle tasks as outlined above. Details of each DNN type configured for these purposes are provided in subsequent sections. The Wi-Fi access is through data rates of 50 Mbps with an optical fiber backhaul of 1 Gbps. Latency is a crucial aspect of the setup. We tested the setup with three Wi-Fi configurations (50 Mbps, 40 Mbps, and 30 Mbps), 4G (10 Mbps), and 3G (1 Mbps), providing latency ranging from 3 milliseconds (at 50 Mbps) to 100 milliseconds. The configuration of the edge node and cloud servers is outlined in Table 1, while the different sensors/ components used in the system are detailed in Table 2.

You only Look once version 4 (YOLOV4) is used at the edge node level for local decision-making based on vision captured by the cameras especially to perform face-mask detection, classification type, proper wearing, and social distance monitoring. The YOLOV4 uses a pre-trained architecture, carries out one-shot training and detection, and is reliable for fast real-time applications. The range of the sensors and other related connectors are shown in Table 2. The cameras (Philips, HSP3500) for face-mask handling are effective upto 25 meters in daylight and 10 meters in low light.

The IoT framework designed for this purpose comprises four layers: the perception layer, transport layer, processing layer, and application layer. The perception layer consists of a camera, IR thermometer, ultrasonic sensor, and proximity sensor, each deployed at the entrance of the gates. The transport layer facilitates communication between the nodes and the cloud server, primarily utilizing Wi-Fi access (XLT240170), which has a maximum range of 150 meters with various backhaul support options. A detailed summary of the sensor types used is provided in Table 2. The processing layer is constituted by the proposed architecture outlined in , which undertakes tasks such as detecting facemasks, classifying masks, ensuring proper mask-wearing, monitoring social distancing norms, executing contact tracing, generating images of violators, and defending against cyber-attacks. The application layer is constituted by the edge nodes (Raspberry Pi + Grove AI-Hat) deployed with an assortment of sensors at the four entrances of a residential premise and connected to a cloud- server so as to ensure synchronized operations with decision support executed by the DL tools. This essentially formulates the IoIIT.

2.2. Proposed Hybrid Multi-Head Attention-Aided Multi-Tasking Deep Network with Diffusion Stability (HMAAHDNDS)

The central focus of our proposed work lies in the design of a Hybrid Multi-Head Attention-Aided Multi-Tasking Deep Network with Diffusion Stability (HMAAHDNDS) framework (Figure 4). At its core, the design incorporates several deep task processing modules which are DPTMs namely RESNET-50, MobileNetV2, and SocialdistancingNet-19, each trained independently to address tasks such as face-mask detection, proper face-mask recognition, face-mask classification, monitoring social distancing norms, and contact tracing. Additionally, the HMAAHDNDS, as a composite setup, is equipped to perform tasks such as image generation from text inputs and detection of cyber-attack types (Table 4). Figure 4 provides a detailed layout of this architecture. The incorporation of attention layers within the network facilitates optimized processing, outlier cancellation, and focused analysis of specific feature maps. Following this, several fully connected (FC) layers generate composite feature maps and text outputs, which are then applied into a generator network for the synthesizing the images of the violators, improper mask-wearing by specific individuals (samples), and creation of the contextual scenarios within the camera range connected to the IoT package. Furthermore, the setup is adept at detecting cyber-attacks, with each network core (DPTM) trained for this purpose. Details regarding the outcomes of cyber-attack detection will be discussed later. Operating on a cloud platform, each primary task, such as face-mask recognition, social distancing monitoring, and contact tracing, has been rigorously tested and is elaborated in the subsequent sections. The inputs are images and sensor feeds. Images are coloured and sensor feeds are taken at intervals of 5 seconds. A summary of the key activities and decision states of the proposed system is shown in Table 3 and Cyber-attacks associated with data driven approaches of healthcare system is shown in 4.

2.3. Face Mask Recognition Utilizing Multiple ML/DL Methods Deployed on Cloud Platforms

As discussed above, proper face mask wearing and its recognition are crucial in the design of pandemic-compliant facilities and related infrastructure. For the design of a system that recognizes a proper face mask, it is necessary to configure an appropriate computer vision-based system with relevant decision-support mechanisms. The fabrication of such a mechanism first in standalone form is key to the design of pandemic-compliant infrastructure. It prepares a DPTM to be part of the proposed HMAAHDNDS (discussed in Section 2.2). The proposed method of proper face mask recognition is summarised using the block diagram shown in Figure 1.

The data used for configuring the model is first separated for training, testing, and validation. This content has a set of images having regions of interest (RoI) marked and appropriately labeled. As already mentioned, we have used DPTMs, namely ResNet-50 and MobileNetV2, for performing the face mask detection to make the system suitable for real-world situations. The other mentioned discrimination methods are used as benchmark approaches (Table 5). The details of the different parameters of the benchmark methods and DPTMs used for face mask detection are shown in Table 5.

Here, the term "batch size" refers to the number of samples processed in a single iteration of a ML/DL network. Instead of feeding the entire dataset into the model at once, which can be computationally expensive and memory-intensive, the dataset is divided into smaller subsets called batches. Further, the layers in the front section of the input block are used to derive feature maps from the available spatial resolution of the images captured by the cameras.

Along with face mask detection, proper wearing of a face mask is also crucial. We have performed some experiments to ascertain the correctness of wearing of the face mask. The methods and parameters used for this purpose are provided in Table 6.

The working of the system is based on certain steps. Kaggle data sets (https://www.kaggle.com/mrviswamitrakaushik/facedatahybrid) are adopted for the work. The data set provides both faces with and without face masks. Around 7000 images overall, of which 3725 depict people wearing face masks and 3828 depict people without them, are gathered into one set. Additionally, the images come in a variety of sizes, colours, backdrops, brightness levels, and contrast levels to suit any circumstance. There are two groupings made up of these datasets. The training set, which has roughly 6042 photos (80%), is one of them. Both the validation and the testing data sets contain 755 photos (20%). The outputs of the pre-processing portions have been applied to train the range of learning-based systems mentioned in Table 5. We have used dropout rates as part of multi-modal factorized bilinear pooling (MFBP) with ResNet-50 [46] and MobileNet [47] to avoid over-fitting. Then, to assess the learning accomplished and to compare the performances before and after training, we employ the mean square error (MSE) as the cost factor and vary the learning rates using the adaptive moment estimation optimizer. Figure 1 shows the position of the facemask detection in the overall scheme of the proposed design. The related parameters and specifications are shown in Table 5 and Table 6.

Similarly, we have performed a series of experiments to ascertain the classification of face masks used by people during pandemic situations in public places. After executing a series of trials, it is found that the ResNet-50 performs best in terms of accuracy and processing time which becomes the main cloud-resident tool for carrying our decision support related to face masks.

2.4. Monitoring Compliance with Social Distancing Norms (SDN)

To prevent the spread of infectious diseases, such as those transmitted through droplets and micro-droplets, adherence to social distancing measures is crucial, as recommended by the WHO [61]. Governments and concerned agencies require mechanisms to continuously monitor compliance with social distancing norms. Automated monitoring frameworks based on pattern recognition, object detection, and AI tools have been considered for this purpose. Some popular methods include region-based and unified-based algorithms [38]. The Region-Based Convolutional Neural Network (RCNN), based on recurrent clustering, is effective but relatively slower due to the need to run procedures for each generated region [60]. An alternative, the Fast-RCNN, enhances speed by processing all regions simultaneously and utilizing a regional proposal network (RPN) instead of targeted search [60,67]. Improved performance is expected from several popular DPTMs, including MobileNetV2, ResNet-50, and Social DistancingNet-19 [67]. The logic for detecting compliance with social distancing norms is depicted in Figure 8, where inputs include images and video streams. Relevant parameters and configuration details are provided in Table 7.

Utilizing the YoLo4 network, renowned for its capability of enabling real-time detection and enhancing the accuracy of identifying small objects, is a specific technology employed in a unified approach [50]. Another efficient method, the Single Shot Multibox Detector (SSD), was also suggested by the authors in [36]. These techniques utilize Convolutional Neural Networks (CNNs) to construct feature maps from images for object detection. By employing surveillance cameras installed in public areas, YOLO or SSD techniques can be utilized to detect individuals. A system linked to these cameras continuously monitors the live image stream, enabling the identification of individuals who violate social distancing regulations. As mentioned above, the YOLO4 network is particularly relevant to the edge nodes constituted by the Raspberry Pi 4 (Table 1) and is deployed in the four entrances to monitor local activity.

As already mentioned, the overall system includes a social distance algorithm. As a result, once the server recognizes an infected person through a cough signal and a sneeze action, it notifies the decision support system, edge computing nodes, local populace and triggers certain procedures linked with the social distancing guidelines. The cough and sneeze actions of a specific person are learned by the system and if the person enters the gate of the residential premises, it records the necessary vitals through the installed sensors. This process helps in placing the person under observation.

In Figure 1, the logical approach to the design and development of the smart social distancing system is summarized. The social distance observance is described by locating people wearing masks and then finding the physical separation between them.

The system initially focuses on detecting masks worn by individuals. Identifying a mask on a face serves as a confirmation of a person’s presence. Subsequently, the distance between two individuals wearing masks is calculated through pixel-to-spatial separation conversion. Facial recognition is then employed to ascertain the identity of potential violators. A record of these violations is generated, and notifications are sent to relevant stakeholders, including the individuals involved. This mechanism is crucial for facilitating contact tracing. After conducting a series of standalone trials, it was determined that SocialDistancingNet-19 is the most suitable model for monitoring compliance with social distancing norms. It is chosen as the cloud-resident tool to oversee adherence to these norms. In this capacity, it functions as a DPTM and undergoes further end-to-end learning using online samples received from edge nodes. At the edge node level, the Yolo4 network provides initial decisions while the DPTM extracts, stores, and shares initial features captured by video cameras. The higher-level inputs are then used by the cloud-resident SocialDistancingNet-19 to continuously train along with the inputs received from all the remaining feed points (as depicted in Figure 4).

2.5. Implementing Contact Tracing in Public Spaces and Residential Communities

During the peak period (2020–2022) of the severe acute respiratory syndrome caused by SARS-CoV-2, traditional contact tracing methods became overwhelmed. Public health organizations resorted to manual contact tracing by interviewing infected individuals to identify potential contacts. Exposed contacts were advised to seek medical attention, undergo treatment, and self-quarantine if necessary. While this strategy helped reduce transmission, it faced challenges and opportunities for technological interventions to enhance reliability. Digital contact tracing emerged as a solution to scalability, notification delays, recall errors, and identifying contacts in public spaces. It utilizes electronic data to trace exposures to illnesses. Most COVID-19 contact-tracing apps estimate proximity between smartphones using Bluetooth signal strength and determine exposure status based on proximity duration and distance from an infected individual. However, privacy and security concerns prompted debates over the architecture used for data gathering in tracking apps. Three main system architectures—centralized, decentralized, and hybrid—have been employed for COVID-19 tracing applications, with the final architecture often blending elements of both centralized and decentralized approaches. The proposed contact tracing technique’s block diagram is depicted in Figure 5. Seven alternative architectures (VGG-16, ResNet-50, MobileNetV2, SVM, DT, KNN, and LR) have been tested to determine the most reliable network for executing contact tracing within a cloud-resident framework integrated with intelligent edge nodes. In this approach, potential contacts are identified based on received inputs, initiating forward tracing. Infected individuals are isolated and treated, while symptom recognition during quarantine ensures continued care. Contacts of isolated cases are identified, and infected individuals are monitored for symptom severity to determine appropriate treatment levels. Quarantine, testing, and medication for contacts of infected individuals occur concurrently, with updates continuously logged in the database. Table 8 outlines the settings and configurations for the various ML and DL architectures utilized in the contact tracing efforts.

2.6. Text to Image Generation

Generating images from text is crucial for visually representing data obtained from continuous monitoring data and deriving proper interpretation of the situations. It aids in raising public awareness by illustrating challenging circumstances and provides training samples for enhancing network efficiency. To accomplish text-to-image generation, the input text is converted into a meaningful representation, such as a feature vector, which is then utilized to produce an image that aligns with the description.

The system initially receives text input and conducts tokenization to extract key textual features. Subsequently, images and text data from the dataset are combined before proceeding to the training phase. Here, the generator and discriminator play pivotal roles: the generator generates images while the discriminator classifies them. This iterative process continues until the generator can proficiently create images based on the input data. Ultimately, a trained model capable of producing visuals from textual input is obtained [80]. The flow logic and constituent blocks are illustrated in detail in Figure 4.

The Long Short-Term Memory (LSTM) layer is a crucial component within the block, serving for sequence modeling and context retention. Within the system, the LSTM layer takes in text embeddings, which are then processed through a dense layer and concatenated with a latent vector for generating a response. To enrich the dataset, a normal distribution is employed for sampling additional data from the latent vector. The output of the dense layers undergoes reshaping and concatenation with the activation volumes of convolution blocks in the generator. Passing the embedded text through various dense layers with outputs of different sizes achieves this. The generator includes an up-sampling layer after a convolution block housing two $3\times 3$ kernel-sized convolution layers. Additionally, a $1\times 1$ convolutional layer with three kernels is applied to the activation volumes of the generator. Pixel-wise normalization is utilized within the generator. In the discriminator, after an average pooling layer, the convolution block comprises two convolution layers with $3\times 3$ kernel sizes. Finally, the resulting output passes through a linear activation block, which determines whether the image is authentic or false.

2.7. Enhancing Cyber-Attack Detection

The risk of cyber-attacks looms large whenever a computer-driven system is connected to a broader network with partial, temporary or complete access to the internet. In this context, the composite system’s cyber-attack detection capability is developed to continuously monitor data traffic, discerning unauthorized access or activities within a networked environment. Cyber-attacks can manifest in two primary phases—during training and testing. Several common cyber-attacks targeting IoT-based setups include phishing, log access breaches, social engineering attacks, and brute force attacks aimed at passwords [80].

The provision for the detection of cyber-attacks has been incorporated into the system because this is an ICT-driven framework. Such a framework is bound to be affected at some point in time by a cyber-intruder which may be external or internal. Moreover, there is always a possibility that mobile device access to information generated by the system should be added as an extension of the work. In such a situation, cyber-threat shall be a reality. Further, internal cyber-threat will always be present.

Database creation is necessary as the records of sensor readouts and decisions generated for individuals will continuously expand. As a result, cyber-threats will always be present. The system can be operated as a standalone system and may not be connected to mobile devices/apps. But then updates and information dissimination will not be automatic and real-time. In the present form, the experiments are only intended to validate the system’s capability against generalized cyber-threats.

There are different types of cyber-threat detection methods. Here, ResNet-50 and MobilenetV2 are used as classifiers to detect cyber-attacks. However, Linear Regression (LR) and VGG-16 classifiers are taken as benchmark methods for similar purposes to ascertain the capabilities of DPTMs. For this, a total of 30,190 (CSC2010HTTP) samples (https://www.kaggle.com/akashkr/phishing-website-dataset) are taken out of which 70% are used for training and 30% are used for testing. These samples are balanced. Social distancing monitoring and cyber-attacks can be done through a mobile app. Residential blocks are required to have internet connectivity. Mobile devices will be connected through the internet and records will be stored on the cloud. So these combinations and features of the networks make the cyber- threat detection a necessary element of the proposed method. Cyber-attacks that are reported to be associated with data-driven approaches of healthcare monitoring systems are shown in Table 4.

3. Results and Discussion

In this section, we discuss the results obtained using different approaches adopted for the detection of facemask wear, the observance of social distance norms, and contact tracing. Further, we report the impact analysis of the proposed work with regard to its deployment in a residential premise. The validation of the system to perform the discrimination of cyber-attacks has also been included. We report the results in a sequence, starting with the work related to facemask detection (Section 3.1), the correct or incorrect wearing of a facemask, and the classification of a facemask, the observance of social distance (Section 3.2), and contact tracing (Section 3.3).

3.1. Facemask Detection Experimental Findings

The cornerstone of an effective design for facemask detection hinges on various subsystems within the system, with the reliability of the ML/DL-aided decision support system being paramount. In our approach, we have leveraged a spectrum of ML and DL techniques to identify the optimal method for dependable facemask detection. Our methodology also encompasses capabilities that are used for distinguishing between correct and incorrect facemask usage. Training and testing have been conducted using a dataset comprising over 6000 samples across multiple iterations. Approximately 80 percent of the data has been allocated for training purposes, with the remaining data reserved for validation and testing. During real-time testing, samples gathered from field visits to potential deployment locations have been utilized. Images of individuals both wearing and not wearing masks, as depicted in Figure 6a and Figure 6b, respectively, have been instrumental in training and testing DL models for facemask detection. Additionally, Figure 6c showcases datasets illustrating improper facemask usage, which serve as valuable resources for refining and enhancing facemask detection algorithms and models, constituting a crucial aspect of our ongoing efforts. These datasets contribute to the robustness of the DL models.

Medical facemasks are commonly crafted from materials like cloth, non-woven fabric, or disposable paper. These can be either disposable or reusable, serving as essential personal protective equipment (PPE) to safeguard both patients and healthcare workers against the transmission of infectious diseases. Consequently, medical face masks find extensive utilization in healthcare settings such as hospitals and clinics. Examples of datasets featuring normal masks and medical facemasks are depicted in Figure 7a and Figure 7b, respectively.

Table 9 shows the summary results obtained from a class of methods trained to detect correct facemask wear. We clearly observe that VGG-16, conventional CNN, ResNet-50, and MobileNetV2 provide better reliability.

In our experiments concerning the detection of proper facemask wear through computer vision and learning-based techniques, we noted that MobileNetV2 outperforms ResNet-50 in terms of accuracy, as evidenced by the summarized results presented in Table 10. Notably, Table 9 showcases a reported 100% accuracy by [15]. Upon implementing the approach detailed in [16], we confirmed achieving 100% accuracy with small datasets. However, when working with larger datasets, this accuracy significantly decreases.

Additional Experiments related to Facemask Type Detection: Table 11 illustrates the outcomes obtained from ResNet-50 and MobileNetV2. Further, it showcases the ResNet-50’s superior accuracy over MobileNetV2. We carried out the implementation of the system as detailed in [47], with parameter adjustments and rigorous training and achieved the accuracy reported in Table 11.

Comparison of ResNet-50 Results with Previous Studies: Our study aligns with prior research, utilizing identical datasets, methodologies, and testing procedures, as depicted in Table 12. Our ResNet-50 based model exhibits a 4% decrease in performance due to extensive data diversity, inclusion of on-field samples, background, interaction, sensor quality fluctuations, and variations in illumination. Anticipated enhancements in results are foreseeable with increased exposure to similar samples, especially those with complex backgrounds. Table 12 exclusively features studies employing ResNet-50, highlighting its widespread utilization in such applications.

We conducted comprehensive model implementations across various datasets and diversity settings, yielding comparable results. [Figure 8,Figure 9,Figure 10] illustrate model accuracies, and training time losses for ANN, CNN, VGG-16, MobileNetV2, and ResNet-50, alongside classification performance metrics for ResNet-50 and MobileNetV2.

3.2. Evaluation of Social Distancing Measures

The goal of social distancing is to slow down the transmission of the disease and reduce the burden on the healthcare system. Using day and night feeds from the cameras effective performance has been obtained upto 20 meters. Accuracy and F1-score are shown in Table 13.

Table 13 summarizes the average performance for person detection and social distancing using specific methods. The model training accuracies for ResNet-50, MobileNetV2, and SocialDistancingNet-19 are compared, with SocialDistancingNet-19 achieving the highest accuracy at 94.5%. Table 7 displays the parameters of different DL methods employed for social distancing, reflecting the combinations and tuned states of the models utilized in this study. Ultimately, SocialDistancingNet-19 emerged as the most suitable option and has been deployed as a cloud-resident network for social distance decision support.

3.3. Contact Tracing: Tracking and Managing Exposure Risks

The objective of contact tracing is to mitigate virus transmission, avert outbreaks, and decelerate disease spread. To facilitate contact tracing, the VGG-16, MobileNetV2, and ResNet-50 networks undergo training using pre-existing data. Table 5 outlines the configurations of these networks employed for this purpose. Current data are continually updated with previously known details, enriching the training process. Among these models, ResNet-50 achieves an accuracy of 96.1%. Table 14 illustrates a comparison of results attained from proposed methods, encompassing VGG-16, MobileNetV2, ResNet-50, and several other previously reported methods.

3.4. Impact Assessment of the Proposed System Architecture

Table 15 provides a comprehensive overview of the optimal performance of the proposed multi-node edge and cloud computing setup tailored for residential premises. It encompasses the analysis of deviations in sensor readings, decision support for facemask recognition, adherence to social distancing norms, contact tracing capabilities, and latency response across various load conditions at both node and system levels. These deviations stem from the diverse array of tasks managed by each node, the continual variation in sample types, the operation of different network types, and synchronization challenges. Despite these complexities, the system exhibits reliability. However, the primary bottleneck emerges from Wi-Fi access limitations and the performance of pickup devices. Notably, varying error levels occur at both node and server levels. Learning and decision-making processes are shared between the nodes and the server. Errors may occur when an individual, who typically utilizes a specific gate on one day, appears at a different gate. In such instances, the server may make a true negative decision, which gradually stabilizes over time toward the desired outcome.

Table 17 presents a comprehensive overview of the effectiveness of various strategies implemented in the proposed system, evaluated in residential settings over a six-month period. The post-validation and deployment phases demonstrate response speeds (in milliseconds) suitable for real-world applications, despite extensive training and testing duration (in seconds). Similarly, a detailed analysis has been conducted regarding social distance measurements using the SOCIALDISTANCINGNET-19. The system’s results exhibit slight discrepancies due to inaccuracies in distance calibration and spatial calculations derived from pixel values. The DL method exhibits a maximum error margin of 12% between projected and actual distances. The devastation caused by the COVID-19 pandemic between 2020 and 2022 underscores the importance of mitigating the spread of contagious diseases by minimizing interpersonal contact. Hence, the development of intelligent infrastructure capable of continuous monitoring of social distance compliance, proper facemask usage, and contact tracing is imperative, particularly in high-risk areas.

The proposed method demonstrates high accuracy rates for facemask detection, ranging from 90% to 96% across different DL models. This facilitates reliable surveillance in crowded environments and provides guidance on correct mask usage. Additionally, the system employs human detection techniques, achieving accuracies of 87% with ResNet-50 and 90% with MobileNetV2, enhancing social distance monitoring reliability. Moreover, the contact tracing accuracy rates using VGG-16, MobileNetV2, and ResNet-50 are reported as 92%, 93%, and 96% respectively, ensuring robustness and dependability. The collective effectiveness of each component underscores the system’s capability to facilitate continuous COVID-related behavioural monitoring as part of intelligent infrastructure compliant with pandemic regulations. Given the challenges and limitations associated with manual intervention, the proposed system emerges as pivotal in the creation of intelligent human environments, offering a solution that is efficient, safe, and accurate.

3.5. Validation of Component Block Capabilities in Detecting Cyber-attacks within the Proposed System

Confusion matrix of various classifiers tested for malicious and normal traffic is shown in Table 18.

A total number of 30,190 samples are taken for detecting cyber-attacks in the proposed system. The accuracy rates for cyber-attack using MobileNetV2, and ResNet-50 are 94.89% and 90.07%, respectively. However, VGG-16 is used as a benchmark to train this model and the summary performance is shown in Table 19.

The ability of the system to provide image output from text input is shown in Figure 11. For four different text descriptions, how the composite system is able to generate the corresponding representative images is shown in Figure 11. The text embedding that contains the decision states obtained from the DPTMs are shown in Table 20. These text descriptions assist the text-to-image transformer in using the principles of diffusion stability to provide synthetic visual representations related to bio-safety norms as part of a pandemic-compliant, resilient and perceptive framework. The key attributes of the proposed method compared to previously reported works have been summarized in Table 21. The advantage of the proposed method is obvious.

4. Conclusion

We present a system architecture comprising edge computing nodes, an IoHT system equipped with sensor packs, and a composite attention-driven framework integrating various DL pre-trained models (DPTM) such as RESNET-50, MobileNetV2, and SocialdistancingNet-19. This framework forms an IoIIT and executes monitoring protocol adherence, including social distancing norms and mask-wearing, as well as for contact tracing utilizing samples from open-source databases. Additionally, the composite framework generates scene images, and context, and identifies violators with text descriptions. Furthermore, it detects cyber-attacks when connected to public networks. Configured and trained to effectively address real-world challenges within residential premises, the constituent systems undergo rigorous testing. Notably, the accuracy rates for detecting mask presence and proper usage are 92% (ResNet-50) and 97.2% (MobileNetV2). However, facial recognition encounters difficulties when faces are inclined at certain angles, regardless of mask usage, necessitating the resolution of this issue. To discern between fabric masks and N-95 masks, ResNet-50 and MobileNetV2 are employed. Parallelism enables the simultaneous execution of mask categorization and social distancing calculations. Despite the system’s robustness, insufficient and non-diverse datasets limit its adaptability. For instance, inadequate representation of beards may lead to occasional confusion with masks. Access to more comprehensive datasets could facilitate the training of more potent models. Contact tracing for COVID-19 and influenza involves identifying individuals with significant contact with confirmed virus carriers and monitoring them for symptoms. This data aids in virus transmission prevention and outbreak management. Public health experts leverage contact tracing results to understand virus spread within communities and inform pandemic response strategies. System performance is constrained by network latency, varying with traffic load. Enhancing bandwidth and physical connectivity effectively mitigates this limitation.