A Systematic Review on Machine Learning Fundamentals for Smart Home: Towards Smart City Solution

1. Introduction

1.1. Background and Motivation

As we all know, there is a lot of emphasis on constructing smart cities all over the world, including in nations like India. Of course, the extent of smart cities varies by nation, and the scope is determined by the priority areas of each of these countries and their governments. In India, a few towns have been recognised in recent years, and funding have been allocated to these cities in stages to create or turn them into smart cities. When we discussed smart cities, what they are, and the ordinary infrastructure in any city, such as residential structures, office buildings, hospitals, schools, transit, police, and so on.

Let us consider such smart methods; what smart means in terms of the services provided to these cities' different stakeholders so residents may accomplish things better than normal, and how that is made feasible. It is feasible thanks to Information and Communication Technologies (ICT), which also comprise embedded electronics and other modern electrical and electronics engineering technologies. As a result, the combination of computers and electronics has the potential to make these cities smart. Let us take an example at the outset to explain smart cities through Figure 1. The smart city has some necessary components like smart home/residential building, smart hospital, smart waste management, smart traffic control, smart banking system, smart transport system, smart police control, smart schools, and smart railway system. We have to transform all these different components of any city to be smart, for which the technology we have to take help of following given below:

• Sensors
• Sensor Network
• Actuators
• Communication Technologies (RFID, NFC, Z-Wave, etc.

Therefore, all these technologies will have to be used to make this transformation. Thus, this work chooses a Smart Home Energy Management System (SHEMS) design concept to contribute to smart city transformation.

When we talk about humans, they have the bones, skin, various organs, the brain, nerves, sensory organs, intellect, and so on. Similarly, a smart city has operated as a human system, as illustrated in Figure 2 by the comparison. A human has a skeleton, skin, and numerous organs, as shown in Figure 2; similarly, smart cities or cities contain structures, industries, people, transportation, logistics, hospitals, police, banks, and schools, among other things. Furthermore, if a human has a skeleton, skin, organs, but no intellect, that human has no life. The same parallel can be taken, and we can argue that in a smart city, the lack of embedded intelligence communication networks, sensors, tags, and software are embedded in these distinct components, and the city's infrastructure would be lifeless. As a result, ICT must be implanted in order to bring existing cities to life with buildings, industries, transportation, police, banking, and so on. Embedded ICT encompasses ubiquitously embedded intelligence, digital communication networks, sensors, actuators, tags, and various software cleverly performing other things to enable these diverse devices to respond intelligently.

1.1.1. Application-Based Research Areas

Cities that employ technological solutions to improve public services and the quality of life for residents can be said to be smart cities. The Internet of Things (IoT) sensors and communications technologies are used by municipal governments to collect relevant data, such as information regarding traffic congestion, energy consumption, and the influence on the environment. These data could also be used by engineering solutions to enhance community services, including infrastructure, transport, and public services. So, these are some of the application focus areas given as (Pellicer et al., 2013):

• Smart Economy based on Competitiveness
• Smart Governance based on Citizen participation
• Smart People based on Social and human capital
• Smart Mobility based on Transport and ICT
• Smart Environment based on Natural Resources
• Smart Living based on Quality of life

These are some of the application focus areas that we have to consider for a smart economy. So, because of the ever-increasing competitiveness, we need to improve our infrastructure, the economy to make it elegant. We also need to improve citizen participation in any good governance with ICT tools, social and human capital. We need to create social and human capital too smarter by giving them different technologies, agencies (ICT tools), and intelligent mobility to improve transportation.

1.1.2. Trending Focus Research Areas (Alhashmi et al., 2019)

• Smart homes (smart health monitoring, smart conservation of resources (electricity, fuel, water), smart security, and safety).
• Smart Parking Lots
• Smart energy (smart metering systems, smart energy allocation, and distributed system, incorporation of traditional and renewable energy sources in the same grid).
• Energy and buildings (highly developed sustainable buildings and smart grids, enhancing the energy efficiency of sustainable housing by optimization).

The above-mentioned trending research areas focused on smart homes management in terms of elderly people health monitoring, energy conservation of resources, and security features. Also, building smart parking lots will protect theft. The smart energy is achieved by developing the smart metering systems, smart energy allocation and distributed system, various incorporation of traditional and renewable energy sources in the grid.

1.1.3. Technological Research Areas

• Data collection (mobile devices, sensors, tags, architecture)
• Data transmission (radios, networking, topologies)
• Data storage (local storage, data warehouse)
• Data processing (data cleaning, analytics, prediction, or forecasting)

These are the technological research areas in data collection, data transmission to the server and end-user, data storage with local and warehouse and data processing techniques in data mining.

1.1.4. Mathematical Methods of Data Fusion

• Probability-Based (Bayesian analysis, statistics, recursive methods)
• Artificial Intelligence (AI) Based (Artificial Neural Network (ANN), machine learning algorithms, Convolution Neural Network (CNN), Deep Neural Network (DNN)).
• Theory of Evidence-Based (belief functions, transferable belief models)

These are the different mathematical methods that are to be used to come up with these bits of intelligence from the various data we know that are used secured from the IoT devices (Battisti et al., 2020). IoT devices are usually restricted devices, creating correlations between hardware, software and protocols used (Kruger et al., 2015). AI comes as a big helper in enabling the IoT devices by maintaining precise decision-making. Consider this particular Figure 3a (Furqan et al., 2017); we have sensors and the sensor data which has to be transmitted over the communication medium. Based on that, some actuation is going to be made possible.

But how that actuation is going to be made possible is it from one or two of these sensors based on these sensor values which will actuate. We can also do something better for positive decision-making by adding intelligence between these different sensors, and actuators can be seen in Figure 3b (Furqan et al., 2017). It is all made possible with the help of AI tools, techniques, algorithms, etc.

2. Historical Developments in Smart Home

The very first smart homes were theories, not real structures. Science fiction has been discussing the concept of home automation for decades. Prolific authors like Ray Bradbury dreamed of a world in which homes were highly interactive and automated. In Bradbury's strongly-worded science fiction, "There Will Come Soft Rains," he defines an autonomous home that continues to work even after humans have died. It's all well and scary before you realize the real advantages of a smart home, and then the idea will become more relaxing than chilling, while the concept of home automation has been around for some time, real intelligent homes have only existed for a short time. This timeline focuses on hardware; that is, actual developments leading to smart homes that we know today and can expect from the near future. The history in developing a smart home is tabulated in Table 1.

3. Smart Home and Related Research Areas

Automated buildings with built-in sensing and control devices, such as climate control, heating and ventilation, lighting, hardware, and security systems, are known as smart homes. These are some of the fundamental components that make up the IoT. Through the use of IoT and communication with a variety of electronic appliances, smart houses provide their occupants with all the necessary amenities (Verma et al., 2019). In the most idealised vision of the wired future, all of the gadgets in smart homes will be able to connect with one another invisibly. IoT-based smart home technology has changed the human life and their efforts by providing connectivity to everyone regardless of time and place (Xiao et al., 2020). Automated home systems have become increasingly sophisticated in recent years (Madakam and Ramaswamy, 2015). These types of system provide infrastructure and methods to exchange all kinds of appliance information and services. Smart Home uses Information Technology (IT), management systems, display technology, and communications technology that will be linked together through a network of different areas to meet the whole system's automated needs and provide more efficient control and management (Fang et al., 2011).

3.1. Modified Machine Learning

In recent years, machine learning has been adjusted to improve the efficiency, precision, and personalisation of smart home systems. Recent adjustments and advances includes the following:

Deep learning algorithms - These algorithms are used to enhance the ability of smart home systems to make decisions (Gollapudi and Cheng, 2020).

Edge Computing- Edge computing enables smart home devices to process data locally, thereby decreasing latency and enhancing responsiveness (Rumelhart et al., 1986).

Federated learning - This approach enables numerous smart home devices to collaborate and share data, leading to better accuracy and generalisation of machine learning models (Rahman et al., 2022).

Reinforcement learning - Algorithms based on reinforcement learning allow smart home systems to learn from their interactions with the environment and optimise their behaviour over time.

Natural Language Processing (NLP) - NLP is utilised to enable smart home devices to interpret and respond to human speech, making interactions more intuitive and user-friendly.

Context-aware systems - These systems are able to evaluate the context of a user's interaction with their smart home, allowing for more precise and individualised replies.

A modified machine learning based smart home with intelligent features is shown in Figure 4. The main important consideration while designing home a smart is machine learning based model. And, it only possible with the several steps that are as follows:

• Data collection
• Anomalies checklist, data missing report and data cleaning report
• Statistical performance analysis and visualize the initial screening
• Modelling of machine learning models
• Accuracy check list
• Results presentation

There are several types of machine learning tasks which can be used in smart home (Li et al., 2017). The supervised machine learning, unsupervised machine learning, semi-supervised machine learning, reinforcement machine learning.

Some key features of Smart Home are as follows (Chan et al., 2009). Efficient utilization of electricity and promote the family's knowledge of energy-saving and environmental protection. A smart home can maximize the comfort, security, accessibility, and interactivity of home life and improve people's lifestyle. A smart home can support remote payments. A smart home understands real-time energy management and safety service with a water meter, an electrical energy meter, and a gas meter offering more comfortable conditions for a high-quality service. Smart homes support the business of "triple networks" and the perfect smart service.

Security and efficiency are the key factors behind the rise in the use of smart home technology. All the devices are connected to each other in the smart home, accessed and controlled by a single central agent or central point’s such as smartphones, tablets, or laptops.

4. Energy Consumption and Human Comfort in Smart Homes

Although Building Management System (BMS) control technology has been nearly consistent and fundamental over the past few years, our knowledge of how people perceive environmental factors inside a building has overgrown. Following pioneer work by scientists such as Fanger (Fanger, 1973), researchers understand how weather variables such as temperature, humidity, airflow, lighting conditions, and even colour can influence the comfort level and individual experience. Notably, our understanding of comfort is not fixed. For example, on a very hot day outside, people might find a temperature of 25⁰C feels very comfortable, while on a cooler day, it would also be slightly warm for so many. With the latest research and "human comfort influences" models, it is now comparatively easy to predict how most people react to a specific building environment (Platt et al., 2013). EIA's International Energy Outlook 2017 projects that India will see the highest increase in building energy demand by 2040 across world regions. In the reference scenario of IEO2017, the energy consumption delivered to residential and commercial buildings in India is projected to rise by an average of 2.7% each year between 2015 and 2040, more than double the worldwide mean rise (IEO, 2017). This growth is attributed to the increasing use of electricity and the rising use of electrical appliances to maintain comfort in the living space. The annual average change in residential buildings' energy consumption is shown in Figure 5 (IEO, 2017).

4.1. Building Management System (BMS) for Smart Cities

BMS is a crucial aspect of the smart home, and intelligent building and IoT-based BMS is the future step in increasing the energy efficiency and also energy conservation. The BMS is a computer-based system that helps in managing, audit, and regulate energy use in a building. It may also collect the information from the buildings in order to operate the HVAC, artificial lighting, natural daylighting operations, and utilities linked with safety devices, fire detection and protection, heating, cooling, water, cooking, and power on a city or house/building cluster level. In building automation, BMS and IoT work together to enhance building efficiency. Researchers are reducing energy use by leveraging the Internet of Things and its bidirectional communication linkages. In response to this, one author designed a new BEMS capable of optimising energy use (Kane, 2018). By keeping inhabitants comfortable and their behaviours in mind, this BEMS develops a unique control mechanism based on adaptive hybrid control approaches over the building's energy usage. As a result of environmental worries, cyber-physical systems and real-time resident activity are embedded. The IAQ and ventilation in smart buildings are critical components that impact human health. As a result, real-time IAQ monitoring is necessary in the BMS system, which analyses key gases like as CO₂, SO_x, NO_x, (x=1, 2, 3, ….), Volatile Organic Compound (VOC), and formaldehyde. Recently, researchers noticed this issue and created a number of radio frequency-based sensor methods for real-time monitoring of IAQ in BMS. In connection with this, an author presented a novel indoor environment monitoring system based on smart sensors that communicate bidirectionally between the base station and smart sensor tag (Yu et al., 2018). According to Javed et al. (2018), the major purpose of a BMS is to save energy and improve environmental quality. That is why smart building researchers were optimising HVAC and lighting energy use (Agarwal et al., 2010; Tushar et al., 2018). A real-time control algorithm for the heating and cooling system was proposed by one of the authors. The suggested model uses the Lyapunov optimization approach to reduce energy consumption in a multi-zone commercial building (Depatla et al., 2015).

The essential prerequisites for improving HVAC and lighting systems in a smart building are occupancy estimate and space use. As a result, utilising a machine learning approach and low-cost IoT sensors, an author showed a test system that extracts high-level building occupancy (Kadri et al., 2017). Another author created and developed a low-cost occupancy detection system that relied on battery-powered wireless sensor nodes (Howard et al., 2017). HVAC energy consumption is lowered by 10% to 15% when this low-cost occupancy system is used. Wi-Fi power measurements (Melfi et al., 2011; Yang et al., 2018) that are continually broadcast by Wi-Fi-enabled smart devices through ICT data streams (Labeodan et al., 2015) and occupancy measurement utilising existing network infrastructure may also be used to estimate occupancy (Ding et al., 2016). In a real-time environment, an IoT-based occupancy sensing platform is developed and tested with 96.8% and 90.6% in occupancy detection and recognition (Kim and Lee, 2015).

Commercial office buildings, as we all know, require a vast floor space and use a lot of energy to keep the occupants comfortable. As a result, (Möller, 2014) explains a few measurement approaches such as CO₂ based detection systems, PIR detection systems, ultrasonic detection systems, image detection systems, sound detection systems, computer activity-based detection systems, and sensor fusion. Fine-grained occupancy information is verified experimentally for demand-driven control techniques in buildings. The fact that this device cannot measure occupants' standing posture is a disadvantage. A few newly created smart building research projects in the field of BMS from various laboratories have been compared and presented in this paper (Verma et al., 2019). These many research laboratories focus on modelling, efficiency, energy management, comfort management, data interoperability, and construction. (Verma et al., 2019 and Verma et al., 2020)

4.2. Future of Research in Smart Home with Machine Learning

Numbers of researchers have carried out several theoretical and practical experiments on the energy management system in homes, buildings, and cities. In recent years' massive prediction models are developed reported by (Zhao et al., 2012; Fumo, 2014; Amasyali and Gohary, 2018). Some authors have implemented the energy management system using network controlled approach & developed an intelligent controller (Han et al., 2010; Singh et al., 2016; Pilloni et al., 2016; Yao et al., 2017). Few authors developed real-time energy management for micro-grid, smart grid (Chen et al., 2012; Qian et al., 2013; Wang et al., 2013) using artificial intelligence methodology. The current trends in literature review on EMS in smart homes and smart buildings lack in many aspects. A reliable and long-term electricity consumption forecast model based on design information is sought throughout the early stages of residential building design. Several literature assessments, however, lack an accurate and long-term prediction model. The lighting load with natural daylighting has not yet been addressed for the energy consumption forecast model. As many researchers developed the energy management system for smart homes and smart buildings based on temperature data, illumination & air quality but relative humidity data and daylighting factor has not been included in HVAC operations in the prediction model. HVAC and lighting are ordinary household operations in homes and buildings but consume a large amount of energy. An optimized model needs to be developed in context with the user's comfort to minimize energy consumption and maximize comfort. Although many metaheuristics optimization algorithms and control approaches have been proposed to optimize environment parameters. However, machine learning-based optimization has not yet been explored in energy and comfort management. Based on the above problem definitions, this review study aims to produce new research areas in developing a long-term energy consumption model based on an environmental parameter (temperature and relative humidity) and an automated energy management system for a smart home (Verma et al., 2022; Maurya et al., 2022).

4.3. Image Protection and Security

The recent development of machine learning and image processing techniques has opened up new doors for research in this sector. The automatic extraction and analysis of information contained inside photographs is now possible thanks to machine learning. The recent confluence of machine learning and image processing has proven to be effective in a wide range of different security applications. Image processing is a critical component of both physical and digital security and plays an important role in both. Homeland security, surveillance applications, identity identification, and other similar tasks fall within the category of physical security applications. Protecting digital data is an essential part of ensuring digital security. Digital security can be enabled through the use of methods such as digital watermarking, network security, and steganography. A real-time detection approach for human actions is presented in the research article titled "RGB + D and deep learning-based real-time detection of a suspicious event in Bank ATMs." The method uses deep learning. This technology is implemented in order to improve the monitoring and safety of automated teller machines (ATMs) found in banks (Khaire and Kumar, 2022). A growing number of illegal operations at automated teller machines has turned into a security problem (Zhang et al., 2013). The currently employed methods of monitoring that include human interaction are inefficient to a significant degree. The effectiveness of the human surveillance techniques is mainly reliant on the actions of the security personnel. The approach that has been developed has the capability of providing real-time surveillance of these equipment. The authors have presented an approach based on deep learning as a means of distinguishing between the various types of motion present in a video stream. In the event that suspicious behaviour is observed, the movements will be labelled as abnormal. A real-time surveillance system that follows people around is presented in the research article titled "A real-time person tracking system based on SiamMask network for intelligent video surveillance." This system monitors people in real time. The method that has been offered is one that may be utilised for the purpose of tracking individuals in a variety of public locations, offices, and other types of structures. A human tracking and segmentation system that takes advantage of an overhead camera perspective has been demonstrated by the authors (Paul et al., 2013). An approach for intelligent harbour surveillance platforms is presented in the paper "Adaptive and stabilised real-time super-resolution control for UAV-assisted smart harbour surveillance platforms" (Jung and Kim, 2021).

Drones are utilised in this strategy in order to provide a variable localisation of the nodes. It has been proposed that an algorithm should be developed for scheduling among the data that is transmitted by various drones and multi-access edge computing devices. All of the drones will then communicate their own data during the second stage of the algorithm, and these transmissions will be used for surveillance purposes. In addition to this, the authors have utilised the idea of super resolution in order to enhance the quality of the data as well as the surveillance. Maximizing the time-average performance of the system while maintaining the stability of the self-adaptive super resolution control is accomplished with the help of a method that is based on Lyapunov optimization.

A technique for real-time video summarising that makes use of image semantic segmentation for CBVR is presented in the publication that bears the title "Real-Time Video Summarizing using Image Semantic Segmentation for CBVR" (Jain et al., 2021).

A method for summarising videos frame by frame is presented in this study. It makes use of layered generalisation, which is generated by an ensemble of many machine learning algorithms. In addition to this, the rankings of the videos are determined by the total amount of time that a particular structure or landmark is shown in the video. The KD Tree software is used to retrieve the videos. The technology is versatile enough to be used in a variety of applications for conducting security surveillance. The authors adopt a video summary technique that focuses on the most notable elements of the video scene. The summary is used to conduct a search within the video in order to retrieve the necessary frames. The tagging is carried out with the assistance of machine learning and image-matching algorithms.

A classification model that is based on joint sparse-collaborative representation is presented in the work that has the title "A real-time classification model based on joint sparse-collaborative representation" (Li et al., 2021). This model is presented in the study. The two-phase test sample representation approach is a proposition made in this paper. The authors have made certain changes and improvements to the first step of the conventional two-set technique.

In the second phase, there is an uneven distribution of the training samples. As a result, the authors have incorporated the training samples that were not picked into the modelling. The approach that has been proposed is utilised on a variety of different face databases. The approach has demonstrated a high degree of accuracy in recognition. A technique for detecting human violent activity through the use of drone surveillance is presented in the paper "Recognizing Human Violent Action Using Drone Surveillance within Real-Time Proximity," which can be found at (Srivastava et al., 2022). A machine-driven recognition and classification of human actions extracted from drone recordings was given by the authors. Drones can also be used to collect data from an open area and compile it into a database. An operation known as key-point extraction is carried out, and 2D skeletons of the people visible in the frame are produced. The categorization module uses these extracted key points as features to recognise the actions being performed. Both the SVM and the Random Forest methods were employed by the authors for the classification process. The proposed method allows for the identification of the aggressive activities.

4.3.1. Improved Security and Privacy through the Use of Various Frameworks, Models, Algorithms, and Protocols

Some studies have proposed various frameworks, models, and algorithms to improve smart cities' security and privacy (Al-Dhubhani et al. 2018; Antonopoulos et al. 2017; Avgerou et al. 2016; Beltran et al. 2017; Burange and Misalkar 2015; Cagliero et al. 2015). This is in response to the fact that smart cities face a number of challenges connected to security and privacy. In this particular element of the research, the focus has been on encryption algorithms as a means of incorporating security measures into smart city systems. The research carried out by Antonopoulos et al. (2017) employs the construction of Wireless Sensor Networks (WSN) in order to test high-level security feature algorithms.

The integration of an end-to-end cryptography system with smart city solutions was proposed by Stromire and Potoczny-Jones (2018). This would be done at the foundational level. By employing this approach, not a single piece of information regarding the data would be exposed in the event of a data breach. In a similar vein, Lai et al. (2017) proposed a method that they called Fully Privacy-Preserving and Revocable Identity-Based Broadcast Encryption, and they did so by making use of an encryption technique (FPPRIB). The objective of the plan that was put forward was to protect the confidentiality of the data as well as the identity of the person who had their access terminated. The data can be safeguarded against unauthorised access in such a way that only the person with the appropriate authorization can view it. The revocation procedure does not make any information about the data contents or the identity of the receiver public, and the general public is also kept in the dark regarding both the identity of the receiver and the identity of the user whose access was revoked. These qualities open the door to applications in the smart city, particularly those that prioritise identity privacy. The research conducted by Patsakis et al. (2015) resulted in the development of a cryptographic protocol that manages the vast amounts of personal information that could be generated through e-participation in a manner that is both scalable and interoperable. This ensures that the privacy of citizens living in smart cities is maintained. The steps which are shown in Figure 6 indicates the process helps to ensure the security and privacy of the smart home, and can be modified and improved over time to meet the changing needs of the homeowner. These steps describe how image processing and security work in smart homes using machine learning (Taiwo et al., 2022).

Controlling who can access a computer network is an essential component of any communication system. It is essential to build sufficient security measures for access to IoT systems in order to stop any unauthorised person from seizing control of IoT devices or revealing confidential information that is stored at the object or node level. Beltran et al. (2017) presented SMARTIE, a user-centric integrating platform for secure Internet of Things applications. It ensures scalability and efficiency while simultaneously protecting the privacy of users. The platform that has been presented allows decentralised access control for Internet of Things devices in an effective manner, taking into account the preferences of individual users regarding their privacy. The goal of the SMARTIE project is to make it easier to integrate user-centered privacy and governance into existing systems.