Artificial Neural Networks Applied to Power Management in Low Power Microprocessors

Computer systems that operate in remote locations such as satellites, remote weather stations and autonomous robots are highly limited in the availability of energy for their operation. This work aims to employ artificial intelligence algorithms in energy management in order to obtain the maximum energy yield and the prediction of energy availability to the system. The work presents the main types of algorithms used in artificial intelligence and presents the creation of a prototype that will operate as a low power system powered by batteries and a small solar plate, the prototype will perform inferences on the LSTM neural network algorithm in order to predict the future availability of energy, consequently the management system will carry out the energy distribution in order to obtain the maximum operation of the prototype without total discharge of the batteries. So that the artificial intelligence system could be embedded in the prototype, the TensorFlow Lite framework was used, which allows the inference to be carried out in devices with low consumption and limited processing power.


Introduction
Modern electrical systems, such as electric vehicles, mobile robots, nano satellites and drones, require multiple power demand operations for user applications and system operation . This, in turn, requires advanced energy management, which jointly considers the energy demand of operations and the supply of energy from multiple sources such as batteries, solar panels and supercapacitors. However, a complete solution for the energy scheduling problem also need to characterize the storage and energy sources , because they affect the energy capacity to the demands of operations and functioning of the system. (Kim et al., 2020) Proper energy management is becoming one of the main factors in device designs that are powered by batteries or with limited energy resources that use renewable energy as an energy source, such as solar and wind energy.
Renewable energy technology is associated with energy estimation schemes for astute energy management. Thus, there is a need to undertake inept energy saving mechanisms in addition to renewable energy technology in order to achieve high reliability status. Sensors can incorporate dynamic behavior trends in the face of estimated energy not being able to sustain them in the next recharge cycle. Therefore, they can optimize critical parameters such as sampling rate, transmission power and duty cycle to adapt their energy consumption according to the frequency and magnitude of the energy source. (ALSHARIF; KIM; KURUOĞLU, 2019) Due to meteorological factors, such as weather conditions, irradiance, ambient temperature and wind speed, as well as non-meteorological factors, such as temperature and place of installation of components and parts, the output power of the photovoltaic energy generation system is strongly intermittent , volatility and uncertainty, in addition to low forecast accuracy of photovoltaic energy generation. However, based on historical energy generation data and real meteorological data from the photovoltaic system, the short-term prediction of photovoltaic energy generation can be performed using the neural network prediction model. (KAIJU et al., 2018) With the advancement of Internet of Things (IoT) technologies, the demand for autonomously powered systems is increasing. Battery autonomy directly influences the operating time of sensor networks, weather stations, and autonomous robots, for example.
Many applications that are typically remote and difficult to access, such as nano satellites, use renewable means as a way to obtain electricity. In the application of nano satellites, the energy management demands and challenges are even greater. In addition to a highly efficient power distribution scheme, onboard power management schemes play a significant role in ensuring that the satellite does not fail due to a power outage. The main source of energy for most small satellites is the sun. Non-implantable solar cells are used to harness this energy. The size, surface area and mass limitations of a nano satellite limit its power generation capacity (THAKURTA et al., 2019) A work published in 2019 by LI; XIAO; FAN demonstrating the efficiency of battery charge prediction using artificial intelligence is one of the few works found in this line of research, but it does not deal with the processing of artificial intelligence embedded in low power devices .
In low power systems, two energy saving schemes are proposed. One is to enforce sleep mode programming on a sensor node, switching between active mode and hibernate mode. The second is the duration of sleep mode that can be chosen based on environmental conditions. Considering an intelligent space where several sensor nodes collaborate in detection work, if one of the sensor nodes detects possible anomalous events, all sensors surrounding the nodes will shorten their sleep period to be aware of the current event. (IIKUBO; LO, 2019)

General Purpose
The objective of the project is to build an embedded system in a printed circuit board (PCB) capable of performing energy management using artificial intelligence such as LSTM neural networks with the ability to predict energy availability and perform measurement, communication and activation operations. loads upon this forecast. The system will have the purpose of proving the energy efficiency obtained by a prediction system based on artificial intelligence that manages energy consumption based on a photovoltaic solar panel, thus proving ways to obtain the maximum use of energy, without it occurring supply failures for the correct operation of the system. In table 1 it is possible to visualize the specific objectives of the project. To manufacture a printed circuit board for a low-power embedded system capable of processing artificial intelligence algorithms.

2
Develop firmware for the system processor in order to collect data from sensors and send the information over a wireless network.

3
Train and optimize LSTM-type recurrent neural networks for use in embedded systems using the Tensor-Flow Lite library. 4 Develop and test the energy management scheduler based on electricity supply forecasts.

Materials
At this stage, the electrical and electronic devices that were used to build the prototype were selected. This set of devices was used to survey the database used later for training the recurrent neural network.
In figure 1 it is possible to view the photos of the components used to carry out the survey of the database: We chose to use the ESP-32 for data acquisition for 3 main reasons , low cost , high commercial availability in the Brazilian market and easy programming with 2 frameworks (ESP-IDF and Arduino).
To store the energy from the photovoltaic board and allow the system to operate during nighttime or cloudy periods, two standard 18650 3.7v and 2600mah Samsung batteries were used in series, both batteries were fully charged when they were put into operation. To protect the batteries, a BMS (Battery Management System) board model: FDC-2S-2 was used, this circuit offers protection against overloads in the battery pack, extending its useful life in the system, the charging voltage allowed by the BMS circuit in the batteries are between 8.4V to 9V and the maximum discharge current allowed by the circuit is 3 amps.
The photovoltaic plate used to supply energy to the system has dimensions of 25cm x 14cm x 1.7cm and can supply up to 3.5 watts of peak, in table 4  In figure 3 it is possible to visualize the circuit board without the components. The Raspberry Pi Zero W plate was also used to execute the trained model. The BCM2835 processor was used in the project due to the following factors: • Low cost; • Vast amount of RAM memory; • Low energy consumption; • Ease of programming in several languages; • High speed processor operating at 1Ghz.
• Full compatibility with Tensor Flow Lite models.
• Various communication ports such as Bluetooth, Wi-fi, SPI and I2C protocols.
• Has multiple programmable I/O pins.

Methods
In this chapter, the methodologies used for the development of this work are presented, thus the classification of the research and the constituent steps of its elaboration will be presented.

Research Process
The work is divided into four stages understood as follows: Table 2. Project Steps.

Stage
Description 1 Presentation of the devices used in the survey of the database. 2 Design and manufacture a printed circuit board. 3 Develop the firmware for the system processor to carry out the database survey. 4 Train and optimize recurrent neural networks. 5 Develop and test power management scheduler with a BCM2835 microprocessor. In figure 4 it is possible to visualize the position of the mini photovoltaic board installed next to a set of photovoltaic boards that serve a residence.
The photovoltaic plate was installed on the roof of a residence located in the Parque Guarus neighborhood, in the city of Campos dos Goytacazes-RJ, Brazil, which has Latitude -21,725011 and Longitude -41.324273 as shown in figure 11. It is mounted at a height of 3 meters next to 10 photovoltaic panels that power a residence and with approximately 30 degrees of inclination in relation to the ground.
An overview of the proposed system can be seen in Figure 5.
Initially, the electronic circuit of the prototype was designed and the simulation was carried out in software to verify the correct functioning of the circuit, then the printed circuit board was designed. Figure 6 shows the final electronic circuit of the project.  In all, 3 programming codes were developed for the correct functioning of the prototype, as follows: 1-Code in C language for database acquisition and survey.
2-Code in Python language for pre-processing, training and testing of the LSTM neural network.
3-Code in Python to make the inferences in the trained model in an embedded device with low power consumption.
To carry out the forecasts, the project used the LSTM-type recurrent neural network model, this choice was made for the following reasons: • The LSTM model operates efficiently through multiple input or output variables.
• In non-linear data patterns the LSTM model offers more precision.
• High accuracy even when trained for a small number of samples.
• LSTM models have the ability to predict both short-term and long-term variations.
Multivariate time series prediction has been widely studied in energy, aerology, meteorology, finance, transportation, etc. Traditional modeling methods have complex patterns and are inefficient at capturing multivariate dependencies of long-term data for prediction with the desired accuracy. To address these concerns, there are several deep learning models based on Recurrent Neural Network Methods (RNN) and Convolutional Neural Network (CNN).(WAN et al., 2019) ISTIAKE published in 2020 an article comparing the accuracy of the ARIMA method and the LSTM model, as representative techniques in predicting data in time series. These two techniques were implemented and applied in a set of financial data and the results indicated by the LSTM model were superior to ARIMA. More specifically, the LSTMbased algorithm improved the prediction by 85% on average compared to ARIMA.
A single time dependent variable means a univariate time series, while a multivariate time series like environmental data has more than one time dependent variable, each variable depends on its previous values and other variables as well. The best solutions to modeling problems for multiple input variables are with recurrent neural networks and are the great solution to many time series prediction problems, where classical linear methods cannot offer high precision. (ALHIRMIZY; QADER, 2019) According to Warden and Situnayake, the workflow to create and use an artificial intelligence model follows these steps: For efficient use of neural networks in mobile devices, the Tensor Flow Lite framework was used, which allows the creation of an optimized model to run on mobile devices such as smartphones and microcontrollers that have limited memory and processing characteristics compared to other computer systems.
To meet these smaller size requirements for mobile platforms, in 2017 Google started a complementary project for the TensorFlow mainline, called TensorFlow Lite. This library is intended to run neural network models efficiently and easily on mobile devices. To reduce the size and complexity of the framework, it eliminates features that are less complex on these platforms. For example, it doesn't support training, just running inferences on models that were previously trained on a cloud platform. It also does not support the full range of data types (such as double) available on the Mainline Tensor. (WARDEN; SITUNAYAKE, 2020) The first firmware of the project was developed for the ESP32 module in order to collect data from voltage and current sensors, and temperature and humidity. Measurements were taken every one minute periods and recorded on the memory card included on the circuit board.
All source code was developed in C language and has the following tasks: • Read and filter analog signals from current and voltage sensors.
• Receive the signal from the relative humidity and temperature sensor.
• Log data to memory card.
The environment used to create the code was Visual Studio Code, the framework used was the Arduino framework. The Arduino framework was chosen because it already has the necessary libraries to carry out I2C and SPI communications, reducing the development time of the data collection firmware.
The data was stored on a 256MB KINGMAX manufacturer's Micro SD card in text format with a .txt extension. In figure 20 it is possible to check the format of the data saved in the memory card.
As the objective of the project is energy management through the generation forecast of the photovoltaic solar panel, the predictive attributes used for training were: • Voltage and current generated by the solar energy board; • Temperature and relative humidity of the air; As energy generation through the sun depends on environmental characteristics, the temperature and humidity sensor will provide data that will contribute to the highest accuracy in energy prediction available on the solar power board. A larger number of input data can be used for model training, such as: • Wind direction and speed.
• Time of year and season; However, the objective of the project is not to exhaust the ways of measuring environmental data, but to prove the efficiency of the LSTM model in energy prediction. .
The power management software was developed in python language on the BCM2835 processor of the Raspberry Pi Zero W board.
The software has the following functions: • Read the INA219A, INA219B and DHT11 sensors.
• Record measurements from the last 36 hours.
• Make the inference in the Tensorflow Flow Lite model.
• Calculate the expected power to drive the load.
• Supervise battery voltage level.
• Supervise the validity of the model's predictions.
To simulate a load, a 12V 5Watts lamp was added to the circuit, the purpose of the software is to allow the lamp to be activated for as long a period of time as possible based on the energy predictions of the LSTM model. For greater safety in operation, the energy management software checks whether the last two predictions made by the LSTM model have at least 65% assertiveness, thus preventing prediction errors from compromising the battery charge level during operation. In addition, the software monitors the voltage of the batteries and suspends all power supply activities for the lamp if the minimum voltage of 7.0Volts is not present in the batteries.
The objective of the algorithm will be to transfer the maximum charge from the solar plates to the battery pack, and allow the prototype to operate in full during the day and partial during the night. In figure 7 it is possible to visualize the block diagram of the inference operation with the predictive attributes.
The LSTM network works with short-term and logo data, so the inference must be performed by providing the model with a number with values prior to the moment of inference. In this project, values from 3 reference periods were used, as each period has 12 hours the total was 36 hours, that is, for the model to return the power value of the next 12 hours, it is necessary to provide the values of the last 36 hours of the predictive attributes. For the purpose of validating the energy management code, a system test was carried out during a period of 5 days.
Minimum battery voltage tests were also carried out, where it was possible to verify that the system is still functional when the battery has a minimum voltage of 6.25 Volts, thus it is still possible to obtain a small safety margin in the operation of the system in case any error occurs in the board operation, since the system enters standby mode when the voltage drops below 7.0Volts. Before starting the tests, the batteries were fully charged, showing a voltage of 8.45Volts without charge and decreasing to 8.23Volts when connected to the printed circuit board and with the lamp in the off state.
For the execution of the trained and converted LSTM model, the BCM2835 processor of the Raspberry Pi Zero W board was used with its HDMI video output disabled. This configuration allows the reduction of the consumption current of the board to a current of 90 milliamps. It is still possible to decrease the card's base clock in order to obtain a greater reduction in current consumption, but this was not done because the card's consumption was already compatible with the power supply system.
To test and validate the circuit board, it underwent individual tests on each of its individual circuits such as power supply, load drive, sensors and micro SD card recording.

Results
The test results and data found in the survey are shown below. The training data were recorded in the memory card in the following sequence: Day/Month/Year Hour:Minute:Second, Temperature, Humidity, Solar Voltage, Solar Current, Battery Voltage, Battery Current. As shown in figure 8.  In the next graph it is possible to visualize the current (I_B) in milliamperes and the voltage (V_B) in volts in the batteries.   Figure 12 shows the box with the components during the system assembly process, as well as the circuit board with the ESP-32 used to collect training data. Figure 12. Photo of the assembly of the assembly and DHT11 sensor located outside the protection box. Figure 12 still shows the temperature and humidity sensor located on the outside of the box.
The model was created based on a two-layer neural network with each layer containing 700 LSTM units. Figure 13 shows the model used in greater detail. As shown in Figure 13, the total parameters of the model were 5,894,701. For greater precision of the trained model, two layers of the Dropout type were added, which allow the model not to be limited to the variables used during training, and to be able to assess with greater precision different values from the real world.
In order to obtain the best possible model in terms of precision and final model size, several tests were performed. Table 3 shows the different configurations used in the tests and their results. As shown in table 6, the increase in LSTM layers did not provide an increase in model accuracy. After the training period, it was possible to verify the LOSS loss curves of the trained model, Figure 14 shows this graph where it is possible to verify that around 50 training periods the model did not obtain major changes in LOSS losses. After training the model, the analysis was performed with the test database, the result can be seen in Figure 15. The training of the model must be as optimized as possible, because when inferences are made with the converted model TensorFlow Lite, small occurrences occur losses due to optimizations carried out by the converter in the model in order to optimize its final size. To make better use of forecast data and reduce the amount of inferences, the forecast period was defined as 12 hours, that is, the model will predict the approximate electrical power value for the next 12 hours after the inference is made. As the inference operation consumes processor power, the adoption of this period made it possible to reduce consumption, as compared to inferences of a shorter period such as minutes or hours, the consumption of energy and processing would be higher.
As the period to be forecast is 12 hours, the information of the predictor attributes must also be in the same period of time, so it was necessary to perform the calculation of the average of each predictor attribute in periods of 12 hours. Here are the predictive attributes used in model training: • Voltage of the solar power board; • Current from the solar power board; • Air temperature; • Relative air humidity; These data were taken from the memory card, where the information was recorded in periods of one minute for 30 days, thus totaling 35277 records. After adapting the database in 12-hour periods, a total of 48 records were obtained. During the tests, it was possible to verify that the amount of 50 records was enough for training the LSTM network, where 40 records were used for model training and the other 10 were used for testing.
By predicting the energy supplied by the photovoltaic board, it was possible to manage the battery charge in order to provide the maximum energy for the board's loads. Battery charge status was relevant in determining the power available for nighttime periods where the solar power board is inactive.

Discussion
During the operational test the board needed 36 hours to produce the forecast data, after this period it was already possible to make the energy predictions. Table 7 presents the data obtained in the test, as well as the error referring to each prediction. The results of the field tests are close to the tests performed on the model, but with a slight difference due to the losses in the model converted to TensorFlow Lite and the very nature of data variation, which makes it impossible to assert 100% of the forecasts. Anyway, the results were solid and prove the efficiency of LSTM neural networks in forecasting temporal data with several variables.
The load autonomy test was also carried out and it was verified that the battery has the capacity to power the system and keep the lamp on for 4 hours and 4 minutes, the lamp powered by a voltage between 7 and 8 Volts consumes an average of 245 milliamps (figure 31) and the board with all its components consumes about 130 milliamperes, that is, the total current in full operation of the lamp is 375 milliamperes.

Conclusions
According to the information presented, it is possible to conclude that, given the need for continuous data acquisition and processing over time, the best artificial intelligence algorithms that can be applied for data prediction are those with temporal processing capacity used in recurrent neural networks, that can provide resources for the prediction of energy generation in low-power systems and allow greater use of the energy generated. It is still possible to classify them as able or not to supply energy to a load. Due to the multivariable nature of applications involving power supply, LSTM-type recurrent neural networks are the most suitable for forecasting temporal data.
After finishing the experiment, it was possible to conclude that the use of LSTM-type recurrent neural networks is efficient and viable in scientific applications and even commercial applications due to the low cost of the components necessary for the operation of the system or even in the integration of existing systems.
The size of the database used for training is a factor of great relevance, being one of the main points that affect the accuracy of the system, although in this work a relatively small database was used, this was already enough to obtain positive results , especially when the model is optimized through the number of layers and LSTM cells.
For the near future, the functionality of the system will be evaluated in places other than the place where the model training data was acquired. It will still be possible to perform new data acquisitions in order to increase the number of training data.
The possibility of making inferences in a low-power embedded device opens up a wide field of applications in engineering, as it allows the use of artificial intelligence models that can make inferences without the need for high-cost, energy-consuming computers like the ones found. on servers, desktops and notebooks.