Energy Efficiency Evaluation of Artificial Intelligence Algorithms

1. Introduction

Inspired by environmental behaviour euphoria to green technologies logically converges to green computing where major player is software. According to the literature green computing aspects involve optimising hardware and software design, reducing power consumption, using renewable energy sources, improving software efficiency, virtualizing servers, and managing e-waste [1]. Green computing should enhance the performance and energy effectiveness of computational systems by optimising hardware and software. Recent trends and concerns on environment and society regarding well-being, reasonably targeted Responsible Artificial Intelligence (RAI) [2] which accelerated demand for energy efficient intelligent software. [3]

Although “Artificial Intelligence (AI) has the potential to drive towards a future in which all of humanity flourishes” [2] the energy consumption of Information Technologies (IT) including, portable devices, data centres and cloud servers, has been dramatically increasing every year [4], which is reflected on global carbon emissions as identified in published global energy reviews [5,6].

Issues with computing energy efficiency need more deep analysis. According to the publications in fundamental research “There is a maximum rate at which data processing can proceed. This maximum rate applies to all data pro-cessing systems, manmade as well as biological. This conjecture aims stimulate the discussion towards a tightening of concepts in artificial data processing and biological evolution alike. The conjecture is the following:

No data processing system whether artificial or living can process more than (2 ×10⁴⁷) bits per second per gram of its mass.” [7]

Formulation of these computational limitations is based on fundamental physical principles: “The capacity of any closed information transmission or processing does not exceed mc²/h bits per second, where m is the mass of the system, c the light velocity, h Planck’s constant” [8].

More recent publications stated that: “suggested by H. J. Bremermann in 1962 limit has to be corrected to make it compatible with general relativity. As a result, Bremermann’s limit, proportional to mass M of system, Mc²/h = ~ (M/gram)10⁴⁷ bits per second, should be replaced by an absolute limit (c⁵/Gh)^1/2= ~ 10⁴³ bits per second, where universal constants c, G, and h are the speed of light, the gravitational constant, and Planck’s constant [9].

Presence of Bremermann’s limit alerts that extensive improvement of computational performance will face insurmountable barrier. Initial indications that computers approaching this barrier are: - increasing need for cooling for extraction of the heat produced from computational systems; - need for extensive growth of electricity sources; - changes of the environmental heat pollution, and impact on the climate which seems irreversible.

A natural solution to cope with limitations could be adapting efficiency similarly to the natural systems and considering two aspects - hardware and software energy efficiency. As far as large computational systems are home of most of AI services including popular OpenAI it is essential to mention that according to [10,11,12], based on the LINPAC Benchmark [13] for the last 20 years Supercomputers hardware energy efficiency is improved more than 200 times. This article focuses on software energy efficiency where is difficult to see such categorical progress, and more specifically on Artificial Intelligence and intelligent algorithms, which are mostly dealing with high volume, largely uncertain, and time dependent data.

2. Survey of Related Literature

Hardware energy efficiency has been improved constantly since computers exists [14,15], however software developers exploit this improvement and concerns about the efficiency of software have appeared only more recently, illustrated by growing number of publications [16]. Software sustainability should be applied to software systems at whole, specific software products, online applications, and data processing amongst others, by minimising power consumption, and harmonising software life cycle processes addressing required human, economic, and energy resources [16].

Currently, due to a growing demand, large research efforts are directed towards reducing energy consumption of portable devices such as smartphones, tablets, and laptops, by improving the software quality using code refactoring which supports the restructuring of existing source codes to obtain more energy-efficient code [17,18,19].

Generalising assessment of the software as sustainable including efficiency, quality and other essential properties seems to be an advanced approach [20,21,22]. It is essential to encourage software practitioners considering sustainability during software design and development [20]. Facilitation of the software development process by providing systematic guidelines, technical dimensions of sustainability [22], a framework that allowing professionals to foresee how and how much impact the software has on sustainability and its dimensions [21]. Taking into account specific for Computational Intelligence and Artificial Intelligence properties proposed models and frameworks are considered and adapted within the study presented in this article.

Intelligent systems and Artificial Intelligence as an advanced type of software play a key role in computing, Cloud Services and Data processing, which affects many aspects of the live. According to recent research [23] current approaches to build AI-based systems target mainly accuracy and reliability which require large data volume, large Artificial Intelligence Models and resource consuming infrastructure. The same publications proposed a hypothesis, which states: “When modelling and developing green AI-based systems, the impact of architectural decisions on energy efficiency must be better understood, defined, reported, and managed in order to deliver AI-based systems with less demanding computational power needs” [23]. Based on past experience in developing intelligent software can be argued that intelligent software needs appropriate data abstractions, adoption of heuristic and metaheuristic algorithms and most importantly reduction of outdated limitations in software development [24,25] which could help adaptation of the intelligent software in resolving tasks using minimal computations resources, for minimal period of time in the range from simple tasks [24,26] to problems with high number of parameters [27,28].

Due to the growing number of services and applications, based om Machine Learning and Artificial Intelligence, research publications [23] proposed approaches for sustainable, green Artificial Intelligence based software systems architecture, which aims to “To provide data scientists and software engineers tool-supported, architecture-centric methods for the modelling and development of green AI-based systems” [23]. However proposed architecture centric approach misses entirely core properties of natural intelligent species such as cognition and adaptation, which will be discussed further in this article.

Comprehensive research on design patterns for Machine Learning applications [29] identifies and distinguishes 15 different patterns. For example, pattern 6 solution - “Support both real-time data processing and continuous reprocessing with a single stream processing engine.” and pattern 7 solution – “Store data, which range from structured to unstructured, as “raw” as possible into a data storage.” Although process of analysis of a range of paters and then identification and distinction of key design patterns all of them are limited to analytical programming and misses properties of natural intelligent species such as abstraction and intuition amongst others, which also will be discussed further in this article. Key aspect in evaluation and assessment of Artificial Intelligence systems sustainability and quality is efficiency is measurement and assessing energy efficiency of software products and components. Recently published research on software energy and resource measurements identifies and compares several existing approaches aiming to generalize, extract, and categorize a comprehensive Green Software Measurement Model. The aim is to allow the categorisation of existing measurement methods and the derivation of adapted methods for individual measurement use cases, such as for software types, hardware and software setups, and individual components of software systems. [30]

This model was adapted for empirical research and evaluation of experimental software presented in this article.

An aspect with need consideration is how level of intelligence and its sustainability depends on used language. Published comparative research identifies significant differences in the energy consumption when certain algorithm is different languages using different compilers [31]. This aspect needs more research not only on computer languages but also on human languages.

Serious concerns regarding the proclivities of current Artificial Intelligence, Machine Learning, and Deep Learning are exponential growing demand for data, training, and infrastructure, amongst others [32,33]. All these are in sharp contradiction with emerging regulations and requirements for efficiency, and sustainability [34,35] and to the laws of nature for natural selection [36].

A harmony with the laws of nature would facilitate sustainability of Artificial Intelligence Systems. Primary questions regarding data processing, which need answers for example are: - Which biological system memorises full data for example images in similar to full definition format? Which natural creatures communicate high volume of data similar to transmission of high-definition video? Etc.

Software energy efficiency and performance are in some extent contradictive but increasingly essential non-functional requirements in software engineering particularly when applied to high-performance computing (HPC) systems.

To achieve performance and energy efficiency goals, software developers require a deep understanding of both the problem at hand and the target computer architecture. Moreover, software developers have to consider a multitude of programming models and languages, tools, and heterogeneous architectures and systems, which increases the development complexity [37].

Artificial intelligence applications are amongst the number of software needing increasing high performance and energy efficiency, and only specialized developers master this necessary knowledge. Thus, methodologies and tools to assist both specialized and typical developers are of paramount importance when targeting high-performance computing systems [37]. Proposed approach for time and energy consumption measurement could be invaluable for evaluation of Intelligent Computing and Artificial Intelligence software systems. Although not automated similar approach, for simultaneous measurement of time, and energy consumption of heuristic algorithms is applied in presented in this article experimental results.

In the field of Artificial Intelligence, Computational Intelligence, and software development in general, similar to the behaviour of biological species, it can be often observed how the same task can be resolved for different period of time using different volume of resources. Typical software examples are sorting algorithms [38] and in Computational Intelligence adaptive heuristic algorithms [39].

A good idea for improving intelligent systems is applying Genetic, Swarm, Evolutionary, Heuristic, Metaheuristics, and Adaptive algorithms. A detailed study on improving machine learning classifiers performance optimised by swarm intelligent algorithms compares more than 10 Metaheuristics applied to code smell detection [40]. Achieved results deserve attention. Summarised algorithms hillites significant progression of metaheuristics in recombination and calculation of distances to desired solutions. This summary [40] (pp 23-29) helps to identify easily common limitation of all used Metaheuristics.

According to [41] ”The ultimate goal of Artificial Intelligence is to create technology that allows computational machines to function in a highly intelligent manner.”

Formulation of a clear aim motivates researchers to develop new algorithms with superior performance and large-scaled datasets with high quality. However, it is still infeasible for Artificial Intelligence systems to cover possible potential situations when applied to real world applications. The core question is how to leverage the strengths of these uncertainties guaranteeing socially responsible behaviour of Artificial Intelligence algorithms [42]. In pursuing this goal appropriate interpretation is facing high diversity answering the question: - What is Artificial intelligence? [43], and this could be serious constraint achieving socially responsible behaviour of Artificial Intelligence algorithms. The question which this study rises is: - Can we classify Artificial Intelligence as socially responsible according to its energy efficiency and sustainability? The answer to this question needs comprehensive further research.

2. Materials, Tools and Methods

For the purpose of this study are selected 7 test problems. Criteria for tests selection are:

• must be scalable for multidimensional format.
• must be with heterogeneous landscape.

The chosen numerical tests are scalable and form different search spaces. All tests are transformed for maximisation and scaled to 100 parameters.

• Griewank test is global [43]. Optimal value is 0.
• Michalewicz test is global test with unknown optimum [44]. Optimal value depends on dimensions number.
• Norwegian test is global test with unknown optimum [45]. Optimal value depends on dimensions number.
• Rastrigin test is global [46]. Optimal value is 0.
• Rosenbrock test is smooth flat test with single solution [47]. Optimal value is 0.
• Schwefel test is global [48]. Optimal value is 0.
• Step test introduces plateaus to the topology and the search process cannot rely on local correlation [49]. Optimal value depends on the dimensions number and for variety of dimension is unknown.

For the purpose of this study 3 algorithms are selected:

• Particle Swarm Optimisation (PSO) [50] – representing the group of swarm algorithms applied to real coded tasks over continues space.
• Differential Evolution (DE) [51] - heuristic approach for optimising nonlinear and non-differentiable continuous space functions.
• Free Search (FS) [52] - adaptive heuristic algorithm for search and optimisation within continuous space.

All algorithms are implemented to operate on 10 solutions and limited to 100000 iterations, 320 sequential runs. The aim of the experiment is to measure time and energy required for completion of the defined number of iterations.

For these experiments, a computer system with the following components and settings is used - processor Intel XEON E5 1660 V2 overclocked at 4.750 GHz, running 1 core – 1 thread, Max TDP – 130 W, CPU water cooler, RAM at 2000 MHz, motherboard ASUS P9X79-E WS and solid-state disk - SanDisk Extreme SSD SATA III.

All experiments are completed individually running one single algorithm on a single test function at time.

Methodology

For this study is adopted Green Software Measurement Model proposed in the literature [30], which is adapted according to the nature of the study by control of the following parameters:

• Duration min
• Number of iterations integer
• Mean system power W
• System Energy Wℎ
• CPU usage %
• CPU power W
• CPU cores - 1 core – 1 thread

Application of full Green Software Measurement Model as proposed in [30] could be a subject of further research and availability of necessary resources.

Power is measured for entire system using digital Power Consumption Energy Meter [53]. Power for each algorithm applied to each test is calculating as a difference between the level during the algorithms’ execution and level on standby - running OS components and tools for CPU parameters measurement – CPUZ [54] and CPU cores temperature measurement CoreTemp [55]. SPUZ and CoreTemp are ringing during all experiments in parallels to the algorithms.

Each experiment is limited to 100000 iterations, and the duration is measured by recording start time manually and end time of the experiment is stored as an attribute of the results file. This ensures the time recording has no effects on the performance of the algorithms or the system’s configurations.

3. Results

Experimental results are presented in Table 1, where: Column Test indicates the test function. Column PSO indicates time of experiments achieved by Particle Swarm Optimisation. Column DE indicates time of experiments achieved by Differential Evolution. Column FS indicates time of experiments achieved by Free Search. Time is presented in format hh:mm:ss.

The Mean system power on standby for Task Monitor 0% workload measured on the socket is 166W. CPU power for 1 core – 1 thread measured by CoreTemp is 33.4W and 21.5W.

The Mean system power for Task Monitor 100% workload for all experiments presented in Table 1 is 185W with variation 3% over the time of execution. Reason for this variation may be due to variation in temperature and other environmental factors but, this could be a subject of further research.

CPU power for 1 core – 1 thread measured by CoreTemp is 42.8W and 30.4W.

Presented in Table 1 data indicates variation of time for execution per test and per algorithm.

Analysis of the experimental data suggest that based on the complexity of the search space time for evaluation per test vary. More complex search space needs more time. Based on capabilities of the search algorithms time for search also vary.

Presented on Figure 1 graphic indicates time per test and on Figure 2 per algorithm for completion of 100000 iterations on selected test. Particle Swarm Optimisation needs more time for all tests. Differential Evolution uses medium interval of time over all tests. Free Search completed faster all tests.

Results for Schwefel and Step test on Figure 2 suggest presence of possible specific factors which reflect on the time for exploration.

Within the time of evaluation can be distinguished specific components such as:

• Time of objective function evaluation in other words this is time for apprehension of the space.
• Time for algorithm execution in other words this is time for interpretation and assessment of the search space.
• Time for algorithm decision making in other word this is time for selection and further action.

Energy use presented in Table 2 is based on algorithms energy consumption (difference between measured power for 100% workload during the experiments and 0% workload on standby), multiplied by period of time for completion of the experiment.

Different algorithms can cope with the same task for different periods which lead to different energy use.

Analysis aiming to identify systematic relations between these components summarized of Table 3 identifies general qualitative difference only.

Presented within Table 3 relative difference between time in % in general suggest that DE is faster than PSO on all test, and FS is faster that RSO and DE. The difference highly varies and cannot be identified precise systematic relation per test and per algorithm. Precise quantitative analysis could be subject of further research.

4. Discussion

This section analyses the results, their interpretation from the perspective of previous studies and of the Role of Intelligent Algorithms Energy Efficiency.

Achieved results confirms and in certain extent clarifies published earlier results [27] on intelligent algorithms aimed to study computational limitations, energy consumptions and time. It can be argued that different efficiency is due to software design, implementation and execution which by its own is absolutely correct. However, it should be clarified how different software engineering techniques implement intelligent behaviour.

First little attention will be given to Epistemology [56,57]. Popular models Data-Information Knowledge (DIK) [58] and Data-Information-Knowledge-Wisdom (DIKW) [59] suggest a way in which intelligent beings and systems can cope with apprehension of the surrounding environment is generation and preprocessing data abstracting essential for certain purposes information which can be used and then abstracting from the information knowledge, which could be memorised for further use. This process first reduces the amount of data, which should be stored and second allow much faster processing of known cases and adapting to and processing unknown cases. Both contribute significantly to efficiency and sustainability of intelligent beings. Implementing this process as a software for Intelligent Computing can contribute to the sustainability of Artificial Intelligence Systems.

For amongst the many definitions of knowledge [60,61,62] probably the closest to understanding of software design and implementation is “Knowledge is the perception of the agreement or disagreement of two ideas” [60].

In the cognitive processes understanding the fundamental principles could help significantly to strengthen machine learning efficiency and sustainability of intelligent systems. According to the literature [63] “Knowledge of the external world can be obtained either by intuition or by abstraction”.

Where appropriate understanding of intuition and abstraction could support significant improvement of the process of learning and more precisely formulated in a process of building knowledge. A good interpretation of intuitive and abstractive cognition is formulated by William of Ockham [64].

“Intuitive cognition is defined as an act of apprehension in virtue of which the intellect can evidently judge that the apprehended object exists or does not exist, or that it has or does not have some particular quality or other condition; in short, an intuitive cognition is an act of immediate awareness in virtue of which an evident judgment of contingent fact can be made.

Abstractive cognition is defined as any act of cognition in virtue of which it cannot be evidently known whether the apprehended object exists or does not exist, and in virtue of which an evident contingent judgment cannot be made” [64].

Applying these concepts to the Blackbox model helps the adaptive heuristic algorithm called Free Search to perform fasted amongst heterogenous landscapes and tasks.

Future research could focus on new models and software implementations for machine training, intelligent computing, apprehension of the environment, building actionable knowledge and adaptive behaviour. For simple tasks computational intelligence con-tributes in competition between algorithms and systems. For large complex problems for example where the search space exceeds the range of 101000000 locations, and the time for exploration for space with this size could tend to infinite, intelligent, adaptive behaviour is essential for reducing the time and energy use of the process of search, and a good example is the result available in [28] achieved by Free Search applied to optimisation of a task with 100 000 parameters.

5. Conclusions

In summary this research completed an empirical evaluation and comparison of intelligent and adaptive algorithms time and energy consumption applied to heterogeneous numerical tests. It identified notable difference in energy efficiency and speed when different algorithms are applied to same tasks.

Experimental results illustrate strengths and limitations of used algorithms. Discussion on concepts and their relationship with computational intelligence and possible value for sustaining intelligent systems concludes the article.