Remaining Useful Life Prediction for Lithium-Ion Batteries Based on Hybrid Ensembles Allied with Data-Driven Approach

1. Introduction

The manufacturing techniques of lithium-ion batteries are rapidly developing globally nowadays, since global corporations are actively positioning themselves in battery technology sector and scholars are attracted to engage in research for the improvement of battery technology. Meanwhile, various green energy policies and reducing greenhouse gas emissions plans are introduced by governments around the world for boosting the development of battery and battery relative industries [1]. As a result, the battery industry can be foreseen to become even more prosperous in the near future. However, there is a significant shortcoming of lithium-ion batteries is that the performance declines gradually over time with usage [2]. This degradation problem is affected by many different factors because of the complexity of lithium-ion batteries systems. Therefore, there is a long way to go for tackling this problem. On another hand, the battery operational data, such as voltage, current, temperature, capacity, and energy, are available during its lifespan. Further, the relation between battery performance features and degradation to a certain extent has been proven by scholars [3]. There is feasibility of predicting the battery remaining useful life influenced by the historical performance features. This estimation is called as remaining useful life (RUL) estimation.

Accurate RUL estimation is crucial aspects of prognostics and health management since the estimation result can provide reference for improving maintenance strategies, optimizing life-cycle costs, and mitigating operational risks or issues [4]. Model-based methods, data-driven approaches and hybrid approaches are the mainstream methods for RUL estimation at present [5]. The model-based method can be further divided into mechanism-based method and empirical-based method. These methods are highly depending on the equipment and requires for long-time experimental exercisers, which is difficult to apply to real-world applications [6]. With the development of data minding technology, data-driven approach is able to constructing aging prediction models by historical operational data. But this approach requires large amount of data for modeling. Its adaptability and reliability are uncertain as well [7]. Due to the rapid evolution of machine learning technology, the hybrid approaches combining optimization and machine learning techniques have gained popularity [8]. Having the advantages of both methodologies, a powerful framework with the ability to resolve complicated problems is built. That leads to the more accurate and efficient batteries RUL estimation modules while applying this kind of approaches.

There are remaining numerous technical and practical obstacles for predicting the state of charge (SOC), state of health (SOH), and remaining useful life (RUL) of batteries, machine learning methods have great advantage in improving the accuracy and efficiency [9]. K-Nearest Neighbor regression model is exploited for remaining useful life estimation by incorporating data from all the cells in a battery pack [10]. Gradient Boosting machine learning approach is suitable for real world applications and handling nonlinear input features [11,12]. An optimized Random Forest Regression model is developed to enhance the learning and generalization ability. This model can achieve high accuracy in a short time using a small number of samples [13].

Recently, in order to lessen the error rate, gain better robustness and achieve better prediction performance, the ensemble learning method is frequently used for battery RUL estimation. A novel ensemble learning method [14] consists of 5 basic learners, including Relevance Vector Machine (RVM), Random Forest (RF), Elastic Net (EN), Auto Regressive (AR), and Long Short-Term Memory (LSTM), is developed. The experiment validation shows that this approach has improvement in robustness and generalization effect. Another ensemble model [15] based on CEEMDAN and CNN-BiGRU is designed to achieve higher precision and reliability.

Remaining useful life (RUL), which is typically random and unknown, is the number of charge-discharge cycles left on a battery at a particular time of operation until its maximum usable capacity decays to a predefined failure threshold. A hybrid deep learning model [16] for early prediction of battery RUL combining handcrafted features with domain knowledge and latent features learned by deep networks is proposed. This model enhances model generalization ability without increasing any additional training cost and has great contribution to the improvement of model prediction accuracy and generalization ability.The main objective of this proposed approach is to predict the batteries RUL from features, such as voltage, current, internal resistance, and capacity. This proposed approach jointly combines Gradient Boosting, Random Forest and K-Nearest Neighbor into voting ensembles, so as to integrate the advantages of ensemble learning.

The battery capacity degradation processes can be divided into two period. Initially, the battery is health and the capacity is stable with a steady decline. When the capacity degrades to a certain rate and then drops to a certain point, the capacity goes through an accelerated degradation. That certain point is normally called knee point [17]. Currently, most of previous works are focusing on predicting the SOC, SOH or RUL for battery health determination. But for knee point, there is seldom researches discussed on it. Although the existing works have enhanced the model to be low error rate, high accurate and efficient, these models are normally evaluated within a single battery dataset, where they are divided into training and evaluation measurement segments. The experiments of applying the model to multiple batteries has not been considered. Therefore, the experiments taken for verification are not sufficient enough.

In this paper, to overcome the above existing issues, an innovated hybrid ensembles approach is proposed for battery RUL prediction and knee point estimation, as shown in Figure 1. Regression methods of Gradient Boosting, Random Forest and K-Nearest Neighbors are integrated into voting ensembles is designed to enhance the generalization performance and improve the accuracy. After generating the prediction result, the binomial fitting algorithm is employed to predict the RUL knee point. With extensive experiments, the proposed approach has demonstrated significant advantages and potential in predicting battery RUL and knee point, contributing to improving the development and application of battery RUL prediction technology.

2. Methods

Three models integrated into voting ensembles are developed for predicting based on regression methods. These three regression methods are Gradient Boosting, Random Forest and K-Nearest Neighbors.

2.1. Gradient Boosting

Gradient Boosting [18] is a popular machine learning technique used for regression tasks. It is an ensemble learning method that combines the predictions of several base estimators sequentially, with each subsequent estimator correcting the errors made by its predecessor. Gradient Boosting algorithmic is as follows.

Let dataset ${\left\{\left(x_i,y_i\right)\right\}}_{i=1,\dots ,n}$. Initialize model with constant value ${\widehat{f}}_{(x)}={\widehat{f}}_0$, ${\widehat{f}}_0=\gamma$, $\gamma \in R$,

$${\widehat{f}}_0={argmin}_{\gamma }{\sum }_{i=1}^{n}L(y_i,\gamma )$$

(1)

Calculate residuals, for each $t=1,\dots ,M$, repeat. Compute pseudo-residuals, for $i=1,\dots ,n$,

$$r_{it}=-{\left[{\displaystyle \frac{\partial L(y_i,f(x_i))}{\partial f(x_i)}}\right]}_{f\left(\mathrm{x}\right)=\widehat{f}(x)}$$

(2)

Build new base algorithm $h_t\left(x\right)$as regression on pseudo-residuals ${\left\{\left(x_i,y_i\right)\right\}}_{i=1,\dots ,n}$. Find optimal coefficient ${\mathsf{\rho }}_t$at $h_t\left(x\right)$regarding initial loss function.

$${\mathsf{\rho }}_t={argmin}_{\mathsf{\rho }}{\sum }_{i=1}^{n}L(y_i,\widehat{f}\left(x_i\right)+\mathsf{\rho }h\left(x_i,\theta \right))$$

(3)

$${\widehat{f}}_t(x)={\mathsf{\rho }}_t{h}_t\left(x\right)$$

(4)

Update current approximation and compose final model,

$$\widehat{f}\left(x\right)\leftarrow \widehat{f}\left(x\right)+{\widehat{f}}_t\left(x\right)={\sum }_{i=1}^t{\widehat{f}}_i\left(x\right)$$

(5)

$$\widehat{f}\left(x\right)={\sum }_{i=0}^M{\widehat{f}}_i\left(x\right)$$

(6)

2.2. Random Forest

Random Forest [19,20] is a powerful ensemble learning method that operates by constructing a multitude of decision trees during training and outputs the class that is the mode of the classes or mean prediction of the individual trees. This method is widely used across various domains due to its flexibility, robustness, and high performance. Particularly, it is favored when accuracy and interpretability of results are both important considerations. The algorithm for Random Forest construction is as follows.

For each $k=1,\dots ,N$, generate a bootstrap sample $X_k$. Build a decision tree $b_k$on the sample $X_k.$ Pick the best feature according to the given criteria. Split the sample by this feature to create a new tree level. Repeat this procedure until the sample is exhausted. Building the tree until any of its leaves contains no more than n min instances or until a certain depth is reached. For each split, we first randomly pick m features from the $d$original ones and then search for the next best split only among the subset. The final model is defined as,

$$a\left(x\right)={\displaystyle \frac{1}{N}}{\sum }_{k=1}^N{b}_k(x)$$

(7)

2.3. K-Nearest Neighbor

K-Nearest Neighbor [21] is a fundamental algorithm in machine learning, providing a baseline for many regression problems. Its simplicity and effectiveness make it a valuable tool in various domains. K-Nearest Neighbor helps identify the nearest points. There are some metrics to determine the nearest points. Minkowski distance is a metric in a normed vector space. Mainly, Minkowski distance is applied in machine learning to find out distance similarity. The formula is indicated as follows,

$$L_p(x_i,x_j)={\left({\sum }_i^n{\left|x_i^{(l)}-x_j^{(l)}\right|}^p\right)}^{{\displaystyle \frac{1}{p}}}$$

(8)

For the above formula, when $p=2$ then it is the same as the formula for the Euclidean distance, and when $p=1$ then we obtain the formula for the Manhattan distance.

2.4. Voting Ensembles

Voting ensembles are generally used in classification or regression problems to improve predictive performance. In addition, voting methods are the appropriate integrating method for bagging and boosting methods [22]. In this paper, method of averaging voting is employed for the proposed approach. The idea of averaging voting is that predictions are extracted from multiple models, and an average of the predictions is used to make the final prediction. Average prediction is calculated using the arithmetic mean, which is the sum of the predictions divided by the total predictions made as shown in the following formula.

$$y^{\mathrm{*}}={argmax}_i{\displaystyle \frac{1}{n}}{\sum }_{j=1}^m{w}_{ij}$$

(9)

where $w_{ij}$ is the probability of the $i^{th}$class label of the $j^{th}$ classifier [23].

3. Experiments

3.1. Experiment and Dataset

In this paper, the voting ensembles integrated with Gradient Boosting, Random Forest, and K-Nearest Neighbor. All these algorithms are implemented by Scikit-learn. The battery test dataset of CS2 is provided by CALCE (Centre for Advanced Life Cycle Engineering) battery team [24,25,26,27]. The CALCE CS2 battery follows a standard constant current/constant voltage protocol and its capacity rating is 1.1 Ah. Initially, this battery is charged with a current of 0.5C until the voltage reaches 4.2V. The voltage sustains until the current drops to 0.05A. Under normal circumstances, the battery discharges at a rate of 1C until the voltage reaches 2.7V. Batteries named CS2_35, CS2_36, CS2_37, and CS2_38 are selected to verify the proposed approach from the dataset.

As can be seen in Figure 2, the feature of voltage has the greatest correlation coefficient (0.94) with capacity among others. This indicates that the change of capacity relates to voltage change trend closely. Take CS2_35 for example (Figure 3), the nominal capacity is 1.1 Ah, the failure threshold is 0.88 Ah. Initially, the capacity drops steadily. Once the capacity reaches the line of failure threshold, the capacity become stable for a few cycles, but diminishes dramatically subsequently. The voltage has the same trend as capacity as well. To explore this phenomenon, due to the loss of charge carriers or active materials, the capacity fade. Further, since the impedance keeps increasing while rising of charge-discharge cycles, the voltage drops [28].

With time and use, the battery storage capacity diminishes and the internal resistance increases due to a wide range of degradation mechanisms, some occurring simultaneously, or triggering further mechanisms [29]. Electrode material aging leads to a decrease in capacity and/or a rise in resistance of the whole cell and thus can dramatically affect the performance of lithium-ion batteries. Furthermore, the aging phenomena are extremely complicated to describe due to the coupling of various factors [30]. Figure 4 evidences this phenomenon quantitatively. As the result of impedance increase, the battery internal resistance keeps rising. The internal resistance rises smoothly, but after certain cycles (failure threshold), the increasing trend accelerates. The trend for voltage is converse. This leads to the decrease of battery capacity as describe previously.

It is worth mentioning that the feature of current has the low correlation coefficient (0.05) with capacity. The reason is that experimental test for battery CS2 was conducted with a constant current of 1C. Hence, the capacity can be considered as not affected by current due to its stable status on data-driven perspective. Nevertheless, current is also one of factors for battery capacity prediction. Although some usage patterns and operating conditions lead to rapid degradation by one or more processes and the interplay between mechanisms is still not well understood [31], the features of voltage and current are still having connection with capacity. There is feasibility to apply these features for prediction.

3.2. Data Standardization

For this study, data for battery numbered CS2_35 are selected for training, and data for batteries CS2_36, CS2_37 and CS2_38 are applied for prediction. In order to streamline data storage, reduce input error, and ensure consistency, data standardization is crucial for the standard preprocessing step. Z-score technique based on the mean and standard deviation of the data is adopted. For this approach, each value is replaced by a score that indicates the standard deviations from the mean. The following formula is defined as follows.

$$x_{norm}={\displaystyle \frac{x-\mathrm{m}\mathrm{i}\mathrm{n}(x)}{\max \left(x\right)-\mathrm{m}\mathrm{i}\mathrm{n}(x)}}$$

(10)

where $x_{norm}$is the new value of $x$ , $\mathrm{m}\mathrm{a}\mathrm{x}(x)$and $\mathrm{m}\mathrm{i}\mathrm{n}(x)$define the maximum and minimum of $x$.

3.3. Estimation Indicator and Process

Currently, there are two key indicators are used normally for battery health prediction, namely, Remaining Useful Life (RUL) and State of Health (SOH) [32]. The RUL of lithium-ion batteries represents the remaining number of battery cycles, including charging and discharging, before the actual capacity drops to the baseline and reaches the end of life (EOL) [33]. The formula for RUL is defined as follows,

$$RUL=N_{eol}-N_t$$

(11)

where $N_{eol}$ denotes the number of battery cycles while reaching the end of life, and $N_t$ represents the total number of charge-discharge cycles occurred.

SOH is the key health indicator for battery aging, and indicates the percentage of current charge-discharge capacity over the nominal capacity [34,35]. The SOH formula [36] is defined as follows,

$${SOH}_t={\displaystyle \frac{C_t}{C_n}}\times 100\%$$

(12)

where $C_t$ denotes the capacity at the $t^{th}$ cycle, and $C_n$ denotes the nominal capacity when the battery is fresh.

While reviewing these two indicators, there is a key point to be noticed that the factor of capacity plays the crucial role for the both calculations. Besides, the RUL is also considered as ended once the actual capacity drops to 80% or below of the initial capacity [37]. Overall, to simplify the estimation process, the battery ages and deteriorates, the battery capacity gradually decreases. When the capacity drops to 80%, the battery has deteriorated to the end of its lifespan. The overall workflow of the estimation process is shown in Figure 5. The ensemble model is integrating three different state-of-the-art machine learning regression models. Data of battery numbered CS2_35 are selected as training data, data of battery CS2_36, CS2_37 and CS2_38 are selected as test data. Both training data and test data are normalized, and the most relevant features are selected from the high dimensional. All three models are experienced first unit training with training data. All the three models are combined to develop the proposed voting ensembles model. The voting ensembles model is evaluated by calculating the performance measures. With the assistance of binomial fitting, the final results, including RUL, knee point, are computed.

3.4. Evaluation Metrics

For the purpose of examining and verifying the results of the improved method, there are four evaluation metrics to be employed, namely mean absolute error, root mean square error, relative error and coefficient of determination. Mean Absolute Error (MAE) is a widely used measure of evaluating the accuracy of a forecasting model. The best possible score is 0.0, smaller value is better. MAE is calculated as the average of the absolute differences between the predicted values and the actual values [38].

$$MAE\left(y,\widehat{y}\right)={\displaystyle \frac{{\sum }_{i=0}^{N-1}\left|y_i-{\widehat{y}}_i\right|}{N}}$$

(13)

where $y_i$ denotes the observed value for the $i^{th}$ observation, ${\widehat{y}}_i$ denotes the predicted value for the $i^{th}$ observation, and $N$ denotes the total sample size.

Root Mean Square Error (RMSE) is widely used to evaluate the performance of forecasting models in various fields. The best possible score is 0.0, smaller value is better. A lower RMSE indicates better forecast accuracy. RMSE is calculated as the square root of the average of the squared differences between the predicted values and the actual values [39].

$$RMSE\left(y,\widehat{y}\right)=\sqrt{{\displaystyle \frac{1}{N}}{\sum }_{i=1}^N{(y_i-{\widehat{y}}_i)}^2}$$

(14)

Relative Error (RE) is used for the evaluation of regression model accuracy. The best possible score is 0.0, smaller value is better. RE is the ratio of the absolute error to the actual value.

$$RE\left(y,\widehat{y}\right)={\displaystyle \frac{\left|y_i-{\widehat{y}}_i\right|}{\left|y_i\right|}}$$

(15)

Coefficient of Determination ($R^2$) is a measure of how well unseen samples are likely to be predicted by the model, through the proportion of explained variance. Best possible score is 1.0, bigger value is better. $R^2$ represents the proportion of variance that has been explained by the independent variables in the model [40].

$$R^2\left(y,\widehat{y}\right)=1-{\displaystyle \frac{{\sum }_{i=1}^N{(y_i-{\widehat{y}}_i)}^2}{{\sum }_{i=1}^N{(y_i-\overline{y})}^2}}$$

(16)

4. Result and Discussion

4.1. Prediction for Battery with Short Voltage Decline Period

Figure 6 describes the voltage change trend for battery CS2_36. The voltage is stable at about 3.64 before the first 200 charge-discharge cycles. Although the voltage drops steadily during cycles between 200 and 420, there is a sudden drop after that. Subsequently, when the cycles go beyond 700, the voltage drops dramatically.

The final prediction result for battery CS2_36 is indicated in Figure 7. The period of RUL is defined as from cycles between 400 and number of capacities reaching 80% of initial capacity. Before RUL period, the prediction curve accompanies the real data curve by a steady decline. When it goes into the RUL period, the prediction curve become fluctuated, as it is affected by the sudden drop of the voltage. Since the actual capacity become steady after RUL and lasts for about 100 cycles, following by the abrupt decline, the prediction curve is affected and the prediction curve stands stably for certain period and drops dramatically after that. For battery CS2_36, the impact of sudden drop of voltage is obvious. As a result, the RUL period becomes short, and it is no more than 150 cycles. Applying the binomial fitting to the prediction curve, the binomial curve fits the real data curve. Particularly, all of those curves and the failure threshold line meets at the same point. That means the prediction results for RUL and knee point for battery CS2_36 is accurate.

4.2. Prediction for Battery with Stable Voltage Decline Period

Figure 8 describes the voltage change trend for battery CS2_37. The voltage is stable at about 3.64 before the first 100 charge-discharge cycles. After that, the voltage drops steadily. By comparing the voltage decline period for battery CS2_37 and battery CS2_36, the voltage decline period for battery CS2_37 lasts longer. By contrary, there is a sudden jump for battery CS2_37 after cycle of 420, and continuing decrease after cycle of 500. After cycle of 700, the voltage drops steeply.

The final prediction result for battery CS2_37 is indicated in Figure 9. Before RUL period, both of the prediction curve and real data curve are stable with a steady decline, and the prediction curve goes ups and downs around the real data curve. When it goes into the RUL period, the prediction curve become fluctuated, as well as a rise up. This is because of the sudden jump of the voltage. After the RUL period, both of the prediction curve and real data curve become stable for about 100 cycles, following by the significant decline. Different from battery CS2_36, the RUL period for battery CS2_37 is longer and it lasts about 190 cycles. This is the result of the long decline cycles of voltage and the sudden jump of the voltage. Applying the binomial fitting to the prediction curve, the binomial curve for battery CS2_37 does not like that for battery CS2_36. This curve is over-fitted for the beginning of the RUL period. But as it goes to the middle of the RUL period, this curve become fitted to the real data curve. And same as the result for battery CS2_36, all of those curves and the failure threshold line meets at the same point.

By comparing the prediction results for battery CS2_37 and prediction result for battery CS2_36, it shows not only the prediction results are precise, but also indicates the performance and quality of the battery. In fact, the RUL period for battery CS2_37 lasts longer and the prediction curve looks steadier. As a result, the knee point for battery CS2_37 is greater, and this battery can be used for more charge-discharge cycles.

4.3. Prediction for Battery with Unstable Voltage Decline Period

Figure 10 describes the voltage change trend for battery CS2_38. The voltage change trend is different. The voltage fluctuated during the first 200 cycles. There is an abrupt drop between cycle of 10 and cycle of 90, and it rises steadily for about 50 cycles and there is a sudden jump after that. After cycle of 200, the voltage becomes stable with a steady decline trend. Although there are some ups and downs during the decline cycles, the curve is smoother than that of for both battery CS2_36 and battery CS2_37. Obviously, after cycle of 700, there is a steep drop for voltage curve.

The final prediction result for battery CS2_38 is indicated in Figure 11. The impact of the fluctuation of the voltage curve at the beginning is significant. Before RUL period, the deflection from the prediction curve and real data curve is large. At the beginning, the prediction curve drops dramatically and rises abruptly after cycle of 150. At about cycle of 200, the prediction curve returns to normal and become stable. This curve accompanies the real data curve with a steady decline. Although both curves decline significantly after the RUL period, they are not as steep as those for battery CS2_36 and battery CS2_37. This result is significant, as the decline period for battery CS2_38 is the longest and more stable if not taking the first 150 cycles into account. Applying the binomial fitting to the prediction curve, the binomial curve for battery CS2_38 is over-fitted nearly during the whole RUL period. But in the end of the RUL period, all of those curves and the failure threshold line still meets at the same point as well.

By comparing the prediction results for all three batteries, it shows at least the performance for battery CS2_38 is the best among all the three batteries. The knee point for battery CS2_38 is the greatest as well, and it is about 630 cycles. This battery can be used for more charge-discharge cycles than those two. On the other hand, the manufacturing quality for battery CS2_38 is not as good as battery CS2_37, due to the fluctuation of the prediction curve during the first 200 cycles. In reality, this unstable voltage affects device performance. Especially, the device using this kind of batteries may not be able to work properly due to low voltage.

4.4. Overall Performance

As shown in Table 1, due to the sudden drop of the voltage, the data fluctuates for a certain period of time, and affects the prediction results. The MAE, RMSE and RE of CS2_36 are lower than the benchmarks, and $R^2$ is higher than benchmark. Although there is a sudden jump for this dataset, it is slight jump and the decline period is long and steady. That means the noise data for this dataset is less. So the performance of CS2_37 is relative good. For battery CS2_38, although the variance from prediction and real data is big, it becomes narrow slowly while cycles increase. On machine learning’s perspective, when the testing data keep increasing and model experiences more prediction cycles, the approach has the capacity for result rectification.

The prediction of the knee point is precise for those three scenarios, and those prediction curve, real data curve and binomial curve cross at the same point, where it is the knee point for the battery. That means this approach is applicable for predicting the RUL and knee point for battery and can be used for more scenario or even in the real-world application. Therefore, this proposed approach is able to predict the RUL of a battery accurately and effectively based on small size of sampling data.

5. Conclusion

In this paper, a methodology for predicting the battery RUL based on a hybrid ensembles approach integrated with Gradient Boosting, Random Forest and K-Nearest Neighbor is presented. The battery features of voltage, current, internal resistance are extracted from dataset of CALCE are applied to this proposed approach for training and prediction. With small size of training and testing data, this approach also performs satisfactory in terms of accuracy, robustness, and adaptability. Particularly, this approach utilizes the binomial fitting for capacity faded trend prediction. The predicted RUL and knee point for batteries are precise and fits the real data very well. Compared with existing RUL methods, this proposed approach achieves better performance as indicated by lower RE, MAE, and RMSE scores and higher $R^2$ scores. This approach can also be promoted to wider usage for other real world application scenario.