A Novel Approach for Reducing Feature Space Dimensionality and Develop a Universal Machine-Learning Model for Coated Tubes in Cross Flow Heat Exchangers

1. Introduction

Coatings are applied to improve various properties, such as corrosion resistance, abrasion resistance, toughness, chemical resistance, etc. One of the most common materials used for coating purposes is polyurethane foams because of their simple handling, economic cost, and proper physical properties [1,2,3]. In the recent decade, researchers have studied how coating thickness affects heat transfer in finned-tube heat exchangers [4,5]. An accurate estimation of coating thickness is crucial to increase the heat exchanger lifetime especially in high working temperatures [6]. Machine-learning methods have recently attracted much attention along usual approaches for heat transfer and thermodynamics analysis. For instance, estimating the heat values of different kinds of fuel with machine-learning methods is faster and computationally cheaper instead of direct calculations and experiments [7,8]. Machine-learning models are also employed to predict the optimal system design. Mohamed et al. suggested multiple machine-learning models to predict eleven different parameters for the optimal design of proton exchange membrane (PEM) electrolyzer cells [9]. Recently, machine learning has shown great promise in computational fluid dynamics (CFD) studies [10] and is becoming more accurate and faster [11,12]. Making machine-learning models with acceptable generalization capability for different heat transfer problems is another approach that made analysis faster. For instance, a universal condensation heat transfer and pressure drop model has been recently proposed by using machine learning techniques [13]. These techniques are also used to analyze heat exchangers for different purposes like predicting the thermal performance of fins for a novel heat exchanger [14]. Lindqvist et al. used machine learning models to optimize heat exchanger designs and developed good correlations with trends in the CFD model [15]. Moreover, machine-learning has shown great potential in predicting heat transfer for high order nonlinear cases [16]. In the absence of a valid physical-based model, machine learning can be utilized to predict heat transfer in many thermal systems [17]. Considering recent advances in machine learning techniques, CFD computational cost can be reduced by using genetic algorithms [18]. Similarly, neural networks were found to be effective for analyzing thermal conductivity assessment of oil-based hybrid nanofluids [19]. Gradient boosting decision trees can reduce the cost of measuring equipment for computing transient heat flux [20]. Although more advanced ensemble models can lead to better results for more complicated datasets [21], the use of such models in heat transfer problems has received less attention. The most common model of this category, random forest, showed a remarkable ability to evaluate heat transfer across various scenarios [22]. Swarts et al. compared three different algorithms for predicting critical heat flux for pillar-modified surfaces and concluded that random forest provides the best results [23]. Its performance on large datasets [24] and a precise ranking of features' importance [25] are the main Random forest's advantages. Despite the benefits of random forest and other machine learning algorithms, collecting training data can be a challenge. The number of data required for training can be minimized by combining features, which enhances the regression accuracy too [26]. As far as the author knows, there are no studies on machine-learning models that investigate the correlation between coating and heat transfer in a heat exchanger. This study fills this gap with machine-learning models with acceptable generalization capability on unseen data.

In this study, a new method of combining input features is proposed to investigate the effect of variable coating thickness on the prediction of heat transfer in a finned-tube heat exchanger at different inlet Reynolds numbers. Almost 1000 different cases are simulated using a 3D finite volume model to generate the data set. The random forest model is initially trained using numerical simulation data. By using the created model, selected features are combined, and new features are added. In addition, various new data are used to validate models’ interpolations. At last, the capability of the introduced method for dimension reduction is investigated with different models. The main contributions of this study are summarized as follows.

• To the best of our knowledge, for the first time, a general machine-learning model is developed for predicting heat transfer in heat exchangers based on coating thickness over a wide range of domains.
• This work proposes a new way to combine input features without losing any sensitive information about the original feature space.
• The proposed method cures the curse of dimensionality of the K-Nearest Neighbors algorithm and as a result, the model accuracy is significantly improved.
• Feature engineering and feature combination methods used in this work make it possible to train machine-learning models over a smaller dataset with acceptable accuracy over unseen data from new domains.

Section 2 investigates the effect of the input features on the heat transfer and discusses the features extraction and data collection procedure. The proposed method, feature, training procedure, and results are described in Section 3. In Section 4, results are discussed to validate the performance of the proposed method with a comparative analysis of different machine-learning models’ performance. Finally, the last subsection concludes this article.

2. Heat transfer analysis and feature extraction

The case study in this work is the passage of hot gas flow over a bunch of tubes with a triangular arrangement. The isometric view of the domain is shown in Figure 1(a), and the geometry dimensions (in mm) are shown in Figure 1(b). The symmetry boundary condition is set around the selected volume, shown with dashed lines in Figure 1(a). The coating material on each tube is polyurethane and the coating thickness is different on each tube but in the range of 10-30 µm. The wall temperature of all tubes is assumed constant. The gas enters the domain with different Reynolds numbers and temperatures, which cause heat transfer. A finite volume model (FVM) was developed to simulate turbulent gas flow. Table 1 reveals some results of this FVM model.

Figure 2 displays the temperature distribution for third case condition presented in the Table.1. In order to understand how coating thickness and Reynolds number influence on the heat transfer, different simulations are performed with variations in coating thickness, Reynolds number and temperature. Figure 3 (a, b) represents total heat flux and heat transfer coefficients for the last tube in fluid flow direction

These numerical simulations are the basis of forming the dataset. The dataset structure is made in two steps: At first, each tube coating thickness is varied within three values, i.e.,10, 20, and 30 μm. Thus, the total number of data (for five tubes) gathered from simulations is 35 = 243. For the next step, after the feature combination process (which will be explained later), the new feature value is varied in a wide range of domains; as a result, the final data set is formed with 1000 data.

3. Proposed method

Machine learning regression algorithms such as random forest are commonly used for quantitative analysis. Random forest results are generated by averaging the predictions of randomly ensembled decision trees and it is a great way to estimate the importance of input features. Conversely, as the total number of features increases, more training data is required. Hence, in the present study, the data set is collected in a fixed domain, and after training the first random forest, features are ranked according to importance. Then, features are combined based on a new approach which will be explained in detail in the next parts. New models are then built based on these new features (illustrated in Figure 4). This approach reduces the required amount of data for training while ensuring the same accuracy. After completing the second step of data collection new regression models such as Random Forest, K-Nearest Neighbors, and Support Vector Regression are trained using four-fold cross-validation. For the hyperparameters tuning section, random grid is used to find the best hyperparameters for each model.

After the creation of models, unseen data are used to validate each model generalization. To investigate the model’s accuracy R-square criterion is used as follows:$1-\frac{\mathrm{r}\mathrm{e}\mathrm{s}\mathrm{i}\mathrm{d}\mathrm{u}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{u}\mathrm{m}\mathrm{o}\mathrm{f}\mathrm{s}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{r}\mathrm{e}}{\mathrm{t}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{u}\mathrm{m}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{h}\mathrm{e}\mathrm{s}\mathrm{q}\mathrm{u}\mathrm{a}\mathrm{r}\mathrm{e}}$ note that the best value for this metric equals 1 but it also can be negative if the model performs worse than the mean of data in predicting the observed value.

3.1. Training model

The random forest quantitatively evaluates each input feature's importance, but factors like untuned hyperparameters and the total number of observed data can cause a negative impact on the value of each feature's rank. The number of categories for each input should also be equal for accurate ranking [27]. Tuning hyperparameters is the next step toward having an accurate model. A model's prediction is more accurate and its ability to rank features' importance is more precise when hyperparameters are tuned. Bayesian optimization is used to tune hyperparameters [28]. A probability model is built using Bayesian optimization based on past evaluation results. Table 2 shows selected hyperparameters with this approach. The designed random forest based on the above steps can closely predict the total heat flux for new input features to calculated results, as shown in Figure 5. The random forest regressor fits q closely to FVM results with R²=0.9832 for the training dataset and R²=0.9864 for the testing dataset, but this model is not intended to predict the total heat flux for new inputs. The main goal of this model is to rank features’ importance as shown in Figure 6. This figure presents how to reduce the size of feature space for the main model.

3.2. Features combination and expanding model domain

In order to have a generalized model that can predict the total heat flux (q), new features such as Reynolds number and temperature are added to the feature matrix. Increasing the number of features will require more data collection since data are collected through either numerical simulation or experiment that has cost. A new approach is employed to decrease the total number of required simulations in this work. To do so a new variable is defined (t') as shown in the equation 1:

$$\stackrel{`}{t}=\frac{\sum _{n=1}^{\mathrm{\infty }}Feature importance\left(n\right)\times feature value\left(n\right)}{\sum _{n=1}^{\mathrm{\infty }}Feature importance\left(n\right)}$$

Considering the results shown in Figure 6, the five first features are combined. The new feature matrix contains four features= [Re, Tpipe, Tfluid, 𝑡́]. As a result, if five thicknesses of coating, [t1, t2, t3, t4, t5], are considered for tubes, the calculated total heat flux is approximately equal to the heat flux of tubes with a coating thickness 𝑡́. After selecting features, data collection is started. To collect data, 168 sample cases are simulated and training and testing matrixes are made. The hyperparameters are tuned with the Bayesian optimization method. Table 3 shows the hyperparameters designed with this optimization scheme. Figure 7 compares the total heat flux predicted by the random forest and calculated results by FVM simulations. The random forest regressor fits q closely to FVM results, with R2 = 0.9994 for the training dataset and R2 = 0.9600 for the testing dataset.

As shown in the previous section, although the training data is limited for the second random forest with combined features, the model performed well over the test dataset but the model still needs to be evaluated over new data. A new dataset is collected to study the model performance over new domains. The dataset contains a total of 549 new cases which are simulated based on eight variables=[Re, T_pipe, T_fluid, t₁, t₂, t₃, t₄, t₅] as inputs, where t_i is the coating thickness of tube #i. The total heat flux is predicted with the second random forest with [Re, T_pipe, T_fluid,, 𝑡́] as input features. Note that 𝑡́ is calculated by equation 1 with [t₁, t₂, t₃, t₄, t₅] as variables. The random forest regressor fits q closely to observed results with R² = 0.9810 for the unseen dataset (see Figure 8).

4. Results and Discussion

In real-world problems in different industries, the feature space contains lots of variables that cause training process slow and, in some cases, make it much harder to find a good solution. Theoretically, it is possible to solve these problems by adding new data to the training data set but collecting new data can be impossible in different scenarios. Reducing the total number of features is another solution to this problem, but available approaches may cause some difficulties, like losing information about the original feature space or even making the model perform worse As indicated in the previous section, combining related features by equation 1 can save the deleted features' information. The model also can accurately predict heat flux over distinct new domain features, as shown in Figure 8. The combination process also significantly reduced the amount of data needed to train the model. Alternatively, using a simpler variable, such as the average value of variables $t_{average}=\frac{\sum _{n=1}^{n=5}Coatingthickness\left(n\right)}{n}$, instead of 𝑡́ for combining features, is not a good choice since this variable does not take into account any differences between the first and last tube's heat flux. In addition, the information about the original feature space is lost (see Table 4).

Different models are made to analyze the feature combination impact on models. The key to a fair comparison of these models is to make sure that each model is evaluated with the same approach using the same data. The second random forest training data set was used to create new models to keep things fair. Each model is trained once with the feature combination technique and without it. The grid search is used to find the best combination of hyperparameters for each model with a common 4-fold cross-validation technique. The results are shown in Table 5.

As shown in Table 5, all models predicted unseen data better after using equation.1 for combining features, but the downside is that a primary model is needed to calculate equation 1 coefficients. This may be disadvantageous if obtaining considerably better results is not possible with this method. Conversely, this method can significantly improve the performance of algorithms that are victims of cures of dimensionality. For example, K-Nearest Neighbors (K-NN) regressor does not perform well with high dimensions inputs [29] but after the feature combination, the K-NN performance increased by 67.66 %. Support vector regression (SVR) may also suffer from the curse of dimensionality when enough data samples are not available. This issue can affect the generalization ability of SVR but after the feature combination, the R2 value for unseen data is increased by 2.82 %. In the case of SVR, one may conclude that increasing the total number of samples may be a better idea than reducing the dimensionality but as a reminder, the author wants to mention SVR doesn't perform well on big data sets so increasing the total number of samples may not be a good strategy for this algorithm. The last model, random forest is immune to the curse of dimensionality and overfitting, so there is no need to combine input features, but by comparing the two random forest models in the table.5, one can see R2 value is just slightly improved after reducing the total number of inputs. This point shows after combining features with equation.1 no information will be lost.

A new feature is also created by combining Re number and t as inputs for equation 1. The trained model with this approach is not as accurate as the previous models. This process raised this question: "When can we use equation 1?". As shown in Figure 3 [t1, t2, t3, t4, t5] have closer importance scores to each other in comparison to the Re number importance score. (Note that Reynolds number has more impact on heat flux than coating thickness, so its importance score is much more than 𝑡́ ' s importance score By knowing this, it is better to combine features within the same order of magnitude or with a similar impact on the target like coating thickness for different tubes.

5. Conclusion

This work explored random forest as a heat transfer correlation for predicting the total heat flux with varied coating thicknesses for different Re numbers. Feature selection was essential before implementing the machine-learning model for the studied case. Despite all benefits of feature selection, there is a chance of losing important information because of deleting some features. The employed approach can decrease the size of feature space without losing information about the original feature space and also showed potential for heat transfer problems in which available data is limited. Training data for the algorithm was obtained by the finite volume model method. The trained model predicted well over the training data set with R² ~ 0.9994 and over the testing data set with R² ~ 0.9600. The algorithm was also able to predict total heat flux, similar to the finite volume model with R² ~ 0.9810 based on its interpolation mechanism. Other algorithms like SVR and K-NN were also employed to check other aspects of the feature combination method used in this work which also showed great promise in predicting the total heat transfer over unseen data with R² ~ 0. 9754 and R² ~ 0.9037. For future works, the results of this study may be beneficial for stacking models with better generalization with fewer data.