Numerical Exploration of Helically Coiled Tube Heat Exchangers’ Shell-Side Nature Through Morpho-Hydrodynamic Variations & A Global Correlation

1. Introduction

A Variety of HEX types are available regarding high levels of compactness and effectiveness, such as gasketed plate HEX, welded plate HEX, plate shell HEX, and Polymeric hollow fiber HEX, which offer low weight, as well. However, even polymeric types with relatively high compactness may not offer a high convection heat transfer coefficient like HCTHEXs [1,2,3,4,5,6,7,8,9,10,11]. Additionally, keeping the regime of the fluid flow laminar while acquiring a high CHTC is another factor that may enormously affect the applicability of certain types of HEXs, especially when it comes to residential (e.g., solar heating systems) systems where the flow is desired to stay laminar (whether because of the available heat sources or power supply). The uniqueness of HCTHEX can be attributed to many aspects of such units: compactness, easy construction, their adaptivity to even further passive manipulations such as insertion of helical wire inside the coil, grooving the shell-side cylinder towards better streamlining of the shell-side flow, etc. In tubular structures that are linear, laminar flow exhibits a parabolic velocity distribution because of the dominance of viscous over inertial forces, which stifles any variation within the fluid. Conversely, when the tubes are curved, the velocity distribution becomes skewed because of a secondary circulation prompted by centrifugal forces, which occurs transversely to the primary axial flow direction. The strength of this secondary flow is characterized by $Dean number \left(De\right)$, $De=Re\sqrt{{\displaystyle \frac{d_{in}}{D_{coil}}}}$ . Curved pipes experience a higher pressure decrease relative to their straight counterparts, and the presence of curvature acts to stabilize the flow, leading to a significant elevation in the critical $Re$ where the transition occurs, compared to that in straight pipes [12,13,14,15]. Such an elevation of the critical $Re$ has been quantitatively expressed by a variety of formulas, in terms of curvature ratio, as follows:

$${Re}_{cr,Coil}=2300\left[1+8.6{\left({\displaystyle \frac{d_{tube}}{D_{coil}}}\right)}^{0.45}\right]$$

(1)

On the other hand, though, shell-side CHTC is shown to be radically affected by the number of turns and pitch of the coil [16,17,18].

2. Numerical Model

Contrary to most of the work on the subject matter, which focuses on the simplified thermal and hydraulic boundary conditions in HCTHEXs, the current study is based on an actual fluid-to-fluid simulation of the HEX. The HEX is modeled to be positioned vertically. The schematic of the HEX is demonstrated in Figure 1. Dimensions of different geometries under study are provided in Table 1. Simulations were conducted in ANSYS-FLUENT 2023, research-licensed by the University of Minnesota. The fluids on both the coil and shell sides of the HEX are modeled by liquid water. No phase change on either side of the coil is assumed.

Thermophysical properties: While temperature-dependent properties would represent an even more accurate example of real hydraulic and thermal situations, the relatively complex flow passage through the shell side, which results in a rather non-uniform and complicated temperature gradient, makes it difficult for the solution to converge as appropriate and desired. To that end, the overall average temperature was estimated based on the previously published studies by the corresponding author. The thermophysical properties of the water were then assumed at that average temperature.

Flow regimes: Shell-side: The structure of the overall flow passage through the shell makes it controversial to use the hydraulic diameter as the characteristic length of the shell-side flow to calculate the corresponding dimensionless parameters, such as $Re$ and $Nu$. The topic of the most thermal and hydraulic representative length has been extensively discussed in previous studies by the corresponding author [1][20]. In the current study, though, the fluid flow regime on the shell side was determined based on the tube outer diameter and hydraulic diameter corresponding to the shadowed surface in Figure 2. Based on either choice, the ${Re}_{sh}$ will be smaller than the critical $Re$ which is ${Re}_{cr}=2300$ for internal flows through a circular pipe., for ${\dot{m}}_{shell}=0.01-0.15 kg/sec$.

Coil-side: The transitional regime of the fluid flow is determined according to equation (1). Accordingly, the flow rate on the coil side of up to ${\dot{m}}_{coil}=0.02 kg/sec$ will be totally in the laminar regiment. To account for the variability of the shell-side thermal characteristics on the tube side Re, ${\dot{m}}_{coil}=0.01-0.02 kg/sec$ are simulated. There are a variety of possible forms of exits through the coil side, including, but not limited to, stretching the last turn on each side of the coil to exit through the outer shell or bending the last turns with 90-degree angle and jot them out through the caps. However, those structures may rule out the presence of fully laminar flow throughout the entire tube length. To this end, in our numerical models, the coils keep turning until their fluid flow cross-sections on both sides are out, without any straight tube length, to avoid turbulence.

Inlet and outlet flow conditions: Water, the only working fluid, enters the shell through the lower circular passage with uniform velocity. Hot flow on the coil side enters from the top exit of the helical coil. Such a flow arrangement is more streamlined with counterflow HEXs, which supposedly results in a higher overall heat transfer coefficient [21]. The uniform temperature at the inlet to the shell and coil sides are $T_{sh,inlet}=10 ℃$ and $T_{coil,inlet}=80 ℃$, respectively. The flows at the exits on both sides of the HEX were modeled by mass flow rate conditions.

Tube wall: The coil tube is modeled by a soft-tempered copper wall thickness corresponding to Type L copper tubes of inner and outer diameters of 9.52 mm and 8.001 mm, respectively. This type of copper tube is used in residential and commercial water supply and pressure applications. Type L is generally available in hard-drawn straight sections and in rolls of soft annealed tubing. Shell conduction throughout the coil wall was assumed to be insignificant, and the radial conduction through the thickness of the wall is simulated only.

The simulation is run under steady-state conditions with the pressure-based algorithm. The ‘coupled’ algorithm is chosen as the pressure-velocity coupling method. Second-order discretization of the continuity equation (pressure correction equation), second-order upwind discretization of the momentum, and second-order upwind discretization of the energy equation are implemented. Notably, varieties of relation factors are used between different models. It is noteworthy that the coil wall is coupled with its shadow. The coil tube was not directly modeled in SOLIDWORKS, but instead was modeled in Ansys as a 1D wall with an input thickness of 0.7595 mm and constant Ansys copper properties. The very small artifacts right at the base of the coil where it protrudes from the shell were also modeled in the same way as the rest of the coil that is inside of the shell. The portions of the coil protruding from the shell were modeled as adiabatic with zero thickness copper. The shell walls were modeled as adiabatic with zero thickness aluminum.

3. Computational Cells

On the coil side of the HEXs, the Poly-Hexacore cells, which are a combination of hexahedral elements with poly-prism elements, are used to grid the computational domain. The cells generated within the model with P=1.8 are demonstrated in Figure 3.

4. Results & Discussion

While the ultimate purpose of the current study was to demonstrate and evaluate the shell-side heat transfer phenomena and hydraulic behavior (to some extent), the results from the coil side are presented along the way, as well. The rationale is that contrary to the shell-side, fluid flow inside helically coiled tubes has been extensively studied (if not exhaustively), which makes the validation of the models achievable through the coil-side behaviors, both locally and overall. Relying on the coil-side behaviors as the points of verification, the shell-side behaviors are assumed valid but novel, as can be seen in the following sections.

5. Numerical Model Validation

5.1. Thermal Characteristics

The numerical model in this study is validated against an established formula on the coil-side thermal and hydraulic characteristics. Coil-side Nu is not a strong function of pitch but dependent on the curvature ratio as follows [22]:

$${\overline{Nu}}_{coil}=3.65+0.08\left[1+0.8{\left({\displaystyle \frac{d_{tube}}{D_{coil}}}\right)}^{0.9}\right]{Pr}^{{\displaystyle \frac{1}{3}}}{Re}^{\left[0.5+0.2903{\left({\displaystyle \frac{d_{inner}}{D_{coil}}}\right)}^{0.194}\right]}$$

(4)

The overall coil-side Nusselt number is calculated as follows:

$$q_{coil}={\dot{m}}_c{\overline{c}}_{P,c}\left({\overline{T}}_{c,out}-{\overline{T}}_{c,in}\right)={\overline{h}}_c{A}_{tube,inner}\left({\overline{T}}_{b,c}-{\overline{T}}_{tube,inner}\right)$$

(5)

$${\overline{Nu}}_{coil}={\displaystyle \frac{{\overline{h}}_c{d}_{inner}}{k}}$$

(6)

where ${\overline{T}}_{b,c}$ and ${\overline{T}}_{tube,inner}$ are volume-based average temperature of the coil-side medium and overall inner coil surface temperature, respectively.

The resultant ${\overline{Nu}}_{coil}$ values and the values from (4) are reported in Table 2. The discrepancy between 9% and 0.8% are obtained from this comparison.

5.2. Hydraulic Characteristics

The overall pressure loss can be estimated by a variety of formulas, even though the reported formulas are only valid for certain ranges of $De$. The models in the current study all have the same $De$. Accordingly, the pressure loss in all cases is depicted against the same formula from the literature [23], equations 7-8.

$$\begin{gathered} \left({\displaystyle \frac{\mathsf{\Delta }P}{2\mathsf{\rho }{V}^2}}\right)\left({\displaystyle \frac{d_{tube, inner}^{0.85}{d}^{0.85}{D_{eq}}^{0.15}}{L_c}}\right)=A{Re}^B \\ \mathrm{where} A\mathrm{and} B \end{gathered}$$

(7)

are identified according to equations (8).

$$\mathsf{\Delta }P=\left\{\begin{array}{c}\left({\displaystyle \frac{2\mathsf{\rho }{V}^2{L}_c}{d^{0.85}{D_{eq}}^{0.15}}}\right)38{Re}^{-1}, Re<500\\ \left({\displaystyle \frac{2\mathsf{\rho }{V}^2{L}_c}{d^{0.85}{D_{eq}}^{0.15}}}\right)5.25{Re}^{-{\displaystyle \frac{2}{3}}} , 500<Re<6300\\ \begin{array}{c}\left({\displaystyle \frac{2\mathsf{\rho }{V}^2{L}_c}{d^{0.85}{D_{eq}}^{0.15}}}\right)0.31{Re}^{-{\displaystyle \frac{1}{3}}} , 6300<Re<\mathrm{10,000}\\ \left({\displaystyle \frac{2\mathsf{\rho }{V}^2{L}_c}{{d_{tube,inner}}^{0.85}{D_{eq}}^{0.15}}}\right)0.045{Re}^{-{\displaystyle \frac{1}{8}}} , \mathrm{10,000}<Re\end{array}\end{array}\right.$$

(8)

In those formulas, $D_{eq}$ is defined as follows:

$$D_{eq}={\displaystyle \frac{\sqrt{\left[{\left(P{d}_{tube,inner}\right)}^2+{\left(\pi D_{coil}\right)}^2\right]}}{\pi }}={\displaystyle \frac{L_c}{N\pi }}$$

(9)

Comparison of the numerically calculated pressure loss on the coil side with the values from the formula (8) are demonstrated in Table 3.

5.3. Shell-Side Heat Transfer Coefficient

Overall, there are 2,400 average values for the flow variables which cover a range of flow rates on the shell and coil sides of the heat exchanger.

The scatter plot in Figure 4 illustrates the relationship between the shell-side Nusselt number ($\mathrm{N}{\mathrm{u}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$) and the shell-side Reynolds number ($\mathrm{R}{\mathrm{e}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$) for the helically coiled tube heat exchanger dataset. As expected for convective heat transfer, the overall trend indicates that $\mathrm{N}{\mathrm{u}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$ tends to increase with $\mathrm{R}{\mathrm{e}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$, reflecting enhanced convective heat transfer at higher flow rates. However, the data exhibit substantial scatter and several discontinuities, suggesting that additional parameters—such as the coil-side Nusselt number ($\mathrm{N}{\mathrm{u}}_{\mathrm{c}\mathrm{o}\mathrm{i}\mathrm{l}}$) and the dimensionless coil pitch ($P$)—also significantly influence the thermal performance. The presence of negative or anomalously low values indicates either transitional flow regimes or experimental/reading inconsistencies, which will be filtered or compensated for in the subsequent model-fitting stage. Overall, this preliminary plot confirms that a single-variable correlation in $\mathrm{R}{\mathrm{e}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$ is insufficient, motivating the development of a generalized multi-parameter correlation.

5.4. Global Nu Correlation

To accurately represent the nonlinear relationship between $\mathrm{N}{\mathrm{u}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$ and the governing parameters of the helically coiled tube heat exchanger, a Generalized Additive Model (GAM) combined with Ensemble Learning was developed. This hybrid regression framework captures the combined nonlinear effects of $\mathrm{R}{\mathrm{e}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$, $\mathrm{N}{\mathrm{u}}_{\mathrm{c}\mathrm{o}\mathrm{i}\mathrm{l}}$, and the dimensionless pitch ratio (P).

5.5. Model Performance & Validation

The predictive capabilities of the three regression frameworks—Power-law (Robust), Log-Quadratic, and GAM/Ensemble—were compared to evaluate their ability to represent the shell-side heat-transfer behavior. Figure 5 illustrates the parity plots of the predicted versus experimental Nusselt numbers for each model. The classical Power-law and Log-Quadratic correlations display poor predictive fidelity, with most points scattered far from the 1:1 reference line, confirming the inadequacy of linearized empirical relationships in capturing the nonlinear coupling among $\mathrm{R}{\mathrm{e}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$, $\mathrm{N}{\mathrm{u}}_{\mathrm{c}\mathrm{o}\mathrm{i}\mathrm{l}}$, and $P$.

In contrast, the GAM/Ensemble regression (Figure 5, right) exhibits a tight clustering of points along the ideal correlation line, demonstrating markedly superior accuracy and generalization. Quantitatively, the GAM/Ensemble achieved an R² ≈ 0.83 and RMSE ≈ 19.6, outperforming both the Power-law (R² < 0) and Log-Quadratic (R² ≈ 0.10) models. These results confirm that the machine-learning-based GAM/Ensemble correlation effectively learns the nonlinear and multivariate nature of shell-side convective heat transfer, yielding a predictive mapping far more robust than conventional curve-fitting methods.

5.6. Model Architecture and Learning Concept

The GAM/Ensemble model represents the Nusselt number as an additive sum of smooth nonlinear functions, each corresponding to one predictor. The model form is given by:

$$N{u}_{shell} = c + f₁\left(R{e}_{shell}\right) + f₂\left(N{u}_{coil}\right) + f₃\left(P\right)$$

Here, $f₁,$ $f₂$, and $f₃$ are smooth shape functions learned through ensemble regression trees, while c is the model bias term. Each function describes the isolated nonlinear influence of its corresponding predictor on $\mathrm{N}{\mathrm{u}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}$, while interactions are implicitly captured through ensemble averaging.

Conceptually, the GAM/Ensemble behaves analogously to an artificial neural network (ANN) regression surface, where each variable activates a nonlinear response. However, it replaces backpropagation-based weights with decision-tree ensembles, resulting in smoother functions that retain interpretability.

For practical implementation and to facilitate inclusion in analytical studies, the GAM/Ensemble model was approximated by expressing each nonlinear function fᵢ(xᵢ) as a third-degree polynomial in log₁₀-space. This yields the following correlation:

$$\mathrm{N}{\mathrm{u}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}} \approx c\_poly + f₁log\left(\mathrm{R}{\mathrm{e}}_{\mathrm{s}\mathrm{h}\mathrm{e}\mathrm{l}\mathrm{l}}\right) + f₂log\left(\mathrm{N}{\mathrm{u}}_{\mathrm{c}\mathrm{o}\mathrm{i}\mathrm{l}}\right) + f₃log\left(P\right)$$

Each term $fᵢlog\left(xᵢ\right)$ is represented by a cubic polynomial of the form:

$$fᵢlog\left(xᵢ\right) = a₃·(log₁₀xᵢ)³ + a₂·(log₁₀xᵢ)² + a₁·(log₁₀xᵢ) + a₀$$

Accordingly, the complete polynomial-based GAM/Ensemble correlation for predicting the shell-side Nusselt number can be expressed as:

$$\begin{gathered} {Nu}_{shell} \approx 59.2364 + (-1337.275·L\_Re³ + 107.518·L\_Re² + 289.845·L\_Re - 233.453) \\ + (13.7486·L\_Nu³ - 113.868·L\_Nu² + 267.977·L\_Nu - 98.848) \\ + (1.915\times 10⁶·L\_P³ - 1.608\times 10⁶·L\_P² + 4.497\times 10⁵·L\_P - 4.188\times 10⁴) \end{gathered}$$

where $L\_Re=log₁₀\left(R{e}_{shell}\right)$, $L\_Nu=log₁₀\left(N{u}_{coil}\right)$, and $L\_P=log₁₀\left(P\right)$.

Table 4. Polynomial coefficients (third-degree, log₁₀-space) for the GAM/Ensemble correlation.

Function	a₃	a₂	a₁	a₀
$f₁(log₁₀Reₛ)$	−1337.275	107.518	289.845	−233.453
$f₂(log₁₀Nu\_c)$	13.7486	−113.868	267.977	−98.848
$f₃(log₁₀P)$	1.915×10⁶	−1.608×10⁶	4.497×10⁵	−4.188×10⁴
$c\_poly$ (Constant)	59.2364

Table 4. Polynomial coefficients (third-degree, log₁₀-space) for the GAM/Ensemble correlation.

Function	a₃	a₂	a₁	a₀
$f₁(log₁₀Reₛ)$	−1337.275	107.518	289.845	−233.453
$f₂(log₁₀Nu\_c)$	13.7486	−113.868	267.977	−98.848
$f₃(log₁₀P)$	1.915×10⁶	−1.608×10⁶	4.497×10⁵	−4.188×10⁴
$c\_poly$ (Constant)	59.2364

5.7. Summary of Model Accuracy

Table 5 summarizes the comparative performance of different models developed for the dataset. The GAM/Ensemble model demonstrates a significant improvement in predictive accuracy.

The GAM/Ensemble correlation provides the highest fidelity in capturing the complex dependence of Nuₛ on the system parameters, outperforming classical empirical fits. The polynomial approximation serves as a convenient analytical form for implementation, while the original GAM/Ensemble model is recommended when computational accuracy is critical.

The classical power-law models (both OLS and robust regressions) produced negative R² values (−0.0076 and −0.1368, respectively). A negative R² value occurs when the sum of squared prediction errors exceeds the variance of the observed data, indicating that the model performs worse than a simple mean predictor. This phenomenon arises when the assumed functional form fails to capture the true underlying relationship between the dependent and independent variables.

In the present case, the narrow variation in Nuₛ and the strongly nonlinear coupling between the governing parameters (Reₛ, Nu_c, and P) render the traditional power-law assumption invalid. The log-linear model thus attempts to fit a planar surface in log-space to data that exhibit significant curvature and interaction effects, leading to poor predictive capability.

Conversely, the GAM/Ensemble model achieved an R² of approximately 0.83, accurately mapping the complex multi-variable dependence of Nuₛ. By employing smooth nonlinear functions for each predictor and leveraging ensemble averaging of decision-tree regressors, the model flexibly captures local curvature and saturation behavior in the heat-transfer response. As such, the GAM/Ensemble correlation demonstrates a substantial improvement in predictive performance and generalization accuracy compared to conventional empirical fits.

6. Closing Remarks

The present study conducted a detailed numerical investigation of the shell-side heat transfer performance of helically coiled tube heat exchangers (HCTHEXs), revealing the complex morpho-hydrodynamic coupling among the coil geometry, flow structure, and convective characteristics. By systematically varying the coil pitch and analyzing its influence on the thermal and hydraulic performance, the simulations captured the inherent irregularity of shell-side behavior, which is strongly influenced by the coil-side Reynolds number and secondary flow interactions.

Building upon this exhaustive numerical dataset, a hybrid Generalized Additive Model (GAM) combined with Ensemble Learning was introduced to establish a global predictive correlation for the shell-side Nusselt number. The GAM/Ensemble model achieved excellent predictive accuracy (R² ≈ 0.83, RMSE ≈ 19.6), successfully mapping the nonlinear multivariate relationships among $N{u}_{shell}$, $R{e}_{shell}$, $N{u}_{coil}$, and the dimensionless pitch ratio $P$. This data-driven framework demonstrated a clear advantage over conventional empirical power-law or log-linear correlations, which exhibited poor generalization and even negative R² values. The machine-learning correlation provides not only a high-fidelity predictive tool but also physical interpretability through its additive, spline-based architecture and its polynomial-in-log analytical form.

In conclusion, this study introduces a robust, computationally grounded approach that merges physics-based simulation with machine-learning-assisted correlation development. The resulting GAM/Ensemble correlation can serve as a reliable design and optimization tool for helically coiled heat exchangers, enabling engineers to predict shell-side thermal performance across a wide range of geometric and flow conditions. Future work may extend this framework to include coil diameter effects, turbulent-to-laminar transitions, and cross-validation with experimental datasets, further advancing the predictive understanding of complex convective systems.