Enhancing Portfolio Allocation: A Random Matrix Theory Perspective

1. Introduction

Markowitz’s seminal paper [1] remains a cornerstone in practical portfolio management, despite facing challenges in out-of-sample analysis due to estimation errors in expected returns and the covariance matrix, as noted in [2,3,4,5,6]. To address these limitations, researchers have investigated the use of robust estimators aimed at providing more accurate and reliable measures of moments and co-moments. Robust estimators, designed to handle outliers and errors, enhance the robustness of portfolio optimization, leading to improved stability and performance, see [7,8,9].

In recent years, there has been a notable shift towards a novel approach known as the market graph for understanding the relationship between asset returns, departing from traditional covariance-based methods. This methodology represents assets as nodes and their relationships as edges in a graph, capturing the broader structure of the finacial market. Pioneering work by Mantegna [10] led to the construction of asset graphs using stock price correlations, revealing hierarchical organization within stock markets. Subsequent research conducted by [11] investigates stock network topology by examining return relationships, with the objective of revealing significant patterns inherent in the correlation matrix. Additionally, [12] introduced constraints based on asset connections in the correlation graph to encourage the inclusion of less correlated assets for diversification, employing assortativity from complex network analysis to guide the optimization process. Recent research explores innovative portfolio optimization techniques using clustering information from asset networks [13,14,15]. These approaches replace the classical Markowitz model’s correlation matrix with a correlation-based clustering matrix. In particular, in [15], a network-based approach is employed to model the financial market’s structure and interdependencies among asset returns. The authors construct a market graph capturing asset connections using various edge weights and introduce an objective function considering both individual security volatility and network connections. [13] proposed a unified Mixed-Integer Linear Programming (MILP) framework integrating clustering and portfolio optimization. Empirical results demonstrate the promising performance of the network-based portfolio selection approach, outperforming classical approaches relying solely on pairwise correlation between assets’ returns. Building upon the research outlined in [14,15], we introduce a novel framework that integrates Random Matrix Theory (RMT) into portfolio allocation procedures. Specifically, we employ a combination of the Hilbert transformation and RMT to estimate the correlation matrix. This approach enables us to effectively separate noise from significant correlations, enhancing the accuracy and robustness of the portfolio allocation process. On one hand, the Hilbert transformation technique is a data pre-processing method which we use to complexify the time series of returns, thus improving the effectiveness in detecting lead/lag correlations in time series data, as shown in [16,17,18]. On the other hand, RMT offers a robust denoising technique on the basis of the Marčenko-Pastur theorem, which describes the eigenvalue distribution of the covariance matrices [19,20,21]. As a result, we obtain a denoised version of the covariance matrix that captures the true underlying correlation structure more accurately and facilitating the computation of clustering coefficients to assess the interconnectedness of assets.

The method proposed is then tested on four equity portfolios, employing a Buy and Hold Rolling Window strategy with various in-sample and out-of-sample periods. The out-of-sample performances are then compared across different methods of estimating the dependence structure between assets, including sample estimates, shrinkage estimates, clustering coefficient, and RMT. Notably, employing the RMT method consistently results in superior out-of-sample performance compared to traditional methods across all portfolios and rolling window strategies. Furthermore, the analysis on transaction costs, measured by turnover, reveals that the RMT approach consistently led to lower portfolio turnover compared to other methods. This reduction in turnover translates to lower transaction costs and underscores the practical applicability of the RMT approach in portfolio management.

The paper is organized as follows: Section 2 introduces the basic notions of Hilbert transformation and RMT-based techniques. Section 3 offers a brief overview of existing portfolio selection models. Section 4 details the in-sample and out-of-sample protocols and analyzes the empirical outcomes. Finally, Section 5 discusses the paper’s key findings.

2. Random Matrix Theory

In the context of financial mathematics it has been established that extensive empirical correlation matrices exhibit significant noise, to the point where, with the exception of their principal eigenvalues and associated eigenvectors, they can be essentially treated as stochastic or random entities. Therefore, it is customary to denoise an empirical correlation matrix before its utilization. In this section, we will elucidate a denoising technique rooted in the principles of RMT where the fundamental result is the Marčenko-Pastur theorem, which describes the eigenvalue distribution of large random covariance matrices.

In the following we denote by $\mathcal{M}\left({\mathbb{C}}^{n\times T}\right)$ the space of complex-valued matrices with dimensions $n\times T$. Let $X\in \mathcal{M}\left({\mathbb{C}}^{n\times T}\right)$ be an observation matrix where columns are the single observations for different n time series. Let $\mathsf{\Sigma }\in \mathcal{M}\left({\mathbb{C}}^{n\times n}\right)$ be the empirical covariance matrix associated to X, i.e. $\mathsf{\Sigma }=\frac{1}{T}X{X}^{*}$1. If $\mathsf{\Sigma }$ is Hermitian (that is, the eigenvalues are real) we can define the empirical spectral distribution function as $F=n^{-1}{\sum }_{j=1}^n{\mathbf{1}}_{{\lambda }_j\le \lambda }$, where ${\lambda }_j$, for $j=1,\cdots ,n$ are the eigenvalues of $\mathsf{\Sigma }$. The distribution of the eigenvalues of a large random covariance matrix is actually universal, meaning that it follows a distribution independent of the underlying observation matrix as stated in the following Marčenko-Pastur theorem [19,20,21]:

Theorem 1 

(Marčenko-Pastur, 1967). Suppose $X_i,i=1,\cdots ,n$ are i.i.d. random variables with mean 0 and variance ${\sigma }^2$. Suppose that $\frac{T}{n}\to q\in (0,+\infty )$ as $n\to +\infty ,T\to +\infty$. Then, as $n\to +\infty$, the empirical spectral distribution $\widehat{F}$ of the sample covariance $\widehat{\mathsf{\Sigma }}$ converges almost surely in distribution to a nonrandom distribution, known as the Marčenko–Pastur law and denoted by $F_q$ whose probability distribution is:

$${\rho }_q\left(\lambda \right)d\lambda =\max \left(1-\frac{1}{q},0\right){\delta }_0+\frac{\sqrt{({\lambda }_{+}-\lambda )(\lambda -{\lambda }_{-})}}{2\pi q\lambda {\sigma }^2}{\mathbf{1}}_{{\lambda }_{-}\le \lambda \le {\lambda }_{+}}d\lambda$$

(1)

where the first term represents a point mass with weight $1-\frac{1}{q}$ at the origin if $q>1$ and ${\lambda }_{\pm }={\sigma }^2{(1\pm \sqrt{q})}^2$ coincide with the lower and the upper edges of the eigenvalue spectrum.

Let us note that, under proper assumptions, the Marčenko-Pastur theorem remains valid for observations drawn from more general distributions, like fat-tailed distributions [22,23].

Utilizing the boundaries of the Marčenko-Pastur distribution within the eigenvalue spectrum facilitates distinguishing between information and noise, enabling the filtration of the sample correlation matrix. Consequently, it is expected that this filtered correlation matrix will provide more stable correlations compared to the standard sample correlation matrix.

2.1. Filtering Covariance by RMT

One can compare the theoretical density of the eigenvalues generated the by the Marčenko-Pastur theorem (in the following indicated as benchmark or null model) with the corresponding empirical one. Thus we can identify the number of empirical eigenvalues, possessing some known and easily interpretable characteristics, that significantly deviate from the null model. There are many methods for filtering the correlation matrix based on the RMT as proposed previously. Essentially, the approaches primarily involve suitable modification of the eigenvalues within the spectrum of the sample correlation matrix while simultaneously preserving its trace, as explained below.

The upper boundary ${\lambda }_{+}$ of the Marčenko-Pastur density serves as a threshold for distinguishing the noisy component of $\mathsf{\Sigma }$. Eigenvalues of $\mathsf{\Sigma }$ within the interval $[{\lambda }_{-},{\lambda }_{+}]$ adhere to the random correlation matrix hypothesis and represent noise-associated eigenvalues, as well as those below ${\lambda }_{-}$. The rationale behind excluding the smallest deviating eigenvalues from these filters lies in the fact that, unlike large eigenvalues – which are separated from the Marčenko-Pastur (MP) upper bound – the same rationale does not always hold true for the smallest deviating eigenvalues. Typically, small eigenvalues can be situated beyond the lower edge of the spectrum, a pattern consistent with the finite dimension of the observed matrix $(n\times T)$. Furthermore, as emphasized in reference, there exists clear evidence about non-randomness and temporal stability of the eigenvectors corresponding to eigenvalues larger than ${\lambda }_{+}$. Such characteristics have not been consistently confirmed for the eigenvectors associated with the eigenvalues smaller than ${\lambda }_{-}$. Therefore, eigenvalues higher than ${\lambda }_{+}$ can be regarded as “meaningful”. This leads to the following denoising method for the estimated covariance matrix $\widehat{\mathsf{\Sigma }}$: all the eigenvalues of $\widehat{\mathsf{\Sigma }}$ lower than or equal to ${\lambda }_{+}$ are replaced by a constant value either equal to the average value of the “noisy” eigenvalues (as in [24]) or equal to zero (as in [25])2. All the eigenvalues of $\widehat{\mathsf{\Sigma }}$ strictly greater than ${\lambda }_{+}$ are left unchanged.

In this paper, the eigenvalues of $\widehat{\mathsf{\Sigma }}$ that fall in the range predicted by the Marčenko-Pastur distribution are set to zero. Then, to build the filtered covariance matrix ${\mathsf{\Sigma }}^F$, the following steps are undertaken: suppose there exist k relevant eigenvalues (i.e., higher than ${\lambda }^{+}$); let us denote them by ${\left\{{\lambda }_j\right\}}_{j=n-k,\dots ,n}$, and also consider the diagonal matrix of filtered eigenvalues ${\mathsf{\Lambda }}^F=\mathrm{diag}(0,\dots ,0,{\lambda }_{n-k},\dots ,{\lambda }_n)$. At this point, we use ${\mathsf{\Lambda }}^F$ back in the eigendecomposition with the original eigenvectors, here denoted by V, so obtaining:

$${\mathsf{\Sigma }}^F=V{\mathsf{\Lambda }}^F{V}^{-1}.$$

(2)

Finally, in order to preserve the trace of the original matrix and prevent system distortion, we adjust the main diagonal elements of the filtered matrix ${\mathsf{\Sigma }}^F$ by replacing them with the sample variances of the portfolio components ${\widehat{\sigma }}_{ii}^2,i=1,\cdots ,n$.

Remark 1. 

When applying this method to portfolio selection, we will work with the correlation matrix. However, the Marčenko-Pastur law remains unchanged with the only difference that ${\sigma }^2=1$.

2.2. Hilbert Transformation

In this paper we use a data pre-processing technique based on Hilbert transformation of time series mainly applied in climatology, signal processing, fi–ce, and economics [16,17,18]. This technique is especially well-suited for detecting correlations with lead/lags, making it highly effective for time series exhibiting shifts in their co-movements.

Let us consider the panel of asset returns (i.e. portfolio) over time as $\mathit{x}\left(t\right)$ and let us indicate with $\tilde{\mathit{x}}\left(t\right)$ the complex series derived by Hilbert transforming the original time series as $\tilde{\mathit{x}}\left(t\right)=\mathit{x}\left(t\right)+i\mathcal{H}\left[\mathit{x}\left(t\right)\right]$ so that the real part of each component $\mathcal{R}e\left(\tilde{\mathit{x}}\left(t\right)\right)=\mathit{x}\left(t\right)$ coincides with the original time series. The Hilbert transformation is a linear transformation defined [26] as the following integral:

$$\begin{array}{c}\hfill \mathcal{H}\left[\mathit{x}\left(t\right)\right]=\frac{1}{\pi }\mathrm{P}\underset{-\infty }{\overset{\infty }{\int }}\frac{\mathit{x}\left(\tau \right)}{\tau -t}\mathrm{d}\tau ,\end{array}$$

where P indicates the Cauchy principal value; then the complexified sequence $\tilde{\mathit{x}}\left(t\right)$ can then be written as:

$$\tilde{\mathit{x}}\left(t\right)=\frac{i}{\pi }\oint \frac{\mathit{x}\left(\tau \right)}{\tau -t}d\tau .$$

(3)

As an example of the effect of the Hilbert transformation on the correlation matrix, let us consider two periodic signals defined as:

$$\begin{array}{cc}\hfill x& =sin\left(\frac{\pi t}{4}\right)\hfill \end{array}$$

(4)

$$\begin{array}{cc}\hfill y& =cos\left(\frac{\pi t}{4}\right)\hfill \end{array}$$

(5)

The correlation matrix between the two time series is:

$$C=\left(\begin{array}{cc}1& -5.86·{10}^{-4}\\ -5.86·{10}^{-4}& 1\end{array}\right)$$

(6)

from which it is clear how negligible is the correlation between the two time series. Meanwhile, after Hilbert transformations of the time-series the two complexified sequences have the following complex correlation matrix:

$$\begin{array}{cc}\hfill \tilde{C}& =\left(\begin{array}{cc}1+i0& -5.86·{10}^{-4}+i0.99\\ -5.86·{10}^{-4}+i0.99& 1-i0\end{array}\right)\hfill \end{array}$$

(7)

$$\begin{array}{cc}\hfill & =\left(\begin{array}{cc}1& 0.99e^{i1.57\pi }\\ 0.99e^{-i1.57\pi }& 1\end{array}\right).\hfill \end{array}$$

(8)

We note from (7) that the magnitude of the correlation is pretty high, as expected if one considers the lags to be just a lack of synchronicity and not a lack of co-movements. The behavior of the two correlation matrices is also evident if one plots the two time series in a real plane and in a Gauss plane, see Figure 1.

If we use the complexified data sequence then the Marčenko-Pastur law has to be modified replacing p with $2p$ since the imaginary part of a complexified data sequence is not independent of its real part, being related by the Hilbert transformation [18].

Utilizing the Hilbert transformation, the covariance (cross-correlation) matrix is represented in a space where the eigenvectors’ components are spread across the complex plane. This facilitates identifying lead-lag relationships between components through angular disparities. The components of the complex covariance matrix can be denoted as follows:

$${\sigma }_{kj}=\mathcal{R}e\left({\sigma }_{kj}\right)+i\mathcal{I}m\left({\sigma }_{kj}\right)=\left|{\sigma }_{kj}\right|e^{i{\varphi }_{kj}},k,j=1,\cdots ,n,$$

where the absolute value of each element of the complex covariance matrix gives the strength of the correlation; meanwhile ${\varphi }_{kj},k,j=1,\cdots ,n,$ indicates the correlation in phase space. The leading or lagging behavior of each component is determined by ${\varphi }_{kj}$, measuring to what extent the time series k leads the time series j.

We can break down the covariance (correlation) matrix as:

$$\begin{array}{cc}\hfill \mathsf{\Sigma }=& V\mathsf{\Lambda }{V}^{-1}=V({\mathsf{\Lambda }}_{noise}+{\mathsf{\Lambda }}^F)V^{-1}\hfill \\ \hfill =& {\mathsf{\Sigma }}_{noise}+{\mathsf{\Sigma }}^F\hfill \end{array}$$

where ${\mathsf{\Lambda }}_{noise}$ is the diagonal matrix of insignificant eigenvalues, ${\mathsf{\Sigma }}^F$ and ${\mathsf{\Sigma }}_{noise}$ denote the principal and noisy components of the correlation matrix, respectively.

In the context of portfolio allocation, the Hilbert transformation technique proves valuable for identifying lead/lag correlations in the performance of different assets over time. This allows investors to uncover temporal patterns and optimize portfolio strategies by understanding the timing of movements in asset values. Essentially, it helps to discern how changes in one asset’s value relate to shifts in another asset, leading to more informed and dynamic portfolio decision-making.

3. Portfolio Selection Models

In this section, we explore the nuances of asset allocation problems. We begin by outlining the classical portfolio model, inspired by Markowitz’s foundational work, which employs the variance/covariance matrix to analyze the interdependencies among asset returns, typically estimated through a sample approach. Following this, we describe the approach rooted in network theory (see [15]), offering a novel perspective on portfolio optimization. As also observed in recent studies [18,27] across various fields of economic literature, this methodological approach presents a modern and sophisticated way to addressing challenges in traditional portfolio selection processes. It aims to unveil an optimal solution by using the interconnected relationships among different assets. Finally, we explore the application of RMT in the portfolio selection model, focusing particularly on the minimum variance portfolio.

3.1. Traditional Global Minimum Variance Portfolio

The classical Global Minimum Variance (GMV) approach is designed to optimize portfolio allocation by determining the fractions $w_i$ of a given capital to be invested in each asset i from a predetermined basket of assets. The objective is to minimize the portfolio risk, identified through its variance. In this context, n represents the number of available assets, $X_i,i=1,\cdots ,n$ symbolizes the random variable of daily returns of the i-th asset and $\mathsf{\Sigma }$ is the sample covariance matrix. The GMV strategy is formulated as follows:

$$\left\{\begin{array}{c}\underset{\mathit{w}}{min}{\mathit{w}}^{\top }\mathsf{\Sigma }\mathit{w}\hfill \\ {\mathit{w}}^{\top }\mathbf{e}=1\hfill \\ 0\le w_i\le 1,\hfill & i=1,\cdots ,n,\hfill \end{array}\right.$$

(9)

where $\mathbf{e}$ is a vector of ones of length n and $\mathit{w}={\left[w_i\right]}_{i=1,...,n}$ is the vector of the fractions invested in each asset. The first equation is the budget constraint and requires that the whole capital should be invested. Conditions $0\le w_i\le 1$, $i=1,\cdots ,n$ preclude the possibility of short selling3. It is evident that the dependence structure between assets, in [1], is assessed through the Pearson correlation coefficient between each pair of asset.

The process of estimating covariance matrices from samples is widely recognized for its susceptibility to noise and high estimation error. This introduces uncertainty and instability when attempting to estimate the true covariance structure, which frequently results in suboptimal performance of portfolios when applied out-of-sample. Portfolios based on unreliable covariance structures may exhibit increased volatility, high risk, and compromised returns. Therefore, ensuring precise and stable covariance estimates is crucial for enhancing the robustness and effectiveness of portfolio optimization models in real-world applications.

3.2. Asset Allocations through Network-Based Clustering Coefficients

To address the challenges of estimating the dependence structure and improve the reliability of covariance matrix estimation, network theory has recently been applied in portfolio allocation models. Here we employ network theory to enhance the reliability of covariance matrix estimation in portfolio allocation. This approach involves representing financial assets through an undirected graph $G=(V,E)$, where V is the set of nodes representing assets, and E is the set of edges denoting dependencies or connections between assets. The weights of the edges are represented by the Pearson correlation matrix ${\rho }_{ij}$, $i,j=1,\cdots ,n$.

At this point, problem (9) is transformed in the following:

$$\left\{\begin{array}{c}\underset{\mathit{w}}{min}{\mathit{w}}^{\top }H\mathit{w}\hfill \\ {\mathbf{e}}^{\top }\mathit{w}=1\hfill \\ 0\le w_i\le 1,\hfill & i=1,\cdots ,n.\hfill \end{array}\right.$$

(10)

where H is a matrix obtained as described below. First, we create a binary undirected adjacency matrix A, associated to the filtered (complex) correlation matrix obtained through the techniques explained in Section 2. The binarization process4 is assessed with the following threshold criteria: for every $i,j=1,\cdots ,n$

$$A_{ij}=\left\{\begin{array}{cc}1\hfill & \mathrm{if}{\rho }_{ij}^F\ge {\rho }_{ij},\hfill \\ 0\hfill & \mathrm{otherwise}\hfill \end{array}\right.,$$

(11)

where ${\rho }_{ij}^F$ are the elements of the filtered correlation matrix.

Second, we calculate for each node i the clustering coefficient $c_i$ as proposed in [28] for binary and weighted graphs.

Third, we construct the interconnectedness matrix C, whose elements are:

$$C_{ij}=\left\{\begin{array}{cc}{c}_i{c}_j\hfill & \mathrm{if}i\ne j\hfill \\ 1\hfill & \mathrm{otherwise}\hfill \end{array}\right..$$

(12)

Finally, we build $H={\Delta }^{\top }C\Delta$ to be the dependency matrix in (10), where $\Delta =\mathrm{diag}\left(s_i\right)$ is the diagonal matrix with entries $s_i=\frac{{\sigma }_i}{\sqrt{{\sum }_{i=1}^n{\sigma }_i^2}}$ as in [15]. This gives the new objective function to be minimized in order to obtain the optimum allocation.

4. Empirical Protocol and Performance Analysis

In this section, we delineate the empirical protocol employed in this paper and subsequently apply it to assess the effectiveness of the proposed approach through empirical applications. To test the robustness and mitigate data mining bias we analyze four energy portfolios, as will be explained in Subsection 4.3.

In portfolio allocation models, in-sample and out-of-sample analyses are crucial components for assessing the performance and robustness of the strategies. The analysis of portfolio strategies centers around key criteria, including portfolio diversification, transaction costs, and risk-return performance measures. This examination is conducted using a rolling window methodology characterized by an in-sample period of length l and an out-of-sample period of length m.

This methodology begins by computing optimal weights during the initial in-sample window spanning from time $t=1$ to $t=l$. These optimal weights remain unchanged during the subsequent out-of-sample period, extending from $t=l+1$ to $t=l+m$. Then the returns and performance metrics for this out-of-sample period are calculated. This process iterates by advancing both the in-sample and out-of-sample periods by m-steps. With each iteration, the weights are recalculated and portfolio performance is evaluated until the end of the dataset.

4.1. Diversification and Transaction Costs: In-Sample Analysis

We assess the performance of the obtained portfolios by analysing diversification and transaction costs, both of which play a crucial role in effective portfolio management. Diversification represents a fundamental risk management tool that seeks to optimize returns while minimizing exposure to undue risks associated with concentrated holdings. This is essential to mitigate risk and enhance the stability of an investment portfolio. By holding a diversified set of assets with potentially uncorrelated returns, investors aim to achieve a balance between risk and return. In this paper as a measure of diversification we use the modified Herfindahl index, defined as:

$$\mathrm{HI}=\frac{{\left({\mathit{w}}^{★}\right)}^{\top }{\mathit{w}}^{★}-\frac{1}{n}}{1-\frac{1}{n}},$$

where ${\mathit{w}}^{★}=\left[w_i^{★}\right],i=1,\dots ,n$ represents the vector of optimal weights. The index ranges from 0 to 1, where the value 0 is reached in case of the EW portfolio (deemed as the most diversified one) and the value of 1 indicates a portfolio concentrated in only one asset.

The second index used in this analysis is related to transaction costs which have a direct impact on portfolio performances: higher transaction costs lead to lower returns. Understanding and minimizing transaction costs is an integral component of effective portfolio management, ensuring that investment decisions are cost-efficient and align with the investor’s goals. By addressing these aspects, investors can enhance the overall competitiveness and sustainability of their investment strategies. In this paper as a proxy for transaction costs we use portfolio turnover, denoted as $\varphi$ and computed as:

$$\varphi =\sum _{i=1}^n\left|w_i^{★}-{\hspace{0.17em}}^{-}{w}_i^{★}\right|,$$

here, $w_i^{★}$ and ${\hspace{0.17em}}^{-}{w}_i^{★}$ are the optimal portfolio weights of the $i^{th}$ asset, respectively, before and after rebalancing (based on the optimization strategy). Higher turnover often comes with increased transaction costs. Monitoring turnover is crucial for understanding the level of trading activity and its impact on the portfolio’s performance. In particular, a lower turnover percentage indicates that the portfolio has experienced relatively minimal changes in asset weights. This might be indicative of a more stable managed portfolio.

4.2. Performance Measurements: Out-of-Sample Analysis

Analyzing risk-adjusted performance measures for different allocation methods is essential as it evaluates the portfolio’s success in generating returns relative to the level of risk assumed. By examining metrics like the Sharpe Ratio and Omega Ratio, investors gain valuable insights into how effectively a portfolio balances risk and return. This analysis assists investors to select the best approach that delivers favorable returns while aligning with their risk tolerance.

Here, we briefly recap the definitions of the risk-adjusted performance measures used in the empirical part of this paper to compare the different models under analysis. The Sharpe Ratio ($\mathrm{SR}$) is defined as:

$$\mathrm{SR}=\frac{\mathbb{E}\left(r_p-r_f\right)}{\sqrt{\mathbb{V}\mathrm{ar}(r_p-r_f)}},$$

where $r_p$ indicates the out-of-sample portfolio returns (obtained using the rolling window methodology) and $r_f$ denotes the risk-free rate5. This ratio indicates the mean excess return per unit of overall risk. The portfolio with the highest $\mathrm{SR}$ is typically considered the best, especially in cases of positive portfolio excess returns concerning the risk-free rate.

The Omega Ratio $\left(\mathrm{OR}\right)$, introduced in [29], is defined as:

$$\mathrm{OR}=\frac{\mathbb{E}\left[{\left(r_p-ϵ\right)}^{+}\right]}{\mathbb{E}\left[{\left(ϵ-r_p\right)}^{+}\right]},$$

where $ϵ$ is a specified threshold. Returns above the threshold are considered gains by investors, while those below are considered as losses. An $\mathrm{OR}$ greater than 1 suggests that the portfolio provides more expected gains than expected losses. The choice of threshold $ϵ$ can vary, and in our empirical analysis, it is set to 0.

4.3. Data Description and Empirical Results

In this empirical analysis, we examine four equity indexes, sourcing our data from Bloomberg. The selected indexes are as follows: (i) the Nasdaq 100 index; (ii) the energy sector of the S&P500 index, which focuses only on the energy companies; (iii) the Dow Jones Industrial Average index; (iv) the benchmark stock market index of the Bolsa de Madrid.

We construct each portfolio by including only those components listed in the specified index from January 2, 2014 to January 19, 2024. Detailed information regarding the Bloomberg tickers of the selected assets in each portfolio can be found in Appendix A. The dataset for each portfolio is composed of 2622 daily observations spanning from January 2, 2014, to January 19, 2024.

As detailed in Section 3, we address optimization problems (9) and (10), both of which account for the dependence structure in constructing the optimal portfolio. In problem (9), this structure is captured by the covariance matrix $\mathsf{\Sigma }$, which is estimated using both the sample and the shrinkage approaches. Conversely, in (10) the dependence structure is represented by the matrix H, leveraging the clustering coefficient method as outlined in [15] and along with the approach proposed in Section 2 (utilizing Hilbert transformation and RMT).

To evaluate the performances of the approaches considered in this paper, we employ four distinct buy-and-hold rolling window methodologies: the first three are 6, 12 or 24 months in-sample and 1 month out-of-sample; the last one is 24 months in-sample and 2 months out-of-sample.

After obtaining the out-of-sample returns for each approach under analysis, we then proceed to compute their respective out-of-sample performances. Figure 2 illustrates the out-of-sample performances of the four portfolios under analysis, specifically for the rolling window methodology of 6 months in-sample and 1 month out-of-sample6.

Observing Figure 2, it becomes clear that across all portfolios under analysis and in case of rolling window methodology with 6 months in-sample and 1 month out-of-sample the CC-RMT strategy exhibits superior out-of-sample performance compared to the “Sample”, “Shrink”, and “CC” strategies. This finding holds true for the other rolling window methodologies as well. However, relying solely on the out-of-sample performance as a singular index may not provide a comprehensive evaluation of the various investment strategies under analysis. This limitation arises because out-of-sample performance prioritizes returns overlooking the aspect of evaluating risk.

Therefore, a more comprehensive assessment requires the consideration of both return and risk metrics to understand the inherent risk-return trade-off in each strategy. Table 1 displays key out-of-sample statistics, including the first four moments, $\mathrm{SR}$, and $\mathrm{OR}$, for the portfolios under analysis across the four rolling window methodologies considered. While the first four moments may not clearly indicate which strategy performs better due to potential inconsistencies in mean, skewness, standard deviation, and kurtosis, a different picture emerges when examining risk-adjusted performance measures like the $\mathrm{SR}$ and $\mathrm{OR}$. In particular, the CC-RMT strategy consistently exhibits superior risk-adjusted performance across all portfolios and rolling windows considered.

In contrast to more traditional methods, the CC-RMT strategy seems to effectively capture the complex interdependencies and structural patterns within the financial markets, providing a more accurate representation of the market dynamics. Moreover, the two network approaches (CC and CC-RMT) yield differing outcomes. Although both employ the clustering coefficient to measure the dependence structure, they differentiate for the threshold criteria used in the binarization process of the correlation adjacency matrix. Unlike our approach, the authors in [15] do not employ the Hilbert transformation of time series or the RMT filtering method to identify significant correlations exclusively. Instead, they consider different thresholds on correlation levels to generate multiple binary adjacencies matrices and compute the corresponding clustering coefficient of nodes. Finally, they compute the average clustering coefficient for each node to derive the interconnectedness matrix as outlined in Equation (12).

Considering transaction costs is crucial in practical portfolio management, as excessive turnover can erode returns. As mentioned earlier, this study evaluates transaction costs by using turnover as a proxy. Figure 3 illustrates the turnover values in each rebalancing period for the four portfolios under examination, employing a rolling window of 6 months in-sample and 1 month out-of-sample. The figure clearly indicates that CC-RMT consistently yield lower portfolio turnover. To provide a comprehensive overview of portfolio turnover, Table 2 presents the average turnover for the four portfolios across all considered rolling windows. These results consistently highlight that CC-RMT tends to produce portfolios with lower turnover, leading to reduced transaction costs.

We have additionally computed the modified Herfindahl index as a proxy for assessing portfolio diversification. Due to space limitations, the average modified Herfindahl index for the analyzed portfolios and rolling window methodologies is presented on the right-hand side of Table 2. Notably, the findings reveal that the CC-RMT approach consistently produces a lower modified Herfindahl index, indicating a more diversified portfolio. This observation highlights the contribution of the CC-RMT model to enhancing diversification within the portfolio, a crucial aspect of effective risk management.

5. Discussions

This study introduces a novel method for estimating the dependence structure, a critical aspect in portfolio selection. It leverages the Hilbert transformation technique for data preprocessing to improve the identification of lead/lag correlations in return time series. Following this, it applies principles from Random Matrix Theory (RMT) to differentiate between noise and meaningful information in the covariance matrix. By integrating these two techniques, the study enables a more accurate and resilient estimation of the portfolio allocation process. The proposed approach aims to obtain a denoised version of the dependence structure, capturing the genuine connections between assets more precisely. The empirical analysis demonstrates that the resulting CC-RMT model not only reduces transaction costs, as evidenced by lower turnover, and enhances portfolio diversification by decreasing the modified Herfindahl index, but also excels in risk-adjusted performance measures. Across different portfolios and rolling windows, the CC-RMT strategy consistently exhibits superior risk-adjusted performance compared to other approaches, including the sample, shrinkage, and clustering coefficient strategies (as discussed in [15]). The CC-RMT model provides investors with a clearer understanding of market dynamics, offering a well-informed framework for making investment decisions. This research significantly contributes to portfolio management by providing insights into the underlying structures and dynamics of investments. It assists investors in making smarter and more effective investment decisions by providing valuable insights and guiding them towards strategies that are better adjusted for risk.