This paper discusses the practical aspects of working with high-dimensional shrinkage portfolios. It presents the R package HDShOP which provides a comprehensive framework for such work. In particular, we cover the construction of portfolios using shrinkage-based estimators for the mean vector, covariance matrix, and precision matrix of asset returns, as well as the shrinkage estimators derived directly for the weights of optimal portfolios. Moreover, shrinkage-based tests on the mean-variance efficiency of a given portfolio are discussed. Aspects related to programming, such as classes and methods used in the construction of optimal portfolios, are described. The description of the software is preceded by underlying theory and it is accompanied by several empirical illustrations based on the data consisting of returns on stocks from the S&P 500 index.

In this paper, we analyze the asymptotic behavior of the main characteristics of the mean-variance efficient frontier employing random matrix theory. Our particular interest covers the case when the dimension p and the sample size n tend to infinity simultaneously and their ratio p/n tends to a positive constant (Formula presented.). We neither impose any distributional nor structural assumptions on the asset returns. For the developed theoretical framework, some regularity conditions, like the existence of the 4th moments, are needed. It is shown that two out of three quantities of interest are biased and overestimated by their sample counterparts under the high-dimensional asymptotic regime. This becomes evident based on the asymptotic deterministic equivalents of the sample plug-in estimators. Using them we construct consistent estimators of the three characteristics of the efficient frontier. Furthermore, the asymptotic normality of the considered estimators of the parameters of the efficient frontier is proved. Verifying the theoretical results based on an extensive simulation study we show that the proposed estimator for the efficient frontier is a valuable alternative to the sample estimator for high dimensional data. Finally, we present an empirical application, where we estimate the efficient frontier based on stocks from the S&P 500 index.

This paper is concerned with deriving a new test on a covariance matrix which is based on its nonlinear shrinkage estimator. The distribution of the test statistic is deduced under the null hypothesis in the large-dimensional setting, that is, when p/n tend to some positive constant c with p variables and n samples both tending to infinity. The theoretical results are illustrated by means of an extensive simulation study where the new nonlinear shrinkage-based test is compared with existing approaches, in particular with the commonly used corrected likelihood ratio test, the corrected John test, and the test based on the linear shrinkage approach. It is demonstrated that the new nonlinear shrinkage test possesses better power properties under heteroscedastic alternative.

Consider a random vector y = Σ ^1/2 x, where the p elements of the vector x are i.i.d. real-valued random variables with zero mean and finite fourth moment, and Σ ^1/2 is a deterministic p × p matrix such that the eigenvalues of the population correlation matrix R of y are uniformly bounded away from zero and infinity. In this paper, we find that the log determinant of the sample correlation matrix R based on a sample of size n from the distribution of y satisfies a CLT (central limit theorem) for p/n → γ ∈ (0, 1] and p ≤ n. Explicit formulas for the asymptotic mean and variance are provided. In case the mean of y is unknown, we show that after re-centering by the empirical mean the obtained CLT holds with a shift in the asymptotic mean. This result is of independent interest in both large dimensional random matrix theory and high-dimensional statistical literature of large sample correlation matrices for non-normal data. Finally, the obtained findings are applied for testing of uncorrelatedness of p random variables. Surprisingly, in the null case R = I, the test statistic becomes distribution-free and the extensive simulations show that the obtained CLT also holds if the moments of order four do not exist at all, which conjectures a promising and robust test statistic for heavy-tailed high-dimensional data.

The chapter is concerned with finding the asymptotic distribution of the estimated shrinkage intensity used in the definition of the linear shrinkage estimator of the covariance matrix, derived by Bodnar et al. (J Multivar Anal 132:215–228, 2014). As a result, a new test statistic is proposed which is deduced from the linear shrinkage estimator. This result is a ready-to-use multivariate hypothesis test in the large-dimensional asymptotic framework and constitutes the main result of the chapter. The theoretical findings are compared by means of a simulation study with existing tests, in particular with the commonly used corrected likelihood ratio test and the corrected John test, both derived by Wang and Yao (Electron J Stat 7:2164–2192, 2013).

In this paper, we show the central limit theorem for the logarithmic determinant of the sample correlation matrix R constructed from the (p × n)-dimensional data matrix X containing independent and identically distributed random entries with mean zero, variance one and infinite fourth moments. Precisely, we show that for p/n → γ ∈ (0, 1) as n, p → ∞ the logarithmic law log det R − (p − n + ¹ ₂ )log(1 − p/n) + p − p/n _{→d N(0, 1)} ^√−2 log(1 − p/n) − 2p/n is still valid if the entries of the data matrix X follow a symmetric distribution with a regularly varying tail of index α ∈ (3, 4). The latter assumptions seem to be crucial, which is justified by the simulations: if the entries of X have the infinite absolute third moment and/or their distribution is not symmetric, the logarithmic law is not valid anymore. The derived results highlight that the logarithmic determinant of the sample correlation matrix is a very stable and flexible statistic for heavy-tailed big data and open a novel way of analysis of high-dimensional random matrices with self-normalized entries.

In the intensive care unit (ICU), infection-related mortality is high. Although adequate antibiotic treatment is essential in infections, beta-lactam target non-attainment occurs in up to 45% of ICU patients, which is associated with a lower likelihood of clinical success. To optimize antibiotic treatment, we aimed to develop beta-lactam target non-attainment prediction models in ICU patients. Patients from two multicenter studies were included, with intravenous intermittent beta-lactam antibiotics administered and blood samples drawn within 12–36 h after antibiotic initiation. Beta-lactam target non-attainment models were developed and validated using random forest (RF), logistic regression (LR), and naïve Bayes (NB) models from 376 patients. External validation was performed on 150 ICU patients. We assessed performance by measuring discrimination, calibration, and net benefit at the default threshold probability of 0.20. Age, sex, serum creatinine, and type of beta-lactam antibiotic were found to be predictive of beta-lactam target non-attainment. In the external validation, the RF, LR, and NB models confirmed good discrimination with an area under the curve of 0.79 [95% CI 0.72–0.86], 0.80 [95% CI 0.73–0.87], and 0.75 [95% CI 0.67–0.82], respectively, and net benefit in the RF and LR models. We developed prediction models for beta-lactam target non-attainment within 12–36 h after antibiotic initiation in ICU patients. These online-accessible models use readily available patient variables and help optimize antibiotic treatment. The RF and LR models showed the best performance among the three models tested.

In this paper, new results in random matrix theory are derived, which allow us to construct a shrinkage estimator of the global minimum variance (GMV) portfolio when the shrinkage target is a random object. More specifically, the shrinkage target is determined as the holding portfolio estimated from previous data. The theoretical findings are applied to develop theory for dynamic estimation of the GMV portfolio, where the new estimator of its weights is shrunk to the holding portfolio at each time of reconstruction. Both cases with and without overlapping samples are considered in the paper. The non-overlapping samples corresponds to the case when different data of the asset returns are used to construct the traditional estimator of the GMV portfolio weights and to determine the target portfolio, while the overlapping case allows intersections between the samples. The theoretical results are derived under weak assumptions imposed on the data-generating process. No specific distribution is assumed for the asset returns except from the assumption of finite 4+ɛ, ɛ >0, moments. Also, the population covariance matrix with unbounded largest eigenvalue can be considered. The performance of new trading strategies is investigated via an extensive simulation. Finally, the theoretical findings are implemented in an empirical illustration based on the returns on stocks included in the S&P 500 index.

In this paper, we derive an analytical solution to the dynamic optimal portfolio choice problem in the case of an investor equipped with a power utility function of wealth. The results are established by solving the Bellman backward recursion under the assumption that the vector of asset returns follows a vector-autoregressive process with predictable variables. In an empirical study, the performance of the derived solution is compared with the one obtained by applying the numerical method. The comparison is performed in terms of the final wealth and its expected utility. It is documented that the application of the analytical solution to the multi-period portfolio choice problem leads to higher values of both the final wealth and the expected utility.

The main contribution of this paper is the derivation of the asymptotic behavior of the out-of-sample variance, the out-of-sample relative loss, and of their empirical counterparts in the high-dimensional setting, i.e., when both ratios p/n and p/m tend to some positive constants as m→∞ and n→∞, where p is the portfolio dimension, while n and m are the sample sizes from the in-sample and out-of-sample periods, respectively. The results are obtained for the traditional estimator of the global minimum variance (GMV) portfolio and for the two shrinkage estimators introduced by Frahm and Memmel (2010) and Bodnar et al. (2018). We show that the behavior of the empirical out-of-sample variance may be misleading in many practical situations, leading, for example, to a comparison of zeros. On the other hand, this will never happen with the empirical out-of-sample relative loss, which seems to provide a natural normalization of the out-of-sample variance in the high-dimensional setup. As a result, an important question arises if the out-of-sample variance can safely be used in practice for portfolios constructed from a large asset universe.

Recently, the shrinkage approach has increased its popularity in theoretical and applied statistics, especially, when point estimators for high-dimensional quantities have to be constructed. A shrinkage estimator is usually obtained by shrinking the sample estimator towards a deterministic target. This allows to reduce the high volatility that is commonly present in the sample estimator by introducing a bias such that the mean-square error of the shrinkage estimator becomes smaller than the one of the corresponding sample estimator. The procedure has shown great advantages especially in the high-dimensional problems where, in general case, the sample estimators are not consistent without imposing structural assumptions on model parameters. In this paper, we review the mostly used shrinkage estimators for the mean vector, covariance and precision matrices. The application in portfolio theory is provided where the weights of optimal portfolios are usually determined as functions of the mean vector and covariance matrix. Furthermore, a test theory on the mean–variance optimality of a given portfolio based on the shrinkage approach is presented as well.

Optimal portfolio selection problems are determined by the (unknown) parameters of the data generating process. If an investor wants to realize the position suggested by the optimal portfolios, he/she needs to estimate the unknown parameters and to account for the parameter uncertainty in the decision process. Most often, the parameters of interest are the population mean vector and the population covariance matrix of the asset return distribution. In this paper, we characterize the exact sampling distribution of the estimated optimal portfolio weights and their characteristics. This is done by deriving their sampling distribution by its stochastic representation. This approach possesses several advantages, e.g. (i) it determines the sampling distribution of the estimated optimal portfolio weights by expressions, which could be used to draw samples from this distribution efficiently; (ii) the application of the derived stochastic representation provides an easy way to obtain the asymptotic approximation of the sampling distribution. The later property is used to show that the high-dimensional asymptotic distribution of optimal portfolio weights is a multivariate normal and to determine its parameters. Moreover, a consistent estimator of optimal portfolio weights and their characteristics is derived under the high-dimensional settings. Via an extensive simulation study, we investigate the finite-sample performance of the derived asymptotic approximation and study its robustness to the violation of the model assumptions used in the derivation of the theoretical results.

In this article, we estimate the mean-variance portfolio in the high-dimensional case using the recent results from the theory of random matrices. We construct a linear shrinkage estimator which is distribution-free and is optimal in the sense of maximizing with probability one the asymptotic out-of-sample expected utility, that is, mean-variance objective function for different values of risk aversion coefficient which in particular leads to the maximization of the out-of-sample expected utility and to the minimization of the out-of-sample variance. One of the main features of our estimator is the inclusion of the estimation risk related to the sample mean vector into the high-dimensional portfolio optimization. The asymptotic properties of the new estimator are investigated when the number of assets p and the sample size n tend simultaneously to infinity such that p/n→c∈(0,+∞). The results are obtained under weak assumptions imposed on the distribution of the asset returns, namely the existence of the 4+ε moments is only required. Thereafter we perform numerical and empirical studies where the small- and large-sample behavior of the derived estimator is investigated. The suggested estimator shows significant improvements over the existent approaches including the nonlinear shrinkage estimator and the three-fund portfolio rule, especially when the portfolio dimension is larger than the sample size. Moreover, it is robust to deviations from normality.

In this paper, using the shrinkage-based approach for portfolio weights and modern results from random matrix theory we construct an effective procedure for testing the efficiency of the expected utility (EU) portfolio and discuss the asymptotic behavior of the proposed test statistic under the high-dimensional asymptotic regime, namely when the number of assets p increases at the same rate as the sample size n such that their ratio p/n approaches a positive constant cin (0,1) as nto infty. We provide an extensive simulation study where the power function and receiver operating characteristic curves of the test are analyzed. In the empirical study, the methodology is applied to the returns of S&P 500 constituents.

We derive new results related to the portfolio choice problem for power and logarithmic utilities. Assuming that the portfolio returns follow an approximate log-normal distribution, the closed-form expressions of the optimal portfolio weights are obtained for both utility functions. Moreover, we prove that both optimal portfolios belong to the set of mean-variance feasible portfolios and establish necessary and sufficient conditions such that they are mean-variance efficient. Furthermore, we extend the derived theoretical finding to the general class of the log-skew-normal distributions. Finally, an application to the stock market is presented and the behaviour of the optimal portfolio is discussed for different values of the relative risk aversion coefficient. It turns out that the assumption of log-normality does not seem to be a strong restriction.

We consider the estimation of the multi-period optimal portfolio obtained by maximizing an exponential utility. Employing the Jeffreys non-informative prior and the conjugate informative prior, we derive stochastic representations for the optimal portfolio weights at each time point of portfolio reallocation. This provides a direct access not only to the posterior distribution of the portfolio weights but also to their point estimates together with uncertainties and their asymptotic distributions. Furthermore, we present the posterior predictive distribution for the investor's wealth at each time point of the investment period in terms of a stochastic representation for the future wealth realization. This in turn makes it possible to use quantile-based risk measures or to calculate the probability of default, i.e the probability of the investor wealth to become negative. We apply the suggested Bayesian approach to assess the uncertainty in the multi-period optimal portfolio by considering assets from the FTSE 100 in the weeks after the British referendum to leave the European Union. The behaviour of the novel portfolio estimation method in a precarious market situation is illustrated by calculating the predictive wealth, the risk associated with the holding portfolio, and the probability of default in each period.

A reflexive generalized inverse and the Moore-Penrose inverse are often confused in statistical literature but in fact they have completely different behaviour in case the population covariance matrix is not a multiple of identity. In this paper, we study the spectral properties of a reflexive generalized inverse and of the Moore-Penrose inverse of the sample covariance matrix. The obtained results are used to assess the difference in the asymptotic behaviour of their eigenvalues.

The paper solves the problem of optimal portfolio choice when the parameters of the asset returns distribution, for example the mean vector and the covariance matrix, are unknown and have to be estimated by using historical data on asset returns. Our new approach employs the Bayesian posterior predictive distribution which is the distribution of future realizations of asset returns given the observable sample. The parameters of posterior predictive distributions are functions of the observed data values and, consequently, the solution of the optimization problem is expressed in terms of data only and does not depend on unknown quantities. By contrast, the optimization problem of the traditional approach is based on unknown quantities which are estimated in the second step, and lead to a suboptimal solution. We also derive a very useful stochastic representation of the posterior predictive distribution whose application not only gives the solution of the considered optimization problem, but also provides the posterior predictive distribution of the optimal portfolio return which can be used to construct a prediction interval. A Bayesian efficient frontier, the set of optimal portfolios obtained by employing the posterior predictive distribution, is constructed as well. Theoretically and using real data we show that the Bayesian efficient frontier outperforms the sample efficient frontier, a common estimator of the set of optimal portfolios which is known to be overoptimistic.

In this paper, we consider the asymptotic distributions of functionals of the sample covariance matrix and the sample mean vector obtained under the assumption that the matrix of observations has a matrix-variate location mixture of normal distributions. The central limit theorem is derived for the product of the sample covariance matrix and the sample mean vector. Moreover, we consider the product of the inverse sample covariance matrix and the mean vector for which the central limit theorem is established as well. All results are obtained under the large-dimensional asymptotic regime, where the dimension p and the sample size n approach infinity such that p/n→c ∈ [0, + ∞) when the sample covariance matrix does not need to be invertible and p/n→c ∈ [0,1) otherwise.