This vignette provides a “jump-start” for users that want to quicklyunderstand how to use entropy-pooling (EP) to constructviewson the market distribution. The methodology fits well in a variety ofscenarios: stress-testing, macro factors, non-linear payoffs, andmore.
To demonstrate entropy-pooling’s firepower, the current vignette usessome of the cases Meucci (2008)1 leaves for the reader in the appendix. Inparticular, the usage of the following functions are covered indepth:
view_on_mean()view_on_volatility()view_on_correlation()view_on_rank()view_on_marginal_distribution()view_on_copula()Along with the examples, theEuStockMarkets dataset -that comes with the installation ofR - is used as aproxy for “the market”:
library(ffp)library(ggplot2)set.seed(321)# invariance / stationarityx<-diff(log(EuStockMarkets))dim(x)#> [1] 1859 4head(x)#> DAX SMI CAC FTSE#> [1,] -0.009326550 0.006178360 -0.012658756 0.006770286#> [2,] -0.004422175 -0.005880448 -0.018740638 -0.004889587#> [3,] 0.009003794 0.003271184 -0.005779182 0.009027020#> [4,] -0.001778217 0.001483372 0.008743353 0.005771847#> [5,] -0.004676712 -0.008933417 -0.005120160 -0.007230164#> [6,] 0.012427042 0.006737244 0.011714353 0.008517217Assume an investor believes theFTSE index return willbe\(20\%\) above average in the nearfuture. In order to process thisview, the investor needs tominimize the relative entropy:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j))\]
Subject to the restriction:
\[ \ x_j V_{j, k} = \mu_{k} \]
In which\(x_{j}\) is a yet to bediscoveredposterior probability;\(V_{j,k}\) is a\(1859 \times 1\) vector with theFTSE returns; and\(\mu_{k}\) is a\(1 \times 1\) scalar with the investorprojected return for theFTSE index. In this case, the\(k\) subscript represents the fourthcolumn inx2.
Views on expected returns can be constructed withview_on_mean:
# Returns 20% higher than averagemu<-mean(x[ ,"FTSE"])*1.2# ffp views constructorviews<-view_on_mean(x =as.matrix(x[ ,"FTSE"]),mean = mu)views#> # ffp view#> Type: View On Mean#> Aeq : Dim 1 x 1859#> beq : Dim 1 x 1Theviews object is alist with twocomponents,Aeq andbeq, that are equivalentto the elements\(V_{j,k}\) transposedand\(\mu_{k}\), respectively.
The investor also needs to formulate a vector ofpriorprobabilities,\(p_j\), which isusually - but not necessary - a equal-weight vector:
# Prior probabilitiesp_j<-rep(1/nrow(x),nrow(x))Once theprior and theviews are established, theoptimization can take place withentropy_pooling:
# Solve Minimum Relative Entropy (MRE)ep<-entropy_pooling(p = p_j,Aeq = views$Aeq,beq = views$beq,solver ="nlminb")ep#> <ffp[1859]>#> 0.0005425631 0.0005340012 0.000544236 0.0005418246 0.0005322989 ... 0.0005451271What is the interpretation forep? Among all thepossible probability vectors,ep is the one that cansatisfy theviews in the “least” evasive way (in which the term“least evasive” is direct reference to theprior). Therefore,ep is the best candidate for aposteriordistribution.
In real world, the investor can have manyviews. Extendingthe current example, say the investor has a new belief: theCAC index returns will be\(10\%\) bellow average3:
mu<-c(mean(x[ ,"FTSE"])*1.2,# Return 20% higher than averagemean(x[ ,"CAC"])*0.9# Return 10% bellow average)# ffp views constructorviews<-view_on_mean(x =as.matrix(x[ ,c("FTSE","CAC")]),mean = mu)views#> # ffp view#> Type: View On Mean#> Aeq : Dim 2 x 1859#> beq : Dim 2 x 1In which the elementsAeq andbeq now have\(2\) rows each, one for everyview.
The minimum relative entropy problem is solved in the exact sameway:
# Solve MREep<-entropy_pooling(p = p_j,Aeq = views$Aeq,beq = views$beq,solver ="nlminb")ep#> <ffp[1859]>#> 0.0005602371 0.0005473082 0.0005573006 0.0005384979 0.000531066 ... 0.0005434801It’s easier to analyse the output ofep visually.
class(ep)#> [1] "ffp" "vctrs_vctr"To that end, theautoplot method is available forobjects of theffp class:
autoplot(ep)+scale_color_viridis_c(option ="C",end =0.75)+labs(title ="Posterior Probability Distribution",subtitle ="Views on Expected Returns",x =NULL,y =NULL)The plot reinforces the intuition of\(x_j\) - theposterior probability- as a distribution that distorts theprior in order toaccommodate theviews.
It’s easy to double-check the investorviews by computingthe ratio of theconditional vs. unconditionalreturns:
conditional<-ffp_moments(x, ep)$muunconditional<-colMeans(x)conditional/ unconditional-1#> DAX SMI CAC FTSE#> 0.01335825 0.02078130 -0.09999999 0.20000009According to expectations,CAC outlook is now\(10\%\) bellow average andFTSEexpected return is\(20\%\) aboveaverage.Violà!
Say the investor is confident that markets will be followed by aperiod of lull and bonanza. Without additional insights about expectedreturns, the investor anticipates that market volatility for theCAC index will drop by\(10\%\).
To imposeviews on volatility, the investor needs tominimize the expression:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j))\] Subject to the constraint:
\[ \sum_{j=1}^{J} x_j V_{j, k}^2 =\mu_{k}^2 + \sigma_{k}^2 \]
In which,\(x_j\) is a vector of yetto be definedposterior probabilities;\(V_{j,k}^2\) is a\(1859 \times 1\) vector withCAC squared returns;\(\mu_{k}^2\) and\(\sigma_{k}^2\) are theCACsample mean and variance; and\(k\) isthe column position ofCAC in the panel\(V\).
Theview_on_volatility function can be used for thispurpose:
# opinionvol_cond<-sd(x[ ,"CAC"])*0.9# viewsviews<-view_on_volatility(x =as.matrix(x[ ,"CAC"]),vol = vol_cond)views#> # ffp view#> Type: View On Volatility#> Aeq : Dim 1 x 1859#> beq : Dim 1 x 1Theviews object holds a list with two components -Aeq andbeq - that are equivalent to theelements\(V_{j,k}^2\) transposed and\(\mu_{k}^2 + \sigma_{k}^2\),respectively.
To solve the relative entropy use theentropy_poolingfunction:
ep<-entropy_pooling(p = p_j,Aeq = views$Aeq,beq = views$beq,solver ="nlminb")ep#> <ffp[1859]>#> 0.0005227495 0.0004701207 0.0005609239 0.0005476659 0.0005631671 ... 0.0005349393Once again, theep vector is what we callposterior: the probability vector that causes the “least”amount of distortion in theprior, but still obeys theconstraints (views).
Theposterior probabilities can be observed with theautoplot method:
autoplot(ep)+scale_color_viridis_c(option ="C",end =0.75)+labs(title ="Posterior Probability Distribution",subtitle ="View on Volatility",x =NULL,y =NULL)To check if theposterior probabilities are valid as a wayto display the investor’sview, compare theconditional vs. unconditional volatilities:
unconditional<-apply(x,2, sd)conditional<-sqrt(diag(ffp_moments(x, ep)$sigma))conditional/ unconditional-1#> DAX SMI CAC FTSE#> -0.08377407 -0.06701178 -0.09977906 -0.04552477Theposterior volatility for theCAC index isreduced by\(10\%\), in line with theinvestor’s subjective judgment. However, by emphasizing tranquil periodsmore often, other assets are also affected, but in smallermagnitudes.
Assume the investor believes the correlation betweenFTSE andDAX will increase by\(30\%\), from\(0.64\) to\(0.83\). To constructviews on thecorrelation matrix, the general expression has to be minimized:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j))\] Subject to the constraints:
\[ \sum_{j=1}^{J} x_j V_{j, k} V_{j, l} =\mu_{k} \mu_{l} + \sigma_{k} \sigma_{l} C_{k,l} \]
In which, the term\(V_{j, k} V_{j,l}\) on the left hand-side of the restriction consists ofcross-correlations among assets; the terms\(\mu_{k} \mu_{l} + \sigma_{k} \sigma_{l}C_{k,l}\) carry the investor target correlation structure; and\(x_j\) is a yet to be definedprobability vector.
To formulate thisview, first compute theunconditional correlation matrix. Second, add a “perturbation”in the corresponding element that is consistent with the perceivedview:
cor_view<-cor(x)# unconditional correlation matrixcor_view["FTSE","DAX"]<-0.83# perturbation (investor belief)cor_view["DAX","FTSE"]<-0.83# perturbation (investor belief)Finally, pass the adjusted correlation matrix into theview_on_correlation function:
views<-view_on_correlation(x = x,cor = cor_view)views#> # ffp view#> Type: View On Correlation#> Aeq : Dim 10 x 1859#> beq : Dim 10 x 1In whichAeq is equal to\(V_{j, k} V_{j, l}\) transposed andbeq is a\(10 \times 1\)vector with\(\mu_{k} \mu_{l} + \sigma_{k}\sigma_{l} C_{k,l}\).
Notice that even though the investor has aview in just oneparameter4 the constraint vector has\(10\) rows, because every element of thelower/upper diagonal of the correlation matrix has to match.
Once again, the minimum entropy is solved byentropy_pooling:
ep<-entropy_pooling(p = p_j,Aeq = views$Aeq,beq = views$beq,solver ="nlminb")ep#> <ffp[1859]>#> 6.709143e-05 0.000659403 0.001085763 0.0003254822 0.0005215729 ... 0.0004748052And theautoplot method is available:
autoplot(ep)+scale_color_viridis_c(option ="C",end =0.75)+labs(title ="Posterior Probability Distribution",subtitle ="View on Correlation",x =NULL,y =NULL)To fact-check the investorview, compute theposterior correlation matrix:
cov2cor(ffp_moments(x,p = ep)$sigma)#> DAX SMI CAC FTSE#> DAX 1.0000000 0.6932767 0.7317714 0.8346939#> SMI 0.6932767 1.0000000 0.6260098 0.6113109#> CAC 0.7317714 0.6260098 1.0000000 0.6535585#> FTSE 0.8346939 0.6113109 0.6535585 1.0000000And notice that the linear association betweenFTSE andDAX is now\(0.83\)!
Assume the investor believes theDAX index willoutperform theSMI by some amount, but he doesn’t know byhow much. If the investor has only a mildview on theperformance of assets, he may want to minimize the followingexpression:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j))\] Subject to the constraint:
\[ \sum_{j=1}^{J} x_j (V_{j, k} - V_{j,l}) \ge 0 \] In which, the\(x_j\) is a yet to be determined probabilityvector;\(V_{j, k}\) is a\(1859 \times 1\) column vector withDAX returns and\(V_{j,l}\) is a\(1859 \times 1\)column vector with theSMI returns. In this case, the\(k\) and\(l\) subscripts refers to the columnpositions ofDAX andSMI in the objectx, respectively.
Views on relative performance can be imposed withview_on_rank:
views<-view_on_rank(x = x,rank =c(2,1))views#> # ffp view#> Type: View On Rank#> A : Dim 1 x 1859#> b : Dim 1 x 1Assets that are expected to outperform enter in therankargument with their column positions to the right: assets in the leftunderperform and assets in the right outperform. Because the investorbelieves thatDAX\(\geq\)SMI (the asset in the first column will outperform theasset in the second column), we fillrank = c(2, 1)5.
The optimization is, once again, guided by theentropy_pooling function:
ep<-entropy_pooling(p = p_j,A = views$A,b = views$b,solver ="nloptr")ep#> <ffp[1859]>#> 0.0005145646 0.0005403155 0.0005470049 0.0005330238 0.0005446858 ... 0.0005469164Two important observations:
nlminb solver. Usenloptr orsolnl, instead;A andb to be fulfilled rather thanAeq andbeq.Theposterior probabilities are presented bellow:
autoplot(ep)+scale_color_viridis_c(option ="C",end =0.75)+labs(title ="Posterior Probability Distribution",subtitle ="View on Ranking/Momentum",x =NULL,y =NULL)To fact-check the ongoingview, compare theconditional vs. unconditional returns:
conditional<-ffp_moments(x, ep)$muunconditional<-colMeans(x)conditional/ unconditional-1#> DAX SMI CAC FTSE#> 0.17274275 -0.06507207 0.13576463 0.06261163Indeed, returns forDAX are adjusted upwards by\(17\%\), while returns forSMIare revised downwards by\(6.5\%\).Notice that returns forCAC andFTSE were alsoaffected. This behavior could be tamed by combining the rankingcondition withviews on expected returns. See the functionbind_views() for an example.
Assume the investor has aview on the entire marginaldistribution. One way to addviews on marginal distributions isby matching the first moments exactly, up to a given order.Mathematically, the investor minimizes the relative entropy:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j))\] With, say, two equality constraints (one for\(\mu\) and one for\(\sigma^2\)):
\[ \sum_{j=1}^J x_jV_{j,k} = \sum_{j=z}^Z p_z \hat{V}_{z,k} \]\[ \sum_{j=1}^J x_j (V_{j,k})^2 = \sum_{z=1}^Zp_z (\hat{V}_{z,k})^2 \]
In which\(x_j\) is a yet to bedefined probability vector;\(V_{j,k}\)is a matrix with the empirical marginal distributions;\(p_z\) is vector ofpriorprobabilities; and\(\hat{V}_{z,k}\) anexogenous panel with simulations that are consistent with the investor’sviews.
When\(j = z\), the panels\(V_{j,k}\) and\(\hat{V}_{z,k}\) have the same number ofrows and the dimensions in both sides of the restrictions match.However, it’s possible to set\(z \gej\) to simulate a larger panel for\(\hat{V}_{z, k}\). Keep in mind though that,if\(z \ne j\), twopriorprobabilities will have to be specified: one for\(p_j\) (the objective function) and one for\(p_z\) (theviews).
Continuing on the example, consider the margins ofx canbe approximated by a symmetric multivariate t-distribution. If this isthe case, the estimation can be conducted by the amazingghyp package, that covers the entire family ofgeneralizedhyperbolic distributions:
library(ghyp)# multivariate t distributiondist_fit<-fit.tmv(data = x,symmetric =TRUE,silent =TRUE)dist_fit#> Symmetric Student-t Distribution:#>#> Parameters:#> nu#> 6.151241#>#> mu:#> [1] 0.0007899665 0.0009594760 0.0004790250 0.0003811669#>#> sigma:#> DAX SMI CAC FTSE#> DAX 1.000013e-04 6.046969e-05 7.933384e-05 5.072439e-05#> SMI 6.046969e-05 8.062728e-05 5.869208e-05 4.119624e-05#> CAC 7.933384e-05 5.869208e-05 1.216927e-04 5.715865e-05#> FTSE 5.072439e-05 4.119624e-05 5.715865e-05 6.398099e-05#>#> gamma:#> [1] 0 0 0 0#>#> log-likelihood:#> 26370.73#>#>#> Call:#> fit.tmv(data = x, symmetric = TRUE, silent = TRUE)The investor then, can construct a large panel with statisticalproperties similar to the margins he envisions:
set.seed(123)# random numbers from `dist_fit`simul<-rghyp(n =100000,object = dist_fit)Where the\(100.000 \times 4\)matrix is used to match theviews withview_on_marginal_distribution:
p_z<-rep(1/100000,100000)views<-view_on_marginal_distribution(x = x,simul = simul,p = p_z)views#> # ffp view#> Type: View On Marginal Distribution#> Aeq : Dim 8 x 1859#> beq : Dim 8 x 1The objectssimul andp_z corresponds tothe terms\(\hat{V}_{z,k}\) transposedand\(p_z\), respectively. Note that\(8\) restrictions are created, fourfor each moment of the marginal distribution (there are four assets in“the market”).
With theprior and theviews at hand, theoptimization can be quickly implemented:
ep<-entropy_pooling(p = p_j,Aeq = views$Aeq,beq = views$beq,solver ="nloptr")ep#> <ffp[1859]>#> 0.000528748 0.0005678894 0.000536657 0.0005231905 0.0005349765 ... 0.0005324979As the visual inspection of the vectorep:
autoplot(ep)+scale_color_viridis_c(option ="C",end =0.75)+labs(title ="Posterior Probability Distribution",subtitle ="View on Marginal Distribution",x =NULL,y =NULL)It’s easy to extend the current example. For instance, the investorcould “tweak” some of the fitted parameters for stress-testing(i.e. change the degrees of freedom, increase/decrease expected returns,etc) to verify the immediate impact on the P&L (VaR, CVaR, etc).
Acknowledge that when the restrictions are to harsh, the scenarioswill be heavily distorted to satisfy theviews. In this case,one may need to compute the effective number of scenarios (ENS). This isvery easy to do withens():
ens(ep)#> [1] 1857.356Rest on the investor to calibrate this statistic in order to get adesired confidence level6.
Assume the investor would like to simulate a different dependencestructure for the market.Views that change the interdependenceof assets can be implemented by minimizing the relative entropy:
\[ \sum_{j=1}^J x_j(ln(x_j) - ln(p_j))\] Subject to the following restrictions:
\[ \sum_{j=1}^J x_i U_{j,k} = 0.5\]\[ \sum_{j=1}^J x_j U_{j,k}U_{j,l} = \sum_{z=1}^Zp_z \hat{U}_{z,k}\hat{U}_{z,l} \]
\[ \sum_{j=1}^J x_jU_{j,k}U_{j,l}U_{j,i} = \sum_{j=1}^J p_j\hat{U}_{j,k}\hat{U}_{j,l}\hat{U}_{j,i} \]
In which, the first restriction matches the first moment of theuniform distribution; the second and third restrictions pair thecross-moments of the empirical copula,\(U\), with the simulated copula,\(\hat{U}\);\(x_j\) is a yet to be discoveredposterior distribution;\(p_z\) is aprior probability; When\(j = z\), the dimensions of\(p_j\) and\(p_z\) match.
Among many of the available copulas, say the investor wants to modelthe dependence of the market as aclaytoncopula to ensure the lowertail dependencydoes not go unnoticed. The estimation is simple to implement with thepackagecopula:
library(copula)# copula (pseudo-observation)u<-pobs(x)# copula estimationcop<-fitCopula(copula =claytonCopula(dim =ncol(x)),data = u,method ="ml")# simulationr_cop<-rCopula(n =100000,copula =claytonCopula(param = cop@estimate,dim =4))First, the copulas are computed component-wise by applying themarginal empirical distribution in every\(k\) column of the dataset. Second, thepseudo-observations are used to fit a Clayton copula by theMaximum-Likelihood (ML) method. Finally, a large panel with dimension\(100.000 \times 4\) is constructed tomatch a Clayton copula with\(\hat \alpha =1.06\), as in the objectcop.
Theviews on the market is constructed with theview_on_copula:
views<-view_on_copula(x = u,simul = r_cop,p = p_z)views#> # ffp view#> Type: View On Copula#> Aeq : Dim 34 x 1859#> beq : Dim 34 x 1Once again, the solution of the minimum relative entropy can be foundin one line of code:
ep<-entropy_pooling(p = p_j,Aeq = views$Aeq,beq = views$beq,solver ="nloptr")ep#> <ffp[1859]>#> 0.0001990935 0.0004161479 0.0008166059 0.0006512624 0.0003776943 ... 0.0003269579And theautoplot method is available for the objects oftheffp class:
autoplot(ep)+scale_color_viridis_c(option ="C",end =0.75)+labs(title ="Posterior Probability Distribution",subtitle ="View on Copulas",x =NULL,y =NULL)To stretch the current example, assume the investor stresses theparameter\(\hat \alpha = 1.06\)7. Higher\(\alpha\) values are linked to extremeoccurrences on the left tails of the distribution. Whence, consider thecase where\(\hat \alpha = 5\):
cop@estimate<-5Rerun the previous steps (simulation,views andoptimization):
# simulationr_cop<-rCopula(n =100000,copula =claytonCopula(param = cop@estimate,dim =4))# #viewsviews<-view_on_copula(x = u,simul = r_cop,p = p_z)# relative entropyep<-entropy_pooling(p = p_j,Aeq = views$Aeq,beq = views$beq,solver ="nloptr")To find the newposterior probability vector the satisfy thestress-test condition:
autoplot(ep)+scale_color_viridis_c(option ="C",end =0.75)+labs(title ="Posterior Probability Distribution",subtitle ="View on Copulas",x =NULL,y =NULL)Meucci, Attilio, Fully Flexible Views: Theory andPractice (August 8, 2008). Fully Flexible Views: Theory and Practice,Risk, Vol. 21, No. 10, pp. 97-102, October 2008, Available at SSRN:https://www.ssrn.com/abstract=1213325↩︎
The commandsx[ , 4] orx[ , "FTSE"] yield the same results.↩︎
An ARIMA, GARCH or VAR model could be used as well. Thefocus is not on the best forecasting method, but how to input theview once the forecast has already been made.↩︎
cor(ftse, dax) = 0.83.↩︎
In thex object,DAX it’s inthe first column and theSMI it’s in the second column.↩︎
Meucci, Attilio, Effective Number of Scenarios in FullyFlexible Probabilities (January 30, 2012). GARP Risk Professional,pp. 32-35, February 2012, Available at SSRN:https://www.ssrn.com/abstract=1971808.↩︎
cop@estimate↩︎