A look at the distortion from predicted to realized.
The efficient frontier is a mainstay of academic quant. I’ve made fun of it before. This post explores the efficient frontier in a slightly less snarky fashion.
The universe is 474 stocks in the S&P 500. The predictions are made using data from 2010. The realized period is 2011.
The return predictor is MACD — essentially a momentum estimate. As before this is used neither to praise nor to denigrate it, but rather because it is a publicly available method that is sometimes effective.
The variance matrix is estimated via Ledoit-Wolf shrinkage towards equal correlation.
The efficient frontier is a function of the constraints. We’ll use one set of constraints here:
- 90 to 100 names in the portfolio
- no more than 3% of the portfolio variance attributed to any asset
Figure 1 shows the predictions for 10,000 random portfolios that obey the constraints — the plot uses efficient frontier style axes. (In this case the return predictor isn’t literally predicting returns, but something thought to be correlated with returns.)
I was surprised at the shape of Figure 1. I expected a fan shape — a wider range of return predictions for higher volatility portfolios. My expectation is reinforced by looking at this relationship for the assets (Figure 2).
Figure 3 adds a few points on the efficient frontier to what is plotted in Figure 1.
We see that the efficient frontier is seriously far from the typical random portfolio.
When we were just predicting, we were at the end of 2010. Now we are in 2012 and we can see what actually happened in 2011.
Figure 4 shows the quality of the volatility prediction for the random portfolios.
2011 turned out to be somewhat more volatile than the predictions. The correlation among portfolios that satisfy the constraints is reasonably high.
A high correlation between predicted and realized volatility is what we need for minimum variance optimization to work. (But this set of random portfolios doesn’t allow us to see the relationship at the extreme low end of volatility.)
Ex ante frontier
Figure 5 shows the realized version of Figure 3.
Figure 5 looks nothing like it is “supposed” to. This appears to be an advertisement for low volatility investing, but part of what we see is that this return predictor did not work in this time period. If the return estimate worked, then the efficient frontier would be above the random portfolio cloud rather than below it.
However, it does seem quite extraordinary that this minimum variance portfolio should return 15% in a year when the underlying index was flat.
Ex post frontier
Figure 6 adds the ex-post efficient frontier to what is in Figure 5. This is what could have been achieved with perfect foresight — this is much more imaginary than even the ex-ante efficient frontier.
The first thing to notice is that the actual minimum variance portfolio achieved almost as small a variance as possible. The difference in returns between the two minimum variance portfolios is accidental in a sense.
The next thing to notice is that the ex-post frontier is severely truncated in terms of volatility. The maximum return portfolio has a volatility that is smaller than 85% of the random portfolios. This is an advertisement for low volatility investing — in retrospect for 2011.
Are there good reasons for the shape of Figure 1?
Here is a summary of the R computations that were done.
The functions that downloaded the data and estimated MACD are among the functions that support the Portfolio Probe User’s Manual. These depend on the TTR package, hence:
sp5.close <- as.matrix(pp.TTR.multsymbol(sp5.names, 20060101, 20120224))
There were some stocks that didn’t have the exact range of data and so consisted of all missing values. We delete these:
sp5.close <- sp5.close[, !is.na(sp5.close[1,])]
It is good practice to inspect data for weirdness. Our eyes are really good at that. Here is the code I used to look through all the data series.
for(i in 1:474) plot(sp5.close[,i], type='l', main=colnames(sp5.close)[i])
plot command was just to get a graphics device running that could then be set to prompt for each new plot.
Nothing untoward was seen.
sp5.macd <- as.matrix(pp.TTR.multmacd(sp5.close))
sp5.ret <- diff(log(sp5.close))
The variance estimation uses a function from the BurStFin package.
sp5.var10 <- var.shrink.eqcor(sp5.ret[1006:1258, ])
generate random portfolios
The tasks done here can be done in any recent version of Portfolio Probe, but some things are easier using features new to the latest beta version.
sp5.price10 <- sp5.close[1259,]
sp5.alpha10 <- sp5.macd[1259,]
rp1.sp5 <- random.portfolio(1e4, sp5.price10, sp5.var10, gross=1e6, long.only=TRUE, risk.fraction=.03, port.size=c(90,100))
random portfolio ex ante values
rp1.sp5.xa <- randport.eval(rp1.sp5, keep=c('var.values', 'alpha.values'), additional.args=list(expected.return=sp5.alpha10))
The result of the command above is a list with a component for each of the 10,000 random portfolios. Each of those components is a list that contains the ex ante variance and expected return for the portfolio. The command below reconfigures that into a more useful matrix.
rp1.sp5.xamat <- do.call('rbind', lapply(rp1.sp5.xa, function(x) c(volatility=sqrt(252 * x$var.values)*100, prediction=x$alpha.values)))
random portfolio ex post values
Create the matrix of closing prices during 2011:
sp5.close11 <- sp5.close[1259:1511,]
Now is when the latest beta version comes in handy —
valuation now can compute returns.
rp1.sp5.xpvol <- apply(valuation(rp1.sp5, prices=sp5.close11, returns="log"), 2, sd) * sqrt(252) * 100
rp1.sp5.xpret <- 100 * valuation(rp1.sp5, prices=sp5.close11[c(1, nrow(sp5.close11)),], returns="simple")
Here is the command to get the minimum variance portfolio. The expected returns (
sp5.alpha10) are ignored in this command except to note what the portfolio expected return is.
op1.mv.sp5 <- trade.optimizer(sp5.price10, sp5.var10, sp5.alpha10, gross=1e6, long.only=TRUE, risk.fraction=.03, port.size=c(90,100), utility="minimum variance")