# A variance campaign that failed

**Portfolio Probe » R language**, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

they ought at least be allowed to state why they didn’t do anything and also to explain the process by which they didn’t do anything.

## First blush

One of the nice things about R is that new statistical techniques fall into it. One such is the glasso (related to the statistical lasso) which converts degenerate variance matrices into positive definite ones.

Once I was in my local R session, all I had to do to try out glasso was:

> install.packages('glasso') > require(glasso) Loading required package: glasso > ls('package:glasso') [1] "glasso" "glassopath" > ?glasso

So four commands:

- Install the package — in general, you will be asked to select a nearby mirror for the CRAN repository.
- Load the package into the current session
- Look at what objects are in the package (and visible) — in this case two objects.
- Get help on the
`glasso`

function

The help says that the first argument should be the degenerate matrix that is to be rehabilitated. But it also wants a second argument saying how much (and how) to shrink. The help file is not lying in that regard:

> wrongo <- glasso(sp5.sampvar06) Error in is.matrix(rho) : 'rho' is missing

## The plan

Since there is a free parameter (actually a matrix, but a single number is allowed) that needs to be selected, I devised a plan:

- Use glasso with a range of values for rho on the sample variance for 2006 for the returns of almost all of the S&P 500.
- See which value of rho was best when tested out of sample in 2007 on a set of random portfolios — a la “The quality of variance matrix estimation”.
- Use the best value of rho on data from 2010, and use a different set of random portfolios to compare the quality of the
`glasso`

in 2011 with Ledoit-Wolf shrinkage and a statistical factor model.

## The reality

So now comes time to pick values of rho. Hint: it has to be non-negative. That’s the only hint I could glean from the help file. And reading is for ninnies — it’s time to start hacking. What number would you pick?

> sp5.sampvar06[1:4,1:4] MMM ACE ABT ANF MMM 1.314376e-04 3.050325e-05 2.225360e-05 4.302205e-05 ACE 3.050325e-05 1.379232e-04 4.379703e-05 5.564900e-05 ABT 2.225360e-05 4.379703e-05 1.091088e-04 2.977900e-05 ANF 4.302205e-05 5.564900e-05 2.977900e-05 4.320315e-04 > stillwrongo <- glasso(sp5.sampvar06, 1) > stillwrongo$w[1:4,1:4] [,1] [,2] [,3] [,4] [1,] 1.000131 0.000000 0.000000 0.000000 [2,] 0.000000 1.000138 0.000000 0.000000 [3,] 0.000000 0.000000 1.000109 0.000000 [4,] 0.000000 0.000000 0.000000 1.000432

So rho equal 1 doesn’t look too promising. But we do have a hint that maybe rho should be smaller than the typical value. What would you try next?

I tried 1e-6. And I waited.

And waited.

And waited.

After roughly an hour it finished. That was, of course, before I thought that timing the computation would be a good thing — it took less than a second with rho equal 1.

Next was:

> system.time(gl06.1en7 <- glasso(sp5.sampvar06, 1e-7)$w) user system elapsed 10685.30 7.77 12457.60

That is, a leisurely lunch provided an insufficient interval (the timing numbers are seconds). Maybe it’s like making chocolate — the longer it takes, the better it is.

## Results

The plan has been mislaid by now, but we can still look at the results of random portfolios in 2007.

> require(PortfolioProbe) > # generate random portfolios > rp06.mw3 <- random.portfolio(1e4, prices=sp5.close[250,], + gross=1e6, long.only=TRUE, max.weight=.03, + port.size=c(90,100)) > > # 2007 realized volatility of the random portfolios > vol.rp06.mw3 <- sqrt(252) * apply(valuation(rp06.mw3, + prices=sp5.close[251:501,], returns='log'), 2, sd) > > # ex-ante variance from glasso > predvar.gl06.1en7 <- unlist(randport.eval(rp06.mw3, + keep="var.values", + additional.args=list(variance=gl06.1en7)))

The correlation between the predicted volatility and realized volatility is 0.595 for both the glasso estimates. The correlation for the Ledoit-Wolf estimate is 0.576. So the glasso estimates give a better result.

## Summary

- R is a good thing.
- glasso possibly has promise but we need some more of Moore’s law or algorithmic innovation before it will be of practical use.
- Thanks to the person who brought glasso to my attention.

## Epilogue

We had flanked the farmhouse. We had made our first military movement and it was a success.

They were fine horsemen and good revolver shots, but their favorite arm was the lasso.

from “The Private History of a Campaign that Failed” by Mark Twain

Subscribe to the Portfolio Probe blog by Email

**leave a comment**for the author, please follow the link and comment on their blog:

**Portfolio Probe » R language**.

R-bloggers.com offers

**daily e-mail updates**about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.

Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.