BDA3 Chapter 14 Exercise 3

[This article was first published on Brian Callander, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

BDA3 Chapter 14 Exercise 3

Here’s my solution to exercise 3, chapter 14, of Gelman’s Bayesian Data Analysis (BDA), 3rd edition. There are solutions to some of the exercises on the book’s webpage.

\(\DeclareMathOperator{\dbinomial}{Binomial} \DeclareMathOperator{\dbern}{Bernoulli} \DeclareMathOperator{\dpois}{Poisson} \DeclareMathOperator{\dnorm}{Normal} \DeclareMathOperator{\dt}{t} \DeclareMathOperator{\dcauchy}{Cauchy} \DeclareMathOperator{\dexponential}{Exp} \DeclareMathOperator{\duniform}{Uniform} \DeclareMathOperator{\dgamma}{Gamma} \DeclareMathOperator{\dinvgamma}{InvGamma} \DeclareMathOperator{\invlogit}{InvLogit} \DeclareMathOperator{\logit}{Logit} \DeclareMathOperator{\ddirichlet}{Dirichlet} \DeclareMathOperator{\dbeta}{Beta}\)

We need to reexpress \((y – X\beta)^T (y – X\beta)\) as \((\mu – \beta)^T \Sigma^{-1} (\mu – \beta)\), for some \(\mu\), \(\Sigma\). Using the QR-decomposition of \(X = QR\), we see

\[ \begin{align} (y – X\beta)^T(y – X\beta) &= (Q^T(y – X\beta))^TQ^T(y – X\beta) \\ &= (Q^Ty – Q^TX\beta)^T (Q^Ty – Q^TX\beta) \\ &= (Q^Ty – R\beta)^T (Q^Ty – R\beta) , \end{align} \]

where \(Q\) is orthogonal and \(R\) an invertible upper triangular matrix. We can read off the minimum of this quadratic form as

\[ \hat\beta = R^{-1}Q^Ty , \]

which shows that \(\mu = \hat\beta = R^{-1}Q^Ty\). Note that

\[ \begin{align} (X^TX)^{-1}X^T &= (R^TR)^{-1}R^T Q^T \\ &= R^{-1}R^{-T}R^T Q^T \\ &= R^{-1}Q^T \end{align} \]

so that \(\hat\beta = (X^TX)^{-1}X^Ty\).

Expanding the brackets of both quadratic form expressions and comparing the quadratic coefficients, we see that

\[ \Sigma^{-1} = R^T R = X^T X , \]

which shows that \(V_\beta = (X^T X)^{-1}\), in the notation of page 355.

To leave a comment for the author, please follow the link and comment on their blog: Brian Callander.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)