BDA3 Chapter 14 Exercise 3

February 9, 2019
By

[This article was first published on Brian Callander, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

BDA3 Chapter 14 Exercise 3

Here’s my solution to exercise 3, chapter 14, of Gelman’s Bayesian Data Analysis (BDA), 3rd edition. There are solutions to some of the exercises on the book’s webpage.

\(\DeclareMathOperator{\dbinomial}{Binomial} \DeclareMathOperator{\dbern}{Bernoulli} \DeclareMathOperator{\dpois}{Poisson} \DeclareMathOperator{\dnorm}{Normal} \DeclareMathOperator{\dt}{t} \DeclareMathOperator{\dcauchy}{Cauchy} \DeclareMathOperator{\dexponential}{Exp} \DeclareMathOperator{\duniform}{Uniform} \DeclareMathOperator{\dgamma}{Gamma} \DeclareMathOperator{\dinvgamma}{InvGamma} \DeclareMathOperator{\invlogit}{InvLogit} \DeclareMathOperator{\logit}{Logit} \DeclareMathOperator{\ddirichlet}{Dirichlet} \DeclareMathOperator{\dbeta}{Beta}\)

We need to reexpress \((y – X\beta)^T (y – X\beta)\) as \((\mu – \beta)^T \Sigma^{-1} (\mu – \beta)\), for some \(\mu\), \(\Sigma\). Using the QR-decomposition of \(X = QR\), we see

\[
\begin{align}
(y – X\beta)^T(y – X\beta)
&=
(Q^T(y – X\beta))^TQ^T(y – X\beta)
\\
&=
(Q^Ty – Q^TX\beta)^T (Q^Ty – Q^TX\beta)
\\
&=
(Q^Ty – R\beta)^T (Q^Ty – R\beta)
,
\end{align}
\]

where \(Q\) is orthogonal and \(R\) an invertible upper triangular matrix. We can read off the minimum of this quadratic form as

\[
\hat\beta
=
R^{-1}Q^Ty
,
\]

which shows that \(\mu = \hat\beta = R^{-1}Q^Ty\). Note that

\[
\begin{align}
(X^TX)^{-1}X^T
&=
(R^TR)^{-1}R^T Q^T
\\
&=
R^{-1}R^{-T}R^T Q^T
\\
&=
R^{-1}Q^T
\end{align}
\]

so that \(\hat\beta = (X^TX)^{-1}X^Ty\).

Expanding the brackets of both quadratic form expressions and comparing the quadratic coefficients, we see that

\[
\Sigma^{-1} = R^T R = X^T X
,
\]

which shows that \(V_\beta = (X^T X)^{-1}\), in the notation of page 355.

To leave a comment for the author, please follow the link and comment on their blog: Brian Callander.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)