BDA3 Chapter 14 Exercise 3

February 9, 2019
By

(This article was first published on Brian Callander, and kindly contributed to R-bloggers)

BDA3 Chapter 14 Exercise 3

Here’s my solution to exercise 3, chapter 14, of Gelman’s Bayesian Data Analysis (BDA), 3rd edition. There are solutions to some of the exercises on the book’s webpage.

\(\DeclareMathOperator{\dbinomial}{Binomial} \DeclareMathOperator{\dbern}{Bernoulli} \DeclareMathOperator{\dpois}{Poisson} \DeclareMathOperator{\dnorm}{Normal} \DeclareMathOperator{\dt}{t} \DeclareMathOperator{\dcauchy}{Cauchy} \DeclareMathOperator{\dexponential}{Exp} \DeclareMathOperator{\duniform}{Uniform} \DeclareMathOperator{\dgamma}{Gamma} \DeclareMathOperator{\dinvgamma}{InvGamma} \DeclareMathOperator{\invlogit}{InvLogit} \DeclareMathOperator{\logit}{Logit} \DeclareMathOperator{\ddirichlet}{Dirichlet} \DeclareMathOperator{\dbeta}{Beta}\)

We need to reexpress \((y – X\beta)^T (y – X\beta)\) as \((\mu – \beta)^T \Sigma^{-1} (\mu – \beta)\), for some \(\mu\), \(\Sigma\). Using the QR-decomposition of \(X = QR\), we see

\[
\begin{align}
(y – X\beta)^T(y – X\beta)
&=
(Q^T(y – X\beta))^TQ^T(y – X\beta)
\\
&=
(Q^Ty – Q^TX\beta)^T (Q^Ty – Q^TX\beta)
\\
&=
(Q^Ty – R\beta)^T (Q^Ty – R\beta)
,
\end{align}
\]

where \(Q\) is orthogonal and \(R\) an invertible upper triangular matrix. We can read off the minimum of this quadratic form as

\[
\hat\beta
=
R^{-1}Q^Ty
,
\]

which shows that \(\mu = \hat\beta = R^{-1}Q^Ty\). Note that

\[
\begin{align}
(X^TX)^{-1}X^T
&=
(R^TR)^{-1}R^T Q^T
\\
&=
R^{-1}R^{-T}R^T Q^T
\\
&=
R^{-1}Q^T
\end{align}
\]

so that \(\hat\beta = (X^TX)^{-1}X^Ty\).

Expanding the brackets of both quadratic form expressions and comparing the quadratic coefficients, we see that

\[
\Sigma^{-1} = R^T R = X^T X
,
\]

which shows that \(V_\beta = (X^T X)^{-1}\), in the notation of page 355.

To leave a comment for the author, please follow the link and comment on their blog: Brian Callander.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Search R-bloggers

Sponsors

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)