Bayesian model II regression

May 27, 2013
By

(This article was first published on Ecology in silico, and kindly contributed to R-bloggers)

Regression is a mainstay of ecological and evolutionary data analysis. For example, a disease ecologist may use body size (e.g. a weight from a scale with measurement error) to predict infection. Classical linear regression assumes no error in covariates; they are known exactly. This is rarely the case in ecology, and ignoring error in covariates can bias regression coefficient estimates. This is where model II (aka errors-in variables and measurement errors) regression models come in handy. Here I’ll demonstrate how to construct such a model in a Bayesian framework, where substantive prior knowledge of covariate error facilitates less-biased parameter estimates.

Here’s a quick illustration of the problem: I’ll generate data from a known simple linear regression model, and fit models that ignore or incorporate error in the covariate.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# simulate covariate data
n <- 50
sdx <- 6
sdobs <- 5
taux <- 1 / (sdobs * sdobs)
truex <- rnorm(n, 0, sdx)
errorx <- rnorm(n, 0, sdobs)
obsx <- truex + errorx

# simulate response data
alpha <- 0
beta <- 10
sdy <- 20
errory <- rnorm(n, 0, sdy)
obsy <- alpha + beta*truex + errory
parms <- data.frame(alpha, beta)

Ignoring error in the covariate:


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# bundle data
jags_d <- list(x = obsx, y = obsy, n = length(obsx))

# write model
cat("
    model{
## Priors
alpha ~ dnorm(0, .001)
beta ~ dnorm(0, .001)
sdy ~ dunif(0, 100)
tauy <- 1 / (sdy * sdy)

## Likelihood
  for (i in 1:n){
    mu[i] <- alpha + beta * x[i]
    y[i] ~ dnorm(mu[i], tauy)
  }
}
",
    fill=TRUE, file="yerror.txt")

require(rjags)
# initiate model
mod1 <- jags.model("yerror.txt", data=jags_d,
                   n.chains=3, n.adapt=1000)

# simulate posterior
out <- coda.samples(mod1, n.iter=1000, thin=1,
                    variable.names=c("alpha", "beta", "sdy"))

# store parameter estimates
require(ggmcmc)
ggd <- ggs(out)
a <- ggd$value[which(ggd$Parameter == "alpha")]
b <- ggd$value[which(ggd$Parameter == "beta")]
d <- data.frame(a, b)

Incorporating error in the covariate: I’m assuming that we have substantive knowledge about covariate measurement represented in the prior for the precision in X. Further, the prior for the true X values reflects knowledge of the distribution of our X value in the population from which the sample was taken.


1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# specify model
cat("
    model {
## Priors
alpha ~ dnorm(0, .001)
beta ~ dnorm(0, .001)
sdy ~ dunif(0, 100)
tauy <- 1 / (sdy * sdy)
taux ~ dunif(.03, .05)

## Likelihood
  for (i in 1:n){
    truex[i] ~ dnorm(0, .04)
    x[i] ~ dnorm(truex[i], taux)
    y[i] ~ dnorm(mu[i], tauy)
    mu[i] <- alpha + beta * truex[i]
  }
}
    ", fill=T, file="xyerror.txt")

# bundle data
jags_d <- list(x = obsx, y = obsy, n = length(obsx))

# initiate model
mod2 <- jags.model("xyerror.txt", data=jags_d,
                   n.chains=3, n.adapt=1000)

# simulate posterior
out <- coda.samples(mod2, n.iter=30000, thin=30,
                    variable.names=c("alpha", "beta", "tauy", "taux"))
# store parameter estimates
ggd <- ggs(out)
a2 <- ggd$value[which(ggd$Parameter == "alpha")]
b2 <- ggd$value[which(ggd$Parameter == "beta")]
d2 <- data.frame(a2, b2)

Now let’s see how the two models perform.


1
2
3
4
5
6
7
8
9
ggplot(d, aes(x=obsx, obsy)) +
  geom_abline(aes(intercept=a, slope=b), data=d, color="red", alpha=0.05) +
  geom_abline(aes(intercept=a2, slope=b2), data=d2, color="blue", alpha=0.05) +
  geom_abline(aes(intercept=alpha, slope=beta),
              data=parms, color="green", size=1.5, linetype="dashed") +
  theme_bw() +
  geom_point(shape=1, size=3) +
  xlab("X values") + ylab("Observed Y values") +
  ggtitle("Model results with and without modeling error in X")

The dashed green line shows the model that generated the data, i.e. the “true” line. The red lines show the posterior for the naive model ignoring error in X, while the less-biased blue lines show the posterior for the model incorporating error in X.

To leave a comment for the author, please follow the link and comment on their blog: Ecology in silico.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: Data science, Big Data, R jobs, visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Comments are closed.

Sponsors

Mango solutions



RStudio homepage



Zero Inflated Models and Generalized Linear Mixed Models with R

Dommino data lab

Quantide: statistical consulting and training



http://www.eoda.de







ODSC

ODSC

CRC R books series





Six Sigma Online Training





Contact us if you wish to help support R-bloggers, and place your banner here.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)