# On p-value

February 2, 2013
By

(This article was first published on Eran Raviv » R, and kindly contributed to R-bloggers)

Albert Schweitzer said: “Example is not the main thing in influencing others. It is the only thing.”, so I start with it.

Generate two random samples from Normal distribution.
Test the hypothesis number one: , do not reject, then test the more general hypothesis number two: .
Logic states that since there is at least one value we do not reject for (in this case, 0), the more general statement would also not be rejected.

This is not always the case.

?View Code RSPLUS
 PVgen = function(mu1,mu2){ n = 100 x1 = rnorm(n,mu1,1) x2 = rnorm(n,mu2,1) X = cbind(x1,x2) mx = apply(X,2,mean)   ### LR test for NULL mu1 = mu2 = 0   lambda1 = exp((-n/2)*( t(mx)%*%mx )) Tstat1 = -2*log(lambda1)   ### LR test for NULL mu1 = mu2   l = c(1,1) II = diag(2) lambda2 = exp((-n/2)*( t(mx)%*% (II - .5*(l%*%t(l))) %*% mx ) ) Tstat2 = -2*log(lambda2) LT1 = ifelse(Tstat1>qchisq(.5,2),T,F) ; LT2 = ifelse(Tstat2>qchisq(.5,1),T,F) pv1 <- 2*(1 - pchisq(Tstat1,2, lower.tail = LT1)) pv2 <- 2*(1 - pchisq(Tstat2,1, lower.tail = LT2)) return(cbind(pv1,pv2)) }   s0 <- s1 <- s2 <- pvalues <- NULL for (i in 1:500){ pvalues <- rbind(pvalues, PVgen(.16,-.16) ) s1[i] <- sign(pvalues[i,1] - .05) s2[i] <- sign(pvalues[i,2] - .05) # which was rejected when the tests do not agree? s0[i] <- ifelse(s1[i] != s2[i],which.min(pvalues[i,]),0 ) } sum(s0==2) # when the second was rejected but the first did not..   Results: 70

Since the code involves simulations, results may differ between users, despite the repetition procedure applied.

So in about 14% = 70/500 cases, we can not reject hypothesis number one, but we reject the more general hypothesis number two. Note, using the same data.

What is the point to take from this?
Well, that there is no logical contradiction here, we just need to be careful with the interpretation of p-values:
“one agent that does not believe in a proposition does NOT imply the she believes in its negation” (Dubois and Prade, 2001).

The idea for the post came from reading the paper: A classical measure of evidence for general null hypotheses. The author suggest an alternative measure to p-value.

Related:
﻿

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...