Design of Experiments – Power Calculations

[This article was first published on Software for Exploratory Data Analysis and Statistical Modelling, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Prior to conducting an experiment researchers will often undertake power calculations to determine the sample size required in their work to detect a meaningful scientific effect with sufficient power. In R there are functions to calculate either a minimum sample size for a specific power for a test or the power of a test for a fixed sample size.

When undertaking sample size or power calculations for a prospective trial or experiment we need to consider various factors. There are two main probabilities of interest that are tied up with calculating a minimum sample size or the power of a specific test, and these are:

  • Type I Error: The probability that the test accepts the null hypothesis, H_0, given that the null hypothesis is actually true. This quantity is often referred to as alpha.
  • Type II Error: The probability that the test rejects the null hypothesis, H_0, given that the null hypothesis is not true. This quantity is often referred to as beta.

A decision needs to be made about what difference between the two groups being compared should be considered as corresponding to a meaningful difference. This difference is usually denoted by delta.

The base package has functions for calculating power or sample sizes, which includes the functions power.t.test, power.prop.test and power.anova.test for various common scenarios.

Consider a scenario where we might be buying batteries for a GPS device and the average battery life that we want to have is 400 minutes. If we decided that the performance is not acceptable if the average is more than 10 minutes (delta) lower than this (390 minutes) then we can calculate the number of batteries to test:

power.t.test(delta = 10, sd = 6, power = 0.95, type = "one.sample",
  alternative = "one.sided")

For this example we have assumed a standard deviation of 6 minutes for batteries (would either be assumed or estimated from previous data) and that we want a power of 95% in the test. Power is defined as 1 – beta, the Type II error probability. The default option for this function is for 5% probability of alpha, a Type I error. The test will involve only one group so we are considering a one-sample t test and only a one sided alternative is relevant as we do not mind if the batteries perform better than required.

The output from this function call is as follows:

     One-sample t test power calculation 
 
              n = 5.584552
          delta = 10
             sd = 6
      sig.level = 0.05
          power = 0.95
    alternative = one.sided

So we would need to test at least 6 batteries to obtain the required power in the test based on the other parameters that have been used.

To leave a comment for the author, please follow the link and comment on their blog: Software for Exploratory Data Analysis and Statistical Modelling.

R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.

Never miss an update!
Subscribe to R-bloggers to receive
e-mails with the latest R posts.
(You will not see this message again.)

Click here to close (This popup will not appear again)