Random input software testing

August 23, 2011
By

(This article was first published on Portfolio Probe » R language, and kindly contributed to R-bloggers)

The usual approach to testing software is to create a specific problem and see if the software gets the correct answer.  Although this is very useful, there are problems with it:

  • It is labor-intensive
  • It almost totally neglects to test the code that throws errors
  • There can be unconscious bias in the test cases created

One alternative is to create problems with random inputs.  The talk I gave on this at useR!2011 was “Random input testing with R”.

There was a question at the talk concerning full coverage — basically that a fully random distribution is not going to be efficient at covering the whole space.  I don’t have any particular experience with this, but here are my thoughts:  If you have a space you are concerned about such that you can keep track of how many times each point in the space has been hit, then you could dynamically change the distributions of inputs to increase the chance of the lesser hit points being hit.

Past versions of the Portfolio Probe software have benefited from this technique on an experimental level.  Future versions will be subjected to more thorough tests of this sort.

Subscribe to the Portfolio Probe blog by Email

To leave a comment for the author, please follow the link and comment on his blog: Portfolio Probe » R language.

R-bloggers.com offers daily e-mail updates about R news and tutorials on topics such as: visualization (ggplot2, Boxplots, maps, animation), programming (RStudio, Sweave, LaTeX, SQL, Eclipse, git, hadoop, Web Scraping) statistics (regression, PCA, time series, trading) and more...



If you got this far, why not subscribe for updates from the site? Choose your flavor: e-mail, twitter, RSS, or facebook...

Tags: , , , ,

Comments are closed.