Relative time microbenchmark
[This article was first published on My R Nightmares, and kindly contributed to R-bloggers]. (You can report issue about the content on this page here)
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Microbenchmark is a very useful function for timing the execution time of one or more programs. I often use it to compare different programs. So I created a function that produces the output relative to the time of one of the programs being tested.Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.
Snippet available at GitHub
Download zip
For example, it is known that the singular value decomposition of a matrix is slower than the eigen-decomposition of its covariance matrix, but what about the time needed to compute the covariance? Of course it depends on the size of the matrix. The example below shows the use of mbm_rel to compare these times.
M = matrix(runif(4000),200)
S = crossprod(M)
dim(M)# [1] 200 20
eigenS computes the covariance
eigenS <- function(X){
eigen(var(X))
}
mbm_out = microbenchmark(svd = svd(M), eigen = eigen(S), eigenS = eigenS(X))
mbm_out
# Unit: microseconds
# expr min lq mean median uq max neval
# svd 704.470 722.0075 911.6752 812.900 878.556 4163.942 100
# eigen 365.281 374.0500 477.9450 425.163 515.628 978.217 100
# eigenS 916.197 941.0050 1746.5251 1085.363 1245.119 50295.797 100
As expected, eigen is faster than svd but eigenS is slower. How much slower? Let’s call mbm_rel
mbm_rel(mbm_out)
# Unit: microseconds
# min lq mean median uq max neval
# svd time 705.754 716.233 806.809 735.267 753.873 3459.045 100
# svd/eigen 1.943 1.936 2.009 1.965 1.919 3.694 100
# svd/eigenS 0.637 0.636 0.414 0.640 0.642 0.054 100
The time units can be changed, just like in the original function. For milliseconds, set unit = “ms”
mbm_rel(mbm_out, unit = “ms”)
# Unit: milliseconds
# min lq mean median uq max neval
# svd time 0.706 0.716 0.807 0.735 0.754 3.459 100
# svd/eigen 1.943 1.936 2.009 1.965 1.919 3.694 100
# svd/eigenS 0.637 0.636 0.414 0.640 0.642 0.054 100
The first row gives the time for svd and the following the svd time relative to the other functions. So svd takes almost twice the time of eigen but 60% of the time of eigenS. A more informative output would be the time of the slowest. easily done, set relpos= 3 (the third function)
# Unit: milliseconds
# min lq mean median uq max neval
# eigenS time 1.109 1.126 1.948 1.149 1.174 64.490 100
# eigenS/svd 1.571 1.572 2.415 1.563 1.557 18.644 100
# eigenS/eigen 3.053 3.044 4.852 3.070 2.987 68.877 100
Easier to interpret.
It’s also possible to reverse the relativity, that is have the proportion of time of the methods with respect to the target method by setting inverse = TRUE
mbm_rel(mbm_out, relpos = 3, inverse = TRUE, unit = “ms”)
# Unit: milliseconds
# min lq mean median uq max neval
# eigenS time 1.109 1.126 1.948 1.149 1.174 64.490 100
# svd/eigenS 0.637 0.636 0.414 0.640 0.642 0.054 100
# eigen/eigenS 0.328 0.329 0.206 0.326 0.335 0.015 100
One can retrieve the time units with
attributes(mt)$unit
# [1] “microseconds”
The argument rel_name assigns a different name to the target method.
mbm_rel returns the relative times table, setting reltime = FALSE, returns ones instead of the target method time summaries.
mbm_r <- mbm_rel(mbm_out, relpos = 3, reltime = FALSE, unit = "ms")
mbm_r
# min lq mean median uq max neval
# eigenS 1.000 1.000 1.000 1.000 1.000 1.000 100
# eigenS/svd 1.571 1.572 2.415 1.563 1.557 18.644 100
# eigenS/eigen 3.053 3.044 4.852 3.070 2.987 68.877 100
barplot(mbm_r$median, names = rownames(mbm_r), main = “relative execution times”)
To leave a comment for the author, please follow the link and comment on their blog: My R Nightmares.
R-bloggers.com offers daily e-mail updates about R news and tutorials about learning R and many other topics. Click here if you're looking to post or find an R/data-science job.
Want to share your content on R-bloggers? click here if you have a blog, or here if you don't.