Following on from my previous post about improving performance of R by linking with optimized linear algebra libraries, I thought it would be useful to try out the five benchmarks Revolutions Analytics have on their Revolutionary Performance pages.
For convenience I collected their tests into a single script revolution_benchmark.R that I can simply run with Rscript --vanilla revolution_benchmark.R.
The results, compared with the speed-up factors Revolution claims for their version:
| R | R + ATLAS | Speed-up |
Revolution’s claimed speed-up |
|
|---|---|---|---|---|
| Matrix Multiply | 360.96 | 9.30 | 37.8 | 41.0 |
| Cholesky Factorization | 27.28 | 5.65 | 3.8 | 21.0 |
| Singular Value Decomposition | 98.73 | 23.57 | 3.2 | 12.6 |
| Principal Components Analysis | 454.55 | 40.92 | 10.1 | 15.2 |
| Linear Discriminant Analysis | 271.44 | 79.61 | 2.4 | 4.4 |
In all instances Revolution’s claimed speed-up is greater, though probably not significantly so for the Matrix Multiply test and hardly so for the Principal Components Analysis. (Of course, I do not have a copy of Revolution Analytics’ product, so I can’t verify their claims or make a comparable test.)
Whether saving 48 seconds on a linear discriminant analysis is enough to justify buying the product is a decision I leave to you: you know what analysis you do. For me, there are (many) orders of magnitudes to be gained by better algorithms and better variable selections so I am not too worried about factors of 2 or even 10. For extra raw power, I run R on a cloud service like AWS which scales well for many problems and is easy to do with stock R while I guess there are some sort of license implications if you wanted to do the same with Revolution’s product. (But I like Revolution and am still trying to find an excuse to use their product.)
Your mileage may vary.