Comparing standard R with Revoutions for performance


17 June 2010

Following on from my previous post about improving performance of R by linking with optimized linear algebra libraries, I thought it would be useful to try out the five benchmarks Revolutions Analytics have on their Revolutionary Performance pages.

For convenience I collected their tests into a single script revolution_benchmark.R that I can simply run with Rscript --vanilla revolution_benchmark.R.

The results, compared with the speed-up factors Revolution claims for their version:

Revolutions benchmarks compared with R on x86_64 system
R R + ATLAS Speed-up Revolution’s
claimed speed-up
Matrix Multiply 360.96 9.30 37.8 41.0
Cholesky Factorization 27.28 5.65 3.8 21.0
Singular Value Decomposition 98.73 23.57 3.2 12.6
Principal Components Analysis 454.55 40.92 10.1 15.2
Linear Discriminant Analysis 271.44 79.61 2.4 4.4

In all instances Revolution’s claimed speed-up is greater, though probably not significantly so for the Matrix Multiply test and hardly so for the Principal Components Analysis. (Of course, I do not have a copy of Revolution Analytics’ product, so I can’t verify their claims or make a comparable test.)

Whether saving 48 seconds on a linear discriminant analysis is enough to justify buying the product is a decision I leave to you: you know what analysis you do. For me, there are (many) orders of magnitudes to be gained by better algorithms and better variable selections so I am not too worried about factors of 2 or even 10. For extra raw power, I run R on a cloud service like AWS which scales well for many problems and is easy to do with stock R while I guess there are some sort of license implications if you wanted to do the same with Revolution’s product. (But I like Revolution and am still trying to find an excuse to use their product.)

Your mileage may vary.