Something on JavaScript Performance and Benchmarking

November 18, 2014

Inspired by the new J2V8 JavaScript runtime by Ian Bull and his benchmark against Rhino, I considered finally to do some benchmarking and performance tests with the JS runtimes I'm currently working with:
Nashorn and DynJS as JS runtimes on top of the JVM, Rhino as comparison to the past and V8/Node as a native runtime.

The test environment:
Virtual Machine running (headless) Ubuntu Linux 12.04 LTS 32bit, 1 CPU, 1GB RAM
(Host: Win7 64bit, i7 Quad-Core, 8GB RAM)

The test script:
I took exactly the fibonacci-/array-function Ian Bull used for his benchmark against Rhino. Then, I called the function 1, 10.000 and 100.000 times and measured the time. I know that this is not exactly correct, because when running the 10k and 100k loops, the JS engine is already warmed up. But, I ran the same script on all plattforms:
var fibonacci = function() {
var i;
var fib = [];
fib[0] = 1;
fib[1] = 1;
for(i = 2; i <= 100; i++) {
fib[i] = fib[i-2] + fib[i-1];
}
fib;
};

var fibExec = function(times) {
var start = new Date().getTime();
for (n = 0; n < times; n++) {
fibonacci();
}
var duration = times + ": " + (new Date().getTime() - start);
print(duration);
};

fibExec(1);
fibExec(10000);
fibExec(100000);

The results:
First of all, let the numbers speak for themselves, then, I have some comments:

JS RuntimeVersion110k100k
V8/Nodev0.10.331ms35ms284ms
Rhinov1.7R4 on J8u256ms265ms2232ms
NashornJava 8u2555ms252ms1606ms
NashornJava 8u40 (ea build b12)52ms215ms1064ms
DynJSv0.3.0 on J8u2544ms6406ms63047ms
DynJSv0.3.0 on J8u40 (b12)38ms6058ms58229ms

Some thoughts and comments:
  • Interesting thing, that V8 is so much faster. I expected that it is faster, but not that much. I'm impressed, there's some work for the JVM runtimes to do.
  • Rhino ist much faster than Nashorn when the engine is cold, after warming-up, Nashorn is performing better (ok, this is well known, nothing new - but why? and why it doesn't get fixed? even DynJS performs here better)
  • The next Java 8 update 40 release will come with performance improvements in Invokedynamic between 17 and 51% for Nashorn. That's quite a lot. I like! :-)
  • Unfortunately DynJS only uses +/- 10% in average of this performance improvement, although it also uses invokedynamic.
  • DynJS is up to 20% faster when executing the fibonacci function only 1 time, but much, much slower when executing it 10k times. And I don't know what the hell they are doing at the 100k loop...!?

So, there's some homework to do for the Nashorn and DynJS teams, if they want to compete with V8 on performance! But still I like theses JavaScript JVM approaches very much! IMO it's more powerful when it comes to enterprise integration than any native solution. Performance is not the only thing that matters!

Tags: dynjs v8 nodejs nashorn javascript