Tamarin Benchmarks Redux - How much faster?

Tamarin-Tracing's interpreter was just totally revamped with this latest patch. More or less every post about the Interpreter is moot, as there is no longer double interpretation. Instead of having a .abc be interpreted in Forth, then interpreting Forth in C, the new patch translates .abc to the Forth intermediate language. This IL is then interpreted. Therefore, there is only one interpreter instead of two. There are also a whole slew of other changes you can read about in the patch.

I just wanted to see how much faster the new interpreter is. This is without a doubt, not a comprehensive nor complete picture. The benchmark I'll be using is heap sort, which randomly generates 1 million random numbers and sorts them using the heap sort algorithm. This is a JavaScript translation from the Java JavaGrande2 benchmark suite. If you want the source, feel free to ask.

Let's take a look at the relative numbers. All benchmarks are on a MacBook Pro 2.4 ghz w/ 2 gigs of ram. Both Tamarin old/new are built with the "Release" configuration, and run with the -interp flag so we can look purely at the interpreter. The SpiderMonkey benchmark is here simply for comparison, as is tamarin-central. Both were taken from the CVS snapshot 5 minutes before this post. SpiderMonkey was built with optimizations on. Tamarin-central was built with the Release configuration. Here are the numbers:

Tamarin with Double Interpretation - Compiler off
time ./avmplus -interp ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.abc

real 3m33.839s
user 3m33.700s
sys 0m0.072s

Tamarin with Double Interpretation - Compiler on
time ./avmplus ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.abc

real 0m12.201s
user 0m12.105s
sys 0m0.095s

Tamarin with Single Interpretation - Compiler off
time ./avmplus -interp ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.abc

real 3m8.205s
user 3m8.080s
sys 0m0.075s

Tamarin with Single Interpretation - Compiler on
time ./avmplus ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.abc

real 0m11.698s
user 0m11.646s
sys 0m0.048s

SpiderMonkey (JavaScript-C 1.7.0 2007-10-03)
time ./js ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.js

real 0m27.779s
user 0m27.611s
sys 0m0.123s

SpiderMonkey (JavaScript-C 1.8.0 pre-release 1 2007-10-03)
Build Date: built on Apr 26 2008 at 18:21:38
time ./js ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.js

real 0m11.671s
user 0m11.542s
sys 0m0.082s

Tamarin-Central - Compiler off
time ./shell -Dinterp ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.abc

real 2m6.523s
user 2m5.880s
sys 0m0.538s

Tamarin-Central - Compiler on
time ./shell ~/Projects/tamarin-central/test/custom/JGFHeapSortBench.abc
real 0m8.566s
user 0m8.538s
sys 0m0.021s


The scariest part about all of this, is how fast SpiderMonkey is. It's only an interpreter! Also, whatever the guys at Mozilla did between JS 1.7 to JS 1.8, bravo. There's still hope though! Tamarin-Tracing is still a work in progress. Adobe said that tamarin-tracing usually beats tamarin-central unless you get some weird looking code. Hopefully we'll start to see that soon.

Edit: Added build date to SpiderMonkey 1.8