How Node.js applications will benefit from replacing V8 in JXcore

tantalor 12 years ago |

Nothing to see here. Author uses Octane benchmark where bigger is better, so v8 ~26 beats the "experimental engine".

Q: What do the scores mean?

A: In a nutshell: bigger is better. Octane measures the time a test takes to complete and then assigns a score that is inversely proportional to the run time (historically, Firefox 2 produced a score of 100 on an old benchmark rig the V8 team used).

https://developers.google.com/octane/faq

moisy 12 years ago | |

Reproduced the lower node 0.11.10 performance on gentoo.

time node core_engine_benchmark.js

as author answered below comment, the performance gain/loss could be from switches hence the combination of latest v8/node

Clearly the blog posting focuses around the second item (as a biggest replacement reasoning) which is available on both new and old v8. I couldn't reproduce the same problem on i.e. spidermonkey cli but it's visible on node 0.10.26 / 0.11.13

azakai 12 years ago | |

Also, they report run times for the process - it's unclear if they are looking at the total score or the run times. If score, then it seems they are looking at it in the wrong direction, as you said. If run times, then those are meaningless in the Octane benchmark, as it estimates how many executions are possible in a fixed time frame, not how long a fixed workload takes to run.

egeozcan 12 years ago |

They should clearly state that this is a private fork (as in, not open source).

obastemur 12 years ago | |

Please read the last paragraph ( we are committed to 'entirely' open source it, right after the engine switch )

egeozcan 12 years ago | | |

I love what you are doing, so don't get me wrong but it's just a declaration of intent and not binding. Also, not being open source can make people question where actually the speed difference comes from. Did you trade stability with speed? A shortcut for a common operation? Who knows!

obastemur 12 years ago | | |

Yet this is just a declaration for now, we do not have any intention to keep it closed source. There is no need to trade stability with speed since most of the 'current' performance difference with JXcore comes from less amount of JS <-> native switches and better memory management. Very soon we will be open sourcing it! Thanks

cibyr 12 years ago |

I thought bigger means faster for Octane scores. Isn't this showing that their engine has about 75% of the performance of V8?

titzer 12 years ago | |

Yes, bigger is better in Octane scores. And by god we are optimizing the hell out of Octane these days, so I doubt the premise of this post.

AshleysBrain 12 years ago |

How exactly does JXCore manage to beat years worth of optimisation engineering by Google? I'm skeptical, especially since I can imagine an engineering tradeoff between performance and stability. Maybe V8 is slower because they fixed crash bugs caused by overoptimistic code generation. What assurances does anyone have that their slower but working V8 code won't be fast and broken in a new engine?

modeless 12 years ago | |

It doesn't. The benchmark clearly shows that V8 is faster than JXCore, because lower scores are worse.

bbllee 12 years ago | | |

Looking at the linked benchmark file appears to bear out this claim:

https://github.com/Nubisa/jxdocs/blob/master/benchmarks/core...

It appears that the reported numbers for benchmark X are _the # of times that X can be run before one second elapses_, so parent comment is correct and the premise of the blogpost (that the new node.js version with V8 has worse performance than the earlier version / the private fork has better performance than the new version and slightly worse performance than the old) is contradicted by the evidence presented.

Not a good way to look competent, posting something like this. Countdown until edit or takedown...

eshyong 12 years ago | | |

Wait, I'm confused. From the following line:

https://github.com/Nubisa/jxdocs/blob/master/benchmarks/core...

Doesn't this line calculate the amount of time it takes to compute a single task? (elapsed time / number of runs = time per run) Doesn't this mean that JXcore is actually faster, since the number would be smaller with more runs?

bbllee 12 years ago | | |

    // Suites of benchmarks consist of a name and the set of benchmarks in
    // addition to the reference timing that the final score will be based
    // on.  This way, all scores are relative to a reference run and higher
    // scores implies better performance.

https://github.com/Nubisa/jxdocs/blob/master/benchmarks/core...

For complete clarity, see also the lines

https://github.com/Nubisa/jxdocs/blob/master/benchmarks/core...

The score is (arbitrary reference constant) / (geometric mean of runtime). Bad performance ==> longer runtime ==> lower scores.

moisy 12 years ago | | |

numbers seem less important considering the run times.

jfk3 12 years ago | |

Performance is measurable not in years.

bbllee 12 years ago |

Seems double-silly. First there's the whole "smaller is better... I mean, whoops!"-ness of the blogpost. Second is the general choice of benchmark: JXcore is supposed to be a multithreaded JS engine, so it seems like you'd want to benchmark its multithreaded performance against V8's single thread perf. on some workload that can take advantage of multiple threads.

Excavator 12 years ago |

Source code protection? I guess this is some form of security by obscurity when distributing to clients? If it's for in-house code, why would you limit it to the set of data covered by this framework rather than proper encryption for your whole project?

chilledheart 12 years ago |

Maybe JXcore will come with a new LLVM Javascript Frontend?

Looking forward to it.

http://jxcore.com/jxcore-llvm-javascript-frontend-b/

alexchamberlain 12 years ago |

What are you doing computing heavy stuff in JS for?

bbllee 12 years ago | |

Looks like http://nubisa.com/ is interested in native (read: mobile, desktop) app development in JS.