What is the most accurate JS performance testing website?

Question

What is the most accurate JS performance testing website?

After conducting a benchmark on both jsperf.com and jsben.ch, I noticed that they are providing significantly different results.

Here are the links to the benchmarks:

JSPerf: https://jsperf.com/join-vs-template-venryx
JSBench:

Even though the code blocks used in both benchmarks are identical, the performance outcomes vary greatly. On jsperf, block 1 is reported to be "61% slower" than the fastest, while on jsbench, it's only 32% slower than the fastest option.

This inconsistency is perplexing as one would expect benchmark sites to yield similar results with minimal variation.

Due to these discrepancies, I am unable to draw a definitive conclusion regarding which method is the most efficient.

EDIT

I have expanded the list of benchmarks for further analysis:

https://jsperf.com/join-vs-template-venryx

While I cannot definitively determine the best performing option, I recommend avoiding jsben.ch due to missing crucial information such as the number of runs, error margin, and operations per second, which are essential for accurate performance assessment and comparison across benchmark sites and browsers.

(Currently, is my preferred choice.)

javascript performance benchmarking jsperf

Answer 1

Answer №1

Update for March 2019: There are discrepancies in results between Firefox and Chrome - perf.zone behaves oddly on Chrome, while jsben.ch behaves strangely on Firefox. Until the reasons behind this are clear, it's best to benchmark on multiple websites (avoiding jsben.ch if possible, as other sites offer more information on run statistics)

In summary: ~~testing your code on perf.zone and jsbench.github.io shows results similar to jsperf.~~ For various reasons, I personally trust these three websites over jsben.ch.

I recently tested string concatenation performance by combining over 1000000 single character strings (join('') is the most efficient for such large numbers). On my system, jsben.ch timed out without producing any result. Here are the links to the benchmarks:

(Dealing with jsperf's not all tests inserted issue is a hassle)

I suspect, though can't prove, that perf.zone provides slightly more reliable benchmark data:

While optimizing lz-string, I used jsbench.github.io for a long time but noticed significantly high error margins at times, exceeding 100%.
Benchmarking on mobile devices works well with jsperf.com and perf.zone, but jsbench.github.io has glitches in CSS during testing.

There might be a connection between these issues and how jsbench.github.io updates the DOM, causing some overhead affecting the benchmarks (they should consider meta-benchmarking this).

Note: While not perfect, perf.zone occasionally times out when saving benchmarks (at the worst moments) and only allows forking of code, not editing. However, its output aligns better with jsperf, and it offers a convenient "quick" mode for temporary benchmarks

Answer 2

Update for March 2019: There are discrepancies in results between Firefox and Chrome - perf.zone behaves oddly on Chrome, while jsben.ch behaves strangely on Firefox. Until the reasons behind this are clear, it's best to benchmark on multiple websites (avoiding jsben.ch if possible, as other sites offer more information on run statistics)

In summary: ~~testing your code on perf.zone and jsbench.github.io shows results similar to jsperf.~~ For various reasons, I personally trust these three websites over jsben.ch.

I recently tested string concatenation performance by combining over 1000000 single character strings (join('') is the most efficient for such large numbers). On my system, jsben.ch timed out without producing any result. Here are the links to the benchmarks:

(Dealing with jsperf's not all tests inserted issue is a hassle)

I suspect, though can't prove, that perf.zone provides slightly more reliable benchmark data:

While optimizing lz-string, I used jsbench.github.io for a long time but noticed significantly high error margins at times, exceeding 100%.
Benchmarking on mobile devices works well with jsperf.com and perf.zone, but jsbench.github.io has glitches in CSS during testing.

There might be a connection between these issues and how jsbench.github.io updates the DOM, causing some overhead affecting the benchmarks (they should consider meta-benchmarking this).

Note: While not perfect, perf.zone occasionally times out when saving benchmarks (at the worst moments) and only allows forking of code, not editing. However, its output aligns better with jsperf, and it offers a convenient "quick" mode for temporary benchmarks

Answer 3

Answer №2

Apologies for bringing this up again, but I thought it might be helpful for those who come across this issue in their search results.

I can't speak on behalf of others, but jsbench.me simply uses benchmark.js for its testing purposes. It's a single-page React application, which means it operates entirely within your browser and chosen engine, resulting in consistent outcomes within the same browser. Running tests in Firefox or on mobile may yield different results, as expected. However, all testing components are managed locally, with the exception of utilizing AWS DynamoDB to store test results.

By the way, I'm the creator of the tool, so it's really just a personal passion project. Currently, it doesn't incur any costs for me since it's designed to be serverless and falls within the limits of the AWS free tier. The amount of effort I put into it is directly proportional to the number of users engaging with the platform :)

Answer 4

Apologies for bringing this up again, but I thought it might be helpful for those who come across this issue in their search results.

I can't speak on behalf of others, but jsbench.me simply uses benchmark.js for its testing purposes. It's a single-page React application, which means it operates entirely within your browser and chosen engine, resulting in consistent outcomes within the same browser. Running tests in Firefox or on mobile may yield different results, as expected. However, all testing components are managed locally, with the exception of utilizing AWS DynamoDB to store test results.

By the way, I'm the creator of the tool, so it's really just a personal passion project. Currently, it doesn't incur any costs for me since it's designed to be serverless and falls within the limits of the AWS free tier. The amount of effort I put into it is directly proportional to the number of users engaging with the platform :)

Answer 5

Answer №3

From what I understand, one issue arises from the fact that different JavaScript engines optimize code in varying ways depending on the environment.

I conducted a test with the same function and found that it yielded different results depending on where the function was created. For instance, in one scenario it looked like this:

const lib = {}
lib.testFn = function() {
   ....
}

While in another scenario it appeared as:

const lib = {
 testFn: function() {
   ....
 },
};

And in yet another case:

function testFn() {
   ....
}

const lib = {}
lib.testFn = testFn

There was a greater than 10% difference in results for a complex function when running in the same browser, and even more variation when testing across different browsers.

This indicates that no JavaScript benchmark can be considered entirely accurate because the way in which the benchmark executes its tests - such as fetching the test script via XHR, using eval, running the test in a worker, or within an iframe - can impact the results due to how each JS engine optimizes these scenarios differently.

Answer 6

From what I understand, one issue arises from the fact that different JavaScript engines optimize code in varying ways depending on the environment.

I conducted a test with the same function and found that it yielded different results depending on where the function was created. For instance, in one scenario it looked like this:

const lib = {}
lib.testFn = function() {
   ....
}

While in another scenario it appeared as:

const lib = {
 testFn: function() {
   ....
 },
};

And in yet another case:

function testFn() {
   ....
}

const lib = {}
lib.testFn = testFn

There was a greater than 10% difference in results for a complex function when running in the same browser, and even more variation when testing across different browsers.

This indicates that no JavaScript benchmark can be considered entirely accurate because the way in which the benchmark executes its tests - such as fetching the test script via XHR, using eval, running the test in a worker, or within an iframe - can impact the results due to how each JS engine optimizes these scenarios differently.

What is the most accurate JS performance testing website?

Answer №1

Answer №2

Answer №3

Similar questions

Is Firefox triggering setTimeout function prematurely (or is Date.getTime() inaccurate)?

How can I destructure the newly added object in a reducer function?

What is the best way to ask users to allow pop-ups in any browser using React.js?

How can I efficiently iterate through the array of index IDs and then iterate individually through the communes, categories, and locations?

What is the behavior of a variable when it is assigned an object?

`Proliferating values through constantly changing addition`

Ways to stop user authentication in Firebase/Vue.js PRIOR to email verification

Incorporating a function from a separate .js file into an index.ejs view using app.js

The function JSON.parse(obj) is malfunctioning and throwing an error, but an online parser is displaying the data correctly

What is the trick to getting an accordion to expand when hovering, rather than when clicking?

Issue: function is not a valid type (during mount)

Secure User Authentication with Node.Js Express

Possible rewrite: "Unable to use jQuery to add elements to data fetched through AJAX requests."

Save the outcome of an ArrayBuffer variable into an array within a Javascript code

Is there a JavaScript equivalent to the explode function in PHP with similar functionality

Encountering exception: Issue occurred post Node version upgrade, indicating "write after end" error

What is the best way to determine the width of a div element and determine if a horizontal scroll bar is present?

Transfer information to ajax request while simultaneously loading

Innovative guidelines originating from a resource attribute

What is the best way to test a React component that triggers a function in its parent, causing the component's props to be modified?