Repository navigation
events: performance regression in event benchmarks #12657
Description
Activity
Cross posting from #12655 (comment):
It's very probable that the regression you are seeing comes from #11930, but IMO it is more of a problem of unrealistic benchmark (see #11930 (comment) for my reasoning).
I think the problem for this regression is https://bugs.chromium.org/p/v8/issues/detail?id=6376. Specifically https://github.com/nodejs/node/blob/master/lib/events.js#L378, as the regression is present in the remove benchmarks.
I'm working on the
Array.prototype.shiftfast-path in TurboFan.- added a commit that references this issue
on May 10, 2017 - added a commit that references this issue
on May 25, 2017 - added a commit that references this issue
on May 25, 2017 - added a commit that references this issue
on May 28, 2017 10 remaining items
- added a commit that references this issue
on Jun 29, 2017 - added a commit that references this issue
on Jul 11, 2017 - added a commit that references this issue
on Jul 18, 2017 - added a commit that references this issue
on Jul 21, 2017 - added a commit that references this issue
on Jul 24, 2017 Comparing 8.3.0 with 7.9.0:
$ cat compare.csv | Rscript benchmark/compare.R improvement confidence p.value events/ee-add-remove.js n=250000 -30.57 % *** 3.570833e-60 events/ee-emit-multi-args.js n=2000000 -13.31 % *** 1.491800e-38 events/ee-emit.js n=2000000 -14.58 % *** 1.977752e-27 events/ee-listener-count-on-prototype.js n=50000000 12.87 % *** 4.639347e-34 events/ee-listeners-many.js n=5000000 34.51 % *** 1.745743e-50 events/ee-listeners.js n=5000000 19.73 % *** 6.707594e-34 events/ee-once.js n=20000000 -40.54 % *** 5.223855e-52 $Those numbers were taken using the 8.3.0 versions of those benchmarks?
Those numbers were taken using the 8.3.0 versions of those benchmarks?
I used the version in the master branch.
Also: I ran it on macOS 10.12.6.
I've tested on Windows 7 x 64, vs V8 6.0 / 6.1 / 6.2 and things seem getting better:
V8 5.5.372.43 (Node.js 7.10.1) vs V8 6.0.286.52 (Node.js 8.3.0) improvement confidence p.value events\\ee-add-remove.js n=250000 -42.77 % *** 4.141429e-90 events\\ee-emit-multi-args.js n=2000000 -7.56 % *** 3.765808e-27 events\\ee-emit.js n=2000000 -9.59 % *** 5.277457e-27 events\\ee-listener-count-on-prototype.js n=50000000 12.72 % *** 9.573892e-38 events\\ee-listeners-many.js n=5000000 21.15 % *** 4.807840e-43 events\\ee-listeners.js n=5000000 6.91 % *** 1.381210e-06 events\\ee-once.js n=20000000 -45.65 % *** 1.205400e-57 V8 5.5.372.43 (Node.js 7.10.1) vs V8 6.1.556 (Node.js 9.0.0 2017-07-20 V8-canary) improvement confidence p.value events\\ee-add-remove.js n=250000 -12.04 % *** 1.846156e-35 events\\ee-emit-multi-args.js n=2000000 10.96 % *** 1.395131e-29 events\\ee-emit.js n=2000000 2.63 % *** 5.275229e-05 events\\ee-listener-count-on-prototype.js n=50000000 8.65 % *** 3.501371e-33 events\\ee-listeners-many.js n=5000000 18.98 % *** 3.670456e-37 events\\ee-listeners.js n=5000000 5.45 % *** 8.625583e-10 events\\ee-once.js n=20000000 -31.56 % *** 1.767708e-43 V8 5.5.372.43 (Node.js 7.10.1) vs V8 6.2.217 (Node.js 9.0.0 2017-08-12 V8-canary) improvement confidence p.value events\\ee-add-remove.js n=250000 -9.71 % *** 7.300658e-24 events\\ee-emit-multi-args.js n=2000000 21.74 % *** 5.781329e-30 events\\ee-emit.js n=2000000 10.67 % *** 3.856524e-20 events\\ee-listener-count-on-prototype.js n=50000000 6.88 % *** 2.059200e-11 events\\ee-listeners-many.js n=5000000 14.08 % *** 3.452212e-45 events\\ee-listeners.js n=5000000 -1.82 % ** 1.769813e-03 events\\ee-once.js n=20000000 -26.43 % *** 5.119348e-73
There doesn't seem to be much actionable here. The benchmarks should probably be updated to be more comprehensive but that's an issue across the board. The performance as per above is now nearly at par or better.
See: #12655 (comment)
I'm seeing a significant performance regression in the events benchmarks comparing master to 7.9.
/cc @mscdex @mcollina @nodejs/benchmarking