async-hooks.test-emit-after-on-destroyed is flaky #50245

Seems to be a child process issue not async hooks.
The assert checks the exit code given via the childprocess close event which is a number according to docs but here it is null.

Seems to be a child process issue not async hooks. The assert checks the exit code given via the childprocess close event which is a number according to docs but here it is null.

Usually the exit code being null means that the child was ended by a signal.

Similar test is flaky as well: #50262

I started to investigate this issue last week.

I asked @mhdawson to spin up some stress tests

ref: https://ci.nodejs.org/job/node-stress-single-test/nodes=rhel8-ppc64le/469/console

We ran this test case 1000 times on rhel8 ppc64le. This did not reproduce the error though.

I then built Node.js 21.0.0 (The version of node at the time of the issue) on my local machine (Fedora 39). From there I ran this test 100k times and I still was not able to reproduce a test failure.

$ tools/test.py -j 16 --repeat=100000 async-hooks/test-emit-after-on-destroyed
[14:17|% 100|+ 100000|-   0]: Done                               

All tests passed.

As noted before in #50245 (comment), the failure occurs due to a signal be raised but will need find a way to reproduce the failure to get more info on the signal and why the test case is getting signaled.

@mhdawson

Any further suggestions on how to reproduce the error?

Maybe it occurs more when running all the test cases together?

OR

Maybe it presents itself more frequently on some platforms? The original issue indicates it occurred on ppc64 AIX. Maybe stress testing AIX would reproduce the error.

@abmusse I think trying the stress test on one of the platforms where we saw the failure makes sense. I think that @richardlau mentioned you still have access to one of the AIX machines from an earlier investigation so trying the 100k run there would be a good next step.

@mhdawson

Today I ran 100k stress test on one our AIX machines.

$ tools/test.py --repeat=100000 async-hooks/test-emit-after-on-destroyed
[59:37|% 100|+ 100000|-   0]: Done   

Running it 100k times on AIX didn't reproduce the error.

I suggest we un-mark this test as flaky as running it 100k times did not reproduce the error.

I will keep an eye on it and if it returns to a flaky state will handle marking it as flaky again.

Re-opening this as I've seen it failing in CI and a stress run on showed 79/1000 failures in this test. Exclude PR created at #61381

@abmusse ref your earlier comment about it not failing for you on 100 attempts on "our AIX machines" did you mean ones in the Node.js infrastructure or others that you have access to?

This issue has been marked as stale due to 90 days of inactivity.
It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

This issue slipped through the cracks because our previous stale bot only tracked issues and couldn't catch all the issues.
Our new stale bot flagged this, and would have closed it shortly after RenderATL, but I'm just doing it a bit early so
maintainer's can focus on new code-and-learn PRs during the event.

If this is still relevant, feel free to reopen it or leave a comment with additional details so we can continue the discussion.