Repository navigation
test-process-uid-gid.js (& euid-egid) failing in Node.js 10.x on Alpine 3.9 with Ubuntu 18.04 hosts #29977
Description
Activity
welp, V8 doesn't support DEBUG against musl apparently, so the above is the best we're going to get.
../deps/v8/src/external-reference-table.cc:14:10: fatal error: execinfo.h: No such file or directory #include <execinfo.h> ^~~~~~~~~~~~ compilation terminated.@nodejs/releasers I'd say for now that these two failures on 10.x on alpine-last-latest-x64 are acceptable and should now hold up a release.
I can take a look if you want. Do I have access? What's the address to ssh to?
#include <execinfo.h>This lives in
libexecinfo-devin alpine.libexecinfo-dev was 👍 but it fails later
/home/iojs/build/workspace/node-test-commit-linux/nodes/alpine-last-latest-x64/out/../deps/v8/src/external-reference-table.cc:45: undefined reference to `backtrace_symbols'installing
libunwinddoesn't help either.@bnoordhuis you should have access, it's
test-digitalocean-ubuntu1804_docker-x64-2(root@159.89.183.200 - but if you haven't runansible-playbook playbooks/write-ssh-config.ymlin build/ansible then maybe you should so you get the ssh config all set up).I don't know how much help you need with Docker & Alpine but here's a quick rundown to get you in and working:
docker pswill show you the running containers, the one I've taken offline for debugging this isnode-ci:test-digitalocean-alpine39_container-x64-2which is tagged asnode-ci-test-digitalocean-alpine39_container-x64-2docker exec -ti node-ci-test-digitalocean-alpine39_container-x64-2 bashwill get you in to the container as useriojs. You can dodocker exec -ti -u rootto get in as root andsu iojsfrom within there as you need./home/iojs/build/workspace/node-test-commit-linux/nodes/alpine-last-latest-x64inside the container is where there's a v10.x checked out, you can mess with that./home/iojs/is mounted from/home/iojs/test-digitalocean-alpine39_container-x64-2/on the host if that helps.- If you need to install packages,
apk add(I've already added libexecinfo-dev as @jbergstroem suggested). I don't mind what you do to this container, I'll reset it all when you're done. Add editors or whatever else you like. /lib/systemd/system/jenkins-test-digitalocean-alpine39_container-x64-2.serviceon the host has the startup for this container, you can see my customulimitin there as well as the mount of/var/crash. You're welcome to modify this and restart as you need, I'll assume you're comfortable with systemd. I don't mind what you do here, I'll reset it when you're done.
Maybe obvious, or too difficult to arrange, but I suspect a
-lexecinfois needed.I've had to rerun ansible on this machine to update the sharedlibs containers, so the systemd config is reset and Dockerfile was reset along with its image too. Feel free to edit them again as needed, that alpine39 container is still marked as offline.
Same failure on both Alpine 3.9 and Alpine 3.10 for 10.x here: https://ci.nodejs.org/job/node-test-commit-linux/30365/
Perhaps there's something particular about the base machines that's causing it to be flaky. It might explain why this didn't show up when I ran the initial tests after updating the container hosts.
and both on 8.x https://ci.nodejs.org/job/node-test-commit-linux/30366/
🤷
@BethGriggs picked this up @ nodejs/build#1945 (comment)
We've switched our Alpine 3.9 containers to Ubuntu 18.04 hosts, from 16.04. So that's a ~4.4.0 kernel to a ~4.15.0 and these two tests are now reliably failing with segfaults: test-process-uid-gid.js & test-process-euid-egid.js.
With some effort I've managed to generate a core dump but I don't think it's very helpful:
The latest Alpine is 3.10 and it's working fine for 10.x (and above), so this is specifically for the last gen Alpine with last gen Node. How much does this matter, and do we have someone with expertise and time to dive into this? I've taken an Alpine 3.9 container out of CI for messing around with this and am happy to give someone access & instructions if you want to toy with it.
If I manage to make a debug build core dump then I'll paste that in, maybe it'll be more interesting?
For my own reference, to help clean up:
--ulimit core=99999999999:99999999999nodeas~iojs/node-10-segfault, uid-gid.js core dump is/var/crash/core.node.78and euid-egid.js core dump is/var/crash/core.node.86.