Repository navigation
node/openssl/ssl.h should compile without any additional defines #40575
Description
Activity
@danbev thoughts on this?
I haven't looked at node-gyp at all during the OpenSSL work so if this is something that is not covered by the OpenSSL tests in test/addons (openssl-binding, openssl-client-cert-engine, or openssl-key-engine) then I've missed it I'm afraid.
I added OPENSSL_API_COMPAT to node.gypi to avoid warnings due to the usage of deprecated APIs. We could remove this and live with the warnings but since we need to keep backward compatibility with OpenSSL 1.1.1 for some time and it might become annoying seeing these.
@danbev You should be able to include node/openssl/ssl.h from a standalone application from outside the Node.js tree
This is the test:
#include "openssl/ssl.h"
int main() {
return 0;
}If you have ever compiled a native module with 17.0.1 you will have this directory:
gcc -I${HOME}/.cache/node-gyp/17.0.1/include/node -o openssl openssl.c${HOME}/Caches on macOS
Normally, the OpenSSL of node-gyp should be a drop-in replacement for a system-installed OpenSSL - it allows to build a native addon that uses OpenSSL without linking in a second copy in the Node executable - which might not work at all
On windows, and alpine linux docker image, updating to node 17...
After getting around OPENSSL_API_COMPAT by just defining it in CMakeList for it (and the binding.gyp), I ran into
error "The requested API level higher than the configured API compatibility level"
which OPENSSL_CONFIGURED_API is undefined... so being 0, anything I set as a compatibility level fails.
adding this to the top of ....cmake-js\node-x64\v17.0.1\include\node\openssl\macros.h
#include "./configuration.h"
(which I don't think is a good or correct solution)
But even then I had to define the OPENSSL_MIN_API=30000 in my build configuration
Edit: Sorry for being terse and vague.
After tracking down this, and getting some hints, I made forward progress... there's still missing symbols (RSA_new() and RSA_free() for examples). There's a openssl/configuration.h file which is new in 17 that was not in 16. This configuration.h is not included by anything in openssl directory, and seems the user is supposed to include this file now too?
Just chiming in that I'm also seeing this with node v17.x ("OPENSSL_API_COMPAT expresses an impossible API compatibility level").
Any (official) solutions to this? /cc @nodejs/crypto
Just chiming in that I'm also seeing this with node v17.x ("OPENSSL_API_COMPAT expresses an impossible API compatibility level").
Any (official) solutions to this? /cc @nodejs/crypto
"OPENSSL_API_COMPAT=10101"
and
#if NODE_MAJOR_VERSION >= 17
# include <openssl/configuration.h>
#endifbefore including anything from OpenSSL. If configuration.h is not defined than the configured version is undefined and becomes a '0' which everything is bigger than the configured level then.
@danbev How about including
#define OPENSSL_API_COMPAT=10101innode.h?
Sorry about the late reply. I'm currently on a leave of absence and will be back working in April, and would be happy to take a closer look at this then (unless someone else beats me to it).
"OPENSSL_API_COMPAT=10101"
and
#if NODE_MAJOR_VERSION >= 17 # include <openssl/configuration.h> #endifbefore including anything from OpenSSL. If configuration.h is not defined than the configured version is undefined and becomes a '0' which everything is bigger than the configured level then.
Unfortunately that doesn't seem to work for me, I still get the same error.
Looks like part of the problem in my particular case is that homebrew on macos is using OpenSSL 1.1.1 with their node v17.x formula, which causes problems for addons because of the OpenSSL 3.0 headers that get pulled in by node-gyp. The addon thinks it's compiling for OpenSSL 3.0 but then during runtime you can get various errors from dyld depending on if you are using new OpenSSL APIs or OpenSSL APIs that are now aliases in 3.0.
I've taken a look at this and the following is what I've found.
Looking at the the error when running node-gyp build (with the addition of some
debugging of the macros involved to help troubleshoot this):
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:83:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’
83 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:141:9: note: ‘#pragma message: OPENSSL_VERSION_MAJOR: 3’
141 | #pragma message "OPENSSL_VERSION_MAJOR: " OPENSSL_MSTR(OPENSSL_VERSION_MAJOR)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:142:9: note: ‘#pragma message: OPENSSL_VERSION_MINOR: 0’
142 | #pragma message "OPENSSL_VERSION_MINOR: " OPENSSL_MSTR(OPENSSL_VERSION_MINOR)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:143:9: note: ‘#pragma message: OPENSSL_VERSION: (3 * 1000 + OPENSSL_MINOR * 100)’
143 | #pragma message "OPENSSL_VERSION: " OPENSSL_MSTR((OPENSSL_VERSION_MAJOR * 1000 + OPENSSL_MINOR * 100))
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:144:9: note: ‘#pragma message: OPENSSL_API_LEVEL: (30000)’
144 | #pragma message "OPENSSL_API_LEVEL: " OPENSSL_MSTR(OPENSSL_API_LEVEL)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:145:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’
145 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:146:9: note: ‘#pragma message: OPENSSL_CONFIGURED_API: OPENSSL_CONFIGURED_API’
146 | #pragma message "OPENSSL_CONFIGURED_API: " OPENSSL_MSTR(OPENSSL_CONFIGURED_API)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:148:4: error: #error "The requested API level higher than the configured API compatibility level"
148 | # error "The requested API level higher than the configured API compatibility level"
| ^~~~~
make: *** [example.target.mk:113: Release/obj.target/example/example.o] Error 1Notice that OPENSSL_CONFIGURED_API has not been set which is the cause of
the error message above.
Now, macros.c has the following include:
#include <openssl/opensslconf.h>
#include <openssl/opensslv.h>In Node.js this is a "redirect" openssl header which choose different headers
depending of whether ASM of NO_ASM was configured:
#if defined(OPENSSL_NO_ASM)
# include "./opensslconf_no-asm.h"
#else
# include "./opensslconf_asm.h"
#endifAnd if we assume this using ASM this would end up in:
#elif defined(OPENSSL_LINUX) && defined(__x86_64__)
# include "./archs/linux-x86_64/asm/include/openssl/opensslconf.hAnd this file does not include configuration.h.
In OpenSSL the following headers are generated:
$ find . -name '*.in'
./include/openssl/conf.h.in
./include/openssl/lhash.h.in
./include/openssl/crmf.h.in
./include/openssl/err.h.in
./include/openssl/safestack.h.in
./include/openssl/ess.h.in
./include/openssl/opensslv.h.in
./include/openssl/ocsp.h.in
./include/openssl/pkcs12.h.in
./include/openssl/asn1.h.in
./include/openssl/bio.h.in
./include/openssl/x509.h.in
./include/openssl/x509v3.h.in
./include/openssl/cmp.h.in
./include/openssl/pkcs7.h.in
./include/openssl/srp.h.in
./include/openssl/x509_vfy.h.in
./include/openssl/asn1t.h.in
./include/openssl/ct.h.in
./include/openssl/fipskey.h.in
./include/openssl/ui.h.in
./include/openssl/configuration.h.in
./include/openssl/crypto.h.in
./include/openssl/cms.h.in
./include/openssl/ssl.h.in
./include/crypto/bn_conf.h.in
./include/crypto/dso_conf.h.inAnd notice that opensslconf.h is not among these.
What I think is happening is that we have an old opensslconf.h which may
have been generated with past versions but is not generated any more, instead
configuration.h is the file being generated.
If we replace this header,
/.cache/node-gyp/17.8.0/include/node/openssl/opensslconf.h, with the header
from OpenSSL 3.0:
$ cp opensslconf.h /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/opensslconf.hAnd the run the build again:
$ node-gyp build
gyp info it worked if it ends with ok
gyp info using node-gyp@9.0.0
gyp info using node@17.8.0 | linux | x64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/home/danielbevenius/work/nodejs/node-gyp-openssl-issue/build'
CC(target) Release/obj.target/example/example.o
In file included from /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/././archs/linux-x86_64/asm/include/openssl/ssl.h:21,
from /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/./ssl_asm.h:11,
from /home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/ssl.h:4,
from ../example.c:1:
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:83:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’
83 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:141:9: note: ‘#pragma message: OPENSSL_VERSION_MAJOR: 3’
141 | #pragma message "OPENSSL_VERSION_MAJOR: " OPENSSL_MSTR(OPENSSL_VERSION_MAJOR)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:142:9: note: ‘#pragma message: OPENSSL_VERSION_MINOR: 0’
142 | #pragma message "OPENSSL_VERSION_MINOR: " OPENSSL_MSTR(OPENSSL_VERSION_MINOR)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:143:9: note: ‘#pragma message: OPENSSL_API_LEVEL: (30000)’
143 | #pragma message "OPENSSL_API_LEVEL: " OPENSSL_MSTR(OPENSSL_API_LEVEL)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:144:9: note: ‘#pragma message: OPENSSL_API_COMPAT: 30000’
144 | #pragma message "OPENSSL_API_COMPAT: " OPENSSL_MSTR(OPENSSL_API_COMPAT)
| ^~~~~~~
/home/danielbevenius/.cache/node-gyp/17.8.0/include/node/openssl/macros.h:145:9: note: ‘#pragma message: OPENSSL_CONFIGURED_API: 30000’
145 | #pragma message "OPENSSL_CONFIGURED_API: " OPENSSL_MSTR(OPENSSL_CONFIGURED_API)
| ^~~~~~~
SOLINK_MODULE(target) Release/obj.target/example.node
COPY Release/example.node
make: Leaving directory '/home/danielbevenius/work/nodejs/node-gyp-openssl-issue/build'
gyp info ok Solution
So we should remove this template from Node.js:
$ git st .
On branch opensslconf
Changes to be committed:
(use "git restore --staged <file>..." to unstage)
deleted: deps/openssl/config/opensslconf.h
deleted: deps/openssl/config/opensslconf.h.tmpl
deleted: deps/openssl/config/opensslconf_no-asm.hWe can create the tar-headers file (by making some updates to the Makefile
to allow this to be built for a non-release):
$ make node-v17.9.1.tar-headers
First we can revert the headers:
$ node-gyp remove v17.8.0And running node-gyp rebuild should fail with the above error message.
And then unzip the new headers to replace the headers used by node-gyp:
$ tar xvzf node-v17.9.1-headers.tar.gz -C ~/.cache/node-gyp/17.8.0 --strip 1And this running node-gyp should succeed:
$ node-gyp build
gyp info it worked if it ends with ok
gyp info using node-gyp@9.0.0
gyp info using node@17.8.0 | linux | x64
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
make: Entering directory '/home/danielbevenius/work/nodejs/learning-nodejs/node-gyp-openssl-issue/build'
make: Nothing to be done for 'all'.
make: Leaving directory '/home/danielbevenius/work/nodejs/learning-nodejs/node-gyp-openssl-issue/build'There are addons in Node.js that include openssl/ssl.h but this error does not
happen for those and I've not had time to find out why this is the case.
Version
17.0.1
Platform
5.11.0-38-generic #42~20.04.1-Ubuntu SMP
Subsystem
No response
What steps will reproduce the bug?
node/openssl/ssl.his exported in thenode-gypbundle to be used by native modules that need to link against the openssl symbols in NodeCurrently, when building Node.js itself some of these defines come from
openssl.gyp:OPENSSL_API_COMPATandOPENSSL_CONFIGURED_APIThis means that a native addon must also define them and must do so in a manner compatible with Node.js across the eventual future versions
How often does it reproduce? Is there a required condition?
No response
What is the expected behavior?
These defines should be present somewhere in an additional include file to be added to the
node-gypbundle distributionWhat do you see instead?
No response
Additional information
No response