ext/openssl: add a dtls:// stream transport. Open as Draft.

[GianfriAur]

Adds DTLS to ext/openssl: the dtls://, dtlsv1.2:// and dtlsv1.3:// stream transports
(client and server) and DTLS over udp:// through stream_socket_enable_crypto(). The ssl
context options, Openssl\Session and the stream API work for DTLS as they do for tls://.

// client
$ctx = stream_context_create(['ssl' => ['cafile' => '/path/ca.pem']]);
$c = stream_socket_client('dtls://198.51.100.1:4433', $errno, $errstr, 5,
        STREAM_CLIENT_CONNECT, $ctx);
fwrite($c, $datagram);
$reply = fread($c, 1500);

// server: one socket, any number of peers
$ctx = stream_context_create(['ssl' => ['local_cert' => '/path/server.pem']]);
$s = stream_socket_server('dtls://0.0.0.0:4433', $errno, $errstr,
        STREAM_SERVER_BIND | STREAM_SERVER_LISTEN, $ctx);
while ($peer = stream_socket_accept($s, 5, $addr)) {
    fwrite($peer, strtoupper(fread($peer, 1500)));
}

Design

The implementation is split by layer rather than by protocol, so DTLS adds no second copy of the
option handling or of the IO loops:

• xp_bio.c, the ciphertext transport. OpenSSL never does IO itself. Its BIO serves records
  from a receive queue and appends to a send queue owned by the connection, and the stream moves
  ciphertext between the queues and the transport around each SSL_*() call. No call into OpenSSL
  ever waits, and the socket stays non-blocking once crypto is set up, with the stream's blocking
  mode emulated by waiting with a monotonic deadline. The transport is the stream's socket, or any
  other stream given by the new inner_stream context option (a tcp:// stream or a user wrapper),
  which makes TLS and DTLS over an arbitrary stream possible. DTLS queues whole datagrams with their
  peer address and folds the retransmit timer into every wait.
• xp_ssl.c, one stream implementation for TLS and DTLS. The datagram schemes select the DTLS
  methods and version range; everything else (verification, SNI, ALPN, PSK, sessions, early data,
  ciphers, security level, meta data) is shared. The old gettimeofday() loops and the switching of
  the descriptor mode are gone.
• xp_dtls.c, the server port. One UDP socket carries every peer of a server. The port receives
  the datagrams, routes them by peer address to their connection, runs the handshakes of new peers
  with a cookie exchange (SSL_OP_COOKIE_EXCHANGE, timestamped HMAC cookies, a pending cap and
  timeout) and hands the connections that completed to stream_socket_accept() as streams sharing
  the socket. It uses neither DTLSv1_listen() (DTLS 1.2 only, single peer) nor the OpenSSL 4.1
  listener API, so it works with every supported OpenSSL from 1.1.1.

Semantics

• DTLS behaves like tls://: verify_peer_name is separate from verify_peer, a peer_fingerprint
  check comes on top of verify_peer => false, sessions are captured with session_new_cb, and
  the handshake completes inside stream_socket_accept().
• A dtls:// client negotiates DTLS 1.2 or 1.3 (OpenSSL 4.1+). A server offers DTLS 1.3 only when
  asked with an explicit crypto_method, because outside the OpenSSL listener a DTLS 1.3 server
  cannot validate the peer address before its first flight.
• New ssl context options: inner_stream, dtls_link_mtu, dtls_max_pending (default 256),
  dtls_pending_timeout (seconds, default 30), keying_material_label and
  keying_material_length (RFC 5705 export in stream_get_meta_data()['crypto']).
• New constants STREAM_CRYPTO_METHOD_DTLSv1_2_*, STREAM_CRYPTO_METHOD_DTLSv1_3_* and
  STREAM_CRYPTO_METHOD_DTLS_ANY_*. The stream type of udp:// streams is udp_socket/dtls when
  the extension is loaded, as tcp:// is tcp_socket/ssl.

Tests

ext/openssl/tests/dtls_*: client against openssl s_server, PHP server and client, mutual
authentication, fingerprints, session resumption on both sides, MTU, robustness against bogus
datagrams, a server with several peers on one socket, DTLS 1.3, DTLS over udp://, a non-blocking
handshake; tls_inner_stream*.phpt for TLS over a tcp:// stream and over a user wrapper. Verified
against OpenSSL 1.1.1, 3.0 and the 4.2-dev tree (for DTLS 1.3).

Discussed on internals: https://externals.io/message/131514.

TODO before removing Draft status

• Windows build and tests
• UPGRADING and NEWS
• php.net documentation
• Coding standards pass and squash of the history

[bukka]

Hmm the amount of duplication is quite big. I think this is more a good PoC but it's very far from anything that we could merge. We might need to re-architecture the whole xp_ssl for this. I will need to do more research and thinking to see what would be the best way.

[bukka]

I don't think this is the right approach. We should re-use (and possibly extend if needed) the underlaying udp stream so it works in the same way as tls that is build on top tcp.

Thinking about it, we should really go with enable_crypto support from the beginning as it will mirror better the TLS code.

[GianfriAur]

Okay, great. I’ve already made an attempts. I’ll work on it over the next few days.

[bukka]

We should not use DTLSv1_listen

[GianfriAur]

Hmm the amount of duplication is quite big. I think this is more a good PoC but it's very far from anything that we could merge. We might need to re-architecture the whole xp_ssl for this. I will need to do more research and thinking to see what would be the best way.

Okay, do you want me to try drafting an xp_common file?

[bukka]

Okay, do you want me to try drafting an xp_common file?

Yeah I think it would be actually better.

[GianfriAur]

Honestly I'm not a big fan of adding is_dgram to php_netstream_data_t, but I couldn't make dtls:// reuse the socket transport the clean way tls:// reuses it for tcp://.

It comes down to an asymmetry in the generic socket transport. tls:// gets the reuse for free because TCP is the default socket type there — the comment in xp_socket.c even spells it out:
/* Note: the test here for php_stream_udp_socket_ops is important, because we want the default to be TCP sockets so that the openssl extension can re-use this code. */

That said, I'm not thrilled with it, if anyone has a cleaner idea, it's very welcome.

[GianfriAur]

@bukka, I've sketched out an xp_common; let me know what you think and what changes you'd make.

The approach: I moved into xp_common.{c,h} only the machinery that xp_ssl and xp_dtls genuinely share as-is, fingerprint matching, peer-name resolution, the passphrase callback, local cert/key loading, the verify callback + enable/disable peer verification, the cipher metadata helper, and the
session-resumption infrastructure (callbacks struct, validate/allocate, and the SSL_CTX ex-data indices the callbacks use to reach the stream without knowing the transport's netstream type).

I deliberately kept the setup_client_session / setup_server_session orchestration per-transport rather than merging it: DTLS has no cross-connection internal cache (fresh context per accepted peer), so the cache-mode policy actually differs, and unifying it would mean re-introducing those
differences behind flags. My guiding rule was to share the reusable building blocks but not force a common abstraction over code paths that only look similar.

[bukka]

I had a deeper look into this and realised that DTLSv1_listen wasn't really the way forward given that it cannot work with DTLS 1.3. Unfortunately SSL_new_listener for DTLS was just introduced in 4.1 so we couldn't properly support older version. So another solution was needed. As I have been working on IO Hooks, I needed to introduce custom BIO and planned for some rewrite of that. The fact that you also needed user wrapper integration also makes it necessary. This actually gives possibility to create a custom listener and map it to peers ourselves which is exactly what I did. There is one omission in OpenSSL as it ignores SSL_OP_COOKIE_EXCHANGE for DTLS 1.3 which I'm trying to address in openssl/openssl#33093 so will see if it gets to 4.1 as a bug fix. This does not impact client - it's just for server that needs to verify address (default). The current version of this patch will default to DTLS 1.2 for server unless crypto_method explicitly includes DTLS 1.3. We can update it once that OpenSSL is resolved. I also aligned the verify and session options with tls:// logic to make it a bit more consistent.

[GianfriAur]

If sslsock->port != NULL, php_openssl_dtls_detach is always called twice when close_handle is ture ,
the first one at line 3207.

[GianfriAur]

@bukka First of all, thank you very much for the beautiful refactor.

That said, I quickly read through the code and pointed out the only thing I noticed at a glance.
I'm waiting for the openssl/33093 to read a bit more in depth.

I also ran some tests and realized there's a small leak issue; in one case, sharing a stream and closing one causes a segfault. I think it's due to 'inner_stream', calling it a "leak" might be a bit of an exaggeration, perhaps, but it’s certainly not the kind of behavior I would expect.

Here is a snippet of code.

$inner = fopen('php://memory', 'w+');
$ctx = stream_context_create(['ssl' => ['inner_stream' => $inner]]);
  
$a = stream_socket_client('tcp://127.0.0.1:1', $e, $es, 1, STREAM_CLIENT_CONNECT, $ctx);
var_dump($a !== false);
fwrite($a, "MY-DATA");
rewind($inner);
var_dump(stream_get_contents($inner)); // MY-DATA

$b = stream_socket_client('tcp://127.0.0.1:1', $e, $es, 1, STREAM_CLIENT_CONNECT, $ctx);
fclose($a);
var_dump(fclose($inner)); // bool(true)
fwrite($b, "after-free");  // segfault

php://memory only makes the first problem visible without a server; any inner stream behaves the same

As for the segfault, it occurs regardless of the stream, even during legitimate operations.
like:

$inner = stream_socket_client('tcp://IP:PORT', $errno, $errstr, 5);
$ctx = stream_context_create(['ssl' => [
    'inner_stream' => $inner,
    'verify_peer' => false,
    'verify_peer_name' => false,
]]);

$first = stream_socket_client('tls://IP:PORT', $errno, $errstr, 5, STREAM_CLIENT_CONNECT, $ctx);
$second = stream_socket_client('tls://IP:PORT', $errno, $errstr, 1, STREAM_CLIENT_CONNECT, $ctx);

var_dump(fclose($inner));  // bool(true)
fclose($first);            // segfault