a new config option _no_rcu_ is added into HT_CONFIG. When _no_rcu_ is
set then hashtable can be guarded with any other locking primitives,
and behives as ordinary hashtable. Also, all the impact of the
atomics used internally to the hash table was mitigated.
RCU performance
# INFO: @ test/lhash_test.c:747
# multithread stress runs 40000 ops in 40.779656 seconds
No RCU, guarded with RWLOCK
# INFO: @ test/lhash_test.c:747
# multithread stress runs 40000 ops in 36.976926 seconds
Signed-off-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28677)
When defining a custom hash function for a hashtable key, you typically start with:
HT_START_KEY_DEFN(key)
HT_DEF_KEY_FIELD(k, unsigned char *)
HT_END_KEY_DEFN(KEY)
In this setup, the hash function signature requires keybuf and len as
parameters rather than the hashtable key itself. As a result,
accessing members of the hashtable structure becomes awkward, since
you must do something like:
#define FROM_KEYBUF_TO_HT_KEY(keybuf, type) (type)((keybuf) - sizeof(HT_KEY))
static uint64_t ht_hash(uint8_t *keybuf, size_t keylen)
{
KEY *k = FROM_KEYBUF_TO_HT_KEY(keybuf, KEY *);
...
}
This kind of pointer arithmetic is both unnecessary and error-prone.
A cleaner approach is to pass the HT pointer directly into the hash
function. From there, you can safely cast it to the required type
without the pointer gymnastics.
Signed-off-by: Nikola Pajkovsky <nikolap@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28677)
The documentation suggested that they were always zero, while the
implementation in <openssl/opensslv.h> suggested that it could be
0xf in OpenSSL releases... which (almost) never happened because
of a bug in said implementation.
Therefore, we solidify that the status bits are indeed always zero,
at least in all OpenSSL 3 versions.
Resolves: https://github.com/openssl/project/issues/1621
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28603)
This reverts commit dc5cd6f70a "rsa: expose pairwise consistency test API",
that has introduced ossl_rsa_key_pairwise_test() function, as the only user
has been removed in 7f7f75816f "import pct: remove import PCTs for most
algorithms".
Complements: 7f7f75816f "import pct: remove import PCTs for most algorithms"
Signed-off-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Richard Levitte <levitte@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28557)
Switching from ANSI-C we can use implementation of printf like
function provided by libc on target platform. This applies
starting from 3.6 and onwards.
The slight exception here is old windows printf functions
before 2015, those are supported.
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28305)
Current QUIC stack may leave connection monitored by SSL_poll() to stale
during regular shutdown. The issue is triggered when ACK for client's
FIN gets delayed. The sequeance of operations to trigger
the stale of QUIC connection at client goes as follows:
- application calls SSL_shutdown() on connection,
the shutdown can not proceed, because bi-directional
stream must be flushed. The client awaits ACK from
server acknowledging reception of FIN on client's stream
- the stream object gets destroyed, because application
received all data from server.
- application updates poll set and passes to SSL_poll()
- ssl poll ticks the engine. Engine receives delayed ACK
and marks stream as flushed. At this point the SSL_shutdown()
operation may proceed given the application calls the
SSL_shutdown(). However there is no mechanism to make SSL_poll()
return so application is unable to proceed with its event
loop where SSL_shutdown() may get called.
This change introduces ossl_quic_channel_notify_flush_done() function
which notifies channel when all streams are flushed (all FINs got ACKed).
The first thing SSL_shudown() does it calls ossl_quic_stream_map_begin_shutdown_flush().
The function walks list of all streams attached to channel and notes how many
streams is missing ACK for their FIN. In our test case it finds one such stream.
Call to SSL_shutdown() returns and application destroys the SSL stream object
and updates a poll set.
SSL_poll() gets called. The QUIC stack (engine) gets ticked and reads data
from socket. It processes delayed ACK now. The ACK-manager updates the
stream notifying the server ACKs the FIN sent by client. The stream
is flushed now. Thw shutdown_flush_done() for stream gets called on
behalf of ACK manager.
The shutdown_flush_done() does two things:
- it marks stream as flushed
- it decrements the num_shutdown_flush counter initialized
be earlier call to ossl_quic_stream_map_begin_shutdown_flush()
called by SSL_shutdown()
The change here calls ossl_quic_channel_notify_flush_done() when
num_shutdown_flush reaches zero.
The ossl_quic_channel_notify_flush_done() then calls function
ossl_quic_channel_notify_flush_done(), which just moves the state
of the channel (connection) from active to terminating state.
The change of channel state is sufficent for SSL_poll() to
signal _EC event on connection.
Once application receives _EC event on connection it should
check the state of the channel/reason of error. In regular case
the error/channel state hints application to call SSL_shutdown()
so connection object can proceed with connection shutdown.
The SSL_shutdown() call done now moves channel to terminated
state. So the next call to SSL_poll() can signal _ECD which
tells application it's time to stop polling on SSL connection
object and destroy it.
Fixesopenssl/project#1291
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28116)
The __attribute__((malloc)) is for functions that return new memory,
and "the memory [returned by the function] has undefined content", which
is a property that doesn't hold for the *dup functions (the same reason
it doesn't apply to realloc).
Fixes: e1035957eb "OSSL_CRYPTO_ALLOC attribute introduction proposal."
Signed-off-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Neil Horman <nhorman@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28220)
during memfail testing:
https://github.com/openssl/openssl/actions/runs/16794088536/job/47561223902
We get lots of test failures in ossl_rcu_read_lock. This occurs
because we have a few cases in the read lock path that attempt mallocs,
which, if they fail, trigger an assert or a silent failure, which isn't
really appropriate. We should instead fail gracefully, by informing the
caller that the lock failed, like we do for CRYPTO_THREAD_read_lock.
Fortunately, these are all internal apis, so we can convert
ossl_rcu_read_lock to return an int indicating success/failure, and fail
gracefully during the test, rather than hitting an assert abort.
Fixesopenssl/project#1315
Reviewed-by: Paul Yang <paulyang.inf@gmail.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
(Merged from https://github.com/openssl/openssl/pull/28195)
Such routines allow alleviating the need to perform explicit integer
overflow check during allocation size calculation and generally make
the allocations more semantic (as they signify that a collection
of NUM items, each occupying SIZE bytes is being allocated), which paves
the road for additional correctness checks in the future.
Signed-off-by: Eugene Syromiatnikov <esyr@openssl.org>
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Neil Horman <nhorman@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28059)
Recently, our overnight QUIC interop runs began failing in CI when an
openssl server was tested against an ngtcp2 client:
https://github.com/openssl/openssl/actions/runs/16739736813
The underlying cause bears some explination for historical purposes
The problem began happening with a recent update to ngtcp2 in which
ngtcp2 updated its wolfssl tls backend to support ML-KEM, which caused
ngtcp to emit a client hello message that offered several groups
(including X25519MLKEM768) but only provided a keyshare for x25519.
This in turn triggered the openssl server to respond with a hello retry
request (HRR), requesting an ML-KEM keyshare instead, which ngtcp2
obliged. However all subsequent frames from the client were discarded by
the server, due to failing packet body decryption.
The problem was tracked down to a mismatch in the initial vectors used
by the client and server, leading to an AEAD tag mismatch.
Packet protection keys generate their IV's in QUIC by xoring the packet
number of the received frame with the base IV as derived via HKDF in the
tls layer.
The underlying problem was that openssl hit a very odd corner case with
how we compute the packet number of the received frame. To save space,
QUIC encodes packet numbers using a variable length integer, and only
sends the changed bits in the packet number. This requires that the
receiver (openssl) store the largest received pn of the connection,
which we nominally do.
However, in default_port_packet_handler (where QUIC frames are processed
prior to having an established channel allocated) we use a temporary qrx
to validate the packet protection of those frames. This temporary qrx
may be incorporated into the channel in some cases, but is not in the
case of a valid frame that generates an HRR at the TLS layer. In this
case, the channel allocates its own qrx independently. When this
occurs, the largest_pn value of the temporary qrx is lost, and
subsequent frames are unable to be received, as the newly allocated qrx
belives that the larges_pn for a given pn_space is 0, rather than the
value received in the initial frame (which was a complete 32 bit value,
rather than just the changed lower 8 bits). As a result the IV
construction produced the wrong value, and the decrypt failed on those
subsequent frames.
Up to this point, that wasn't even a problem, as most quic
implementations start their packet numbering at 0, so the next packet
could still have its packet number computed properly. The combination
of ngtcp using large random values for initial packet numbers, along
with the HRR triggering a separate qrx creation on a channel led to the
discovery of this discrepancy.
The fix seems pretty straightforward. When we detect in
port_default_packet_handler, that we have a separate qrx in the new
channel, we migrate processed packets from the temporary qrx to the
canonical channel qrx. In addition to doing that, we also need to
migrate the largest_pn array from the temporary qrx to the channel_qrx
so that subsequent frame reception is guaranteed to compute the received
frame packet number properly, and as such, compute the proper IV for
packet protection decryption.
Fixesopenssl/project#1296
Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28115)
Functions like ERR_set_mark(), ERR_clear_last_mark(), and ERR_pop_to_mark()
are already passed to the a provider via the 'in' dispatch array of the
provider initialization function (although the documentation did not
mention them).
Also pass ERR_count_to_mark() to the provider the same way, and update
the documentation to mention all four functions.
Signed-off-by: Ingo Franzki <ifranzki@linux.ibm.com>
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/28073)
Also add support for ML-KEM in CMS (draft-ietf-lamps-cms-kyber).
Add the -recip_kdf and -recip_ukm parameters to `openssl cms -encrypt`
to allow the user to specify the KDF algorithm and optional user
keying material for each recipient.
A provider may indicate which RecipientInfo type is supported
for a key, otherwise CMS will try to figure it out itself. A
provider may also indicate which KDF to use in KEMRecipientInfo
if the user hasn't specified one.
Reviewed-by: Dmitry Belyavskiy <beldmit@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/27681)
