The return value of RSA_*_{en,de}crypt() is signed

The functions RSA_(public|private)_(en|de)crypt() return a signed
result, in particular `-1` may be returned on error, so the caller
MUST treat the value as signed.

Reviewed-by: Saša Nedvědický <sashan@openssl.org>
Reviewed-by: Matt Caswell <matt@openssl.org>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/29323)
This commit is contained in:
Viktor Dukhovni
2025-12-06 15:04:36 +11:00
committed by Tomas Mraz
parent 013e1174df
commit f247d36074

View File

@@ -693,12 +693,12 @@ static int rsa_keygen(OSSL_LIB_CTX *libctx, RSA *rsa, int bits, int primes,
static int rsa_keygen_pairwise_test(RSA *rsa, OSSL_CALLBACK *cb, void *cbarg)
{
int ret = 0;
unsigned int plaintxt_len;
unsigned char *plaintxt = NULL;
unsigned int ciphertxt_len;
unsigned char *ciphertxt = NULL;
unsigned char *decoded = NULL;
unsigned int decoded_len;
int plaintxt_len;
int ciphertxt_len;
int decoded_len;
int padding = RSA_NO_PADDING;
OSSL_SELF_TEST *st = NULL;