Fix coverity-1604666

Coverity recently flaged an error in which the return value for
EVP_MD_get_size wasn't checked for negative values prior to use, which
can cause underflow later in the function.

Just add the check and error out if get_size returns an error.

Reviewed-by: Paul Dale <ppzgs1@gmail.com>
Reviewed-by: Tomas Mraz <tomas@openssl.org>
(Merged from https://github.com/openssl/openssl/pull/24896)

(cherry picked from commit 22e08c7cdc)
This commit is contained in:
Neil Horman 2024-07-15 14:30:16 -04:00 committed by Tomas Mraz
parent 842959a0af
commit 1d37b00b28

View file

@ -184,7 +184,7 @@ int RSA_padding_check_PKCS1_OAEP_mgf1(unsigned char *to, int tlen,
mdlen = EVP_MD_get_size(md);
if (tlen <= 0 || flen <= 0)
if (tlen <= 0 || flen <= 0 || mdlen <= 0)
return -1;
/*
* |num| is the length of the modulus; |flen| is the length of the