Add kem support #4

rajeev-0 · 2023-08-28T09:28:10Z

Add RSA KEM support to CMP client.

Checklist

documentation is added or updated
[ x] tests are added or updated

…STRINGS) A recursive OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CRYPTO_STRINGS) call may happen if an out-of-memory error happens at the first callstack, and the dead-lock happens at the second callstack, because ossl_err_get_state_int calls OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CRYPTO_STRINGS) although that call is currently already executing. At least on posix system this causes the process to freeze at this point, and must be avoided whatever it takes. The fix is using err_shelve_state around the critical region, which makes ossl_err_get_state_int return early and not call the recursive OPENSSL_init_crypto(OPENSSL_INIT_LOAD_CRYPTO_STRINGS). This can be reproduced with my error injection patch. The test vector has been validated on the master branch: $ ERROR_INJECT=1692279870 ../util/shlib_wrap.sh ./asn1parse-test ./corpora/asn1parse/027f6e82ba01d9db9a9167b83e56cc9f2c602550 ERROR_INJECT=1692279870 #0 0x7f280b42fef8 in __sanitizer_print_stack_trace ../../../../src/libsanitizer/asan/asan_stack.cpp:86 #1 0x5610a3f396b4 in my_malloc fuzz/test-corpus.c:114 #2 0x7f280a2eb94c in CRYPTO_malloc crypto/mem.c:177 #3 0x7f280a2dafdb in OPENSSL_LH_insert crypto/lhash/lhash.c:114 #4 0x7f280a1c87fe in err_load_strings crypto/err/err.c:264 #5 0x7f280a1c87fe in err_load_strings crypto/err/err.c:259 #6 0x7f280a1c87fe in ERR_load_strings_const crypto/err/err.c:301 openssl#7 0x7f280a6f513b in ossl_err_load_PROV_strings providers/common/provider_err.c:233 openssl#8 0x7f280a1cf015 in ossl_err_load_crypto_strings crypto/err/err_all.c:109 openssl#9 0x7f280a2e9b8c in ossl_init_load_crypto_strings crypto/init.c:190 openssl#10 0x7f280a2e9b8c in ossl_init_load_crypto_strings_ossl_ crypto/init.c:181 openssl#11 0x7f2808cfbf67 (/lib/x86_64-linux-gnu/libc.so.6+0x99f67) openssl#12 0x7f280a32301e in CRYPTO_THREAD_run_once crypto/threads_pthread.c:154 openssl#13 0x7f280a2ea1da in OPENSSL_init_crypto crypto/init.c:553 openssl#14 0x5610a3f38e2f in FuzzerInitialize fuzz/asn1parse.c:29 openssl#15 0x5610a3f38783 in main fuzz/test-corpus.c:194 openssl#16 0x7f2808c8bd8f (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) openssl#17 0x7f2808c8be3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) openssl#18 0x5610a3f38d34 in _start (/home/runner/work/openssl/openssl/fuzz/asn1parse-test+0x3d34) AddressSanitizer:DEADLYSIGNAL ================================================================= ==27629==ERROR: AddressSanitizer: ABRT on unknown address 0x03e900006e23 (pc 0x7f2808cfbef8 bp 0x7f280b36afe0 sp 0x7ffd545b2460 T0) #0 0x7f2808cfbef8 (/lib/x86_64-linux-gnu/libc.so.6+0x99ef8) #1 0x7f280a32301e in CRYPTO_THREAD_run_once crypto/threads_pthread.c:154 #2 0x7f280a2ea1da in OPENSSL_init_crypto crypto/init.c:553 #3 0x7f280a1c935e in ossl_err_get_state_int crypto/err/err.c:705 #4 0x7f280a1cf1f9 in ERR_new crypto/err/err_blocks.c:20 #5 0x7f280a2eb9ac in CRYPTO_malloc crypto/mem.c:205 #6 0x7f280a2dafdb in OPENSSL_LH_insert crypto/lhash/lhash.c:114 openssl#7 0x7f280a1c87fe in err_load_strings crypto/err/err.c:264 openssl#8 0x7f280a1c87fe in err_load_strings crypto/err/err.c:259 openssl#9 0x7f280a1c87fe in ERR_load_strings_const crypto/err/err.c:301 openssl#10 0x7f280a6f513b in ossl_err_load_PROV_strings providers/common/provider_err.c:233 openssl#11 0x7f280a1cf015 in ossl_err_load_crypto_strings crypto/err/err_all.c:109 openssl#12 0x7f280a2e9b8c in ossl_init_load_crypto_strings crypto/init.c:190 openssl#13 0x7f280a2e9b8c in ossl_init_load_crypto_strings_ossl_ crypto/init.c:181 openssl#14 0x7f2808cfbf67 (/lib/x86_64-linux-gnu/libc.so.6+0x99f67) openssl#15 0x7f280a32301e in CRYPTO_THREAD_run_once crypto/threads_pthread.c:154 openssl#16 0x7f280a2ea1da in OPENSSL_init_crypto crypto/init.c:553 openssl#17 0x5610a3f38e2f in FuzzerInitialize fuzz/asn1parse.c:29 openssl#18 0x5610a3f38783 in main fuzz/test-corpus.c:194 openssl#19 0x7f2808c8bd8f (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) openssl#20 0x7f2808c8be3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) openssl#21 0x5610a3f38d34 in _start (/home/runner/work/openssl/openssl/fuzz/asn1parse-test+0x3d34) AddressSanitizer can not provide additional info. SUMMARY: AddressSanitizer: ABRT (/lib/x86_64-linux-gnu/libc.so.6+0x99ef8) ==27629==ABORTING Reviewed-by: Tomas Mraz <[email protected]> Reviewed-by: Tom Cosgrove <[email protected]> (Merged from openssl#21683)

DDvO

Here finally is an initial part of my review.

DDvO · 2023-09-06T07:15:57Z

apps/lib/cmp_mock_srv.c

+    for (i = 0; i < sk_X509_num(certs); i++) {
+        X509 *cert = sk_X509_value(certs, i);
+
+        if ((X509_get_key_usage(cert) & X509v3_KU_KEY_ENCIPHERMENT)) {


Taking the key usage can easily go wrong - maybe for RSA keys there is not much better choice,
but at least add a TODO for other key types (such as PQ keys) that checking the key type (as far as makes sense) needs to be preferrred.

DDvO · 2023-09-06T07:21:00Z

apps/lib/cmp_mock_srv.c

+        if (OSSL_CMP_ITAV_get0_value(req) == NULL) {
+            X509 *kemcert;
+
+            kemcert = extracert_withKEM(OSSL_CMP_MSG_get_extraCerts(genm));


Trying to determine if the sender side uses a KEM cert this way can go wrong.
You should instead use OSSL_CMP_CTX_get0_validatedSrvCert() at least on the client side -
despite its original intention and documentation, I hope to works on the server side, too.
Then you can remove again the new genm parameter.

I agree it is not the best way and also certificate validation need to be performed ( will add TODO).
In case of client side having KEM cert, first genm message is without protection with extracert, So it does not populate validatedSevCert.
In case of server having KEM cert, it is assumed client already have valid server cert (in this implementation).

DDvO · 2023-09-06T07:24:51Z

crypto/cmp/cmp_asn.c

+ASN1_SEQUENCE(OSSL_CMP_RSAKEMPARAMETERS) = {
+    ASN1_SIMPLE(OSSL_CMP_RSAKEMPARAMETERS, KeyDerivationFunction, X509_ALGOR),
+    ASN1_SIMPLE(OSSL_CMP_RSAKEMPARAMETERS, KeyLength, ASN1_INTEGER)
+} ASN1_SEQUENCE_END(OSSL_CMP_RSAKEMPARAMETERS)
+IMPLEMENT_ASN1_FUNCTIONS(OSSL_CMP_RSAKEMPARAMETERS)


I think. this is entirely independent of CMP and should go into some more general OpenSSL source file.

DDvO · 2023-09-06T07:27:53Z

crypto/cmp/cmp_asn.c

+    itav->infoValue.KemCiphertextInfoValue = OSSL_CMP_KEMCIPHERTEXTINFO_new();
+    if (itav->infoValue.KemCiphertextInfoValue == NULL)
+        goto err;


For avoiding redundancy, such statements (including the following ones) may be simplified to

if ((itav->infoValue.KemCiphertextInfoValue = OSSL_CMP_KEMCIPHERTEXTINFO_new())== NULL) ...

Single line length is greater than 80 , and I could not find way to reduce it.

DDvO · 2023-09-06T07:57:24Z

crypto/cmp/cmp_asn.c

@@ -197,6 +230,37 @@ int OSSL_CMP_ITAV_push0_stack_item(STACK_OF(OSSL_CMP_ITAV) **itav_sk_p,
    return 0;
 }

+OSSL_CMP_ITAV *ossl_cmp_itav_new_KemCiphertext(X509_ALGOR *kem,


For consistency, either move definitions of similar constructors such as ossl_cmp_kem_KemOtherInfo_new() (and any selector functions) from cmp_kem.c here (preferred) or move also this definition to cmp_kem.c.

Moved definition of ossl_cmp_kem_KemOtherInfo_new( ) to cmp_asn.c

DDvO · 2023-09-06T21:04:52Z

crypto/objects/obj_dat.h

@@ -10,7 +10,7 @@
 */

 /* Serialized OID's */
-static const unsigned char so[8476] = {
+static const unsigned char so[8518] = {


I have not yet reviewed the changes to obj_dat.c

DDvO · 2023-09-06T21:05:16Z

include/openssl/obj_mac.h

@@ -53,6 +53,14 @@
 #define NID_gmac                1195
 #define OBJ_gmac                OBJ_iso,0L,9797L,3L,4L

+#define SN_id_kem_rsa           "id-kem-rsa"


I have not yet reviewed the changes to obj_mac.h

DDvO · 2023-09-06T21:06:19Z

test/recipes/80-test_cmp_http_data/Mock/server_withkem.cnf

@@ -0,0 +1,17 @@
+[cmp] # mock server configuration


Where is this file used?

This is a temporary test file, it was provided to pqs for testing. will be removed.

DDvO · 2023-09-06T21:06:55Z

test/recipes/80-test_cmp_http_data/Mock/test_withserverkem.cnf

@@ -0,0 +1,141 @@
+[default]


Where is this file used?

This is a temporary test file, it was provided to pqs for testing. will be removed.

DDvO · 2023-09-06T21:08:41Z

test/recipes/80-test_cmp_http_data/test_credentials.csv

@@ -44,3 +44,6 @@ expected,description, -section,val, -ref,val, -secret,val, -cert,val, -key,val,
 0,multiple digests, -section,,BLANK,,BLANK,, -cert,signer.crt, -key,signer.p12, -keypass,pass:12345,BLANK,,BLANK,, -digest,sha256 sha512,BLANK,
 ,,,,,,,,,,,,,,,,,,,,,,
 0,unprotected request, -section,,BLANK,,BLANK,, -cert,"""", -key,"""", -keypass,"""",BLANK,,BLANK,,BLANK,, -unprotected_requests,
+,,,,,,,,,,,,,,,,,,,,,,
+1,client with KEM, -section,,,,,, -cert,kem.crt, -key,kem.key,,,BLANK,,BLANK,,BLANK,,BLANK,,,,
+1,server with kem, -section,,BLANK,,-secret,"""", -cert,signer.crt, -key,signer.p12, -keypass,pass:12345,BLANK,,BLANK,,BLANK,,BLANK,,-srvcert, server_kem.crt


Using -srvcert this way gets silently ignored.
Unfortunately, we'll need to spin up a new server instance for making it use other (KEM) credentials.

fake_now in the quictestlib is read/written by potentially many threads, and as such should have a surrounding lock to prevent WAR/RAW errors as caught by tsan: 2023-11-03T16:27:23.7184999Z ================== 2023-11-03T16:27:23.7185290Z WARNING: ThreadSanitizer: data race (pid=18754) 2023-11-03T16:27:23.7185720Z Read of size 8 at 0x558f6f9fe970 by main thread: 2023-11-03T16:27:23.7186726Z #0 qtest_create_quic_connection_ex <null> (quicapitest+0x14aead) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7187665Z #1 qtest_create_quic_connection <null> (quicapitest+0x14b220) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7188567Z #2 test_quic_write_read quicapitest.c (quicapitest+0x150ee2) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7189561Z #3 run_tests <null> (quicapitest+0x2237ab) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7190294Z #4 main <null> (quicapitest+0x223d2b) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7190720Z 2023-11-03T16:27:23.7190902Z Previous write of size 8 at 0x558f6f9fe970 by thread T1: 2023-11-03T16:27:23.7191607Z #0 qtest_create_quic_connection_ex <null> (quicapitest+0x14aecf) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7192505Z #1 run_server_thread quictestlib.c (quicapitest+0x14b1d6) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7193361Z #2 thread_run quictestlib.c (quicapitest+0x14cadf) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7193848Z 2023-11-03T16:27:23.7194220Z Location is global 'fake_now.0' of size 8 at 0x558f6f9fe970 (quicapitest+0x1af4970) 2023-11-03T16:27:23.7194636Z 2023-11-03T16:27:23.7194816Z Thread T1 (tid=18760, running) created by main thread at: 2023-11-03T16:27:23.7195465Z #0 pthread_create <null> (quicapitest+0xca12d) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7196317Z #1 qtest_create_quic_connection_ex <null> (quicapitest+0x14adcb) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7197214Z #2 qtest_create_quic_connection <null> (quicapitest+0x14b220) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7198111Z #3 test_quic_write_read quicapitest.c (quicapitest+0x150ee2) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7198940Z #4 run_tests <null> (quicapitest+0x2237ab) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7199661Z #5 main <null> (quicapitest+0x223d2b) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7200083Z 2023-11-03T16:27:23.7200862Z SUMMARY: ThreadSanitizer: data race (/home/runner/work/openssl/openssl/test/quicapitest+0x14aead) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) in qtest_create_quic_connection_ex Reviewed-by: Hugo Landau <[email protected]> Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#22616)

Sometimes the error handling returns an ASN1_STRING object in *out although that was not passed in by the caller, and sometimes the error handling deletes the ASN1_STRING but forgets to clear the *out parameter. Therefore the caller has no chance to know, if the leaked object in *out shall be deleted or not. This may cause a use-after-free error e.g. in asn1_str2type: ==63312==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000073280 at pc 0x7f2652e93b08 bp 0x7ffe0e1951c0 sp 0x7ffe0e1951b0 READ of size 8 at 0x603000073280 thread T0 #0 0x7f2652e93b07 in asn1_string_embed_free crypto/asn1/asn1_lib.c:354 #1 0x7f2652eb521a in asn1_primitive_free crypto/asn1/tasn_fre.c:204 #2 0x7f2652eb50a9 in asn1_primitive_free crypto/asn1/tasn_fre.c:199 #3 0x7f2652eb5b67 in ASN1_item_free crypto/asn1/tasn_fre.c:20 #4 0x7f2652e8e13b in asn1_str2type crypto/asn1/asn1_gen.c:740 #5 0x7f2652e8e13b in generate_v3 crypto/asn1/asn1_gen.c:137 #6 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#7 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#8 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#9 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#10 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#11 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#12 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#13 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#14 0x564ed19d5f25 in req_main apps/req.c:806 openssl#15 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#16 0x564ed1985165 in main apps/openssl.c:183 openssl#17 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 openssl#18 0x564ed1985acd in _start (/home/ed/OPCToolboxV5/Source/Core/OpenSSL/openssl/apps/openssl+0x139acd) 0x603000073280 is located 16 bytes inside of 24-byte region [0x603000073270,0x603000073288) freed by thread T0 here: #0 0x7f265413440f in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:122 #1 0x7f265315a429 in CRYPTO_free crypto/mem.c:311 #2 0x7f265315a429 in CRYPTO_free crypto/mem.c:300 #3 0x7f2652e757b9 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:191 #4 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 #5 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 #6 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#7 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#8 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#9 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#10 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#11 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#12 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#13 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#14 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#15 0x564ed19d5f25 in req_main apps/req.c:806 openssl#16 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#17 0x564ed1985165 in main apps/openssl.c:183 openssl#18 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 previously allocated by thread T0 here: #0 0x7f2654134808 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144 #1 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:221 #2 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:198 #3 0x7f265315a945 in CRYPTO_zalloc crypto/mem.c:236 #4 0x7f2652e939a4 in ASN1_STRING_type_new crypto/asn1/asn1_lib.c:341 #5 0x7f2652e74e51 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:150 #6 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 openssl#7 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 openssl#8 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#9 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#10 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#11 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#12 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#13 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#14 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#15 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#16 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#17 0x564ed19d5f25 in req_main apps/req.c:806 openssl#18 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#19 0x564ed1985165 in main apps/openssl.c:183 openssl#20 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 Reviewed-by: Paul Yang <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23138)

fake_now in the quictestlib is read/written by potentially many threads, and as such should have a surrounding lock to prevent WAR/RAW errors as caught by tsan: 2023-11-03T16:27:23.7184999Z ================== 2023-11-03T16:27:23.7185290Z WARNING: ThreadSanitizer: data race (pid=18754) 2023-11-03T16:27:23.7185720Z Read of size 8 at 0x558f6f9fe970 by main thread: 2023-11-03T16:27:23.7186726Z #0 qtest_create_quic_connection_ex <null> (quicapitest+0x14aead) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7187665Z #1 qtest_create_quic_connection <null> (quicapitest+0x14b220) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7188567Z #2 test_quic_write_read quicapitest.c (quicapitest+0x150ee2) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7189561Z #3 run_tests <null> (quicapitest+0x2237ab) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7190294Z #4 main <null> (quicapitest+0x223d2b) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7190720Z 2023-11-03T16:27:23.7190902Z Previous write of size 8 at 0x558f6f9fe970 by thread T1: 2023-11-03T16:27:23.7191607Z #0 qtest_create_quic_connection_ex <null> (quicapitest+0x14aecf) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7192505Z #1 run_server_thread quictestlib.c (quicapitest+0x14b1d6) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7193361Z #2 thread_run quictestlib.c (quicapitest+0x14cadf) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7193848Z 2023-11-03T16:27:23.7194220Z Location is global 'fake_now.0' of size 8 at 0x558f6f9fe970 (quicapitest+0x1af4970) 2023-11-03T16:27:23.7194636Z 2023-11-03T16:27:23.7194816Z Thread T1 (tid=18760, running) created by main thread at: 2023-11-03T16:27:23.7195465Z #0 pthread_create <null> (quicapitest+0xca12d) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7196317Z #1 qtest_create_quic_connection_ex <null> (quicapitest+0x14adcb) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7197214Z #2 qtest_create_quic_connection <null> (quicapitest+0x14b220) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7198111Z #3 test_quic_write_read quicapitest.c (quicapitest+0x150ee2) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7198940Z #4 run_tests <null> (quicapitest+0x2237ab) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7199661Z #5 main <null> (quicapitest+0x223d2b) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) 2023-11-03T16:27:23.7200083Z 2023-11-03T16:27:23.7200862Z SUMMARY: ThreadSanitizer: data race (/home/runner/work/openssl/openssl/test/quicapitest+0x14aead) (BuildId: d06f7b04830b55de9c8482b398a1781472d1c7d5) in qtest_create_quic_connection_ex Reviewed-by: Hugo Landau <[email protected]> Reviewed-by: Paul Dale <[email protected]> Reviewed-by: Matt Caswell <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#22616) (cherry picked from commit 11179b3)

Sometimes the error handling returns an ASN1_STRING object in *out although that was not passed in by the caller, and sometimes the error handling deletes the ASN1_STRING but forgets to clear the *out parameter. Therefore the caller has no chance to know, if the leaked object in *out shall be deleted or not. This may cause a use-after-free error e.g. in asn1_str2type: ==63312==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000073280 at pc 0x7f2652e93b08 bp 0x7ffe0e1951c0 sp 0x7ffe0e1951b0 READ of size 8 at 0x603000073280 thread T0 #0 0x7f2652e93b07 in asn1_string_embed_free crypto/asn1/asn1_lib.c:354 #1 0x7f2652eb521a in asn1_primitive_free crypto/asn1/tasn_fre.c:204 #2 0x7f2652eb50a9 in asn1_primitive_free crypto/asn1/tasn_fre.c:199 #3 0x7f2652eb5b67 in ASN1_item_free crypto/asn1/tasn_fre.c:20 #4 0x7f2652e8e13b in asn1_str2type crypto/asn1/asn1_gen.c:740 #5 0x7f2652e8e13b in generate_v3 crypto/asn1/asn1_gen.c:137 #6 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#7 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#8 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#9 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#10 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#11 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#12 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#13 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#14 0x564ed19d5f25 in req_main apps/req.c:806 openssl#15 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#16 0x564ed1985165 in main apps/openssl.c:183 openssl#17 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 openssl#18 0x564ed1985acd in _start (/home/ed/OPCToolboxV5/Source/Core/OpenSSL/openssl/apps/openssl+0x139acd) 0x603000073280 is located 16 bytes inside of 24-byte region [0x603000073270,0x603000073288) freed by thread T0 here: #0 0x7f265413440f in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:122 #1 0x7f265315a429 in CRYPTO_free crypto/mem.c:311 #2 0x7f265315a429 in CRYPTO_free crypto/mem.c:300 #3 0x7f2652e757b9 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:191 #4 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 #5 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 #6 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#7 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#8 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#9 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#10 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#11 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#12 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#13 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#14 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#15 0x564ed19d5f25 in req_main apps/req.c:806 openssl#16 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#17 0x564ed1985165 in main apps/openssl.c:183 openssl#18 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 previously allocated by thread T0 here: #0 0x7f2654134808 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144 #1 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:221 #2 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:198 #3 0x7f265315a945 in CRYPTO_zalloc crypto/mem.c:236 #4 0x7f2652e939a4 in ASN1_STRING_type_new crypto/asn1/asn1_lib.c:341 #5 0x7f2652e74e51 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:150 #6 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 openssl#7 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 openssl#8 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#9 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#10 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#11 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#12 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#13 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#14 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#15 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#16 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#17 0x564ed19d5f25 in req_main apps/req.c:806 openssl#18 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#19 0x564ed1985165 in main apps/openssl.c:183 openssl#20 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 Reviewed-by: Paul Yang <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23138) (cherry picked from commit 73ebaac)

if the private key is output to stdout using the HARNESS_OSSL_PREFIX, out is a stack of BIOs and must therefore free'd using BIO_free_all. Steps to reproduce: $ HARNESS_OSSL_PREFIX=x OPENSSL_CONF=apps/openssl.cnf util/shlib_wrap.sh apps/openssl req -new -keyout - -passout pass: </dev/null [...] Direct leak of 128 byte(s) in 1 object(s) allocated from: #0 0x7f6f692b89cf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x7f6f686eda00 in CRYPTO_malloc crypto/mem.c:202 #2 0x7f6f686edba0 in CRYPTO_zalloc crypto/mem.c:222 #3 0x7f6f68471bdf in BIO_new_ex crypto/bio/bio_lib.c:83 #4 0x7f6f68491a8f in BIO_new_fp crypto/bio/bss_file.c:95 #5 0x555c5f58b378 in dup_bio_out apps/lib/apps.c:3014 #6 0x555c5f58f9ac in bio_open_default_ apps/lib/apps.c:3175 openssl#7 0x555c5f58f9ac in bio_open_default apps/lib/apps.c:3203 openssl#8 0x555c5f528537 in req_main apps/req.c:683 openssl#9 0x555c5f50e315 in do_cmd apps/openssl.c:426 openssl#10 0x555c5f4c5575 in main apps/openssl.c:307 openssl#11 0x7f6f680461c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 SUMMARY: AddressSanitizer: 128 byte(s) leaked in 1 allocation(s). Reviewed-by: Tom Cosgrove <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23365)

if the private key is output to stdout using the HARNESS_OSSL_PREFIX, out is a stack of BIOs and must therefore free'd using BIO_free_all. Steps to reproduce: $ HARNESS_OSSL_PREFIX=x OPENSSL_CONF=apps/openssl.cnf util/shlib_wrap.sh apps/openssl req -new -keyout - -passout pass: </dev/null [...] Direct leak of 128 byte(s) in 1 object(s) allocated from: #0 0x7f6f692b89cf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x7f6f686eda00 in CRYPTO_malloc crypto/mem.c:202 #2 0x7f6f686edba0 in CRYPTO_zalloc crypto/mem.c:222 #3 0x7f6f68471bdf in BIO_new_ex crypto/bio/bio_lib.c:83 #4 0x7f6f68491a8f in BIO_new_fp crypto/bio/bss_file.c:95 #5 0x555c5f58b378 in dup_bio_out apps/lib/apps.c:3014 #6 0x555c5f58f9ac in bio_open_default_ apps/lib/apps.c:3175 openssl#7 0x555c5f58f9ac in bio_open_default apps/lib/apps.c:3203 openssl#8 0x555c5f528537 in req_main apps/req.c:683 openssl#9 0x555c5f50e315 in do_cmd apps/openssl.c:426 openssl#10 0x555c5f4c5575 in main apps/openssl.c:307 openssl#11 0x7f6f680461c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 SUMMARY: AddressSanitizer: 128 byte(s) leaked in 1 allocation(s). Reviewed-by: Tom Cosgrove <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23365) (cherry picked from commit ff78d94)

cmp_asn.c: add ASN1 structure for OSSL_CMP_KEMCIPHERTEXTINFO cmp{_asn.c, .h.in}: add function OSSL_CMP_ITAV_new_KemCiphertext cmp{_msg.c, .h.in}: add function OSSL_CMP_MSG_get_extraCerts cmp_server.c: extend unprotected_exception for KemCiphertextInfo without value objects.txt, oids.txt, obj_{dah.h, mac.num, mac.h}: add id_it_KemCiphertextInfo (preliminary value for prototyping) cmp{.h.in, _local.h, _asn.c}: add structure OSSL_CMP_KEMOTHERINFO for kdf objects.txt, cmp_{asn.c, local.h}: add OSSL_CMP_KEMBMPARAMETER and preliminary value for id-KemBasedMac objects.txt : add NID_hkdfWithSHA256 openssl.txt, cmperr.h: add KBM error. cmp_{client.c, ctx.c, local.h}, cmp.h.in: extend OSSL_CMP_CTX for KEM. cmp_mock_srv.c: add support of Key encapculation and ITAV with kemCiphertext build.info, cmp.h.in, cmp_{kem.c, local.h}: add functions for kemBasedMac cmp_{genm.c, local.h}: convert static function to ossl_cmp_genm_get_itav cmp_{hdr.c, local.h}: add func ossl_cmp_hdr_has_KemCiphertextInfo cmp_protect.c: extend protection for KemBasedMac cmp_client.c: add kembasedMac protection to OSSL_CMP_exec_certreq cmp_vfy.c: add msg validation for NID_id_KemBasedMac cmp_client.c: extend unprotected_exception for NID_id_it_KemCiphertextInfo. cmp_server.c: adapt server for kem objects.txt, obj_{dat.h, mac.num. mac.h}, oids.txt: add oids for id_kem_rsa & id_kdf_kdf2 cmp_{asn.c, local.h}: add asn structure for OSSL_CMP_RSAKEMPARAMETERS cmp_{kem.c, local.h}: add function ossl_cmp_kem_get_ssk_using_srvcert kem{.crt, .key, _only.crt}: add certificate with kem.

…m cmp_kem.c

…EM testing

… testing.

The following issue was found in automatic tests with thread sanitizer builds in ClickHouse (which uses OpenSSL 3.2.1) [0]. The first stack [1] does proper locking (function 'x509_store_add', x509_lu.c) but in the second stack [2], function 'get_cert_by_subject_ex' (by_dir.b) forgets to lock when calling 'sk_X509_OBJECT_is_sorted'. [0] ClickHouse/ClickHouse#63049 [1] WARNING: ThreadSanitizer: data race (pid=1870) Write of size 4 at 0x7b08003d6810 by thread T552 (mutexes: write M0, write M1, write M2, write M3): #0 OPENSSL_sk_insert build_docker/./contrib/openssl/crypto/stack/stack.c:280:16 (clickhouse+0x203ad7e4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 OPENSSL_sk_push build_docker/./contrib/openssl/crypto/stack/stack.c:401:12 (clickhouse+0x203ad7e4) #2 x509_store_add build_docker/./contrib/openssl/crypto/x509/x509_lu.c:419:17 (clickhouse+0x203d4a52) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 X509_STORE_add_cert build_docker/./contrib/openssl/crypto/x509/x509_lu.c:432:10 (clickhouse+0x203d48a2) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #4 X509_load_cert_file_ex build_docker/./contrib/openssl/crypto/x509/by_file.c:127:18 (clickhouse+0x203b74e6) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:333:22 (clickhouse+0x203b684c) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#7 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) openssl#8 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#11 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#16 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#20 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#21 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#22 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#23 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) [2] Previous read of size 4 at 0x7b08003d6810 by thread T553 (mutexes: write M4, write M5, write M6): #0 OPENSSL_sk_is_sorted build_docker/./contrib/openssl/crypto/stack/stack.c:490:33 (clickhouse+0x203adcff) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:423:10 (clickhouse+0x203b6d8f) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #2 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) #4 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#7 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#8 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#11 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#16 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) CLA: trivial Reviewed-by: Todd Short <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#24295)

if the private key is output to stdout using the HARNESS_OSSL_PREFIX, out is a stack of BIOs and must therefore free'd using BIO_free_all. Steps to reproduce: $ HARNESS_OSSL_PREFIX=x OPENSSL_CONF=apps/openssl.cnf util/shlib_wrap.sh apps/openssl req -new -keyout - -passout pass: </dev/null [...] Direct leak of 128 byte(s) in 1 object(s) allocated from: #0 0x7f6f692b89cf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x7f6f686eda00 in CRYPTO_malloc crypto/mem.c:202 #2 0x7f6f686edba0 in CRYPTO_zalloc crypto/mem.c:222 #3 0x7f6f68471bdf in BIO_new_ex crypto/bio/bio_lib.c:83 #4 0x7f6f68491a8f in BIO_new_fp crypto/bio/bss_file.c:95 #5 0x555c5f58b378 in dup_bio_out apps/lib/apps.c:3014 #6 0x555c5f58f9ac in bio_open_default_ apps/lib/apps.c:3175 openssl#7 0x555c5f58f9ac in bio_open_default apps/lib/apps.c:3203 openssl#8 0x555c5f528537 in req_main apps/req.c:683 openssl#9 0x555c5f50e315 in do_cmd apps/openssl.c:426 openssl#10 0x555c5f4c5575 in main apps/openssl.c:307 openssl#11 0x7f6f680461c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 SUMMARY: AddressSanitizer: 128 byte(s) leaked in 1 allocation(s). Reviewed-by: Tom Cosgrove <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23365) (cherry picked from commit ff78d94)

The following issue was found in automatic tests with thread sanitizer builds in ClickHouse (which uses OpenSSL 3.2.1) [0]. The first stack [1] does proper locking (function 'x509_store_add', x509_lu.c) but in the second stack [2], function 'get_cert_by_subject_ex' (by_dir.b) forgets to lock when calling 'sk_X509_OBJECT_is_sorted'. [0] ClickHouse/ClickHouse#63049 [1] WARNING: ThreadSanitizer: data race (pid=1870) Write of size 4 at 0x7b08003d6810 by thread T552 (mutexes: write M0, write M1, write M2, write M3): #0 OPENSSL_sk_insert build_docker/./contrib/openssl/crypto/stack/stack.c:280:16 (clickhouse+0x203ad7e4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 OPENSSL_sk_push build_docker/./contrib/openssl/crypto/stack/stack.c:401:12 (clickhouse+0x203ad7e4) #2 x509_store_add build_docker/./contrib/openssl/crypto/x509/x509_lu.c:419:17 (clickhouse+0x203d4a52) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 X509_STORE_add_cert build_docker/./contrib/openssl/crypto/x509/x509_lu.c:432:10 (clickhouse+0x203d48a2) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #4 X509_load_cert_file_ex build_docker/./contrib/openssl/crypto/x509/by_file.c:127:18 (clickhouse+0x203b74e6) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:333:22 (clickhouse+0x203b684c) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#7 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) openssl#8 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#11 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#16 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#20 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#21 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#22 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#23 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) [2] Previous read of size 4 at 0x7b08003d6810 by thread T553 (mutexes: write M4, write M5, write M6): #0 OPENSSL_sk_is_sorted build_docker/./contrib/openssl/crypto/stack/stack.c:490:33 (clickhouse+0x203adcff) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:423:10 (clickhouse+0x203b6d8f) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #2 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) #4 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#7 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#8 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#11 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#16 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) CLA: trivial Reviewed-by: Todd Short <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#24295) (cherry picked from commit af75373)

Running the x509_req_test with address sanitizer shows a memory leak: ==186455==ERROR: LeakSanitizer: detected memory leaks Direct leak of 53 byte(s) in 1 object(s) allocated from: #0 0x3ffad5f47af in malloc (/lib64/libasan.so.8+0xf47af) (BuildId: 93b3d2536d76f772a95880d76c746c150daabbee) #1 0x3ffac4214fb in CRYPTO_malloc crypto/mem.c:202 #2 0x3ffac421759 in CRYPTO_zalloc crypto/mem.c:222 #3 0x100e58f in test_mk_file_path test/testutil/driver.c:450 #4 0x1004671 in test_x509_req_detect_invalid_version test/x509_req_test.c:32 #5 0x100d247 in run_tests test/testutil/driver.c:342 #6 0x10042e3 in main test/testutil/main.c:31 openssl#7 0x3ffaad34a5b in __libc_start_call_main (/lib64/libc.so.6+0x34a5b) (BuildId: 461b58df774538594b6173825bed67a9247a014d) openssl#8 0x3ffaad34b5d in __libc_start_main@GLIBC_2.2 (/lib64/libc.so.6+0x34b5d) (BuildId: 461b58df774538594b6173825bed67a9247a014d) openssl#9 0x1004569 (/root/openssl/test/x509_req_test+0x1004569) (BuildId: ab6bce0e531df1e3626a8f506d07f6ad7c7c6d57) SUMMARY: AddressSanitizer: 53 byte(s) leaked in 1 allocation(s). The certFilePath that is obtained via test_mk_file_path() must be freed when no longer used. While at it, make the certFilePath variable a local variable, there is no need to have this a global static variable. Fixes: openssl@7d2c0a4 Signed-off-by: Ingo Franzki <[email protected]> Reviewed-by: Dmitry Belyavskiy <[email protected]> Reviewed-by: Neil Horman <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#24715)

Sometimes the error handling returns an ASN1_STRING object in *out although that was not passed in by the caller, and sometimes the error handling deletes the ASN1_STRING but forgets to clear the *out parameter. Therefore the caller has no chance to know, if the leaked object in *out shall be deleted or not. This may cause a use-after-free error e.g. in asn1_str2type: ==63312==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000073280 at pc 0x7f2652e93b08 bp 0x7ffe0e1951c0 sp 0x7ffe0e1951b0 READ of size 8 at 0x603000073280 thread T0 #0 0x7f2652e93b07 in asn1_string_embed_free crypto/asn1/asn1_lib.c:354 #1 0x7f2652eb521a in asn1_primitive_free crypto/asn1/tasn_fre.c:204 #2 0x7f2652eb50a9 in asn1_primitive_free crypto/asn1/tasn_fre.c:199 #3 0x7f2652eb5b67 in ASN1_item_free crypto/asn1/tasn_fre.c:20 #4 0x7f2652e8e13b in asn1_str2type crypto/asn1/asn1_gen.c:740 #5 0x7f2652e8e13b in generate_v3 crypto/asn1/asn1_gen.c:137 #6 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#7 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#8 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#9 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#10 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#11 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#12 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#13 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#14 0x564ed19d5f25 in req_main apps/req.c:806 openssl#15 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#16 0x564ed1985165 in main apps/openssl.c:183 openssl#17 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 openssl#18 0x564ed1985acd in _start (/home/ed/OPCToolboxV5/Source/Core/OpenSSL/openssl/apps/openssl+0x139acd) 0x603000073280 is located 16 bytes inside of 24-byte region [0x603000073270,0x603000073288) freed by thread T0 here: #0 0x7f265413440f in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:122 #1 0x7f265315a429 in CRYPTO_free crypto/mem.c:311 #2 0x7f265315a429 in CRYPTO_free crypto/mem.c:300 #3 0x7f2652e757b9 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:191 #4 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 #5 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 #6 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#7 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#8 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#9 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#10 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#11 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#12 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#13 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#14 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#15 0x564ed19d5f25 in req_main apps/req.c:806 openssl#16 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#17 0x564ed1985165 in main apps/openssl.c:183 openssl#18 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 previously allocated by thread T0 here: #0 0x7f2654134808 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144 #1 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:221 #2 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:198 #3 0x7f265315a945 in CRYPTO_zalloc crypto/mem.c:236 #4 0x7f2652e939a4 in ASN1_STRING_type_new crypto/asn1/asn1_lib.c:341 #5 0x7f2652e74e51 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:150 #6 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 openssl#7 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 openssl#8 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#9 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#10 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#11 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#12 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#13 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#14 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#15 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#16 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#17 0x564ed19d5f25 in req_main apps/req.c:806 openssl#18 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#19 0x564ed1985165 in main apps/openssl.c:183 openssl#20 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 Reviewed-by: Tom Cosgrove <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23165) (cherry picked from commit dfa1e49)

if the private key is output to stdout using the HARNESS_OSSL_PREFIX, out is a stack of BIOs and must therefore free'd using BIO_free_all. Steps to reproduce: $ HARNESS_OSSL_PREFIX=x OPENSSL_CONF=apps/openssl.cnf util/shlib_wrap.sh apps/openssl req -new -keyout - -passout pass: </dev/null [...] Direct leak of 128 byte(s) in 1 object(s) allocated from: #0 0x7f6f692b89cf in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cpp:69 #1 0x7f6f686eda00 in CRYPTO_malloc crypto/mem.c:202 #2 0x7f6f686edba0 in CRYPTO_zalloc crypto/mem.c:222 #3 0x7f6f68471bdf in BIO_new_ex crypto/bio/bio_lib.c:83 #4 0x7f6f68491a8f in BIO_new_fp crypto/bio/bss_file.c:95 #5 0x555c5f58b378 in dup_bio_out apps/lib/apps.c:3014 #6 0x555c5f58f9ac in bio_open_default_ apps/lib/apps.c:3175 openssl#7 0x555c5f58f9ac in bio_open_default apps/lib/apps.c:3203 openssl#8 0x555c5f528537 in req_main apps/req.c:683 openssl#9 0x555c5f50e315 in do_cmd apps/openssl.c:426 openssl#10 0x555c5f4c5575 in main apps/openssl.c:307 openssl#11 0x7f6f680461c9 in __libc_start_call_main ../sysdeps/nptl/libc_start_call_main.h:58 SUMMARY: AddressSanitizer: 128 byte(s) leaked in 1 allocation(s). Reviewed-by: Tom Cosgrove <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23365) (cherry picked from commit ff78d94)

Sometimes the error handling returns an ASN1_STRING object in *out although that was not passed in by the caller, and sometimes the error handling deletes the ASN1_STRING but forgets to clear the *out parameter. Therefore the caller has no chance to know, if the leaked object in *out shall be deleted or not. This may cause a use-after-free error e.g. in asn1_str2type: ==63312==ERROR: AddressSanitizer: heap-use-after-free on address 0x603000073280 at pc 0x7f2652e93b08 bp 0x7ffe0e1951c0 sp 0x7ffe0e1951b0 READ of size 8 at 0x603000073280 thread T0 #0 0x7f2652e93b07 in asn1_string_embed_free crypto/asn1/asn1_lib.c:354 #1 0x7f2652eb521a in asn1_primitive_free crypto/asn1/tasn_fre.c:204 #2 0x7f2652eb50a9 in asn1_primitive_free crypto/asn1/tasn_fre.c:199 #3 0x7f2652eb5b67 in ASN1_item_free crypto/asn1/tasn_fre.c:20 #4 0x7f2652e8e13b in asn1_str2type crypto/asn1/asn1_gen.c:740 #5 0x7f2652e8e13b in generate_v3 crypto/asn1/asn1_gen.c:137 #6 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#7 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#8 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#9 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#10 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#11 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#12 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#13 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#14 0x564ed19d5f25 in req_main apps/req.c:806 openssl#15 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#16 0x564ed1985165 in main apps/openssl.c:183 openssl#17 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 openssl#18 0x564ed1985acd in _start (/home/ed/OPCToolboxV5/Source/Core/OpenSSL/openssl/apps/openssl+0x139acd) 0x603000073280 is located 16 bytes inside of 24-byte region [0x603000073270,0x603000073288) freed by thread T0 here: #0 0x7f265413440f in __interceptor_free ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:122 #1 0x7f265315a429 in CRYPTO_free crypto/mem.c:311 #2 0x7f265315a429 in CRYPTO_free crypto/mem.c:300 #3 0x7f2652e757b9 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:191 #4 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 #5 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 #6 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#7 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#8 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#9 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#10 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#11 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#12 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#13 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#14 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#15 0x564ed19d5f25 in req_main apps/req.c:806 openssl#16 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#17 0x564ed1985165 in main apps/openssl.c:183 openssl#18 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 previously allocated by thread T0 here: #0 0x7f2654134808 in __interceptor_malloc ../../../../src/libsanitizer/asan/asan_malloc_linux.cc:144 #1 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:221 #2 0x7f265315a4fd in CRYPTO_malloc crypto/mem.c:198 #3 0x7f265315a945 in CRYPTO_zalloc crypto/mem.c:236 #4 0x7f2652e939a4 in ASN1_STRING_type_new crypto/asn1/asn1_lib.c:341 #5 0x7f2652e74e51 in ASN1_mbstring_ncopy crypto/asn1/a_mbstr.c:150 #6 0x7f2652e75ec5 in ASN1_mbstring_copy crypto/asn1/a_mbstr.c:38 openssl#7 0x7f2652e8e227 in asn1_str2type crypto/asn1/asn1_gen.c:681 openssl#8 0x7f2652e8e227 in generate_v3 crypto/asn1/asn1_gen.c:137 openssl#9 0x7f2652e9166c in ASN1_generate_v3 crypto/asn1/asn1_gen.c:92 openssl#10 0x7f2653307b9b in do_othername crypto/x509v3/v3_alt.c:577 openssl#11 0x7f2653307b9b in a2i_GENERAL_NAME crypto/x509v3/v3_alt.c:492 openssl#12 0x7f26533087c2 in v2i_subject_alt crypto/x509v3/v3_alt.c:327 openssl#13 0x7f26533107fc in do_ext_nconf crypto/x509v3/v3_conf.c:100 openssl#14 0x7f2653310f33 in X509V3_EXT_nconf crypto/x509v3/v3_conf.c:45 openssl#15 0x7f2653311426 in X509V3_EXT_add_nconf_sk crypto/x509v3/v3_conf.c:312 openssl#16 0x7f265331170c in X509V3_EXT_REQ_add_nconf crypto/x509v3/v3_conf.c:360 openssl#17 0x564ed19d5f25 in req_main apps/req.c:806 openssl#18 0x564ed19b8de0 in do_cmd apps/openssl.c:564 openssl#19 0x564ed1985165 in main apps/openssl.c:183 openssl#20 0x7f2651c4a082 in __libc_start_main ../csu/libc-start.c:308 Reviewed-by: Tom Cosgrove <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#23165)

The following issue was found in automatic tests with thread sanitizer builds in ClickHouse (which uses OpenSSL 3.2.1) [0]. The first stack [1] does proper locking (function 'x509_store_add', x509_lu.c) but in the second stack [2], function 'get_cert_by_subject_ex' (by_dir.b) forgets to lock when calling 'sk_X509_OBJECT_is_sorted'. [0] ClickHouse/ClickHouse#63049 [1] WARNING: ThreadSanitizer: data race (pid=1870) Write of size 4 at 0x7b08003d6810 by thread T552 (mutexes: write M0, write M1, write M2, write M3): #0 OPENSSL_sk_insert build_docker/./contrib/openssl/crypto/stack/stack.c:280:16 (clickhouse+0x203ad7e4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 OPENSSL_sk_push build_docker/./contrib/openssl/crypto/stack/stack.c:401:12 (clickhouse+0x203ad7e4) #2 x509_store_add build_docker/./contrib/openssl/crypto/x509/x509_lu.c:419:17 (clickhouse+0x203d4a52) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 X509_STORE_add_cert build_docker/./contrib/openssl/crypto/x509/x509_lu.c:432:10 (clickhouse+0x203d48a2) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #4 X509_load_cert_file_ex build_docker/./contrib/openssl/crypto/x509/by_file.c:127:18 (clickhouse+0x203b74e6) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:333:22 (clickhouse+0x203b684c) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#7 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) openssl#8 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#11 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#16 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#20 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#21 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#22 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#23 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) [2] Previous read of size 4 at 0x7b08003d6810 by thread T553 (mutexes: write M4, write M5, write M6): #0 OPENSSL_sk_is_sorted build_docker/./contrib/openssl/crypto/stack/stack.c:490:33 (clickhouse+0x203adcff) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #1 get_cert_by_subject_ex build_docker/./contrib/openssl/crypto/x509/by_dir.c:423:10 (clickhouse+0x203b6d8f) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #2 X509_LOOKUP_by_subject_ex build_docker/./contrib/openssl/crypto/x509/x509_lu.c:105:16 (clickhouse+0x203d46ec) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #3 ossl_x509_store_ctx_get_by_subject build_docker/./contrib/openssl/crypto/x509/x509_lu.c:360:17 (clickhouse+0x203d46ec) #4 X509_STORE_CTX_get1_issuer build_docker/./contrib/openssl/crypto/x509/x509_lu.c:782:10 (clickhouse+0x203d56cb) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #5 get1_trusted_issuer build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3194:10 (clickhouse+0x203db4a9) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) #6 build_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:3324:40 (clickhouse+0x203db4a9) openssl#7 verify_chain build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:240:15 (clickhouse+0x203dbe27) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#8 x509_verify_x509 build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:358 (clickhouse+0x203d7fd8) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#9 X509_verify_cert build_docker/./contrib/openssl/crypto/x509/x509_vfy.c:293:56 (clickhouse+0x203d8215) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#10 ssl_verify_internal build_docker/./contrib/openssl/ssl/ssl_cert.c:496:13 (clickhouse+0x2019a2a4) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#11 ssl_verify_cert_chain build_docker/./contrib/openssl/ssl/ssl_cert.c:543:12 (clickhouse+0x2019a402) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#12 tls_post_process_server_certificate build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:2072:9 (clickhouse+0x20227658) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#13 ossl_statem_client_post_process_message build_docker/./contrib/openssl/ssl/statem/statem_clnt.c:1159:16 (clickhouse+0x202272ee) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#14 read_state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:712:35 (clickhouse+0x2021e96d) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#15 state_machine build_docker/./contrib/openssl/ssl/statem/statem.c:478:21 (clickhouse+0x2021e96d) openssl#16 ossl_statem_connect build_docker/./contrib/openssl/ssl/statem/statem.c:297:12 (clickhouse+0x2021ddce) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#17 SSL_do_handshake build_docker/./contrib/openssl/ssl/ssl_lib.c:4746:19 (clickhouse+0x201a5781) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#18 SSL_connect build_docker/./contrib/openssl/ssl/ssl_lib.c:2208:12 (clickhouse+0x201a5893) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) openssl#19 Poco::Net::SecureSocketImpl::connectSSL(bool) build_docker/./base/poco/NetSSL_OpenSSL/src/SecureSocketImpl.cpp:206:11 (clickhouse+0x1d179567) (BuildId: 3ceefd39df36d762f06bf9aab19cfc3467e4558b) CLA: trivial Reviewed-by: Todd Short <[email protected]> Reviewed-by: Tomas Mraz <[email protected]> (Merged from openssl#24295) (cherry picked from commit af75373)

rajeev-0 force-pushed the add_KEM_support branch from 198e800 to 4939df5 Compare August 28, 2023 09:49

rajeev-0 force-pushed the add_KEM_support branch from 4939df5 to 97b96e0 Compare August 28, 2023 10:13

rajeev-0 requested a review from DDvO August 28, 2023 10:46

github-actions bot added the severity: fips change label Aug 28, 2023

rajeev-0 force-pushed the add_KEM_support branch from 97b96e0 to aac09e1 Compare August 31, 2023 07:37

DDvO requested changes Sep 6, 2023

View reviewed changes

rajeev-0 force-pushed the add_KEM_support branch 2 times, most recently from eaded1a to 734af7c Compare September 14, 2023 14:14

rajeev-0 force-pushed the add_KEM_support branch from a8367b6 to b450fe3 Compare December 28, 2023 11:43

rajeev-0 force-pushed the add_KEM_support branch from aeceaf2 to 5ebbc34 Compare January 8, 2024 08:31

github-actions bot added the severity: ABI change label Jan 8, 2024

rajeev-0 added 12 commits March 25, 2024 09:19

TODO! server_withkem.cnf, test_withserverkem.cnf: added for testing

71b9b35

libcrypto.num: add CMP APIs

3974259

server_kem.crt: add server cert with keyusage=keyEncipherment

5de3e13

test_credentials.csv: add test with kem cert

0a32ca5

cmp_{asn.c, kem.c, local.h}: move ossl_cmp_kem_KemOtherInfo_new() fro…

19b5539

…m cmp_kem.c

cmp_mock_srv.c: add TODO for verifying client Kem certificate.

9545e41

cmp_ctx.c: add TODO to generalize KEM default parameter

181afc4

cmp_ctx.c: add helper function to set octet secret in ctx.

191ed08

cmp_{ctx.c,local.h}: remove ossl_cmp_ctx_get0_transactionID()

367bcf5

cmp_{ctx.c, local.h}: update kem variable name

b072001

cmp_client.c: update kem variable name

77fd2df

rajeev-0 added 17 commits March 25, 2024 09:19

cmp_kem.c: update kem variable name

43f7b08

cmp_protect.c: update kem variable name

3b53f3b

cmp_server.c: update kem variable name

464079e

cmp_kem.c: fix return in case of error

ba6dcd5

cmp_kem.c: remove RSA kem code from cmp_kem.c

8f53865

cmp_{kemrsa.c, local.h}: add rsa kem and kdf2 implementation

58b8b0f

build.info: add file cmp_kemrsa.c

0a56fe6

cmp_kem.c: updated to support KEM from oqsprovider

a00cf23

kem_{RSA,kyber512}.{crt,key}: add RSA and kyber512 cert and key for K…

7587a73

…EM testing

cmp_{util.c, local.h}: add function ossl_cmp_x509_algor_set0

43e4703

cmp_asn.c: fix memory leak

e090035

cmp_kem.c: fix memory leak

8e9151d

cmp_kemrsa.c: fix memory leak

36505aa

to be removed, cmp_kem.c:add debug message

4b14d3d

kem_kyber1024.{crt, priv, pub}: add kyber1024 key and certificate for…

ed6fb45

… testing.

update structure as per updated RFC

28bc2b3

Update kem.crt certificate

b219a7d

rajeev-0 force-pushed the add_KEM_support branch from c8a157f to b219a7d Compare March 25, 2024 08:20

rajeev-0 closed this May 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add kem support #4

Add kem support #4

rajeev-0 commented Aug 28, 2023

DDvO left a comment

DDvO Sep 6, 2023

DDvO Sep 6, 2023

rajeev-0 Sep 7, 2023

DDvO Sep 6, 2023

DDvO Sep 6, 2023

rajeev-0 Sep 8, 2023

DDvO Sep 6, 2023

rajeev-0 Sep 8, 2023

DDvO Sep 6, 2023

DDvO Sep 6, 2023

DDvO Sep 6, 2023

rajeev-0 Sep 14, 2023

DDvO Sep 6, 2023

rajeev-0 Sep 14, 2023

DDvO Sep 6, 2023

Add kem support #4

Add kem support #4

Conversation

rajeev-0 commented Aug 28, 2023

Checklist

DDvO left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment