SSL error when connecting to CE 7.9.0.2 on Wildfly 24

[ecarlseen]

This is an issue I already solved, but I'm posting it here because it sucked up a ton of time before I figured it out. This was a working install upgraded from Wildfly 10 / OpenJDK 8 to Wildfly 24 / OpenJDK 11, 3-port install.

Servers on ports 8080 and 8442 worked fine, so my certificates were working correctly. Connecting to 8443 generated an error on the browser. Wireshark showed that the certificates were exchanging just fine, but the connection was being terminated by the server. Fun.

I enabled SSL / TLS debugging at the Java level by adding this line to standalone.conf:

JAVA_OPTS="$JAVA_OPTS -Djavax.net.debug=ssl,handshake"

Once this happened, I saw the certificates being exchanged and when the client certificate was starting to verify I got this exception:

2022-08-14 18:07:04,463 ERROR [stderr] (default I/O-53) javax.net.ssl|ALL|88|default I/O-53|2022-08-14 18:07:04.463 PDT|SSLEngineImpl.java:724|Closing inbound of SSLEngine 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) javax.net.ssl|ERROR|88|default I/O-53|2022-08-14 18:07:04.464 PDT|TransportContext.java:345|Fatal (INTERNAL_ERROR): closing inbound before receiving peer's close_notify ( 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) "throwable" : { 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) javax.net.ssl.SSLException: closing inbound before receiving peer's close_notify 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:133) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:117) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:340) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:296) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:287) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at java.base/sun.security.ssl.SSLEngineImpl.closeInbound(SSLEngineImpl.java:733) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//io.undertow.protocols.ssl.SslConduit.notifyReadClosed(SslConduit.java:636) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//io.undertow.protocols.ssl.SslConduit.closed(SslConduit.java:1064) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//io.undertow.protocols.ssl.SslConduit.close(SslConduit.java:1196) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//io.undertow.protocols.ssl.SslConduit.doUnwrap(SslConduit.java:852) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//io.undertow.protocols.ssl.SslConduit.doHandshake(SslConduit.java:672) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//io.undertow.protocols.ssl.SslConduit.access$900(SslConduit.java:70) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//io.undertow.protocols.ssl.SslConduit$5$1.run(SslConduit.java:1129) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//org.xnio.nio.WorkerThread.safeRun(WorkerThread.java:612) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) at [email protected]//org.xnio.nio.WorkerThread.run(WorkerThread.java:479)} 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) 2022-08-14 18:07:04,464 ERROR [stderr] (default I/O-53) )

I'll skip over all of the banging my head against various walls and my desk and get to the answer:

I had already created .p12 files and used those instead of .jks. My problem, apparently, is that I created the truststore.p12 file with openssl instead of exporting it using keytool. I did not imagine that this would make a difference. It was sending the correct CA certificate out to the client for connection, and the client was sending the correct user CA. There are some differences in the resulting formats. openssl created a .p12 file with these attributes:

`
MAC: sha1, Iteration 2048
MAC length: 20, salt length: 8
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 2048
Certificate bag
Bag Attributes:
subject=CN = REDACTED

issuer=REDACTED

-----BEGIN CERTIFICATE-----
`

keytool created a .p12 files with these attributes:

`
MAC: sha1, Iteration 100000
MAC length: 20, salt length: 20
PKCS7 Encrypted data: pbeWithSHA1And40BitRC2-CBC, Iteration 50000
Certificate bag
Bag Attributes
friendlyName: REDACTED
2.16.840.1.113894.746875.1.1: <Unsupported tag 6>
subject=CN = REDACTED

issuer=CN = REDACTED
-----BEGIN CERTIFICATE-----
`

There are three differences:

• Hash salt length
• Encryption iterations
• Bag attributes

After some searching, it appears that Java really needs Bag attribute 2.16.840.1.113894.746875.1.1 with tag 6 or it throws a fit (and an exception).

[primetomas]

This sounds really weird. There should be some article on stack overflow for this, if it's an issue with WildFly. Did you think about asking in the WildFly forum?

[ecarlseen]

I agree that it's weird, but if you search the Interwebs for "2.16.840.1.113894.746875.1.1" you'll see several references to this problem. It seems to be Java-related and not specific to Wildfly.

I just built Wildfly 24.0.1 from source to solve what appears to be an OS-related issue (I'll type it up later), and I'll play around and see if that had any effect.

[primetomas]

Very interesting. Like a bug in the JRE almost? I can see it causing lots of issues for WildFly as they move away from JKS trust stores...

[ecarlseen]

I have no idea. I know enough about Java / JRE / J2EE to be dangerous and I have enough generic troubleshooting skills to brute-force my way through most issues. The weird thing is that Java would read the P12 file just fine and send the certificate out and then refuse to authenticate the client certificate against it. It's either a JRE bug or it's some weird security feature. The annoying thing is that according to my searching openssl can't create P12 files with that parameter. I've seen some ... interesting ... workarounds for this based on manipulating the P12 file directly. At the end of the day this is a minor / scriptable annoyance, as long as you know what to look for.

[primetomas]

Thanks for the input. Good to have this documented here. Downloading the P12 from EJBCA, instead of creating if using openssl, should then be working, does it?