Fix assertion for test_katello_certs_check_output_invalid_input

[jameerpathan111]

Problem Statement

• test_katello_certs_check_output_invalid_input[error0-certs/invalid.crt-certs/invalid.key-certs/ca.crt] is failing with AssertionError because the error message has changed.

Solution

• Fix assertion

Related Issues

• SAT-29725

[jameerpathan111]

trigger: test-robottelo
pytest: tests/foreman/sys/test_katello_certs_check.py

[evgeni]

This looks like a raw OpenSSL error. Did that change in both EL8 and EL9?

[jameerpathan111]

This looks like a raw OpenSSL error. Did that change in both EL8 and EL9?

Hmm, it's failing only for EL9 on 6.16

[evgeni]

This looks like a raw OpenSSL error. Did that change in both EL8 and EL9?

Hmm, it's failing only for EL9 on 6.16

Then ACK on master/Stream, but you're going to need a different patch for 6.16 :/

[Satellite-QE]

PRT Result

Build Number: 9464
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/sys/test_katello_certs_check.py --external-logging
Test Result : ========== 5 passed, 1 deselected, 38 warnings in 1475.36s (0:24:35) ===========

[jameerpathan111]

This looks like a raw OpenSSL error. Did that change in both EL8 and EL9?

Hmm, it's failing only for EL9 on 6.16

Then ACK on master/Stream, but you're going to need a different patch for 6.16 :/

Can I just keep error 26 at 0 depth lookup: or error /root/certs/invalid.crt: verification failed(It's in both EL9 and EL8)?😄

[evgeni]

If the assertion can use a substring, sure

[jameerpathan111]

If the assertion can use a substring, sure

maybe let's not, kept the original message. Updating it based on rhel version.

[jameerpathan111]

trigger: test-robottelo
pytest: tests/foreman/sys/test_katello_certs_check.py

[Satellite-QE]

PRT Result

Build Number: 9470
Build Status: SUCCESS
PRT Comment: pytest tests/foreman/sys/test_katello_certs_check.py --external-logging
Test Result : ========== 5 passed, 1 deselected, 37 warnings in 1411.15s (0:23:31) ===========

[ekohl]

This error surprises me a bit. If we don't assert the explicit message we emit ourselves:
https://github.com/theforeman/foreman-installer/blob/7c2b54267a933bcf0261396c0950d51c8d5bec5c/bin/katello-certs-check#L157-L169

Perhaps that's not needed and we can simply assert the right code path is used by checking for exit code 4?

[jameerpathan111]

this is what I see in logs, no exit code 4

    
    Checking CA bundle against the certificate file: 
    [FAIL]
    
    The /root/cacert.crt does not verify the /root/certs/invalid.crt
    CN=foreman.example.com
    error 20 at 0 depth lookup: unable to get local issuer certificate
    error /root/certs/invalid.crt: verification failed
    
    Checking CA bundle size: 1
    [OK]
    
    Checking if CA bundle has trust rules: 0
    [OK]
    
    Checking Subject Alt Name on certificate 
    [FAIL]
    
    The /root/certs/invalid.crt does not contain a Subject Alt Name. Common Name is deprecated, use Subject Alt Name instead. See: https://tools.ietf.org/html/rfc2818#section-3.1
    Checking Key Usage extension on certificate for Key Encipherment 
    [OK]
    
    Checking for use of shortname as CN
    
    stderr:
    
    status: 1

[ekohl]

That signals something is broken. The error function sets the exit code:
https://github.com/theforeman/foreman-installer/blob/7c2b54267a933bcf0261396c0950d51c8d5bec5c/bin/katello-certs-check#L65-L70

Then It's also surprising the command exits unexpectedly, without showing [OK] or [FAIL]. That looks like check-shortname isn't following an expected code path:
https://github.com/theforeman/foreman-installer/blob/7c2b54267a933bcf0261396c0950d51c8d5bec5c/bin/katello-certs-check#L229-L250

Given there was already a warning that it doesn't contain a SAN I think we can conclude it's missing an else statement with success.

[jameerpathan111]

So it's a bug😅. DO you want me to file an issue?

[ekohl]

It would be great to have it covered in our tests: https://github.com/theforeman/foreman-installer/blob/develop/spec/katello_certs_check_spec.rb is what we use in CI.

Do I read it well that these certs are generated using tests/foreman/data/certs.sh? In particular:

robottelo/tests/foreman/data/certs.sh

Lines 48 to 49 in fdad16a

	# generate invalid certs with hostname as 'foreman.example.com'
	generate_certs "invalid" "/CN=foreman.example.com" "client_extensions"

[jameerpathan111]

yes