Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature crypto backend2 #202

Closed
wants to merge 10 commits into from
Closed
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,13 @@ on:

jobs:
build:
name: test-${{ matrix.python }}-${{ matrix.build-type }}
runs-on: ubuntu-latest
strategy:
fail-fast: false
matrix:
python: ["3.8", "3.9", "3.10", "3.11", "3.12", "3.13"]
build-type: ["noopenssl", "legacy"]
steps:
- uses: actions/checkout@v3
- name: Setup Python
Expand All @@ -34,7 +36,7 @@ jobs:
- name: Install Poetry & Tox
run: pip install poetry>1.0.0 tox>3.3.0
- name: Run tox
run: tox
run: tox -e github-${{ matrix.build-type }}
# This job runs our tests like external parties such as packagers.
external:
runs-on: ubuntu-latest
Expand Down
15 changes: 15 additions & 0 deletions CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,21 @@ Changelog
* Dropped support for Python 3.7.
* Support for Python 3.8 has been deprecated and will be removed in the next
scheduled release.
* Deprecated pyOpenSSL in favor of Cryptography and removed the required
dependency. The underlying storage format of the `josepy.util.ComparableX509`
has been switched to `cryptography.x509` objects, and the
`josepy.util.ComparableX509.wrapped` attribute will now be either a
`cryptography.x509.Certificate` or `cryptography.x509.CertificateSigningRequest`
- objects from the `opensssl.crypto` package will be automatically transcoded
to their Cryptography counterparts on initialization. A new convenience
attribute, `josepy.util.ComparableX509.wrapped_legacy` will return an
`opensssl.crypto` object for affected projects that are unable to immediately
migrate code to the Cryptography objects. This is offered as a minimally
breaking change to aid in migration to Cryptography. Affected projects should
either pin to `1.14.0` or utilize the new attribute in a "hotfix" release.
Please note, due to the removal of `X509_V_FLAG_NOTIFY_POLICY` in pyOpenSSL
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain a bit more? Where does X509_V_FLAG_NOTIFY_POLICY come into play? Not seeing it on a grep.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A pretty smart developer on this project, @ohemorange, figured this out. Check out their comment in the other PR ;) : #196 (comment)

In terms of this causing issues...

This breaks the linting routine (the Python Tests for Josepy / external (pull_request) test), if it picks up a system pyOpenSSL that is incompatible. I got around that by setting a compatible pyopenssl as a dev/testing requirement - however (going back to my concern 2 above), even though the code does not require PyOpenSSL to operate, it would probably be wise to list the minimum compatible version as a regular dependency.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, but you see that was last-year @ohemorange, a person that this-year @ohemorange retains no memory of.

I still don't see why there would be a version conflict though, what's causing the conflict? Is this just saying that there will be a version conflict only if they don't update to the new required version (or remove pyopenssl entirely)? Or is there a conflicting pyopenssl requirement somewhere else, that is has to be under some version? We usually just write required version updates as "josepy now requires project version x.x.x," see https://github.com/certbot/certbot/pull/10130/files for an ongoing example.

And as I mentioned in my other comment, it's probably easiest to remove the pyopenssl requirement after this PR in one fell swoop rather than making it conditional, especially if it's both a conditional requirement and a dev/testing requirement.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still don't see why there would be a version conflict though, what's causing the conflict?

The only time I've been able to trigger this is that test suite, and bunch of debugging/analysis suggests this is what happens:

  • PyOpenSSL is not required by josepy or any dependencies, so is not installed into the virtualenv
  • The linter sees the PyOpenSSL import, and loads the system installation of PyOpenSSL or something from the last cached download - which sometimes (but not always) is an earlier version.
  • The linter picks up the incompatibility,

I don't think this would be triggered in production code - I think it's just a happenstance of the linter.

And as I mentioned in my other comment, it's probably easiest to remove the pyopenssl requirement after this PR in one fell swoop rather than making it conditional, especially if it's both a conditional requirement and a dev/testing requirement.

Agreed. I'm going to migrate PyOpenSSL into a requirement, but not use it. Then it can be dropped on the next release. If someone is still using PyOpenSSL, forcing them to use a more modern version makes sense.

`23.2.0`, projects migrating to the new backend may experience a version
conflict during the code transition.

1.14.0 (2023-11-01)
-------------------
Expand Down
5 changes: 3 additions & 2 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

13 changes: 11 additions & 2 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -42,9 +42,11 @@ python = "^3.8"
# load_pem_private/public_key (>=0.6)
# rsa_recover_prime_factors (>=0.8)
# add sign() and verify() to asymetric keys (RSA >=1.4, ECDSA >=1.5)
cryptography = ">=1.5"
# add not_valid_after_utc() in (>=42.0.0)
cryptography = ">=42.0.0"
# Connection.set_tlsext_host_name (>=0.13)
pyopenssl = ">=0.13"
# incompatibilities with cryptography; X509_V_FLAG_NOTIFY_POLICY removed (>=23.2.0)
pyopenssl = {version = ">=23.2.0", optional=true}
# >=4.3.0 is needed for Python 3.10 support
sphinx = {version = ">=4.3.0", optional = true}
sphinx-rtd-theme = {version = ">=1.0", optional = true}
Expand Down Expand Up @@ -73,10 +75,17 @@ docs = [
"sphinx",
"sphinx-rtd-theme",
]
legacy = [
"pyopenssl",
]

[tool.poetry.scripts]
jws = "josepy.jws:CLI.run"

# pyopenssl is only needed for testing
[tool.poetry.group.dev.dependencies]
pyopenssl = ">=23.2.0"

# Black tooling configuration
[tool.black]
line-length = 100
Expand Down
31 changes: 14 additions & 17 deletions src/josepy/json_util.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
TypeVar,
)

from OpenSSL import crypto
from cryptography import x509
from cryptography.hazmat.primitives.serialization import Encoding

from josepy import b64, errors, interfaces, util

Expand Down Expand Up @@ -429,56 +430,52 @@ def decode_hex16(value: str, size: Optional[int] = None, minimum: bool = False)
def encode_cert(cert: util.ComparableX509) -> str:
"""Encode certificate as JOSE Base-64 DER.

:type cert: `OpenSSL.crypto.X509` wrapped in `.ComparableX509`
:type cert: `x509.Certificate` wrapped in `.ComparableX509`
:rtype: unicode

"""
if isinstance(cert.wrapped, crypto.X509Req):
if isinstance(cert._wrapped_new, x509.CertificateSigningRequest):
raise ValueError("Error input is actually a certificate request.")

return encode_b64jose(crypto.dump_certificate(crypto.FILETYPE_ASN1, cert.wrapped))
return encode_b64jose(cert._wrapped_new.public_bytes(Encoding.DER))


def decode_cert(b64der: str) -> util.ComparableX509:
"""Decode JOSE Base-64 DER-encoded certificate.

:param unicode b64der:
:rtype: `OpenSSL.crypto.X509` wrapped in `.ComparableX509`
:rtype: `x509.Certificate` wrapped in `.ComparableX509`

"""
try:
return util.ComparableX509(
crypto.load_certificate(crypto.FILETYPE_ASN1, decode_b64jose(b64der))
)
except crypto.Error as error:
return util.ComparableX509(x509.load_der_x509_certificate(decode_b64jose(b64der)))
except Exception as error:
raise errors.DeserializationError(error)


def encode_csr(csr: util.ComparableX509) -> str:
"""Encode CSR as JOSE Base-64 DER.

:type csr: `OpenSSL.crypto.X509Req` wrapped in `.ComparableX509`
:type csr: `x509.CertificateSigningRequest` wrapped in `.ComparableX509`
:rtype: unicode

"""
if isinstance(csr.wrapped, crypto.X509):
if isinstance(csr._wrapped_new, x509.Certificate):
raise ValueError("Error input is actually a certificate.")

return encode_b64jose(crypto.dump_certificate_request(crypto.FILETYPE_ASN1, csr.wrapped))
return encode_b64jose(csr._wrapped_new.public_bytes(Encoding.DER))


def decode_csr(b64der: str) -> util.ComparableX509:
"""Decode JOSE Base-64 DER-encoded CSR.

:param unicode b64der:
:rtype: `OpenSSL.crypto.X509Req` wrapped in `.ComparableX509`
:rtype: `cryptography.x509.CertificateSigningRequest` wrapped in `.ComparableX509`

"""
try:
return util.ComparableX509(
crypto.load_certificate_request(crypto.FILETYPE_ASN1, decode_b64jose(b64der))
)
except crypto.Error as error:
return util.ComparableX509(x509.load_der_x509_csr(decode_b64jose(b64der)))
except Exception as error:
raise errors.DeserializationError(error)


Expand Down
14 changes: 5 additions & 9 deletions src/josepy/jws.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,8 @@
cast,
)

from OpenSSL import crypto
from cryptography import x509
from cryptography.hazmat.primitives.serialization import Encoding

import josepy
from josepy import b64, errors, json_util, jwa
Expand Down Expand Up @@ -138,21 +139,16 @@ def crit(unused_value: Any) -> Any:

@x5c.encoder # type: ignore
def x5c(value):
return [
base64.b64encode(crypto.dump_certificate(crypto.FILETYPE_ASN1, cert.wrapped))
for cert in value
]
return [base64.b64encode(cert._wrapped_new.public_bytes(Encoding.DER)) for cert in value]

@x5c.decoder # type: ignore
def x5c(value):
try:
return tuple(
util.ComparableX509(
crypto.load_certificate(crypto.FILETYPE_ASN1, base64.b64decode(cert))
)
util.ComparableX509(x509.load_der_x509_certificate(base64.b64decode(cert)))
for cert in value
)
except crypto.Error as error:
except Exception as error:
raise errors.DeserializationError(error)


Expand Down
Loading
Loading