Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Content signed with an expired certifcate after the expiration date is considered unsigned #363

Closed
21 tasks done
merks opened this issue May 23, 2024 · 37 comments
Closed
21 tasks done

Comments

@merks
Copy link
Contributor

merks commented May 23, 2024

I will edit this description with additional details as they become available.

This issue affects all projects that sign jars, not just SimRel projects.

This issue directly related to the problem is already opened:

https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4662


Here is a (hopefully correct) short list of affected projects:

  • org.eclipse.cdt
  • org.eclipse.cdt.mylyn.ui (Mylyn)
  • org.eclipse.dltk
  • org.eclipse.draw2d (GEF)
  • org.eclipse.eclemma
  • org.eclipse.egit
  • org.eclipse.emf.parsley
  • org.eclipse.gef
  • org.eclipse.launchbar (CDT)
  • org.eclipse.lsp4e
  • org.eclipse.mylyn
  • org.eclipse.ocl
  • org.eclipse.papyrus
  • org.eclipse.qvt
  • org.eclipse.qvtd
  • org.eclipse.rap
  • org.eclipse.remote (CDT)
  • org.eclipse.tm (CDT)
  • org.eclipse.tracecompass
  • org.eclipse.wildwebdeveloper
  • org.eclipse.zest (GEF)

The signing certificate expired May 21, 2024:

image

Any content signed with this certificate after that date is considered unsigned, e.g.,

$/c/Program\ Files/Java/jdk-21.0.2+13/bin/jarsigner.exe  -verbose -verify /d/stuff/org.eclipse.e4.ui.dialogs_1.5.0.v20240424-0957.jar

s       4608 Wed Apr 24 22:09:26 CEST 2024 META-INF/MANIFEST.MF
        3961 Wed Apr 24 22:09:26 CEST 2024 META-INF/ECLIPSE_.SF
        9554 Wed Apr 24 22:09:26 CEST 2024 META-INF/ECLIPSE_.RSA
           0 Wed Apr 24 22:09:24 CEST 2024 META-INF/
           0 Wed Apr 24 22:09:24 CEST 2024 org/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/textbundles/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/
           0 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/full/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/full/dtool16/
           0 Wed Apr 24 22:09:24 CEST 2024 icons/full/etool16/
sm      2520 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/BasicUIJob.class
sm     13542 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree.class
sm      3362 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$NotifyingTreeViewer.class
sm      1397 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$5.class
sm      1199 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$4.class
sm      2604 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$2.class
sm      1270 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$3.class
sm      4743 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/FilteredTree$1.class
sm      7594 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/filteredtree/PatternFilter.class
sm      1258 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/dialogs/textbundles/E4DialogMessages.class
sm      1663 Wed Apr 24 22:03:14 CEST 2024 org/eclipse/e4/ui/dialogs/textbundles/messages.properties
sm      9637 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutDialogE4.class
sm      1838 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutText$1.class
sm     11033 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutText.class
sm      2222 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/AboutText$2.class
sm      4111 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/BrandingProperties.class
sm      2556 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/HyperlinkExtractor.class
sm       719 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/HyperlinkRange.class
sm       421 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/IProductConstants.class
sm      1911 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/ParsedAbout.class
sm      2617 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/ProductInformation.class
sm      7441 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/ProductProperties.class
sm      1084 Wed Apr 24 22:09:24 CEST 2024 org/eclipse/e4/ui/internal/dialogs/about/UnavailableProduct.class
sm       397 Wed Apr 24 22:03:14 CEST 2024 icons/full/dtool16/clear_co.png
sm       755 Wed Apr 24 22:03:14 CEST 2024 icons/full/dtool16/[email protected]
sm       463 Wed Apr 24 22:03:14 CEST 2024 icons/full/etool16/clear_co.png
sm      1015 Wed Apr 24 22:03:14 CEST 2024 icons/full/etool16/[email protected]
sm       214 Wed Apr 24 22:09:24 CEST 2024 .api_description
sm      1460 Wed Apr 24 22:03:14 CEST 2024 about.html
sm       614 Wed Apr 24 22:03:14 CEST 2024 plugin.properties

  s = signature was verified
  m = entry is listed in manifest
  k = at least one certificate was found in keystore

- Signed by "[email protected], CN="Eclipse.org Foundation, Inc.", OU=IT, O="Eclipse.org Foundation, Inc.", L=Ottawa, ST=Ontario, C=CA"
    Digest algorithm: SHA-256
    Signature algorithm: SHA384withRSA, 4096-bit key
  Timestamped by "CN=Symantec SHA256 TimeStamping Signer - G3, OU=Symantec Trust Network, O=Symantec Corporation, C=US" on Mi. Apr. 24 22:09:27 UTC 2024
    Timestamp digest algorithm: SHA-256
    Timestamp signature algorithm: SHA256withRSA, 2048-bit key

jar verified.

Warning:
This jar contains entries whose TSA certificate chain is invalid. Reason: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
POSIX file permission and/or symlink attributes detected. These attributes are ignored when signing and are not protected by the signature.

Re-run with the -verbose and -certs options for more details.

The signer certificate expired on 2024-05-22. However, the JAR will be valid until the timestamp expires on 2029-03-23.

We see SimRel staging is badly affected by this:

image

Note that some shown with strikeout and some are not. That's because some artifacts were signed by the certificate when it was still valid while others are signed by the certificate after is expired.


Here are JUnit-style test results:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/lastCompletedBuild/testReport/

@cdietrich
Copy link
Contributor

is there already a new cert available?

@Kellindil
Copy link
Contributor

Hello,

I will assume this means we will have to rebuild all projects currently contributed to the simrel, even those which didn't plan new versions?

Laurent

@merks
Copy link
Contributor Author

merks commented May 23, 2024

@cdietrich

No, I believe the new cert is not yet available.


@Kellindil

No, not all projects need to rebuild. It's perfectly find to have a jar signed with an expired signature as long as the siganature itself was produced within the certificates validity range. This is why I mentioned that some artifacts are in strikeout but some are not. I will collect a list of affected projects ASAP.

Please look for your artifacts in this list:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/lastCompletedBuild/testReport/

@merks
Copy link
Contributor Author

merks commented May 23, 2024

This is the impact on the users:

image

@mbarbero
Copy link
Contributor

The certificate expired yesterday, so anything built and signed before should be valid.

There is ongoing work to update the certificate. However, this update is not routine because the secret key must now be stored on an HSM, a new requirement from the code-signing CA.

https://www.eclipsestatus.io/incident/373129

@iloveeclipse
Copy link

Please look for your artifacts in this list:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/lastCompletedBuild/testReport/

It does not include org.eclipse.e4.ui.dialogsmentioned in your error above.
Can we get a report for the platform somewhere?

@merks
Copy link
Contributor Author

merks commented May 23, 2024

@iloveeclipse

I think you already noticed that I provided the platform-specific report here:

The Platform's M3 contribution was last Friday, so that content is not a problem; the Platform's own pend RC1 candidate content is a problem.

@akurtakov
Copy link
Contributor

I wonder is it possible for simrel verification to fail if contributed artifacts contain bad signature?

@pcdavid
Copy link
Contributor

pcdavid commented May 23, 2024

Shouldn't signing itself fail if the certificate is expired?

@Bralic
Copy link
Contributor

Bralic commented May 23, 2024

If possible, of course, that'd be nice.

@merks
Copy link
Contributor Author

merks commented May 23, 2024

@akurtakov

Right now the verification only uses the p2 planner to ensure there is a solution. Checking all the artifact signatures could be done by the aggregation step, which takes much longer. Right now actual testing of signatures is done by Oomph's repository analyzer as done by this job:

https://ci.eclipse.org/simrel/job/simrel.oomph.repository-analyzer.test/

Unfortunately that didn't send me an email because it didn't actually fail. 😞

The Platform's content verification sent me an email this morning so that's what I noticed first. Of course I check the report before promoting the content so that's what I noticed second.


Several factors contributed to this problem. Proactively updating the certificates would prevent all these problems. The signer verifying that the artifact is really properly signed would help catch that oversight when that proactive update hasn't taken place. Projects using the Oomph analyzer themselves would also catch this problem. And finally my complacency also contributed. That's because the aggregation builds PGP signs everything that is unsigned thanks to Tycho's support for PGP signing with skipIfJarsignedAndAnchored. So I generally never expect to see a signing failures for SimRel because that feature always kicks in. Unfortunately that Tycho feature does not call org.eclipse.osgi.signedcontent.SignedContent.checkValidity(SignerInfo) as part of signature checking, so it did not notice that this content, while signed, is not actually considered signed because it's not signed validly. Blah, blah, blah. I'm working to fix that Tycho problem and @laeubi already planned a new Tycho release for tomorrow, so I'm trying to get that in ASAP.

@Bralic
Copy link
Contributor

Bralic commented May 23, 2024

Thank you for all this information! Very instructive!
And of course all the effort.

@merks
Copy link
Contributor Author

merks commented May 23, 2024

FYI, I added this content to the start of this issue:


Here is a (hopefully correct) short list of affected projects:

  • org.eclipse.cdt
  • org.eclipse.draw2d (GEF)
  • org.eclipse.eclemma
  • org.eclipse.egit
  • org.eclipse.emf.parsley
  • org.eclipse.gef
  • org.eclipse.launchbar (CDT)
  • org.eclipse.mylyn
  • org.eclipse.ocl
  • org.eclipse.papyrus
  • org.eclipse.qvtd
  • org.eclipse.rap
  • org.eclipse.remote (CDT)
  • org.eclipse.tm (CDT)
  • org.eclipse.tracecompass
  • org.eclipse.zest (GEF)

@iloveeclipse
Copy link

Here is a (hopefully correct) short list of affected projects:

What about Platform?

@merks
Copy link
Contributor Author

merks commented May 23, 2024

@iloveeclipse

I answered that here: 😱

#363 (comment)

@iloveeclipse
Copy link

I answered that here: 😱

#363 (comment)

OK, so this ticket is for all projects except platform?

@merks
Copy link
Contributor Author

merks commented May 23, 2024

The Platform's own build is affected, i.e, the current RC1 candidate has bad signatures. In fact all projects using the CBI jar signing service yesterday and today are affected. The intent of the list is for the SimRel participants who will need to rebuild yet another new contribution once the new signing certificate is available for the CBI jar signing service. As such, the Platform will need to respin RC1 before contributing that to SimRel on Friday as currently scheduled.

@ruspl-afed
Copy link
Contributor

who will need to rebuild yet another new contribution once the new signing certificate is available for the CBI jar signing service.

Do you plan to provide cross-project notification so that we all know that we can start rebuilding Ed @merks ?

@merks
Copy link
Contributor Author

merks commented May 23, 2024

Yes, I will break my own rule and send an email when folks can respin. I've seen no updates here:

https://www.eclipsestatus.io/incident/373129

As you've probably seen on cross-projects, M3 is canceled so no one needs to lose sleep or hair over this.

@merks
Copy link
Contributor Author

merks commented May 24, 2024

FYI, there is currently no ETA for the incident:

https://www.eclipsestatus.io/incident/373129

@netomi
Copy link

netomi commented May 24, 2024

so the jar signing service is operational again.

We are still working on bringing back the authenticode signing service for windows executables, but it would be great if people can already test the jar signing service and report if everything works as expected.

@jonahgraham
Copy link
Contributor

it would be great if people can already test the jar signing service and report if everything works as expected.

I kicked off a CDT build https://ci.eclipse.org/cdt/job/cdt/job/main/400/ - but if the windows signing is still offline the build will probably fail as CDT contains a handful of dlls that need to be signed.

@jonahgraham
Copy link
Contributor

The build is running still (take a long time) but jar signing is happening and windows signing is running without error. I'll try to find time to test and report back once the build is finished later today

@cdietrich
Copy link
Contributor

/Library/Java/JavaVirtualMachines/temurin-21.jdk/Contents/Home/bin/jarsigner --verify org.eclipse.emf.mwe2.language.ide_2.18.0.20240524-1707.jar

JAR-Datei verifiziert.

=> looks good

@netomi
Copy link

netomi commented May 24, 2024

the windows / authenticode service has also been redeployed and should be working again, please test and report.

@merks
Copy link
Contributor Author

merks commented May 24, 2024

Thanks @netomi and the IT team!

I will close this issue after I've verified that the staging repository no longer contains content signed with the expired certificate.

@LorenzoBettini
Copy link
Contributor

EMF Parsley should also be fixed now.

@Godin
Copy link
Contributor

Godin commented May 26, 2024

Prior to #369

jarsigner -verify -verbose org.eclipse.eclemma.core_3.1.9.202405220136.jar | grep expire

This jar contains entries whose signer certificate has expired.
The signer certificate expired on 2024-05-22. However, the JAR will be valid until the timestamp expires on 2029-03-23.

and after

jarsigner -verify -verbose org.eclipse.eclemma.core_3.1.9.202405260028.jar | grep expire

The signer certificate will expire on 2026-06-12.
The timestamp will expire on 2031-11-10.

so EclEmma should also be fixed now.

jonahgraham added a commit to jonahgraham/simrel.build that referenced this issue May 26, 2024
@msohn
Copy link
Contributor

msohn commented May 29, 2024

I contributed a new build for JGit/EGit: #385

@merks
Copy link
Contributor Author

merks commented May 29, 2024

FYI, I have a little analyzer hacked tougher to read the test output and then produce output like this:

image

I compare that to the state of the checked items at the start of this issue so I can easily and accurately track the process.

I also see all PRs being created and merged, so I know when to check content again. 😁

@merks merks closed this as completed May 30, 2024
@titou10titou10
Copy link

It seems this is not fully solved for eclipse RCP 2024-06

The certificate associated to "eclipse.exe" RCP v2024-06 for windows,downloaded from eclipse.org is ok , expiring in 2026-06-21

But when we build an app with tycho v4.0.8 and the tycho-p2-director-pluginplugin, the generated windows executable created has a certificate that expired 2024-05-21
see jmstoolbox/jmstoolbox#173

Is there a workaround to this problem? thanks

@jonahgraham
Copy link
Contributor

@titou10titou10 I think you are seeing something slightly different. The eclipse.exe discussed in jmstoolbox/jmstoolbox#173 was signed with a valid certificate, and the screenshots show that that the digital signature is ok. I'm not sure what we can do about that unfortunately. Can you ask your user to share the details of the error they are seeing? I'll follow on your issue to see if I can help and if we find something generally useful we can report it back here.

@titou10titou10
Copy link

titou10titou10 commented Jul 2, 2024

The eclipse.exe discussed in jmstoolbox/jmstoolbox#173 was signed with a valid certificate, and the screenshots show that that the digital signature is ok.

@jonahgraham I'm not sure to understand your point
When I build JMSToolBox with tycho ("classic" build for eclipse RCP apps), the signature of the generated exe is not valid
When you right click on the generated executable > Digital Signature Information, the windows that appears clearly state that "the digital signature is not valid"

It is the same if I build, for example DBeaver, another RCP based app (https://github.com/dbeaver/dbeaver)

(sorry for the bad quality of the picture)
The printscreen for JMSToolBox freshly build this morning with tycho 4.08 and eclipse RCP 2024-06

Sans titre

To be clear, in french:
"Cette signature numérique n'est pas valide" = This digital signature is not valid
"La signature numérique de l'objet n'a pas pu se verifier" = The digital signature could not be verified
"Valide du/au" = Valid from/to

@jonahgraham
Copy link
Contributor

I don't understand why the screenshot in jmstoolbox/jmstoolbox#173 (comment) is showing valid, but your screenshot is showing invalid. As @mbarbero said in jmstoolbox/jmstoolbox#173 (comment) the thing that matters is when it was signed, not today's date. Your screenshot shows it was signed on 7 May, and that is within the validity dates.

When I check the launcher I see the same file size as you, and Windows reports certs as valid. Here is what I checked:

$ sha512sum $PWD/*
6b1a7cb2faee77804b889797793833a4c80dc6d3a0f04f43936e93c174f7c355e149a53a32c410b2d7c1b9638d9b9aeb5e2113402cc8142fdd9d4d3242ff47f9  /tmp/a/jmstoolbox/org.titou10.jtb.product/target/org.eclipse.equinox.executable-3.8.2500.v20240513-1750/bin/win32/win32/x86_64/eclipsec.exe
1a099b90844a51065687ea3c352732649b8cdb1a34647d77c8c758970a6d694d86aeaa88442a811ee123a81b835b8b1dbb597067fe1a0a0981689d464fce2a64  /tmp/a/jmstoolbox/org.titou10.jtb.product/target/org.eclipse.equinox.executable-3.8.2500.v20240513-1750/bin/win32/win32/x86_64/launcher.exe

Can you check that nothing in your build process is modifying anything after the executables are extracted?

@titou10titou10
Copy link

titou10titou10 commented Jul 2, 2024

Honestly the build process is very straight forward:

Maybe it is tycho that is modifying the final exe?
I must say, I'm far from an expert in tycho/maven etc...

If you download the artefacts from the action build from github, the JMSToolBox.exe has the same "problem", ie "The digital signature is not valid" : https://github.com/jmstoolbox/jmstoolbox/actions/runs/9598536715

@jonahgraham
Copy link
Contributor

(Conversation moved back to jmstoolbox/jmstoolbox#173 because the issue is unrelated to #363)

@netomi
Copy link

netomi commented Jul 3, 2024

I added some comment in the referenced ticket, I strongly believe the expired signing certificate is just a red herring, the signature of the file is invalid due to a digest mismatch due to the changed icon. This is unrelated to the problems we have been facing with the expired signing certificate to which this issue relates.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests