Inconsistencies on SBOM vulnerabilities scanning

[DmitriyLewen]

Discussed in #5790

^{Originally posted by juan131 December 15, 2023}

Description

SBOM scanning with more than one Python applications result on inconsistent reported vulns.

Note: it also affected Node pkgs, Conda pkgs, Ruby gems & JARs.

Given a SPDX file with two Python apps (such as the one shown below) each of them containing dozens of packages, Trivy reports a different number of vulnerabilities on consecutive executions.

"packages": [{
  {
    "SPDXID": "SPDXRef-Application-gnrtd7",
    "downloadLocation": "NONE",
    "filesAnalyzed": false,
    "licenseConcluded": "NOASSERTION",
    "licenseDeclared": "NOASSERTION",
    "name": "python-pkg",
    "primaryPackagePurpose": "APPLICATION",
    "sourceInfo": "Python"
  }, {
    "SPDXID": "SPDXRef-Application-gnrtd8",
    "downloadLocation": "NONE",
    "filesAnalyzed": false,
    "licenseConcluded": "NOASSERTION",
    "licenseDeclared": "NOASSERTION",
    "name": "python-pkg",
    "primaryPackagePurpose": "APPLICATION",
    "sourceInfo": "Python"
  }

The problem seems be related with setting the application file path with an empty string for these apps:

• https://github.com/aquasecurity/trivy/blob/main/pkg/sbom/spdx/unmarshal.go#L230-L240

When the nested map below is populated at ApplyLayers, given both apps don't have a file path, the resulting "key" used in the map is the same /type:python-pkg, therefore the last app extracted from the layer overwrites the previous one:

• https://github.com/aquasecurity/trivy/blob/main/pkg/fanal/applier/docker.go#L116-L120

As a consequence, the info about the packages included on that application gets lost.

Desired Behavior

Consistency on reported vulnerabilities.

Actual Behavior

Number of reported vulnerabilities differs between executions.

Reproduction Steps

Run the Trivy scanner several times to receive different amount of reported vulnerabilities:

$ trivy sbom airflow-spdx.json --quiet --format json | grep VulnerabilityID | wc -l
     385
$ trivy sbom airflow-spdx.json --quiet --format json | grep VulnerabilityID | wc -l
     380
$ trivy sbom airflow-spdx.json --quiet --format json | grep VulnerabilityID | wc -l
     385

Target

SBOM

Scanner

Vulnerability

Output Format

JSON

Mode

Standalone

Debug Output

Irrelevant

Operating System

macOS sonoma

Version

Version: 0.48.0
Vulnerability DB:
  Version: 2
  UpdatedAt: 2023-12-15 12:12:09.918117448 +0000 UTC
  NextUpdate: 2023-12-15 18:12:09.918116827 +0000 UTC
  DownloadedAt: 2023-12-15 15:57:47.294271 +0000 UTC
Java DB:
  Version: 1
  UpdatedAt: 2023-12-12 00:46:36.610548653 +0000 UTC
  NextUpdate: 2023-12-15 00:46:36.610548473 +0000 UTC
  DownloadedAt: 2023-12-12 12:29:27.481363 +0000 UTC

Checklist

• Run trivy image --reset
• Read the troubleshooting

[juan131]

Thanks @DmitriyLewen ! I noticed you self-assigned this issue, I really appreciate it. Please don't hesitate to let me know it if you're busy to address it and you want me to drop a PR proposing a solution.

[DmitriyLewen]

We are discussing changes to Package structure.
These changes should also solve this problem.

But thanks for your suggestion and help!
If we decide to fix this problem separately, I will write to you.

[mpermar]

Just wondering. Was there any further discussion done on this topic @DmitriyLewen ? I was checking with Trivy's latest release and this is still noticeable.

[DmitriyLewen]

We are still looking for the best way to solve this problem.

[beltran-rubo]

Is that issue fixed in version 0.50.0?

[DmitriyLewen]

We've updated the logic for SPDX (#6310).
So I think this issue should be resolved in version v0.50.0.

Let me know if you are still having problems with packages/applications being overwritten.

[itaysk]

@DmitriyLewen does it mean we can close this issue?

[DmitriyLewen]

@itaysk Looks like users no longer have this problem with the new version.
So yes - let's close this issue.

[pablogalegoc]

@DmitriyLewen thank you and the team for taking care of this! I've tested scanning many SBOMs of container images and I've found no issues whatsoever, but I did see some conflicting results with Helm chart SBOMs. For example, running Trivy on this nginx-helm.json:

❯ trivy sbom nginx-helm.json --quiet --format json | grep VulnerabilityID | wc -l
     198
❯ trivy sbom nginx-helm.json --quiet --format json | grep VulnerabilityID | wc -l
     179

Let me know if I can help any further and thanks in advance!

[DmitriyLewen]

Hello @pablogalegoc
Thanks for your report!

This problem is related with multiple OSes in your spdx file.
I created #6907 for this task.

Regards, Dmitriy

[pablogalegoc]

@DmitriyLewen is there anything I can help with in order to get that PR merged? Thanks in advance!

[DmitriyLewen]

Hello @pablogalegoc

some of the team were on vacation, so there is a delay in reviewing the PRs - #7303