Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Retrieve SARIF errors and warnings correctly #4837

Draft
wants to merge 17 commits into
base: main
Choose a base branch
from

Conversation

bdovaz
Copy link
Collaborator

@bdovaz bdovaz commented Feb 23, 2025

With @nvuillam's request I have:

  • Return the number of warnings (which it didn't do)
  • Correct the calculation of counting the errors or warnings according to: https://github.com/microsoft/sarif-tutorials/blob/main/docs/2-Basics.md#results
    • If the property “level” exists in the object “result” (some linters like gitleaks does not fill it) it counts the number of objects in the array “locations”.
    • If the property “level” does not exist, it only counts the results for the level “warning”, because by default in the specification it is set like this: https://json.schemastore.org/sarif-2.1.0.json
  • Add sarif to cli_lint_errors_count in json scheme (was implemented in code but not added to schema)
  • Add sarif to cli_lint_warnings_count in json scheme
  • Share in two functions (regex and sarif) this logic

There are linters that do not fill out the SARIF level field:

llaville/sarif-php-converters#1 (Fixed!)

pmd/pmd#5573

anchore/grype#2511 (Fixed but not released yet)

In these cases if they do not fill in the “level” field as I have put in the first message, according to the specification the default value is “warning” so it will be reported as such.

There are other cases Python Bandit that fills levels that we do not show in the reporters like “note” level, in this case I make it to be interpreted as warning that is the closest one.

That is, before this PR everything was interpreted as error, now:

  • If they do not fill in the property “level” it will be reported as warning as described in the specification.
  • If they fill it as “error” it will be reported as error.
  • If they fill it as “warning” it will be reported as warning.
  • If they fill it as “note” it will be reported as a warning because it is the closest one we have in the reporters.

Copy link
Contributor

github-actions bot commented Feb 23, 2025

🦙 MegaLinter status: ⚠️ WARNING

Descriptor Linter Files Fixed Errors Warnings Elapsed time
✅ API spectral 1 0 0 1.57s
⚠️ BASH bash-exec 6 1 0 0.03s
✅ BASH shellcheck 6 0 0 0.24s
✅ BASH shfmt 6 0 0 0 0.95s
✅ COPYPASTE jscpd yes no no 2.75s
✅ DOCKERFILE hadolint 129 0 0 31.29s
✅ JSON jsonlint 20 0 0 0.19s
✅ JSON v8r 22 0 0 15.35s
⚠️ MARKDOWN markdownlint 267 0 302 0 21.39s
✅ MARKDOWN markdown-table-formatter 267 0 0 0 168.16s
⚠️ PYTHON bandit 215 66 0 2.81s
✅ PYTHON black 215 0 0 0 5.11s
✅ PYTHON flake8 215 0 0 2.14s
✅ PYTHON isort 215 0 0 0 1.48s
✅ PYTHON mypy 215 0 0 12.07s
✅ PYTHON pylint 215 0 0 33.67s
✅ PYTHON ruff 215 0 0 0 0.68s
✅ REPOSITORY checkov yes no no 38.12s
✅ REPOSITORY git_diff yes no no 0.74s
⚠️ REPOSITORY grype yes 28 no 25.87s
✅ REPOSITORY secretlint yes no no 8.66s
✅ REPOSITORY trivy yes no no 15.28s
✅ REPOSITORY trivy-sbom yes no no 0.58s
⚠️ REPOSITORY trufflehog yes 1 no 55.25s
✅ SPELL cspell 718 0 0 12.31s
⚠️ SPELL lychee 349 33 0 7.47s
✅ XML xmllint 3 0 0 0 0.93s
✅ YAML prettier 160 0 0 0 4.13s
✅ YAML v8r 103 0 0 31.72s
✅ YAML yamllint 161 0 0 3.26s

See detailed report in MegaLinter reports

MegaLinter is graciously provided by OX Security

@bdovaz
Copy link
Collaborator Author

bdovaz commented Apr 10, 2025

This PR is blocked because of the issues / PRs I mention. I have managed to merge in grype the PR and the pmd one created by someone else is still in progress, with that one I think there would be no more blockages.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants