Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CWE number as enum #31

Closed
wants to merge 14 commits into from
19 changes: 9 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ Public documentation for the Benchmark is on the OWASP site at <a href="https://

This project provides a Maven plugin for OWASP Benchmark that currently has the following capabilities:

## Scorecard Generator
## Scorecard Generator
When invoked, it analyzes all the tool scan results in the /results folder as compared to the expected results file for that test suite, and generates a scorecard for all those tools in the `/scorecard` folder. Scorecard generation can be invoked like so:

```bash
Expand All @@ -23,16 +23,16 @@ Scripts like these are typically created, and included with each Benchmark test

The `CUSTOMconfig.yaml` file primarily needs to specify the version of the `expectedresults-##.csv` file. That's typically all that's needed.

## Crawler
## Crawler
Used to invoke every HTTP endpoint in a Benchmark test suite. Typically used to exercise the entire test suite so IAST and other code execution monitoring tools can identify vulnerabilities in the test suite. The Crawler can be invoked like so:

```
mvn -Djava.awt.headless=true org.owasp:benchmarkutils-maven-plugin:run-crawler -DcrawlerFile=data/TESTSUITENAME-crawler-http.xml
```
mvn -Djava.awt.headless=true org.owasp:benchmarkutils-maven-plugin:run-crawler -DcrawlerFile=data/TESTSUITENAME-crawler-http.xml
```

Note that the `TESTSUITENAME-crawler-http.xml` is generated as part of the generation of the test suite itself, so you simply need to point to the crawlerFile for that test suite.

## Verify presence of this Maven plugin.
## Verify presence of this Maven plugin.
A script is usually provided with each test suite to verify this BenchmarkUtils maven plugin has been installed locally, and if it hasn't, it tells you where to get it and how to install it (which is really easy). You'll see the following line as the 1st line of most scripts that invoke this plugin:

```
Expand All @@ -41,15 +41,14 @@ source "scripts/verifyBenchmarkPluginAvailable.sh"

## Usage

All use of these utilities should be through scripts already rovided with each Benchmark style test suite. To use this, simply clone, navigate and install the plugin:
```
git clone https://github.com/OWASP-Benchmark/BenchmarkUtils.git`
All use of these utilities should be through scripts already rovided with each Benchmark style test suite. To use this, simply clone, navigate and install the plugin:
```
git clone https://github.com/OWASP-Benchmark/BenchmarkUtils.git`
cd BenchmarkUtils
mvn install
```
```

Periodically, you should run: git pull, then: mvn install, to download any updates and build/install the latest version.

## Example
Some example invocation scripts and scoring configuration .yaml files are provided in `examplescripts_configfiles/`

Original file line number Diff line number Diff line change
Expand Up @@ -725,15 +725,15 @@ private static void process(
@SuppressWarnings("unused")
private static void printExtraCWE(
TestSuiteResults expectedResults, TestSuiteResults actualResults) {
Set<Integer> expectedCWE = new HashSet<Integer>();
Set<CweNumber> expectedCWE = new HashSet<CweNumber>();
for (int i : expectedResults.keySet()) {
List<TestCaseResult> list = expectedResults.get(i);
for (TestCaseResult t : list) {
expectedCWE.add(t.getCWE());
}
}

Set<Integer> actualCWE = new HashSet<Integer>();
Set<CweNumber> actualCWE = new HashSet<CweNumber>();
for (int i : actualResults.keySet()) {
List<TestCaseResult> list = actualResults.get(i);
if (list != null) {
Expand All @@ -743,8 +743,8 @@ private static void printExtraCWE(
}
}

Set<Integer> extras = difference(actualCWE, expectedCWE);
for (int cwe : extras) {
Set<CweNumber> extras = difference(actualCWE, expectedCWE);
for (CweNumber cwe : extras) {
System.out.println("Extra: " + cwe);
}
}
Expand Down Expand Up @@ -958,20 +958,21 @@ private static boolean compare(TestCaseResult exp, List<TestCaseResult> actList,
// System.out.println( " Evidence: " + act.getCWE() + " " + act.getEvidence() + "[" +
// act.getConfidence() + "]");

int actualCWE = act.getCWE();
int expectedCWE = exp.getCWE();
CweNumber actualCWE = act.getCWE();
CweNumber expectedCWE = exp.getCWE();

boolean match = actualCWE == expectedCWE;
boolean match = actualCWE.equals(expectedCWE);

// Special case: many tools report CWE 89 (sqli) for Hibernate Injection (hqli) rather
// than actual CWE of 564 So we accept either
if (!match && (expectedCWE == 564)) {
match = (actualCWE == 89);
if (!match && (CweNumber.HIBERNATE_INJECTION.equals(expectedCWE))) {
match = CweNumber.SQL_INJECTION.equals(actualCWE);
}

// special hack since IBM/Veracode don't distinguish different kinds of weak algorithm
if (tool.startsWith("AppScan") || tool.startsWith("Vera")) {
if (expectedCWE == 328 && actualCWE == 327) {
if (CweNumber.WEAK_HASH_ALGO.equals(expectedCWE)
&& CweNumber.WEAK_CRYPTO_ALGO.equals(actualCWE)) {
match = true;
}
}
Expand Down Expand Up @@ -1040,7 +1041,7 @@ private static TestSuiteResults readExpectedResults(File file) {
tcr.setTestCaseName(parts[0]);
tcr.setCategory(parts[1]);
tcr.setReal(Boolean.parseBoolean(parts[2]));
tcr.setCWE(Integer.parseInt(parts[3]));
tcr.setCWE(CweNumber.lookup(Integer.parseInt(parts[3])));

String tcname = parts[0].substring(TESTCASENAME.length());
tcr.setNumber(Integer.parseInt(tcname));
Expand Down
Loading