Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

create runnable app for attack paths #43

Open
wants to merge 30 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
f35c333
non-diff
nmichalov Feb 21, 2024
a288727
test secrets
nmichalov Mar 13, 2024
5e60095
update workflow
nmichalov Apr 23, 2024
914edf0
split actions
nmichalov Apr 24, 2024
2ddea0f
update workflows
nmichalov Apr 24, 2024
238e51c
update permissions
nmichalov Apr 24, 2024
bc74fee
pr check on merge
nmichalov Apr 24, 2024
478f788
env wasn't set
nmichalov Apr 24, 2024
0cb284a
Update endor_push_scan.yml
nmichalov Apr 25, 2024
1e91618
Update endor_push_scan.yml
nmichalov Apr 25, 2024
491aed2
Update endor_push_scan.yml
nmichalov May 10, 2024
4a671b6
test workflows
nmichalov Jun 11, 2024
4082395
Update supervisory-scan.yml
nmichalov Jun 11, 2024
88d2ee7
Update scan-with-endorlabs.yml
nmichalov Jun 11, 2024
54a9dbf
Update supervisory-scan.yml
nmichalov Jun 11, 2024
52fc724
Update supervisory-scan.yml
nmichalov Jun 11, 2024
1285b52
Update endor_push_scan.yml
nmichalov Jul 30, 2024
619d724
try this one
nmichalov Jul 30, 2024
744d579
Update test.yml
nmichalov Jul 30, 2024
ba28f81
Update test.yml
nmichalov Jul 30, 2024
8875028
Update test.yml
nmichalov Jul 30, 2024
131a889
Update endor_push_scan.yml
nmichalov Jul 30, 2024
d2504ff
Update endor_push_scan.yml
nmichalov Jul 30, 2024
14f0d39
Update endor_pr_scan.yml
nmichalov Jul 30, 2024
9256b5d
remove vunerable use and hard code string out
nmichalov Aug 6, 2024
d7b2f29
try that one
nmichalov Aug 6, 2024
25e46f9
Merge branch 'endorlabs:main' into new_main
nmichalov Nov 19, 2024
df0de96
builds and runs
nmichalov Nov 20, 2024
bf4e5a1
it runs, kind of
nmichalov Nov 20, 2024
3bb21d9
updated Dockerfile
nmichalov Nov 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 34 additions & 0 deletions .github/workflows/endor_pr_scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Endor Labs PR Check for Vulns and Secrets
on:
pull_request:
branches: [ new_main]
workflow_dispatch:
jobs:
scan:
permissions:
security-events: write
contents: read
id-token: write
issues: write # Required to automatically comment on PRs for new policy violations
pull-requests: write # Required to automatically comment on PRs for new policy violations
actions: read
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v3
- name: Setup Java
uses: actions/setup-java@v3
with:
distribution: 'microsoft'
java-version: '17'
- name: Build Package
run: mvn clean install
- name: Endor Labs Scan Pull Request
uses: endorlabs/[email protected]
with:
namespace: 'nate-learn'
scan_dependencies: true
scan_secrets: true
pr: true
enable_pr_comments: true
github_token: ${{ secrets.GITHUB_TOKEN }} # Required for PR comments on new policy violations
38 changes: 38 additions & 0 deletions .github/workflows/endor_push_scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
name: Endor Labs Main Branch Check for Secrets and Vulns
on:
push:
branches: [ new_main ]
workflow_dispatch:
jobs:
scan:
permissions:
security-events: write
contents: read
actions: read
id-token: write
issues: write
runs-on: ubuntu-latest
steps:
- name: Checkout Repository
uses: actions/checkout@v3
- name: Setup Java
uses: actions/setup-java@v3
with:
distribution: 'microsoft'
java-version: '17'
- name: Build Package
run: mvn clean install
- name: Endor Labs Scan Main
uses: endorlabs/[email protected]
with:
namespace: 'nate-learn'
scan_dependencies: true
scan_secrets: true
pr: false
scan_summary_output_type: 'table'
sarif_file: 'findings.sarif'
additional_args: '--bypass-host-check'
- name: Upload findings to github
uses: github/codeql-action/upload-sarif@v3
with:
sarif_file: 'findings.sarif'
207 changes: 207 additions & 0 deletions .github/workflows/scan-with-endorlabs.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,207 @@
# this workflow is a reusable flow to scan a repo using autobuild as a sidecar job
# _REQUIRES a LINUX runner on an amd64-compatible arch, with bash >= 4.0 available and curl installed
name: Scan Repo with Endor Labs @ v0
on:
workflow_call:
inputs:
git-url:
description: git URL for repository to scan
required: true
type: string
namespace:
description: Endor Labs namespace (tenant name) for authentication/findings
required: true
type: string
is-pr:
description: Do not monitor this version; set true when you're scanning a Pull request
type: boolean
default: false
git-branch:
description: the branch of the target repo we wish to scan; if empty, scan the default branch
required: false
type: string
upload-logs:
description: "upload logs as artifacts (default: false)"
type: boolean
default: true
upload-json:
description: "upload json results as artifact (default: false)"
type: boolean
default: true
upload-sarif:
description: "upload SARIF results to GitHub for consumption in Security Tab (requires public repo or GHAS license); use only when sidecar-scanning within a repo!"
type: boolean
default: true
endorlabs-api-url:
description: "URL to use for Endor Labs root API (default = 'https://api.endorlabs.com')"
required: false
type: string
default: "https://api.endorlabs.com"
env-file:
description: Path to file that contains endor environment configuration
required: false
type: string
default: ".endorlabs/environment"
runner:
description: "Runner to use (default: 'ubuntu-22.04')"
required: false
type: string
default: "ubuntu-22.04"

secrets:
endorlabs-api-auth:
description: "API auth data in the form KEY:SECRET -- if present, disables GitHub Action OIDC auth"
required: false
github-access-token:
description: "github access token to use instead of default Actions token; may be required to clone private repos"
required: false

jobs:
endorlabs-auto-scan:
runs-on: ${{ inputs.runner }}
permissions:
id-token: write # allows authentication to Endor Labs using Actions OIDC JWT Token
contents: read # allows this job to clone org repos
security-events: write
env:
ENDOR_API: ${{ inputs.endorlabs-api-url }}
ENDOR_NAMESPACE: ${{ inputs.namespace }}
ENDOR_HOME: ".endorlabs-workflow"
ENDOR_OS: linux
ENDOR_ARCH: amd64
ENDOR_GITHUB_ACTION_TOKEN_ENABLE: "true"
ENDOR_SCAN_SUMMARY_OUTPUT_TYPE: "json"
ENDOR_SCAN_PR: ${{ inputs.is-pr }}
GH_TOKEN: ${{ secrets.github-access-token || github.token }}
steps:
- id: safety-check
name: Check safety of inputs
shell: bash
run: |
shopt -s nocasematch
if ! [[ "$ENDOR_NAMESPACE" =~ ^[a-z0-9_-]+([.][a-z0-9_-]+)*$ ]]; then
echo "::error::namespace input does not conform to namespace format"
exit 1
fi
shopt -u nocasematch
if ! [[ "$ENDOR_API" =~ ^https://[^/]+\.endorlabs.com$ ]]; then
echo "::error::endor API URL input does not use a *.endorlabs.com host, or does not start with https://"
exit 2
fi

- id: endorlabs-clone
name: clone ${{ inputs.git-url }}
shell: bash
env:
CLONE_URL: "${{ inputs.git-url }}"
run: |
if (git clone "${CLONE_URL}" "project"); then
>&2 echo "Cloned '${CLONE_URL}' successfully using bare git"
else
>&2 echo "Failed to clone '${CLONE_URL}' with bare git, trying gh"
if ! (gh repo clone "${CLONE_URL}" "project"); then
echo "::error::unable to clone '${CLONE_URL}' with any supported method"
exit 1
fi
fi

- id: endorlabs-setup
name: install endorlabs client
shell: bash
env:
ENDORLABS_API_AUTH: ${{ secrets.endorlabs-api-auth }}
ENDORLABS_ENV_FILE: ${{ inputs.env-file }}
run: |
if [[ -f "${ENDORLABS_ENV_FILE}" ]]; then
grep -E '^ENDOR_' "${ENDORLABS_ENV_FILE}" >> $GITHUB_ENV
echo "::group::added repo environment hints from ${ENDORLABS_ENV_FILE}'"
grep -E '^ENDOR_' "${ENDORLABS_ENV_FILE}"
echo "::endgroup::"
fi
mkdir -p "${ENDOR_HOME}"
curl -o "${ENDOR_HOME}/endorctl" ${ENDOR_API}/download/latest/endorctl_${ENDOR_OS}_${ENDOR_ARCH}
echo "$(curl -s https://api.endorlabs.com/sha/latest/endorctl_${ENDOR_OS}_${ENDOR_ARCH}) ${ENDOR_HOME}/endorctl" | sha256sum -c || exit 1
chmod +x "${ENDOR_HOME}/endorctl"
if [[ -n "$ENDORLABS_API_AUTH" ]]; then
echo "::notice::Disabling GitHub Actions OIDC auth for Endor Labs, using API key auth instead"
echo "ENDOR_API_CREDENTIALS_KEY=$(cut -d: -f1 <<< ${ENDORLABS_API_AUTH})" >> $GITHUB_ENV
echo "ENDOR_API_CREDENTIALS_SECRET=$(cut -d: -f2 <<< ${ENDORLABS_API_AUTH})" >> $GITHUB_ENV
echo "ENDOR_GITHUB_ACTION_TOKEN_ENABLE=false" >> $GITHUB_ENV
fi

- id: endorlabs-host-check
name: perform host check
shell: bash
run: |
LANGUAGES=""
"${ENDOR_HOME}/endorctl" host-check --path=project --droid-gpt 2> >(tee "${ENDOR_HOME}/host-check.log" >&2)
for lang in $(sed -nr 's/^.*Checking ([a-z_-]+) toolset.*$/\1/p' "${ENDOR_HOME}/host-check.log"); do LANGUAGES="${lang},$LANGUAGES"; done
if [[ -n "$LANGUAGES" ]]; then
if [[ -z "$ENDOR_SCAN_LANGUAGES" ]]; then
# only do this if the repo env hasn't manually set languages
# echo "ENDOR_SCAN_LANGUAGES=${LANGUAGES::-1}" >> $GITHUB_ENV ## TODO: decide if we really want to do this
>&2 echo "::notice::Detected languages: ${LANGUAGES::-1}"
fi
fi

- id: endorlabs-scan-ref
name: scan branch of ${{ inputs.git-url }}
shell: bash
env:
SCAN_BRANCH: ${{ inputs.git-branch }}
CLONE_URL: "${{ inputs.git-url }}"
MAKE_SARIF: ${{ inputs.upload-sarif }}
run: |
if [[ -z "$ENDOR_SCAN_LANGUAGES" ]]; then
echo "::warning::No supported languages were detected to scan; check prior step for details. Scanning without language constraint list"
fi
if [[ -n "$SCAN_BRANCH" ]]; then
echo "::notice::Scanning requested ref '${SCAN_BRANCH}'"
git checkout "${SCAN_BRANCH}"
else
echo "::notice::Scanning default branch"
fi
if [[ "$MAKE_SARIF" == "true" ]]
then
echo "::notice::SARIF output requested for submission to GitHub"
ENDOR_SCAN_SUMMARY_SARIF_FILE="${PWD}/scan-results.sarif"
fi
"${ENDOR_HOME}/endorctl" scan --path ./project --bypass-host-check --build --verbose 2> >(tee "${ENDOR_HOME}/default-scan.log" >&2) > "${ENDOR_HOME}/scan-results-${GITHUB_RUN_ID}.json"
echo "results-json=${ENDOR_HOME}/scan-results-${GITHUB_RUN_ID}.json" >> $GITHUB_OUTPUT
echo "results-sarif=$ENDOR_SCAN_SUMMARY_SARIF_FILE" >> $GITHUB_OUTPUT
echo "github-run-id=$(echo $CLONE_URL|sed -r 's/[^a-z0-9_-]+/_/g')-${GITHUB_RUN_ID}" >> $GITHUB_OUTPUT
ls -r

- id: endorlabs-upload-json-results
name: upload scan results as artifacts
if: inputs.upload-json
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: endorlabs-scan-results-${{ steps.endorlabs-scan-ref.outputs.github-run-id }}
path: ${{ steps.endorlabs-scan-ref.outputs.results-json }}
if-no-files-found: warn

- id: endorlabs-upload-scan-logs
name: upload scan logs as artifacts
if: inputs.upload-logs == true
uses: actions/upload-artifact@v4
continue-on-error: true
with:
name: endorlabs-scan-logs-${{ steps.endorlabs-scan-ref.outputs.github-run-id }}
path: ${{ env.ENDOR_HOME }}/*.log
if-no-files-found: warn

- id: endorlabs-submit-sarif
name: submit SARIF output to GitHub
# env:
# GH_TOKEN: ${{ secrets.write_pat }}
if: inputs.upload-sarif == true
uses: github/codeql-action/upload-sarif@v3
continue-on-error: true
with:
# token: ${{ secrets.write_pat }}
sarif_file: ${{ steps.endorlabs-scan-ref.outputs.results-sarif }}
checkout_path: "${{ github.workspace }}/project"
wait-for-processing: false

44 changes: 44 additions & 0 deletions .github/workflows/supervisory-scan.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
name: Scan Specified Projects
on:
workflow_dispatch:
# uncomment below and adjust to frequency to automatically run
# schedule:
# - cron: 1 */2 * * *

jobs:
# generate_matrix:
# runs-on: ubuntu-latest
# outputs:
# matrix: ${{ steps.set-matrix.outputs.matrix }}
# steps:
# - name: Checkout code
# uses: actions/checkout@v4
# with:
# sparse-checkout: |
# repos.csv
# sparse-checkout-cone-mode: false

# - name: Read repos and namespaces from CSV
# id: set-matrix
# run: |
# OUTPUT="$(awk -F',' '{print "{\"repo\":\"" $1 "\", \"namespace\":\"" $2 "\"}"}' repos.csv | jq -s '.' | tr '\n' ' ')"
# echo "matrix=$OUTPUT" >> $GITHUB_OUTPUT

scan-matrix:
# needs: generate_matrix
# strategy:
# # fail-fast should be set false so that one scan failure doesn't stop the whole matrix
# fail-fast: false
# max-parallel: 5
# matrix:
# projects: ${{ fromJSON(needs.generate_matrix.outputs.matrix) }}
uses: nmichalov/app-java-demo/.github/workflows/scan-with-endorlabs.yml@new_main
permissions:
id-token: write # allows authentication to Endor Labs using Actions OIDC JWT Token
pull-requests: write # allows scanner to leave a pull request comment, if enabled
issues: write # allows scanner to leave a PR comment, if enabled
contents: read # allows this job to clone org repos
security-events: write
with:
git-url: "https://github.com/nmichalov/app-java-demo.git"
namespace: nate-learn.testb #${{ matrix.projects.namespace }}
21 changes: 21 additions & 0 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
on: [push, workflow_dispatch]
name: build
jobs:
use-endorctl:
name: Usage of Endorctl
runs-on: ubuntu-latest
permissions:
id-token: write
packages: write
contents: read
steps:
- name: Setup with Endor Labs
# uses: endorlabs/[email protected]
uses: endorlabs/[email protected]
with:
namespace: "nate-learn"
enable_github_action_token: true

- name: Use Endorctl
run: |
endorctl api list -r Project
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
target
1 change: 0 additions & 1 deletion Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ WORKDIR /app

# Copy the built artifact from the build stage
COPY --from=build /app/target/endor-java-webapp-demo.jar .
COPY --from=build /app/target/endor-java-webapp-demo-jar-with-dependencies.jar .
# Expose any necessary ports
EXPOSE 443

Expand Down
4 changes: 4 additions & 0 deletions META-INF/MANIFEST.MF
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
Manifest-Version: 1.0
Created-By: Maven JAR Plugin 3.4.1
Build-Jdk-Spec: 23

Loading