diff --git a/README.md b/README.md index a2b4215..b65fdca 100644 --- a/README.md +++ b/README.md @@ -1,38 +1,94 @@ -# Prisma Cloud Inspect +# Prisma Cloud Inspection Script ## Description -The `inspect.sh` script queries the Prisma Cloud API for all enabled Policies, +The `pc-inspect.py` script queries the Prisma Cloud API for all enabled Policies and for all Alerts within a Relative Time Range (with a default of one month), -and outputs the results to `${CUSTOMER_NAME}-policies.txt` and `${CUSTOMER_NAME}-alerts.txt` files. +and outputs the results to `*-policies.txt` and `*-alerts.txt` files. +It can process those files, outputting: -The `inspect.py` script parses the policies and alerts files created by the `inspect.sh` script, -calculating various results. It's output utilizes tabs, allowing for import into a spreadsheet. +* Open and Closed Alerts By Compliance Standard +* Open and Closed Alerts By Policy +* Summary of Open and Closed Alerts Totals -## Usage +It's output utilizes tabs, allowing for import into a spreadsheet. -* Download the `inspect.sh` and `inspect.py` scripts. -* Execute the `inspect.sh` script to collect the data. -* Execute the `inspect.py` script to process the data. -* Import the data into Google Sheets, and/or Google Slides (For example: [PCS Inspect](https://docs.google.com/presentation/d/10x_PGAu0ZPUGZMc4Tfevf9gpXvhIUOwGrBuRBkI6Jjc/edit?usp=sharing)) +### Usage + +* Download the `pc-inspect.py` script. +* Execute `pc-inspect.py --mode collect` to collect the data. +* Execute `pc-inspect.py --mode process` to process the data. +* Import the data into Google Sheets, and/or Google Slides ( for example: [PCS Inspect Report](https://docs.google.com/presentation/d/10x_PGAu0ZPUGZMc4Tfevf9gpXvhIUOwGrBuRBkI6Jjc/edit?usp=sharing) ) * Profit! -Note that API access requires an access key with `ACCOUNT GROUP READ ONLY` privileges configured for all accounts, or `SYSTEM ADMIN` privileges. +Use `./pc-inspect.py -h` for a complete list of parameters. + +Note that collection requires a Prisma Cloud Access Key with `ACCOUNT GROUP READ ONLY` privileges configured for all accounts, or `SYSTEM ADMIN` privileges. + +### Example + +``` +chmod +x pc-inspect.py + +./pc-inspect.py --customer_name example -u "https://api.prismacloud.io" -a "aaaaaaaa-1111-aaaa-1111-aaaaaaaa1111" -s "ssss1111ssss1111ssss1111=" -m collect -## Example +./pc-inspect.py --customer_name example -m process +./pc-inspect.py --customer_name example -m process > example.tab ``` -vi inspect.sh -chmod +x inspect.sh inspect.py -./inspect.sh -c example -u "https://api.prismacloud.io" -a "aaaaaaaa-1111-aaaa-1111-aaaaaaaa1111" -s "ssss1111ssss1111ssss1111=" -./inspect.py -c example -./inspect.py -c example > example.tab + +See [example.tab](example.tab) for example output. + +# Prisma Cloud Usage Delta Script + +## Description + +The `pc-usage-delta.py` script queries the Prisma Cloud API for License/Usage data, +saving the data to a historical file, calculating the mean of the historical data, +and comparing that mean to the current usage. +If the current usage exceeds the mean usage by a (configurable) percentage, +it will output a notification. + +This is valuable for detecting a drop or spike in usage, +such as when a cloud account is onboarded or offboarded, +or the number of resources/workloads changes unexpectedly. + +### Usage + +* Download the `pc-usage-delta.py` script. +* Customize the `notify` function in the script to meet your notification requirements. +* Execute `pc-usage-delta.py` in the context of a cron job (TODO: or a serverless function). +* Profit! + +Use `./pc-usage-delta.py -h` for a complete list of parameters. + +Note that this script requires a Prisma Cloud Access Key with `ACCOUNT GROUP READ ONLY` privileges configured for all accounts, or `SYSTEM ADMIN` privileges. + +### Example + ``` +chmod +x pc-usage-delta.py + +./pc-usage-delta.py -u "https://api.prismacloud.io" -a "aaaaaaaa-1111-aaaa-1111-aaaaaaaa1111" -s "ssss1111ssss1111ssss1111=" -## Example Output +Generating Prisma Cloud API Token +Querying Cloud Accounts +Querying Usage for 150 Cloud Accounts +...................................................................................................................................................... +Current (Licensable) Resource Count: 515 -[example.tab](example.tab) +Historical (Licensable) Resource Count: -## To Do: +{'Date': '2021-01-26', 'Resources': '1'} +{'Date': '2021-01-27', 'Resources': '552'} +{'Date': '2021-01-27', 'Resources': '552'} +{'Date': '2021-01-27', 'Resources': '515'} +{'Date': '2021-01-27', 'Resources': '515'} +{'Date': '2021-01-27', 'Resources': '104'} +{'Date': '2021-01-27', 'Resources': '104'} +{'Date': '2021-01-27', 'Resources': '515'} -* Allow the `inspect.py` script to output directly to Google Sheets and/or Google Slides, or to a file directly importable into one or both of those formats. +NOTIFY: Spike !!! +NOTIFY: Current resource count (515) is 200 percent greater that the mean resource count (168). +NOTIFY: This notification is triggered by a delta greater than 10 percent, measured over (17) samples. +``` \ No newline at end of file diff --git a/inspect.sh b/inspect.sh deleted file mode 100755 index 560b429..0000000 --- a/inspect.sh +++ /dev/null @@ -1,255 +0,0 @@ -#!/bin/bash - -DEBUG=false - -########################################################################################## -# PREREQUISITES -########################################################################################## - -if ! type "jq" > /dev/null; then - error_and_exit "jq not installed or not in execution path, jq is required for script execution." -fi - -########################################################################################## -# CONFIGURATION -########################################################################################## - -# Prisma Cloud API URL -API="" - -# Prisma Cloud Login Credentials Access Key -ACCESS_KEY="" - -# Prisma Cloud Login Credentials Secret Key -SECRET_KEY="" - -# Customer Name -CUSTOMER_NAME="" - -# Optionally limit the Alert API query to one Cloud Account -CUSTOMER_ACCOUNT="" - -# Used for the (relative) time range for the Alert API query. -# https://api.docs.prismacloud.io/reference#time-range-model - -TIME_RANGE_AMOUNT=1 -TIME_RANGE_UNIT="month" - -# Increase if necessary. -CURL_TIMEOUT=300 - -########################################################################################## -# UTILITY FUNCTIONS -########################################################################################## - -debug() { - if $DEBUG; then - echo - echo "DEBUG: ${1}" - echo - fi -} - -error_and_exit() { - echo - echo "ERROR: ${1}" - echo - exit 1 -} - -prisma_usage() { - echo "" - echo "USAGE:" - echo "" - echo " ${0} " - echo "" - echo "OPTIONS:" - echo "" - echo " --url, -u Prisma Cloud API URL" - echo " --access_key, -a API Access Key" - echo " --secret_key, -s API Secret Key" - echo " --customer, -c Customer Name, used for output file names" - echo " --cloud_account, -ca (Optional) Cloud Account ID to limit the Alert query" - echo " --time_range_amount, -ta (Optional) Time Range Amount [1, 2, 3] to limit the Alert query. Default: ${TIME_RANGE_AMOUNT}" - echo " --time_range_unit, -tu (Optional) Time Range Unit ['day', 'week', 'month', 'year'] to limit the Alert query. Default: '${TIME_RANGE_UNIT}'" - echo "" -} - -########################################################################################## -# PARAMETERS -########################################################################################## - -while (( "${#}" )); do - case "${1}" in - -u|--url) - if [ -n "${2}" ] && [ "${2:0:1}" != "-" ]; then - API=$2 - shift 2 - else - prisma_usage - error_and_exit "Argument for ${1} not specified" - fi - ;; - -a|--access_key) - if [ -n "${2}" ] && [ "${2:0:1}" != "-" ]; then - ACCESS_KEY=$2 - shift 2 - else - prisma_usage - error_and_exit "Argument for ${1} not specified" - fi - ;; - -s|--secret_key) - if [ -n "${2}" ] && [ "${2:0:1}" != "-" ]; then - SECRET_KEY=$2 - shift 2 - else - prisma_usage - error_and_exit "Argument for ${1} not specified" - fi - ;; - -c|--customer) - if [ -n "${2}" ] && [ "${2:0:1}" != "-" ]; then - CUSTOMER_NAME=$2 - shift 2 - else - prisma_usage - error_and_exit "Argument for ${1} not specified" - fi - ;; - -ca|--cloud_account) - if [ -n "${2}" ] && [ "${2:0:1}" != "-" ]; then - CUSTOMER_ACCOUNT=$2 - shift 2 - else - prisma_usage - error_and_exit "Argument for ${1} not specified" - fi - ;; - -ta|--time_range_amount) - if [ -n "${2}" ] && [ "${2:0:1}" != "-" ]; then - if ! is_numeric "${2}"; then - prisma_usage - error_and_exit "Argument for ${1} is not a number" - fi - TIME_RANGE_AMOUNT=$2 - shift 2 - else - prisma_usage - error_and_exit "Argument for ${1} not specified" - fi - ;; - -tu|--time_range_unit) - if [ -n "${2}" ] && [ "${2:0:1}" != "-" ]; then - TIME_RANGE_UNIT=$2 - shift 2 - else - prisma_usage - error_and_exit "Argument for ${1} not specified" - fi - ;; - -h|--help) - prisma_usage - exit - ;; - -*) - # Unsupported flags. - prisma_usage - error_and_exit "Unsupported flag ${1}" - ;; - esac -done - - -if [ -z "${API}" ]; then - error_and_exit "Prisma Cloud API URL not specified" -fi - -if [ -z "${ACCESS_KEY}" ]; then - error_and_exit "API Access Key not specified" -fi - -if [ -z "${SECRET_KEY}" ]; then - error_and_exit "API Secret Key not specified" -fi - -if [ -z "${CUSTOMER_NAME}" ]; then - error_and_exit "Customer Name not specified" -fi - -# Logon Data -PC_API_LOGIN_FILE=$(mktemp /tmp/prisma-api-login.XXXXXXXXX) - -########################################################################################## -# MAIN -########################################################################################## - -echo "Logging on and creating an API Token" - -curl --fail --silent \ - --request POST "${API}/login" \ - --header "Content-Type: application/json" \ - --data "{\"username\":\"${ACCESS_KEY}\",\"password\":\"${SECRET_KEY}\"}" \ - --output "${PC_API_LOGIN_FILE}" - -if [ $? -ne 0 ]; then - error_and_exit "API Login Failed" -fi - -# Check the output instead of checking the response code. - -if [ ! -s "${PC_API_LOGIN_FILE}" ]; then - rm -f "${PC_API_LOGIN_FILE}" - error_and_exit "API Login Returned No Response Data" -fi - -TOKEN=$(jq -r '.token' < "${PC_API_LOGIN_FILE}") -if [ -z "${TOKEN}" ]; then - rm -f "${PC_API_LOGIN_FILE}" - error_and_exit "Token Missing From 'API Login' Response" -fi - -debug "Token: ${TOKEN}" - -# Policies - -echo "Querying Policies (Timeout ${CURL_TIMEOUT} Seconds)" - -curl -s --request GET \ - --max-time "${CURL_TIMEOUT}" \ - --url "${API}/policy?policy.enabled=true" \ - --header 'Accept: */*' \ - --header "x-redlock-auth: ${TOKEN}" \ - | jq > ${CUSTOMER_NAME}-policies.txt - -# Alerts - -echo "Querying Alerts (Timeout ${CURL_TIMEOUT} Seconds)" - -if [ -z "${CUSTOMER_ACCOUNT}" ]; then - curl -s --request POST \ - --max-time "${CURL_TIMEOUT}" \ - --url "${API}/alert" \ - --header 'Accept: */*' \ - --header 'Content-Type: application/json; charset=UTF-8' \ - --header "x-redlock-auth: ${TOKEN}" \ - --data "{\"timeRange\":{\"value\":{\"unit\":\"${TIME_RANGE_UNIT}\",\"amount\":${TIME_RANGE_AMOUNT}},\"type\":\"relative\"}}" \ - | jq > "${CUSTOMER_NAME}-alerts.txt" -else - curl -s --request POST \ - --max-time "${CURL_TIMEOUT}" \ - --url "${API}/alert" \ - --header 'Accept: */*' \ - --header 'Content-Type: application/json; charset=UTF-8' \ - --header "x-redlock-auth: ${TOKEN}" \ - --data "{\"timeRange\":{\"value\":{\"unit\":\"${TIME_RANGE_UNIT}\",\"amount\":${TIME_RANGE_AMOUNT}},\"type\":\"relative\"},\"filters\":[{\"name\":\"cloud.accountId\",\"value\":\"${CUSTOMER_ACCOUNT}\",\"operator\":\"=\"}]}" \ - | jq > "${CUSTOMER_NAME}-alerts.txt" -fi - -# Token - -rm -f "${PC_API_LOGIN_FILE}" - -echo -echo "Done: Please review ${CUSTOMER_NAME}-policies.txt and ${CUSTOMER_NAME}-alerts.txt" -echo \ No newline at end of file diff --git a/inspect.py b/pc-inspect.py similarity index 57% rename from inspect.py rename to pc-inspect.py index 3de51b7..ec7de41 100755 --- a/inspect.py +++ b/pc-inspect.py @@ -3,68 +3,191 @@ import argparse import json import os +import requests +from requests.exceptions import RequestException import sys ########################################################################################## # Configuration ########################################################################################## -pc_parser = argparse.ArgumentParser(prog='pcsinspect') +pc_parser = argparse.ArgumentParser(description='This script collects or processes Policies and Alerts.', prog=os.path.basename(__file__)) pc_parser.add_argument( - '-c', - '--customer', + '-c', '--customer_name', + type=str, required=True, + help='*Required* Customer Name, used for Alert and Policy files') + +pc_parser.add_argument('-m', '--mode', + type=str, required=True, choices=['collect', 'process'], + help="*Required* Mode: collect Policies and Alerts, or process collected data.") + +pc_parser.add_argument('-u', '--url', type=str, - required=True, - help='*Required* Customer Name, used for input policy and alert file names') + help="(Required with '--mode collect') Prisma Cloud API URL") -# https://api.docs.prismacloud.io/reference#time-range-model +pc_parser.add_argument('-a', '--access_key', + type=str, + help="(Required with '--mode collect') API Access Key") -pc_parser.add_argument( - '-ta', - '--time_range_amount', - type=int, - default=1, - choices=[1, 2, 3], - help="(Optional) Time Range Amount of the data in the alert file. Default: 1") +pc_parser.add_argument('-s', '--secret_key', + type=str, + help="(Required with '--mode collect') API Secret Key") -pc_parser.add_argument( - '-tu', - '--time_range_unit', +pc_parser.add_argument('-ca', '--cloud_account', type=str, - default='month', - choices=['day', 'week', 'month', 'year'], - help="(Optional) Time Range Unit of the data in the alert file. Default: 'month'") + help='(Optional) Cloud Account ID to limit the Alert query') + +pc_parser.add_argument('-ta', '--time_range_amount', + type=int, default=1, choices=[1, 2, 3], + help="(Optional) Time Range Amount to limit the Alert query. Default: 1") + +pc_parser.add_argument('-tu', '--time_range_unit', + type=str, default='month', choices=['day', 'week', 'month', 'year'], + help="(Optional) Time Range Unit to limit the Alert query. Default: 'month'") + +pc_parser.add_argument('-d', '--debug', + action='store_true', + help='(Optional) Enable debugging.') args = pc_parser.parse_args() -customer = args.customer -policy_file = '%s-policies.txt' % customer -alert_file = '%s-alerts.txt' % customer -time_range_label = 'Time Range - Past %s %s' % (args.time_range_amount, args.time_range_unit.capitalize()) +#### + +DEBUG_MODE = args.debug + +RUN_MODE = args.mode +PRISMA_API_ENDPOINT = args.url # or os.environ.get('PRISMA_API_ENDPOINT') +PRISMA_ACCESS_KEY = args.access_key # or os.environ.get('PRISMA_ACCESS_KEY') +PRISMA_SECRET_KEY = args.secret_key # or os.environ.get('PRISMA_SECRET_KEY') +PRISMA_API_HEADERS = { + 'Accept': 'application/json; charset=UTF-8', + 'Content-Type': 'application/json' +} +PRISMA_API_REQUEST_TIMEOUTS = (30, 300) # (CONNECT, READ) +CUSTOMER_NAME = args.customer_name +CLOUD_ACCOUNT_ID = args.cloud_account +POLICY_FILE = '%s-policies.txt' % CUSTOMER_NAME +ALERT_FILE = '%s-alerts.txt' % CUSTOMER_NAME +TIME_RANGE_AMOUNT = args.time_range_amount +TIME_RANGE_UNIT = args.time_range_unit +TIME_RANGE_LABEL = 'Time Range - Past %s %s' % (TIME_RANGE_AMOUNT, TIME_RANGE_UNIT.capitalize()) + +########################################################################################## +# Utilities. +########################################################################################## + +def make_api_call(method, url, requ_data = None): + try: + requ = requests.Request(method, url, data = requ_data, headers = PRISMA_API_HEADERS) + prep = requ.prepare() + sess = requests.Session() + resp = sess.send(prep, timeout=(PRISMA_API_REQUEST_TIMEOUTS)) + if resp.status_code == 200: + return resp.content + else: + return {} + except RequestException as e: + print('Error with API: %s: %s' % (url, str(e))) + sys.exit() + +#### + +def get_prisma_login(): + request_data = json.dumps({ + "username": PRISMA_ACCESS_KEY, + "password": PRISMA_SECRET_KEY + }) + api_response = make_api_call('POST', '%s/login' % PRISMA_API_ENDPOINT, request_data) + resp_data = json.loads(api_response) + token = resp_data.get('token') + if not token: + print('Error with API Login: %s' % resp_data) + sys.exit() + return token + +#### + +def get_policies(): + api_response = make_api_call('GET', '%s/policy?policy.enabled=true' % PRISMA_API_ENDPOINT) + policy_file = open(POLICY_FILE, 'w') + policy_file.write(api_response) + policy_file.close() + +#### + +def get_alerts(): + body_params = {"timeRange": {"value": {"unit":"%s" % TIME_RANGE_UNIT, "amount":TIME_RANGE_AMOUNT}, "type":"relative"}} + if CLOUD_ACCOUNT_ID: + body_params["filters"] = [{"name":"cloud.accountId","value":"%s" % CLOUD_ACCOUNT_ID, "operator":"="}] + request_data = json.dumps(body_params) + api_response = make_api_call('POST', '%s/alert' % PRISMA_API_ENDPOINT, request_data) + alert_file = open(ALERT_FILE, 'w') + alert_file.write(api_response) + alert_file.close() + +########################################################################################## +########################################################################################## +# Collect mode: Query the API and write the results to files. +########################################################################################## +########################################################################################## + +if RUN_MODE == 'collect': + if not PRISMA_API_ENDPOINT: + print("Error: '--url' is required with '--mode input'") + sys.exit(0) + if not PRISMA_ACCESS_KEY: + print("Error: '--access_key' is required with '--mode input'") + sys.exit(0) + if not PRISMA_SECRET_KEY: + print("Error: '--secret_key' is required with '--mode input'") + sys.exit(0) + + print('Generating Prisma Cloud API Token') + token = get_prisma_login() + if DEBUG_MODE: + print + print(token) + print + PRISMA_API_HEADERS['x-redlock-auth'] = token + print + + print('Querying Policies') + get_policies() + print('Results saved as: %s' % POLICY_FILE) + print + + print('Querying Alerts') + get_alerts() + print('Results saved as: %s' % ALERT_FILE) + print + + print("Run '%s --customer_name %s --mode process' to process the collected data." % (os.path.basename(__file__), CUSTOMER_NAME)) + print("To save the processed data to a file, redirect the above command by adding ' > {name}-summary.tab'".format(name = CUSTOMER_NAME)) + sys.exit(0) -# Use inspect.sh or the commented curl commands in this script to create the policy and alert files. +########################################################################################## +########################################################################################## +# Inspect mode: Read the result files and output summary results. +########################################################################################## +########################################################################################## ########################################################################################## # Validation. ########################################################################################## -if not os.path.isfile(policy_file): - print('Error: Policy file does not exist: %' % policy_file) +if not os.path.isfile(POLICY_FILE): + print('Error: Policy file does not exist: %' % POLICY_FILE) sys.exit(1) -if not os.path.isfile(alert_file): - print('Error: Alert file does not exist: %' % alert_file) +if not os.path.isfile(ALERT_FILE): + print('Error: Alert file does not exist: %' % ALERT_FILE) sys.exit(1) - ########################################################################################## # Counters and Structures. ########################################################################################## -# Note it appears that `audit_event` alerts are returned from the /policy endpoint, not from the /alert endpoint. -# The `policy_counts` structure counts results from the /alert endpoint. So, included only for reference. - policy_counts = { 'high': 0, 'medium': 0, @@ -97,15 +220,7 @@ # Loop through all Policies and collect the details of each Policy. ########################################################################################## -# Example API request to generate policies.txt: -# -# curl -s --request GET \ -# --url "${API}/policy?policy.enabled=true" \ -# --header 'Accept: */*' \ -# --header "x-redlock-auth: ${TOKEN}" \ -# | jq > ${CUSTOMER_NAME}-policies.txt - -with open(policy_file, 'r') as f: +with open(POLICY_FILE, 'r') as f: policy_list = json.load(f) for policy in policy_list: @@ -138,18 +253,8 @@ # Loop through all Alerts and collect the details of each Alert. # Some details come from the Alert, some from the associated Policy. ########################################################################################## - -# Example API request to generate alerts.txt: -# -# curl -s --request POST \ -# --url "${API}/alert" \ -# --header 'Accept: */*' \ -# --header 'Content-Type: application/json; charset=UTF-8' \ -# --header "x-redlock-auth: ${TOKEN}" \ -# --data "{\"timeRange\":{\"value\":{\"unit\":\"${TIME_RANGE_UNIT}\",\"amount\":${TIME_RANGE_AMOUNT}},\"type\":\"relative\"}}" \ -# | jq > ${CUSTOMER_NAME}-alerts.txt -with open(alert_file, 'r') as f: +with open(ALERT_FILE, 'r') as f: alert_list = json.load(f) for alert in alert_list: @@ -192,7 +297,7 @@ print print('#################################################################################') -print('# SHEET: By Compliance Standard, Open and Closed Alerts, %s' % time_range_label) +print('# SHEET: By Compliance Standard, Open and Closed Alerts, %s' % TIME_RANGE_LABEL) print('#################################################################################') print print('%s\t%s\t%s\t%s' % ('Compliance Standard', 'High-Severity Alert Count', 'Medium-Severity Alert Count', 'Low-Severity Alert Count')) @@ -203,7 +308,7 @@ print print('#################################################################################') -print('# SHEET: By Policy, Open and Closed Alerts, %s' % time_range_label) +print('# SHEET: By Policy, Open and Closed Alerts, %s' % TIME_RANGE_LABEL) print('#################################################################################') print print('%s\t%s\t%s\t%s\t%s\t%s\t%s' % ('policyName', 'policySeverity', 'policyType', 'policyShiftable', 'policyRemediable', 'alertCount', 'policyComplianceStandards') ) @@ -222,7 +327,7 @@ print print('#################################################################################') -print('# SHEET: Summary, Open and Closed Alerts, %s' % time_range_label) +print('# SHEET: Summary, Open and Closed Alerts, %s' % TIME_RANGE_LABEL) print('#################################################################################') print print("Compliance Standard with Alerts: Total\t%s" % len(alerts_by_compliance_standard)) @@ -240,7 +345,9 @@ print("Alerts: Anomaly\t%s" % policy_counts['anomaly']) print("Alerts: Config\t%s" % policy_counts['config']) print("Alerts: Network\t%s" % policy_counts['network']) -# print("Alerts: Audit\t%s" % policy_counts['audit_event']) # See Note above. +# Note it appears that `audit_event` alerts are returned from the /policy endpoint, not from the /alert endpoint. +# The `policy_counts` structure counts results from the /alert endpoint. So, included only for reference. +# print("Alerts: Audit\t%s" % policy_counts['audit_event']) print("Alerts: with IaC\t%s" % alert_counts['shiftable']) print("Alerts: with Remediation\t%s" % alert_counts['remediable']) print