diff --git a/README.md b/README.md index 4613d3fc6a..9490473825 100644 --- a/README.md +++ b/README.md @@ -24,6 +24,7 @@ The tools folder contains ready-made utilities which can simpilfy Google Cloud P * [DNS Sync](tools/dns-sync) - Sync a Cloud DNS zone with GCE resources. Instances and load balancers are added to the cloud DNS zone as they start from compute_engine_activity log events sent from a pub/sub push subscription. Can sync multiple projects to a single Cloud DNS zone. * [GCE Quota Sync](tools/gce-quota-sync) - A tool that fetches resource quota usage from the GCE API and synchronizes it to Stackdriver as a custom metric, where it can be used to define automated alerts. * [GCS Bucket Mover](tools/gcs-bucket-mover) - A tool to move user's bucket, including objects, metadata, and ACL, from one project to another. +* [GCP Architecture Visualizer](/tools/gcp-arch-viz) - A tool that takes CSV output from a Forseti Inventory scan and draws out a dynamic hierarchical tree diagram of org -> folders -> projects -> gcp_resources using the D3.js javascript library. * [GKE Billing Export](tools/gke-billing-export) - Google Kubernetes Engine fine grained billing export. * [GSuite Exporter](tools/gsuite-exporter/) - A Python package that automates syncing Admin SDK APIs activity reports to a GCP destination. The module takes entries from the chosen Admin SDK API, converts them into the appropriate format for the destination, and exports them to a destination (e.g: Stackdriver Logging). * [LabelMaker](tools/labelmaker) - A tool that reads key:value pairs from a json file and labels the running instance and all attached drives accordingly. diff --git a/tools/gcp-arch-viz/README.md b/tools/gcp-arch-viz/README.md new file mode 100644 index 0000000000..49761239a8 --- /dev/null +++ b/tools/gcp-arch-viz/README.md @@ -0,0 +1,42 @@ +# GCP Architecture Visualizer + +This tool serves a single, simple purpose: visualizing a user's GCP architecture and environment, so that they have a dynamic and simple-to-use way to view their world in GCP. + +It does this by: + +* Ingesting a strictly-formatted CSV input file, which is generally output from a [Forseti Security](http://forsetisecurity.org) Inventory scan. +* Dynamically building the parent->child tree of inventory resources in GCP. +* Drawing an interactive tree structure using D3.js, containing useful GCP info and per-resource icons. + + +## Usage + +The tool has been tested with [Forseti Security](http://forsetisecurity.org) 2.0, and can be used to draw out any CSV input (currently stored in gcp-data.csv) that conforms to the following schema: + +``` +id, resource_type, category, resource_id, parent_id, resource_name +``` + +CSV generation currently performed using Google Cloud SQL export from Forseti Security Inventory tables, using the query below: + +``` +SELECT id, resource_type, category, resource_id, parent_id, IFNULL(resource_data->>'$.displayName', '') as resource_data_displayname, IFNULL(resource_data->>'$.name', '') as resource_data_name FROM gcp_inventory WHERE inventory_index_id = (SELECT id FROM inventory_index ORDER BY completed_at_datetime DESC LIMIT 1) AND (category='resource') AND (resource_type='organization' OR resource_type='project' OR resource_type='folder' OR resource_type='appengine_app' OR resource_type='kubernetes_cluster' OR resource_type='cloudsqlinstance'); +``` + +Other useful queries: + +__Get id of latest inventory scan (timestamp):__ +``` +SELECT id FROM inventory_index ORDER BY completed_at_datetime DESC LIMIT 1; +``` + +__Check schema of gcp_inventory table (in case schema changes, and query needs to be updated):__ +``` +Describe forseti_security.gcp_inventory; +``` + +## Examples + +Fully functional example available [here](https://storage.googleapis.com/strike3-gcp-arch-viz/gcp-arch-viz.html). Looks like this: + +![gcp-arch-viz animation](https://storage.googleapis.com/strike3-gcp-arch-viz/gcp-arch-viz.gif) \ No newline at end of file diff --git a/tools/gcp-arch-viz/gcp-arch-viz.gif b/tools/gcp-arch-viz/gcp-arch-viz.gif new file mode 100644 index 0000000000..d2b6af1e0f Binary files /dev/null and b/tools/gcp-arch-viz/gcp-arch-viz.gif differ diff --git a/tools/gcp-arch-viz/gcp-arch-viz.html b/tools/gcp-arch-viz/gcp-arch-viz.html new file mode 100644 index 0000000000..454e7a71a0 --- /dev/null +++ b/tools/gcp-arch-viz/gcp-arch-viz.html @@ -0,0 +1,294 @@ + + + + + +
+ + + \ No newline at end of file diff --git a/tools/gcp-arch-viz/gcp-data.csv b/tools/gcp-arch-viz/gcp-data.csv new file mode 100644 index 0000000000..e53f419c60 --- /dev/null +++ b/tools/gcp-arch-viz/gcp-data.csv @@ -0,0 +1,18 @@ +1,"organization","resource","180703686097",,"strike3software.com","organizations/180703686097" +2,"folder","resource","1037858470618",1,"vendors","folders/1037858470618" +3,"folder","resource","447738853093",1,"products","folders/447738853093" +4,"folder","resource","259115136721",1,"demo","folders/259115136721" +5,"folder","resource","776007621624",1,"migrate","folders/776007621624" +6,"folder","resource","93375575187",1,"experimental","folders/93375575187" +7,"folder","resource","830051761309",1,"admin","folders/830051761309" +8,"folder","resource","98635841550",1,"acquisitions","folders/98635841550" +9,"folder","resource","898988358812",6,"mikez","folders/898988358812" +10,"folder","resource","45281402264",9,"forseti","folders/45281402264" +35,"project","resource","mikez-test-project",5,"","mikez-test-project" +50,"project","resource","strike3-helloworld",4,"","strike3-helloworld" +55,"project","resource","strike3-billing-project",5,"","strike3-billing-project" +60,"project","resource","mikez-strike3project-test",5,"","mikez-strike3project-test" +73,"project","resource","strike3-forseti",10,"","strike3-forseti" +252,"cloudsqlinstance","resource","forseti-server-db-ff5789a",73,"","forseti-server-db-ff5789a" +354,"appengine_app","resource","11804977894556867746",50,"","apps/strike3-helloworld" +361,"appengine_app","resource","1330910310474806251",73,"","apps/strike3-forseti" \ No newline at end of file