Skip to content

Commit

Permalink
nit
Browse files Browse the repository at this point in the history
  • Loading branch information
ddeleo committed Aug 15, 2023
1 parent 1733839 commit 9c14cc4
Showing 1 changed file with 10 additions and 24 deletions.
34 changes: 10 additions & 24 deletions scripts/optimization/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,29 +4,15 @@ This folder contains scripts that (when executed) create several tables within a
dataset named, `optimization_workshop`. These tables are populated with information to help you
optimize your BigQuery tables, views, and queries.

The scripts are broken down by categories as
shown below:

* Project Analysis
* [Daily project metrics](#daily-project-metrics)
* Table Analysis
* [BigQuery Clustering/Partitioning Recommender Tool](#bigquery-clusteringpartitioning-recommender-tool)
* [Tables with high slot consumption](#tables-with-high-slot-consumption)
* [Tables without partitioning or clustering](#tables-without-partitioning-or-clustering)
* [Frequently read tables without partitioning or clustering](#frequently-read-tables-without-partitioning-or-clustering)
* [Tables receiving high quantity of daily DML statements](#tables-receiving-high-quantity-of-daily-dml-statements)
* Query Analysis
* [Queries grouped by hash](#queries-grouped-by-hash)
* [Queries grouped by script](#queries-grouped-by-script)
* [Queries with performance insights](#queries-with-performance-insights)
The scripts are broken down by categories in the following sections.

---

# Project Analysis
Project level analysis enables us to understand key metrics such as slot_time, bytes_scanned, bytes_shuffled and bytes_spilled on a daily basis within a project. The metrics are examined as averages, medians and p80s. This enables us to understand at a high level what jobs within a project consume 80% of the time and 50% of the time daily.

<details>
<summary>Daily project metrics</summary>
<summary>## Daily project metrics</summary>

## Daily project metrics

Expand All @@ -49,7 +35,7 @@ The [daily_project_analysis.sql](daily_project_analysis.sql) script creates a ta
# Table Analysis

<details>
<summary>BigQuery Clustering/Partitioning Recommender Tool</summary>
<summary>## BigQuery Clustering/Partitioning Recommender Tool</summary>

## BigQuery Clustering/Partitioning Recommender Tool

Expand Down Expand Up @@ -86,7 +72,7 @@ resource "google_project_service" "recommender_service" {
</details>
<details>
<summary>Tables with high slot consumption</summary>
<summary>## Tables with high slot consumption</summary>
## Tables with high slot consumption
Expand Down Expand Up @@ -117,7 +103,7 @@ The [table_read_patterns.sql](table_read_patterns.sql) script creates a table ca
</details>
<details>
<summary>Tables without partitioning or clustering</summary>
<summary>## Tables without partitioning or clustering</summary>
## Tables without partitioning or clustering
Expand All @@ -141,7 +127,7 @@ that contains a list of tables which meet any of the following conditions:
</details>
<details>
<summary>Frequently read tables without partitioning or clustering</summary>
<summary>## Frequently read tables without partitioning or clustering</summary>
## Frequently read tables without partitioning or clustering
Expand All @@ -167,7 +153,7 @@ that contains a list of the most frequently read tables which meet any of the fo
</details>
<details>
<summary>Tables receiving high quantity of daily DML statements</summary>
<summary>## Tables receiving high quantity of daily DML statements</summary>
## Tables receiving high quantity of daily DML statements
Expand Down Expand Up @@ -195,7 +181,7 @@ The [frequent_daily_table_dml.sql](frequent_daily_table_dml.sql) script creates
# Query Analysis
<details>
<summary>Queries grouped by hash</summary>
<summary>## Queries grouped by hash</summary>
## Queries grouped by hash
Expand Down Expand Up @@ -236,7 +222,7 @@ SELECT * FROM my_table WHERE date = '2020-01-03';
</details>
<details>
<summary>Queries grouped by script</summary>
<summary>## Queries grouped by script</summary>
## Queries grouped by script
Expand Down Expand Up @@ -282,7 +268,7 @@ LIMIT 100
</details>

<details>
<summary>Queries with performance insights</summary>
<summary>## Queries with performance insights</summary>

## Queries with performance insights

Expand Down

0 comments on commit 9c14cc4

Please sign in to comment.