Skip to content

Commit

Permalink
test collapsible sections
Browse files Browse the repository at this point in the history
  • Loading branch information
ddeleo committed Aug 15, 2023
1 parent 8e3b699 commit 20ce746
Showing 1 changed file with 14 additions and 8 deletions.
22 changes: 14 additions & 8 deletions scripts/optimization/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ shown below:
# Project Analysis
Project level analysis enables us to understand key metrics such as slot_time, bytes_scanned, bytes_shuffled and bytes_spilled on a daily basis within a project. The metrics are examined as averages, medians and p80s. This enables us to understand at a high level what jobs within a project consume 80% of the time and 50% of the time daily.

<details>

<summary>Daily project metrics</summary>

## :mag: Daily project metrics

The [daily_project_analysis.sql](daily_project_analysis.sql) script creates a table called,
Expand All @@ -41,9 +45,11 @@ The [daily_project_analysis.sql](daily_project_analysis.sql) script creates a ta
LIMIT 100
```

</details>

# Table Analysis

## BigQuery Clustering/Partitioning Recommender Tool
## :mag: BigQuery Clustering/Partitioning Recommender Tool

The BigQuery partitioning and clustering recommender analyzes workflows on your BigQuery tables and offers recommendations to better optimize your workflows and query costs using either table partitioning or table clustering. The recommender uses your BigQuery's workload execution data from the past 30 days to analyze each BigQuery table for suboptimal partitioning and clustering configurations.
Expand Down Expand Up @@ -75,7 +81,7 @@ resource "google_project_service" "recommender_service" {
}
```
## Tables with high slot consumption
## :mag: Tables with high slot consumption
The [table_read_patterns.sql](table_read_patterns.sql) script creates a table called, `table_read_patterns`, and populate it with usage data to help you determine:
* Which tables (when queried) are resulting in high slot consumption.
Expand All @@ -101,7 +107,7 @@ The [table_read_patterns.sql](table_read_patterns.sql) script creates a table ca
LIMIT 100
```
## Tables without partitioning or clustering
## :mag: Tables without partitioning or clustering
The [tables_without_partitioning_or_clustering.sql](tables_without_partitioning_or_clustering.sql) script creates a table named, `tables_without_part_clust`,
that contains a list of tables which meet any of the following conditions:
Expand All @@ -120,7 +126,7 @@ that contains a list of tables which meet any of the following conditions:
LIMIT 100
```
## Frequently read tables without partitioning or clustering
## :mag: Frequently read tables without partitioning or clustering
**Note:** The [freq_read_tables_without_partitioning_or_clustering.sql](freq_read_tables_without_partitioning_or_clustering.sql) script depends on the `table_read_patterns` table so you must first run the [table_read_patterns.sql](table_read_patterns.sql) script.
Expand All @@ -141,7 +147,7 @@ that contains a list of the most frequently read tables which meet any of the fo
LIMIT 100
```
## Tables receiving high quantity of daily DML statements
## :mag: Tables receiving high quantity of daily DML statements
The [frequent_daily_table_dml.sql](frequent_daily_table_dml.sql) script creates a table named, `frequent_daily_table_dml`, that contains tables that have had more than 24 daily DML statements run against them in the past 30 days.
Expand All @@ -164,7 +170,7 @@ The [frequent_daily_table_dml.sql](frequent_daily_table_dml.sql) script creates
# Query Analysis
## Queries grouped by hash
## :mag: Queries grouped by hash
The [queries_grouped_by_hash.sql](queries_grouped_by_hash.sql) script creates a table named,
`queries_grouped_by_hash`. This table groups queries by their normalized query pattern, which ignores
Expand Down Expand Up @@ -200,7 +206,7 @@ SELECT * FROM my_table WHERE date = '2020-01-03';
LIMIT 100
```
## Queries grouped by script
## :mag: Queries grouped by script
The [queries_grouped_by_script.sql](queries_grouped_by_script.sql) script creates a table named,
`queries_grouped_by_script`. This table groups queries by their parent job id which means that any queries that
Expand Down Expand Up @@ -241,7 +247,7 @@ ORDER BY total_slots DESC
LIMIT 100
```

## Queries with performance insights
## :mag: Queries with performance insights

The [query_performance_insights.sql](query_performance_insights.sql) script creates a table named, `query_performance_insights` retrieves all queries that have had performance insights
generated for them in the past 30 days.
Expand Down

0 comments on commit 20ce746

Please sign in to comment.