Skip to content

Commit

Permalink
Documentation update for the top text values check.
Browse files Browse the repository at this point in the history
  • Loading branch information
piotrczarnas committed Jan 31, 2024
1 parent 263e149 commit f94bc9a
Show file tree
Hide file tree
Showing 7 changed files with 246 additions and 146 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -59,13 +59,52 @@ spec:

![Asserting that all expected text values are present in a column](https://dqops.com/docs/images/concepts/categories-of-data-quality-checks/all-expected-column-values-are-in-use-data-quality-check-min.png){ loading=lazy }

``` { .yaml linenums="1" hl_lines="12-15" }
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
spec:
columns:
monitoring_checks:
daily:
accepted_values:
daily_expected_text_values_in_use_count:
parameters:
expected_values:
- TRAVIS
- WILLIAMSON
- HAYS
- BASTROP
error:
max_missing: 0
```

![Detecting expected values that are missing in a column](https://dqops.com/docs/images/concepts/categories-of-data-quality-checks/text-values-found-vs-text-values-in-use-min.png){ loading=lazy }

![Top values in a column to assert in a data quality check](https://dqops.com/docs/images/concepts/categories-of-data-quality-checks/column-profiling-result-top-two-column-values-min.png){ width="619px"; loading=lazy }


![Asserting that expected values are in the top of most popular values in a column](https://dqops.com/docs/images/concepts/categories-of-data-quality-checks/text-values-in-top-most-popular-min.png){ loading=lazy }

``` { .yaml linenums="1" hl_lines="12-15" }
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
spec:
columns:
monitoring_checks:
daily:
accepted_values:
daily_expected_texts_in_top_values_count:
parameters:
expected_values:
- TRAVIS
- WILLIAMSON
top: 3
error:
max_missing: 0
```

## Detecting accepted values issues
How to detect accepted values data quality issues.

Expand Down
60 changes: 40 additions & 20 deletions docs/checks/column/accepted_values/expected-numbers-in-use-count.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,7 +101,7 @@ Please expand the section below to see the [DQOps command-line](../../../dqo-con
The sample *schema_name.table_name.dqotable.yaml* file with the check configured is shown below.


```yaml hl_lines="7-17"
```yaml hl_lines="7-19"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -115,8 +115,10 @@ spec:
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -793,7 +795,7 @@ Expand the *Configure with data grouping* section to see additional examples for
**Sample configuration with data grouping enabled (YAML)**
The sample below shows how to configure the data grouping and how it affects the generated SQL query.

```yaml hl_lines="5-15 29-34"
```yaml hl_lines="5-15 31-36"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -816,8 +818,10 @@ Expand the *Configure with data grouping* section to see additional examples for
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -1604,7 +1608,7 @@ Please expand the section below to see the [DQOps command-line](../../../dqo-con
The sample *schema_name.table_name.dqotable.yaml* file with the check configured is shown below.


```yaml hl_lines="7-18"
```yaml hl_lines="7-20"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -1619,8 +1623,10 @@ spec:
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -2297,7 +2303,7 @@ Expand the *Configure with data grouping* section to see additional examples for
**Sample configuration with data grouping enabled (YAML)**
The sample below shows how to configure the data grouping and how it affects the generated SQL query.

```yaml hl_lines="5-15 30-35"
```yaml hl_lines="5-15 32-37"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -2321,8 +2327,10 @@ Expand the *Configure with data grouping* section to see additional examples for
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -3109,7 +3117,7 @@ Please expand the section below to see the [DQOps command-line](../../../dqo-con
The sample *schema_name.table_name.dqotable.yaml* file with the check configured is shown below.


```yaml hl_lines="7-18"
```yaml hl_lines="7-20"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -3124,8 +3132,10 @@ spec:
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -3802,7 +3812,7 @@ Expand the *Configure with data grouping* section to see additional examples for
**Sample configuration with data grouping enabled (YAML)**
The sample below shows how to configure the data grouping and how it affects the generated SQL query.

```yaml hl_lines="5-15 30-35"
```yaml hl_lines="5-15 32-37"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -3826,8 +3836,10 @@ Expand the *Configure with data grouping* section to see additional examples for
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -4614,7 +4626,7 @@ Please expand the section below to see the [DQOps command-line](../../../dqo-con
The sample *schema_name.table_name.dqotable.yaml* file with the check configured is shown below.


```yaml hl_lines="12-23"
```yaml hl_lines="12-25"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -4634,8 +4646,10 @@ spec:
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -5321,7 +5335,7 @@ Expand the *Configure with data grouping* section to see additional examples for
**Sample configuration with data grouping enabled (YAML)**
The sample below shows how to configure the data grouping and how it affects the generated SQL query.

```yaml hl_lines="10-20 40-45"
```yaml hl_lines="10-20 42-47"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand Down Expand Up @@ -5350,8 +5364,10 @@ Expand the *Configure with data grouping* section to see additional examples for
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -6141,7 +6157,7 @@ Please expand the section below to see the [DQOps command-line](../../../dqo-con
The sample *schema_name.table_name.dqotable.yaml* file with the check configured is shown below.


```yaml hl_lines="12-23"
```yaml hl_lines="12-25"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand All @@ -6161,8 +6177,10 @@ spec:
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down Expand Up @@ -6848,7 +6866,7 @@ Expand the *Configure with data grouping* section to see additional examples for
**Sample configuration with data grouping enabled (YAML)**
The sample below shows how to configure the data grouping and how it affects the generated SQL query.

```yaml hl_lines="10-20 40-45"
```yaml hl_lines="10-20 42-47"
# yaml-language-server: $schema=https://cloud.dqops.com/dqo-yaml-schema/TableYaml-schema.json
apiVersion: dqo/v1
kind: table
Expand Down Expand Up @@ -6877,8 +6895,10 @@ Expand the *Configure with data grouping* section to see additional examples for
expected_values:
- 2
- 3
error:
warning:
max_missing: 0
error:
max_missing: 1
fatal:
max_missing: 2
labels:
Expand Down
Loading

0 comments on commit f94bc9a

Please sign in to comment.