Release 1.6.1, documentation regenerated.

.run/dqo run.run.xml

@@ -5,7 +5,7 @@
       <option name="region" />
       <option name="useCurrentConnection" value="false" />
     </extension>
-    <option name="JAR_PATH" value="$PROJECT_DIR$/dqops/target/dqo-dqops-1.6.0.jar" />
+    <option name="JAR_PATH" value="$PROJECT_DIR$/dqops/target/dqo-dqops-1.6.1.jar" />
     <option name="VM_PARAMETERS" value="-XX:MaxRAMPercentage=60.0 --add-opens java.base/java.nio=ALL-UNNAMED --add-opens java.base/java.util.concurrent=ALL-UNNAMED" />
     <option name="PROGRAM_PARAMETERS" value="--server.port=8888" />
     <option name="WORKING_DIRECTORY" value="C:\dev\dqoado" />

CHANGELOG.md

@@ -1,4 +1,5 @@
 # 1.6.1
-* Small fixes in the data quality check editor - styling fixes
-* Parallel invocation of notification events
-* The calculation logic for counting incidents on the global incident screen modified to use dates correctly
+* Incident notification supports emails
+* Small bug fixes in the check editor
+* Additional check to detect empty columns
+* Copying data quality check patterns

VERSION

@@ -1 +1 @@
-1.6.0
+1.6.1

distribution/pom.xml

@@ -11,7 +11,7 @@
 
 	<groupId>com.dqops</groupId>
 	<artifactId>dqo-distribution</artifactId>
-    <version>1.6.0</version>  <!-- DQOps Version, do not touch (changed automatically) -->
+    <version>1.6.1</version>  <!-- DQOps Version, do not touch (changed automatically) -->
 	<name>dqo-distribution</name>
 	<description>DQOps Data Quality Operations Center final assembly</description>
 	<packaging>pom</packaging>

distribution/python/dqops/version.py

@@ -15,8 +15,8 @@
 # limit
 
 # WARNING: the next two lines with the version numbers (VERSION =, PIP_VERSION =) should not be modified manually. They are changed by a maven profile at compile time.
-VERSION = "1.6.0"
-PIP_VERSION = "1.6.0"
+VERSION = "1.6.1"
+PIP_VERSION = "1.6.1"
 GITHUB_RELEASE = "v" + VERSION + ""
 JAVA_VERSION = "17"
 

docs/categories-of-data-quality-checks/how-to-detect-empty-or-incomplete-columns-with-nulls.md

@@ -319,8 +319,8 @@ The status identifies the highest severity issue by color.
 |[*nulls_count*](../checks/column/nulls/nulls-count.md)|Maximum count of rows containing null values (incomplete column)|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects incomplete columns that contain any *null* values. Counts the number of rows having a null value. Raises a data quality issue when the count of null values is above a *max_count* threshold.|:material-check-bold:|
 |[*nulls_percent*](../checks/column/nulls/nulls-percent.md)|Maximum percentage of rows containing null values (incomplete column)|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects incomplete columns that contain any *null* values. Measures the percentage of rows having a null value. Raises a data quality issue when the percentage of null values is above a *max_percent* threshold. Configure this check to accept a given percentage of null values by setting the *max_percent* parameter.|:material-check-bold:|
 |[*nulls_percent_anomaly*](../checks/column/nulls/nulls-percent-anomaly.md)|Abnormal change in percentage of null values. Measured as a percentile of anomalous measures.|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects day-to-day anomalies in the percentage of *null* values. Measures the percentage of rows having a *null* value. Raises a data quality issue when the rate of null values increases or decreases too much.|:material-check-bold:|
-|[*not_nulls_count*](../checks/column/nulls/not-nulls-count.md)|Minimum count of rows containing non-null values (find empty column)|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column.|:material-check-bold:|
-|[*not_nulls_percent*](../checks/column/nulls/not-nulls-percent.md)|Minimum percentage of rows containing non-null values (find empty column)|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below *min_percentage*. The default value of the *min_percentage* parameter is 100.0, but DQOps supports setting a lower value to accept some nulls.|:material-check-bold:|
+|[*not_nulls_count*](../checks/column/nulls/not-nulls-count.md)|Minimum count of rows containing non-null values|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column.|:material-check-bold:|
+|[*not_nulls_percent*](../checks/column/nulls/not-nulls-percent.md)|Minimum percentage of rows containing non-null values|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below *min_percentage*. The default value of the *min_percentage* parameter is 100.0, but DQOps supports setting a lower value to accept some nulls.|:material-check-bold:|
 |[*empty_column_found*](../checks/column/nulls/empty-column-found.md)|Find an empty column|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects empty columns that contain only *null* values. Counts the number of rows that have non-null values. Raises a data quality issue when the count of non-null values is below *min_count*. The default value of the *min_count* parameter is 1, but DQOps supports setting a higher number to assert that a column has at least that many non-null values.|:material-check-bold:|
 |[*nulls_percent_change*](../checks/column/nulls/nulls-percent-change.md)|Maximum percentage of change in the count of null values|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects relative increases or decreases in the percentage of null values since the last measured percentage. Measures the percentage of null values for each day. Raises a data quality issue when the change in the percentage of null values is above *max_percent* of the previous percentage.| |
 |[*nulls_percent_change_1_day*](../checks/column/nulls/nulls-percent-change-1-day.md)|Maximum percentage of change in the count of null values vs 1 day ago|[Completeness](../dqo-concepts/data-quality-dimensions.md#data-completeness)|Detects relative increases or decreases in the percentage of null values since the previous day. Measures the percentage of null values for each day. Raises a data quality issue when the change in the percentage of null values is above *max_percent* of the previous percentage.| |

docs/checks/column/nulls/index.md

@@ -62,11 +62,11 @@ Verifies that a column contains a minimum number of non-null values.
 
 | Data quality check name | Friendly name | Check type | Description | Standard |
 |-------------------------|---------------|------------|-------------|----------|
-|[<span class="no-wrap-code">`profile_not_nulls_count`</span>](./not-nulls-count.md#profile-not-nulls-count)|Minimum count of rows containing non-null values (find empty column)|[profiling](../../../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count.|:material-check-bold:|
-|[<span class="no-wrap-code">`daily_not_nulls_count`</span>](./not-nulls-count.md#daily-not-nulls-count)|Minimum count of rows containing non-null values (find empty column)|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores the most recent captured value for each day when the data quality check was evaluated.|:material-check-bold:|
-|[<span class="no-wrap-code">`monthly_not_nulls_count`</span>](./not-nulls-count.md#monthly-not-nulls-count)|Minimum count of rows containing non-null values (find empty column)|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores the most recent check result for each month when the data quality check was evaluated.|:material-check-bold:|
-|[<span class="no-wrap-code">`daily_partition_not_nulls_count`</span>](./not-nulls-count.md#daily-partition-not-nulls-count)|Minimum count of rows containing non-null values (find empty column)|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores a separate data quality check result for each daily partition.|:material-check-bold:|
-|[<span class="no-wrap-code">`monthly_partition_not_nulls_count`</span>](./not-nulls-count.md#monthly-partition-not-nulls-count)|Minimum count of rows containing non-null values (find empty column)|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores a separate data quality check result for each monthly partition.|:material-check-bold:|
+|[<span class="no-wrap-code">`profile_not_nulls_count`</span>](./not-nulls-count.md#profile-not-nulls-count)|Minimum count of rows containing non-null values|[profiling](../../../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count.|:material-check-bold:|
+|[<span class="no-wrap-code">`daily_not_nulls_count`</span>](./not-nulls-count.md#daily-not-nulls-count)|Minimum count of rows containing non-null values|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores the most recent captured value for each day when the data quality check was evaluated.|:material-check-bold:|
+|[<span class="no-wrap-code">`monthly_not_nulls_count`</span>](./not-nulls-count.md#monthly-not-nulls-count)|Minimum count of rows containing non-null values|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores the most recent check result for each month when the data quality check was evaluated.|:material-check-bold:|
+|[<span class="no-wrap-code">`daily_partition_not_nulls_count`</span>](./not-nulls-count.md#daily-partition-not-nulls-count)|Minimum count of rows containing non-null values|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores a separate data quality check result for each daily partition.|:material-check-bold:|
+|[<span class="no-wrap-code">`monthly_partition_not_nulls_count`</span>](./not-nulls-count.md#monthly-partition-not-nulls-count)|Minimum count of rows containing non-null values|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Verifies that a column contains a minimum number of non-null values. The default value of the *min_count* parameter is 1 to detect at least one value in a monitored column. Raises a data quality issue when the count of non-null values is below min_count. Stores a separate data quality check result for each monthly partition.|:material-check-bold:|
 
 
 
@@ -78,11 +78,11 @@ Detects incomplete columns that contain too few non-null values. Measures the pe
 
 | Data quality check name | Friendly name | Check type | Description | Standard |
 |-------------------------|---------------|------------|-------------|----------|
-|[<span class="no-wrap-code">`profile_not_nulls_percent`</span>](./not-nulls-percent.md#profile-not-nulls-percent)|Minimum percentage of rows containing non-null values (find empty column)|[profiling](../../../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage.|:material-check-bold:|
-|[<span class="no-wrap-code">`daily_not_nulls_percent`</span>](./not-nulls-percent.md#daily-not-nulls-percent)|Minimum percentage of rows containing non-null values (find empty column)|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores the most recent captured value for each day when the data quality check was evaluated.|:material-check-bold:|
-|[<span class="no-wrap-code">`monthly_not_nulls_percent`</span>](./not-nulls-percent.md#monthly-not-nulls-percent)|Minimum percentage of rows containing non-null values (find empty column)|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores the most recent check result for each month when the data quality check was evaluated.|:material-check-bold:|
-|[<span class="no-wrap-code">`daily_partition_not_nulls_percent`</span>](./not-nulls-percent.md#daily-partition-not-nulls-percent)|Minimum percentage of rows containing non-null values (find empty column)|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores a separate data quality check result for each daily partition.|:material-check-bold:|
-|[<span class="no-wrap-code">`monthly_partition_not_nulls_percent`</span>](./not-nulls-percent.md#monthly-partition-not-nulls-percent)|Minimum percentage of rows containing non-null values (find empty column)|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores a separate data quality check result for each monthly partition.|:material-check-bold:|
+|[<span class="no-wrap-code">`profile_not_nulls_percent`</span>](./not-nulls-percent.md#profile-not-nulls-percent)|Minimum percentage of rows containing non-null values|[profiling](../../../dqo-concepts/definition-of-data-quality-checks/data-profiling-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage.|:material-check-bold:|
+|[<span class="no-wrap-code">`daily_not_nulls_percent`</span>](./not-nulls-percent.md#daily-not-nulls-percent)|Minimum percentage of rows containing non-null values|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores the most recent captured value for each day when the data quality check was evaluated.|:material-check-bold:|
+|[<span class="no-wrap-code">`monthly_not_nulls_percent`</span>](./not-nulls-percent.md#monthly-not-nulls-percent)|Minimum percentage of rows containing non-null values|[monitoring](../../../dqo-concepts/definition-of-data-quality-checks/data-observability-monitoring-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores the most recent check result for each month when the data quality check was evaluated.|:material-check-bold:|
+|[<span class="no-wrap-code">`daily_partition_not_nulls_percent`</span>](./not-nulls-percent.md#daily-partition-not-nulls-percent)|Minimum percentage of rows containing non-null values|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores a separate data quality check result for each daily partition.|:material-check-bold:|
+|[<span class="no-wrap-code">`monthly_partition_not_nulls_percent`</span>](./not-nulls-percent.md#monthly-partition-not-nulls-percent)|Minimum percentage of rows containing non-null values|[partitioned](../../../dqo-concepts/definition-of-data-quality-checks/partition-checks.md)|Detects incomplete columns that contain too few non-null values. Measures the percentage of rows that have non-null values. Raises a data quality issue when the percentage of non-null values is below min_percentage. Stores a separate data quality check result for each monthly partition.|:material-check-bold:|