v1.0.1
This release includes the following changes:
- Added the new feature which adds
failed_records_query
field to thedq_summary
andtarget table
. The query will give the records from source table that failed the data quality check. - Added the feature to specify
reference_columns
in input configs and include them from source table tofailed_records_query
indq_summary
andtarget
table. This will help in getting the required information from source table by joining it on referenced columns. The reference columns will mostly include unique identifier fields from source table. - Bug-Fixes for incremental data validation.
What's Changed
- added get high watermark fields and current execution timestamp to co… by @AmandeepSinghCS in #173
- Update composer docs by @AmandeepSinghCS in #176
- Save failed records query by @AmandeepSinghCS in #177
- fix to make reference columns optional by @AmandeepSinghCS in #178
- Update custom sql rule failed records logic by @AmandeepSinghCS in #179
- Save failed records query per rule by @AmandeepSinghCS in #181
- updated failed records query field names. by @AmandeepSinghCS in #182
- Add invocation id to failed records query by @AmandeepSinghCS in #183
Full Changelog: v1.0.0...v1.0.1