Release v0.14.2 #345
Merged
Release v0.14.2 #345
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
* Fix compariosn report when one column is all NAs
With situations when one of the column in the comparison was
all NA's then this would break the reporting. For some reason
when the matching (boolean) of the actual and expected columns happened when there was a categorical value compared to a NA, the result was NA, rather than False, as it would happen for the other elements in the cols compared.
This has now been addressed at the intersect rows level which doesn't seem to break the reporting anymore.
* Fix column_equal to work with StringArrays with pd.NA values not returning booleans
* Add test for fn column_equal to work with StringArrays with pd.NA
Add test for fn column_equal to work with StringArrays containing pd.NA values not returning booleans when compared with other df's with rows of StringArrays
* Fix linter error
Printing out the report would've been useful for this test, but looks like it makes the linter fail the build. This has now been fixed.
* Fix column_equal to work with StringArrays with pd.NA values
Fix column_equal to work with StringArrays with pd.NA values not returning booleans, and update formatting to match the linter expectation
* Add test for fn column_equal to work with StringArrays with pd.NA
Add test for fn column_equal to work with StringArrays containing pd.NA values not returning booleans when compared with other df's with rows of StringArrays, and format test to match the linter.
* adding snowflake/snowpark compare * undo change * add partial case sensitive support * update doc * pr comments * remainder PR comments * update readme * mocking abs, trim * remove local testing, fix text * ignore snowflake tests * catch snowpark import in test config * python 3.12 actions without snowflake * clean * fix * catch snowflake imports, fix snowflake type annotations * conditional install * conditional install * conditional install * fix conditional
* changes * doc fix * version bump
fdosani
requested review from
ak-gupta,
jdawang and
gladysteh99
as code owners
ak-gupta
approved these changes
Oct 30, 2024
| @@ -43,6 +43,24 @@ Run ``python -m pytest`` to run all unittests defined in the subfolder | |||
| `pytest-runner <https://pypi.python.org/pypi/pytest-runner>`_. | |||
|
|
|||
|
|
|||
| Snowflake testing | |||
There was a problem hiding this comment.
Thank you for updating the developer docs -- we aren't always consistent with this so it's very helpful.
| @@ -56,16 +56,19 @@ python-tag = "py3" | |||
| [project.optional-dependencies] | |||
| duckdb = ["fugue[duckdb]"] | |||
| spark = ["pyspark[connect]>=3.1.1; python_version < \"3.11\"", "pyspark[connect]>=3.4; python_version >= \"3.11\""] | |||
| snowflake = ["snowflake-connector-python", "snowflake-snowpark-python"] | |||
There was a problem hiding this comment.
[NON-BLOCKING]: how does snowflake-snowpark-python affect the dependency tree? Does it add a bunch of packages to the snowflake extra?
There was a problem hiding this comment.
I think it most depends on the snowflake-connector-python if you look here: https://github.com/snowflakedb/snowpark-python/blob/main/setup.py#L16
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Release v0.14.2
New features
SnowflakeComparesupport is now available!Changes
SnowflakeCompare(Snowflake/Snowpark compare ( Fix comparison report when one column is all NAs #343)