You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
By default in Spark parquet does not write out bloom filters. CUDF is in the process of adding in support for using bloom filters when doing predicate push down. rapidsai/cudf#17289
I don't expect that to impact us because we don't use the CUDF predicate push down yet. But we probably want some tests to at least verify that we are doing the right thing on reads. This is especially true for combining readers. We need to make sure that if there were bloom filter references, that we either copied/updated them or deleted them from the footers.
I think this becomes more important if we do start to try and use CUDF for predicate push down, which we have plans to try and do.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
By default in Spark parquet does not write out bloom filters. CUDF is in the process of adding in support for using bloom filters when doing predicate push down. rapidsai/cudf#17289
I don't expect that to impact us because we don't use the CUDF predicate push down yet. But we probably want some tests to at least verify that we are doing the right thing on reads. This is especially true for combining readers. We need to make sure that if there were bloom filter references, that we either copied/updated them or deleted them from the footers.
I think this becomes more important if we do start to try and use CUDF for predicate push down, which we have plans to try and do.
The text was updated successfully, but these errors were encountered: