You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Our current test suite uses two distinct methods to create checkpoints in MockTable:
The method parquet_checkpoint accepts engine data
The method json_checkpoint accepta actions to create a JSON checkpoint file.
This inconsistency causes the test cases—such as in test_create_checkpoint_stream_reads_parquet_checkpoint_batch versus test_create_checkpoint_stream_reads_json_checkpoint_batch — to look very different, which increases maintenance overhead and introduces potential discrepancies in handling different checkpoint formats.
Describe the functionality you are proposing.
Create a single method (e.g. create_checkpoint) for the MockTable struct test utility that accepts checkpoint actions as engine data. This method should internally handle the conversion for both JSON and Parquet formats, dependent on a passed parameter.
Instead of having one function that takes a batch of engine data and another that takes actions, standardize on one as the input. This change will streamline the test setup and ensure consistency across both checkpoint types.
Update tests accordingly
Additional context
No response
The text was updated successfully, but these errors were encountered:
sebastiantia
changed the title
test: unify MockTable test utility for mock checkpoint creation (JSON & Parquet)
test: unify checkpoint creation methods for json & parquet in the MockTableFeb 10, 2025
sebastiantia
changed the title
test: unify checkpoint creation methods for json & parquet in the MockTable
test: unify json & parquet checkpoint creation methods in the MockTableFeb 10, 2025
We are moving away from adding the checkpointing functionality in the mock table until a broader conversation is held as to whether or not we want to continue with the MockTable testing infrastructure
Please describe why this is necessary.
Our current test suite uses two distinct methods to create checkpoints in
MockTable
:The method
parquet_checkpoint
accepts engine dataThe method
json_checkpoint
accepta actions to create a JSON checkpoint file.This inconsistency causes the test cases—such as in
test_create_checkpoint_stream_reads_parquet_checkpoint_batch
versustest_create_checkpoint_stream_reads_json_checkpoint_batch
— to look very different, which increases maintenance overhead and introduces potential discrepancies in handling different checkpoint formats.Describe the functionality you are proposing.
Create a single method (e.g.
create_checkpoint
) for theMockTable
struct test utility that accepts checkpoint actions as engine data. This method should internally handle the conversion for both JSON and Parquet formats, dependent on a passed parameter.Instead of having one function that takes a batch of engine data and another that takes actions, standardize on one as the input. This change will streamline the test setup and ensure consistency across both checkpoint types.
Update tests accordingly
Additional context
No response
The text was updated successfully, but these errors were encountered: