Skip to content

Commit

Permalink
Update data/processors/export.py
Browse files Browse the repository at this point in the history
  • Loading branch information
CommanderStorm authored Oct 27, 2024
1 parent 0a8b34b commit c03a84b
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion data/processors/export.py
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ def export_for_api(data: dict) -> None:
_make_sure_is_safe(export_data)
with (OUTPUT_DIR_PATH / "api_data.json").open("w", encoding="utf-8") as file:
json.dump(export_data, file, cls=EnhancedJSONEncoder)
df = pl.read_json(OUTPUT_DIR_PATH / "api_data.json", infer_schema_length=16384)
df = pl.read_json(OUTPUT_DIR_PATH / "api_data.json", infer_schema_length=None)
df.write_parquet(OUTPUT_DIR_PATH / "api_data.parquet", use_pyarrow=True, compression_level=22)


Expand Down

0 comments on commit c03a84b

Please sign in to comment.