Sort output of data

[phalt]

Related to this issue is the fact that ditto produces data but when it pushes it up - the ordering isn't maintained and the diff is very large and hard to read - with most of it just being existing content shuffled around.

Ideally we'd like to provide a --sort option to the ditto tool that orders the output of data somehow.

[Naramsim]

Yeah, we should use ordered dicts instead of the standard dicts. Ideally implementing an order logic

[phalt]

Hmmm now you mention it, what version of python is it running on? Dicts should be ordered by default since like, 3.6 or something?

[Naramsim]

We are on the python installed here:

https://github.com/PokeAPI/api-data/blob/master/updater%2FDockerfile#L4

Which I think might be the latest.

But not 100% sure. I'll check when I have time

[Naramsim]

We had this discussion already in the past via email or GH issue, but we couldn't determine why they would get shuffled anyways

[phalt]

It should definitely be beyond the version we need.

I do remember the discussion, and reviewing a few of the diffs today made me realise it is still a pain point.

I opened the issue with a bit of hope that someone in the community could pick it up.

[Naramsim]

Sometimes the data gets scrambled even if there's no modification to it. This isn't the best also for caching purposes.

---

We could also look into tools that diff JSON files, such as https://github.com/josephburnett/jd

[Naramsim]

interestingly, the issue seems resolved. I updated Python and now it's two build that everything works as expected:

last one: https://app.circleci.com/pipelines/github/PokeAPI/pokeapi/1470/workflows/af8e58bc-8622-4872-982a-acadbfddc559/jobs/3285 No CSVs were modified and the regeneration of the JSONs didn't bring any updates.

previous to last one: https://github.com/PokeAPI/api-data/pull/271/files Only the changed values were effectively updated

[phalt]

Which version of Python did you update to? This is a cool side effect if that solved it.

[Naramsim]

Apparently from 3.10.9 to 3.12.8. And with 3.13 pokeapi doesn't work!

[phalt]

No idea what could've caused that improvement but happy to take it.