Compressed Data for Easier Use

[falknerdominik]

Hi!

Thank you for all your hard work on this project. Over the weekend, I had some time to experiment with the data and decided to compress it using gunzip, reducing the size from 150MB to approximately 15MB. I've made the compressed data available in this repository (As the 'compressed' versions you use at the webpage are not documented). This is mainly helpful if you need the whole dump (e.g. maybe for analysis and to minimize network traffic):

https://github.com/falknerdominik/heisse-preise-data/releases/

I hope this is okay with you – if not, I'm happy to remove it. Feel free to integrate it into a repository within this organization if you find it useful.

Best regards

[badlogic]

Oh, it's super OK with me :) The other compression format can be reversed by looking at the code. it's not terribly complex. You could just rip the function that can uncompress it.

…

On Wed, Jan 8, 2025, 14:43 Dominik Falkner ***@***.***> wrote: Hi! Thank you for all your hard work on this project. Over the weekend, I had some time to experiment with the data and decided to compress it using gunzip, reducing the size from 150MB to approximately 15MB. I've made the compressed data available in this repository (As the 'compressed' versions you use at the webpage are not documented). This is mainly helpful if you need the whole dump (e.g. maybe for analysis and to minimize network traffic): https://github.com/falknerdominik/heisse-preise-data/releases/ I hope this is okay with you – if not, I'm happy to remove it. Feel free to integrate it into a repository within this organization if you find it useful. Best regards — Reply to this email directly, view it on GitHub <#177>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAD5QBET4EDLE7HRAKYOGRL2JUTO7AVCNFSM6AAAAABUZ6UFGOVHI2DSMVQWIX3LMV43ASLTON2WKOZSG43TKNBTGEYTINI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[falknerdominik]

Perfect 👍
FYI: The data is automatically compressed daily at midnight. Hope this tracks your export. Additionally, I think i will provide a data package in python which provides the full dump (cached locally, based upon the data repository).

I will take a look at this on the weekend.

EDIT:

For those interested, I’ve developed a straightforward, dependency-free data package for python, which you can check out here:

https://pypi.org/project/heisse-preise-data

(Examples can be found in the README.md).

That’s all for now, as downloading compressed files for individual stores isn’t essential for data analysis.

Feel free to close this issue if necessary. Thank you for your input!