getData compression option(s)

[gikeymarcia]

getData should add the ability to use zip compression on the data.csv file being sent. This compression considerably reduces file size to about ~5% of the original file size.

Once compression is added it would be nice to have an archive option that zips the entire root folder (and all subfolders) then sends that file. I'm not sure if this should be part of get data or it's own Archive tool.

[gikeymarcia]

http://davidwalsh.name/create-zip-php

[TysonKerr]

So, would the strategy be to create the file locally, zip it, and then send out the zip? Once that is done, should I delete the unzipped file?

Once the new file structure is in place, one cool thing might be the ability to export just one of the experiments in there. So for example, if I have a lineup of 3 unrelated experiments, but then I want to put one experiment up so that anyone can download the data/program, without seeing the 2 unrelated experiments, I can just export the selected experiment to a different URL and let people run amok. Or maybe the global config file should just be able to configure which data sets and programs are downloadable/viewable?

[gikeymarcia]

Ideally this tool would create a file named Data-[ExpName]-[Date].csv and put it in /Data/ root. Each time getData runs it will delete the old .csv in /Data/ and replace it with the new file. Once the file is created the code zips and sends that .zip then deletes the zip.

Archive tool would compress everything from $_root and that would necessarily include the most recent Data-[ExpName]-[Date].csv in /Data/.

[TysonKerr]

I think if the archive tool is grabbing stuff outside of Data/, it should be its own tool. Perhaps it would have a version that copies the experiment with/without the currently existing data?