results download for the whole project

[sfluo]

There is a way to download results of individual tasks from task browse.
But is there a way to download all results of one project? Should I open a issue for this?

[teleyinex]

Yes, there's. I think we might have not added the results.zip into the default theme. Can you go to the tasks section -> Export and see if there, you can see a section for Results? If there's not such a thing, then we have a bug. Can you check that?

In the meantime, you can download the results by copying the same url of the download of tasks or taskruns and change the name to results:

https://server/project/shortname/tasks/export?type=result&format=csv

You can get the zip file for CSV and JSON.

[sfluo]

I checked. There are export tasks and export task runs. Those are all meta data about tasks, no results included. So, that is a bug.

Thanks for the trick tip:)

Update: I did get a file downloaded using the URL however, the data is not correct. The info field is null, which is supposed to be the result for the task. I believe this is a bug.

Also, the result does not come with the original entry but a task id. It is not that user friendly.

[teleyinex]

Hi,

Regarding the results: results are generated empty. That's why you get the info completely empty unless you are using the webhooks solution to update it.

We don't include the original entry in the ZIP files because we don't want to make them really large. Also, all the ZIP files come with their IDs, so you can open a spreadsheet, and load task, task_run, and results on different sheets and link them with your software.

Also, when we do data analysis we use the PYBOSSA api which allows us to get all the results (including the related data). Please, check the following docs:

• https://docs.pybossa.com/api/crud/#related-data
• https://docs.pybossa.com/installation/configuration/#specify-a-new-root-key-instead-of-info-for-csv-exporter

[teleyinex]

Refs

[sfluo]

​Thanks Daniel. ​see below

On Fri, Mar 2, 2018 at 7:37 AM, Daniel Lombraña González < ***@***.***> wrote: Hi, Regarding the results: results are generated empty. That's why you get the info completely empty unless you are using the webhooks solution to update it.

​ users have to setup their web hook in order to ​download results? Will it be good to have a button, saying, [download project results]? We don't include the original entry in the ZIP files because we don't want

to make them really large. Also, all the ZIP files come with their IDs, so you can open a spreadsheet, and load task, task_run, and results on different sheets and link them with your software.

​Make sense!!​ Will read the docs

…

Also, when we do data analysis we use the PYBOSSA api which allows us to get all the results (including the related data). Please, check the following docs: - https://docs.pybossa.com/api/crud/#related-data - https://docs.pybossa.com/installation/configuration/# specify-a-new-root-key-instead-of-info-for-csv-exporter <https://docs.pybossa.com/installation/configuration/#specify-a-new-root-key-instead-of-info-for-csv-exporter> — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1807 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACiYoQ0xvxlkeiX6YUCHI5b0lCEFqPjyks5taUr9gaJpZM4SXyBq> .

[teleyinex]

Regarding your first issue: it's a bug on our side and will fix it. You will be able to download the zip file, but the info field will be empty :-)

[chrisranderson]

I think it would be really useful to put a link somewhere that let you download your CSV with results in it.

I think it would also be good to rename the "export tasks" to "export results". Tasks are what are assigned to users - the admins create them; why would they want to "export" a task? What they want are results.

My zip file is empty after I download it. Any ideas why?

[teleyinex]

We have that ZIP file as well for task, task runs and results. Go to the export Tasks section.

Regarding export tasks: we embrace open science, so anyone should be able to download the tasks to review them if they want. Also, as PYBOSSA import CSV and JSON files, you can re-use data to re-import it to another project and build new projects.

Regarding empty ZIP files: you need to be sure that your server's background jobs are running, otherwise, it will not work.

[chrisranderson]

For some reason, none of the export links work. They just add a # to the URL and scroll to the top of the page.

[teleyinex]

It was an issue, that I think we fixed not so long ago. Can you check if you are running the latest pybossa-default-theme?

[chrisranderson]

@teleyinex I just pulled and restarted PyBossa, but the same thing happens.

"EXPORT" just takes me to the top of the page. I would expect results from labeling to be downloaded.

[chrisranderson]

There are no event listeners on "EXPORT", and the href is "#".

[teleyinex]

Can you try the following? Type this URL in your browser and let us know if you can download a zip file:

https://YOURSERVER/project/PROJECT_SHORT_NAME/tasks/export?type=result&format=csv

That should work.

[chrisranderson]

It does download a zip, but for some reason it's empty. I'll have to check to see if I'm saving things correctly. Here's my code:

pybossa.taskLoaded(function(task, deferred){
    this.set_image(task.info.image_url)
    this._objects = []
    this.add_turk_input_to_canvas(prepare_input_csv(task.info.data))

    submit_button.addEventListener('click', function (event) {
      data = {}
      document.querySelectorAll('input#output, textarea#comments').forEach(function (node) {
        data[node.name] = node.value
      })
      pybossa.saveTask(task.id, data).done(function() {
        deferred.resolve()
        window.location.reload(false)
      })
    }.bind(this))
  }.bind(this));

  pybossa.presentTask(function(task, deferred){
    console.log(2, task)
  }.bind(this));

  pybossa.run(window.PYBOSSA_PROJECT);
}

[chrisranderson]

The CSVs are empty, but it looks like things are getting saved somewhere. I can hit http://172.23.3.175:5000/project/PROJECT_NAME/209/results.json and see what I passed to saveTask.

[sfluo]

Currently, if you request individual task, you will have the result. But if you request the whole project, the answer field is empty in zip file. I have confirmed.

…

--- There is no such a thing called randomness.

On Wed, May 2, 2018 at 3:11 PM, Chris Anderson ***@***.***> wrote: The CSVs are empty, but it looks like things are getting saved somewhere. I can hit http://172.23.3.175:5000/project/PROJECT_NAME/209/results.json and see what I passed to saveTask. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#1807 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ACiYofWgLcUgWOAT71JUFzUhyT6EwVXfks5tuhMDgaJpZM4SXyBq> .

[teleyinex]

Yes, that behavior is expected, because the ZIP files are generated in the background. Can you check if you are running the background jobs?

[chrisranderson]

@teleyinex It's expected that when I request a ZIP with results, it's empty? Can you provide a step-by-step guide of how to get data out of pybossa? Do I need to manually run some kind of background job? Here's what my jobs look like:

The one job is one that failed a few months ago.

1. call saveTask with labeled data
2. ?
3. Download the CSV at project/PROJECT_SHORT_NAME/tasks/export?type=result&format=csv

[teleyinex]

Basically, you need to run some workers. Your workers section is completely empty, so that's why the ZIP files are not being generated. All the information regarding the background jobs is in the installation guide. Here you have the specific section about it.

[teleyinex]

This was an issue with the default theme, now fixed.