Service: Harvard Dataverse Repository Large Data Services

[cmbz]

Overview

Finalize and launch the Harvard Dataverse Repository Large Data Services program

Related

• Proposal: Large Data Support #106
• Globus integration: further improvements/refactoring dataverse#10623
• Project: Pilot Large Data Support Service #178

Investigations

• Use a model like this for large data storage input https://finder.research.cornell.edu/

Resources

• Project: NIH GREI - Generalist Repository Ecosystem Initiative #124
• GREI 5: HDV Task - Large Data Support #176
• Google Drive Folder

[cmbz]

Status: June 2024

• Drafts of internal and external service descriptions were created (see Large Data Services folder)
  • Harvard Dataverse Repository Large Data Services
  • Harvard Dataverse Repository Large Data Services: Internal
  • Budget Planner

[cmbz]

Status: August 2024

• Pull request Improved handling of Globus uploads (experimental async framework) dataverse#10781 is in QA and will be available in the 6.4 release
• In July, work continued on backend labor estimates for specific service components. Currently under review by IQSS leadership.
  • Additional updates made based on IQSS leadership feedback
• @dliburd is creating new Storage Chooser figures

[cmbz]

Status: September 2024

• Continued revisions of service summary and website graphics (e.g., storage chooser flowchart)
• Also see updates here:
  • GREI 5: HDV Task - Large Data Support #176 (comment)
  • Project: Pilot Large Data Support Service #178 (comment)

[cmbz]

Status: October 2024

• See related updates here: Project: Pilot Large Data Support Service #178 (comment)

[cmbz]

Status: November 2024

• Vivli met with DV technical staff to discuss large data support infrastructure and workflows
• Finalized services description documentation
• Finalized intake form
• Began developing services website

[cmbz]

Status: December 2024

• The Harvard Dataverse Support website was updated with the large data documentation and Qualtrics intake form. A webinar to introduce the services has been scheduled for 2025.
• A new project from a Harvard Lab accepted our offer for support.
• A new GitHub repository was created to support tracking of large data projects: https://github.com/IQSS/hdv-large-data-services
• A new GitHub project was created to support large data tracking: https://github.com/orgs/IQSS/projects/32
• @sbarbosadataverse and @cmbz gave a presentation about the new service at the 2024/12/12 IQSS All Staff Meeting.