Skip to content

Commit

Permalink
fix: GH-68 new "Data Depot" hierarchy (#76)
Browse files Browse the repository at this point in the history
* fix: GH-68 new "Data Depot" hierarchy

* fix: GH-68 split "Data Depot" overview
  • Loading branch information
wesleyboar authored Aug 21, 2024
1 parent 204f7d8 commit f23c04b
Show file tree
Hide file tree
Showing 20 changed files with 82 additions and 147 deletions.
29 changes: 0 additions & 29 deletions user-guide/docs/curating.md

This file was deleted.

2 changes: 2 additions & 0 deletions user-guide/docs/curating/bestpractices.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
## Best Practices

### Data Collections Development

#### Accepted Data { #accepteddata }
Expand Down
2 changes: 2 additions & 0 deletions user-guide/docs/curating/faq.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Frequently Asked Questions

### Selecting Files & Data { #selecting }

**Q: What are the best file formats for data publications?**
Expand Down
2 changes: 2 additions & 0 deletions user-guide/docs/curating/guides.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
## Guides

Below are step-by-step guides on how to create projects in the Data Depot, and curate and publish work/data across DesignSafe. We offer the following project types when publishing: Experimental, Simulation, Hybrid Simulation, Field Research, and Other. More information on Data Depot policies, project types, and curation/publication can be found at:

* <a href="#policies">Data Depot Repository (DDR) Policies</a></li>
Expand Down
11 changes: 11 additions & 0 deletions user-guide/docs/curating/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Curating & Publishing Projects

{% include-markdown '../redirect.md' %}

- [Curation & Publication Guides](guides.md)
- [Best Practices](bestpractices.md)
- [Data Depot/Curation Office Hours](https://www.designsafe-ci.org/facilities/virtual-office-hours/)
- [Metrics Documentation](metrics.md)
- [Curation & Publication FAQ](faq.md)
- [Policies](policies.md)
- [Metadata Dictionaries](../dictionary.md)
2 changes: 2 additions & 0 deletions user-guide/docs/curating/metrics.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
## Metrics

### Data Metrics { #data }

Data metrics are research impact indicators complementary to other forms of evaluation such as number of paper citations, allowing researchers to assess the repercussions and influence of their work.
Expand Down
70 changes: 0 additions & 70 deletions user-guide/docs/curating/officehours.md

This file was deleted.

2 changes: 2 additions & 0 deletions user-guide/docs/curating/policies.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
## Policies

### DesignSafe Data Depot Repository Mission and History

#### Mission { #mission }
Expand Down
14 changes: 14 additions & 0 deletions user-guide/docs/datadepot.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# DesignSafe Data Depot

The <a href="https://www.designsafe-ci.org/data/browser/public/" target="_blank">Data Depot</a> is the data repository for DesignSafe. The web interface to the Data Depot allows you to browse, upload, download, share, curate and publish data stored within the repository. You are encouraged to use the Data Depot not only for curation and publication of research results, but as a working "scratch" area for any of your own data and work in progress. Scientific applications in the <a href="https://www.designsafe-ci.org/rw/workspace/" target="_blank">Tools & Applications</a> area can access your Data Depot files, enabling data analysis directly in the DesignSafe portal that minimizes the need to transfer data to your laptop. The Data Depot hosts both public and private data associated with a project, with the following directories:

* **My Data**: Private directory for your data.
* **HPC Work**: Work directory on TACC HPC machines for use with Jupyter.
* **My Projects**: Data to be curated and published must be in this directory. Also has group access that you control.\
* **Shared with Me**: DEPRECATED. Use My Projects. Legacy directory we are no longer utilizing, but some very early users may have data.
* **Box.com**: Access to your Box files for copying to DesignSafe.
* **Dropbox.com**: Access to your Dropbox for copying to DesignSafe.
* **Google Drive**: CURRENTLY NOT FUNCTIONAL. Google has made changes that we are working through to reenable.
* **Published**: Curated data/projects with DOI's.
* **Published (NEES)**: Curated data/projects from NEES program that ran from 1999 - 2015.
* **Community Data**: Non-curated user-contributed data.
2 changes: 2 additions & 0 deletions user-guide/docs/dictionary.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
## Dictionaries

### Experimental

{% include-markdown 'dictionary/experimental.md' %}
Expand Down
21 changes: 0 additions & 21 deletions user-guide/docs/managingdata.md

This file was deleted.

18 changes: 2 additions & 16 deletions user-guide/docs/managingdata/datadepot.md
Original file line number Diff line number Diff line change
@@ -1,17 +1,5 @@
The <a href="https://www.designsafe-ci.org/data/browser/public/" target="_blank">Data Depot</a> is the data repository for DesignSafe. The web interface to the Data Depot allows you to browse, upload, download, share, curate and publish data stored within the repository. You are encouraged to use the Data Depot not only for curation and publication of research results, but as a working "scratch" area for any of your own data and work in progress. Scientific applications in the <a href="https://www.designsafe-ci.org/rw/workspace/" target="_blank">Tools & Applications</a> area can access your Data Depot files, enabling data analysis directly in the DesignSafe portal that minimizes the need to transfer data to your laptop. The Data Depot hosts both public and private data associated with a project, with the following directories:


* **My Data**: Private directory for your data.
* **HPC Work**: Work directory on TACC HPC machines for use with Jupyter.
* **My Projects**: Data to be curated and published must be in this directory. Also has group access that you control.\
* **Shared with Me**: DEPRECATED. Use My Projects. Legacy directory we are no longer utilizing, but some very early users may have data.
* **Box.com**: Access to your Box files for copying to DesignSafe.
* **Dropbox.com**: Access to your Dropbox for copying to DesignSafe.
* **Google Drive**: CURRENTLY NOT FUNCTIONAL. Google has made changes that we are working through to reenable.
* **Published**: Curated data/projects with DOI's.
* **Published (NEES)**: Curated data/projects from NEES program that ran from 1999 - 2015.
* **Community Data**: Non-curated user-contributed data.

## DesignSafe Managing Data

### Browsing, Upload, and Download { #browsing }

![Figure 1. Data Depot](./imgs/datadepotfigure.jpg)
Expand All @@ -31,5 +19,3 @@ A number of data transfer methods are supported for uploading and downloading fi
### Data Sharing, Collaboration, Curation & Publication { #sharing }

My Projects is the simplest way to share data with your collaborators and to curate and ultimately publish your data and receive a Digital Object Identifier (DOI). Any team member in a project has both read and write access to the entire contents of the project. The Data Curation & Publication User Guide provides instructions for creating projects, managing team members, curating and publishing your data.

---
2 changes: 2 additions & 0 deletions user-guide/docs/managingdata/datamanagementplan.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Data Management Plan Guidance

This document is intended as a Data Management Plan (DMP) guide that you can customize for the specific details of your research project that will use the NHERI DesignSafe cyberinfrastructure (CI). There is guidance on the five main DMP areas required by the National Science Foundation (NSF), along with information about the DesignSafe CI functionalities that can support your data management needs.

[Data Management Plan](../documents/DesignSafe_Data_Management_Plan_Guidance.docx)
2 changes: 2 additions & 0 deletions user-guide/docs/managingdata/datatransfer.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
## Data Transfer

DesignSafe supports multiple ways of moving data in and out of the Data Depot, the data transfer method that is best for you will depend on the quantity of data you wish to move. There are two broad categories of data transfer methods available; we will refer to these categories as large data transfer methods and normal data transfer methods. Large data transfer methods are for situations where you want to move a large amount of data (&gt; 2GB), a large numbers of files (&gt; 25), or folders. Whereas normal data transfer methods are for situations where you wish to move a small amount of data (&lt; 2GB) stored across a small number of files (&lt; 25).

This document provides a brief description of the various methods available for moving data to DesignSafe to assist you in identifying the right data transfer method for your research needs. Once you have selected your data transfer method, each description concludes with a link to detailed instructions for initiating your transfer.
Expand Down
4 changes: 2 additions & 2 deletions user-guide/docs/managingdata/experimentalfacilitychecklist.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
## Experimental Facility Checklist

### DesignSafe-EF Onboarding Checklist for Data Curation { #onboarding }

DesignSafe has been developed as a comprehensive research environment supporting a range of activities from research planning to cloud-based data analysis to data curation/publication. We encourage users to take full advantage of the DesignSafe capabilities associated with both the Data Depot data repository and the Tools and Apps. To learn more about all of these capabilities, watch this <a href="https://www.youtube.com/watch?v=5Yus9MjtcTM&amp;feature=youtu.be" target="_blank">Introductory Webinar</a>.
Expand Down Expand Up @@ -60,5 +62,3 @@ DesignSafe has been developed as a comprehensive research environment supporting
* DesignSafe provides the possibility to publish one experiment at a time, so you do not need to finish your entire research project to publish all the experiments.
* You may version your data and thus you can publish the raw data and add later analysis or processed results as version 2.
* The project PI and co PI should be involved in the process to make sure they agree with the data presentation. Clarify with the team the authorship and order of authors.

---
10 changes: 10 additions & 0 deletions user-guide/docs/managingdata/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
# Managing Data

{% include-markdown '../redirect.md' %}

- [Data Depot Overview](../datadepot.md)
- [Managing Data](./datadepot.md)
- [Setting Path to DS on Corral](./settingpathtodesignsafe.md)
- [Data Transfer Guides](./datatransfer.md)
- [Data Management Plan Guidance](./datamanagementplan.md)
- [Experimental Facility Checklist](./experimentalfacilitychecklist.md)
1 change: 1 addition & 0 deletions user-guide/docs/managingdata/settingpathtodesignsafe.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
## Setting Path to DesignSafe on Corral

The data stored on DesignSafe resides on the large (40 PB), shared data resource Corral located at the Texas Advanced Computing Center. Importantly, Corral services many different projects, not only DesignSafe, and as such utilizes a complex file structure for organization. The purpose of this documentation is to explain how to navitage this complex file structure to locate the directories pertinent to your data transfer needs on DesignSafe.

Expand Down
4 changes: 3 additions & 1 deletion user-guide/docs/recon.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
# Recon Portal

{% include-markdown 'tools/recon.md' %}
{% include-markdown './redirect.md' %}

- [Recon Portal User Guide](../tools/recon/)
5 changes: 5 additions & 0 deletions user-guide/docs/redirect.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
!!! attention "This page has moved."
Please use navigation panel or links below to find the content.

!!! caution "Update bookmarks and links."
This page is temporary, unmaintained, and will be deleted.
26 changes: 18 additions & 8 deletions user-guide/mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,6 @@ theme:
navigation_depth: 4
features:
- navigation.top
nav_redirects: # TACC Feature
- before: '#data-depotcuration-office-hours'
after: https://www.designsafe-ci.org/facilities/virtual-office-hours/
analytics:
gtag: G-D96RT1T24K

Expand All @@ -37,11 +34,24 @@ plugins:

nav:
- Data Depot:
- Managing Data: managingdata.md
- Curating &amp; Publishing Projects: curating.md
# NOTE: External link is achieved via `js/changeNavMarkup.js`, NOT here
# - Data Depot/Curation Office Hours: https://www.designsafe-ci.org/facilities/virtual-office-hours/
- Recon Portal: recon.md
- Overview: datadepot.md
- Managing Data:
- Overview: managingdata/datadepot.md
- Setting Path to DS on Corral: managingdata/settingpathtodesignsafe.md
- Data Transfer Guides: managingdata/datatransfer.md
- Data Management Plan: managingdata/datamanagementplan.md
- Experimental Facility Checklist: managingdata/experimentalfacilitychecklist.md
- Data Depot Repository:
- Office Hours: https://www.designsafe-ci.org/facilities/virtual-office-hours/
- Curating &amp; Publication:
- Guides: curating/guides.md
- Frequently Asked Questions: curating/faq.md
- Best Practices: curating/bestpractices.md
- Metrics Documentation: curating/metrics.md
- Policies: curating/policies.md
- Metadata Dictionaries: dictionary.md
- Recon Portal:
- Recon Portal User Guide: tools/recon.md

- Tools and Apps:
- Analysis Applications: analysis.md
Expand Down

0 comments on commit f23c04b

Please sign in to comment.