Merge pull request #244 from OpenEnergyPlatform/feature-embargo

Feature embargo

docs/courses/04_upload.md

@@ -4,36 +4,44 @@ tags:
   - api
 ---
 
-# 04 - Upload - Contribute data to the OEP
+# 04 - Upload and Publish - Contribute data to the OEP
 
-Learn how to publish data on the OEP.
+Learn how to publish data on the [OpenEnergyPlatform](https://openenergyplatform.org/) (OEP).
 You can provide feedback to this course on GitHub: [https://github.com/OpenEnergyPlatform/academy/issues/166](https://github.com/OpenEnergyPlatform/academy/issues/166)
 
 ## For whom is this training and what can you learn?
 
-:oep-icon-info: **This course is aimed at researchers, no matter whether you have programming skills or not, who**
+:oep-icon-info: **This course is aimed at researchers, no matter whether you have programming skills or not, who want to**
 
-- want to publish their data on the OEP
+- upload their data on the OEP
+- publish their tables on the OEP
+- apply an embargo period on their data
 
-:oep-icon-info: **After reading the sections of this training course you will be able to publish data**
+:oep-icon-info: **After reading the sections of this training course you will be able to upload and publish data**
 
 - on the OEP using a visual interface - this is for those of you who do not want to program
 - on the OEP with the REST-full-HTTP API in python - this is for those of you who want to akquire some programming skills
 - using the oeclient or the OEP REST-API using python - this one is for those of you who can program already
 
-## How can I publish data to the OEP without programming skills?
+## What does 'upload' mean?
+
+Generally 'upload' refers to transimitting any data from a computer system onto another through means of a network. In this case specifically it stands for:
+- Move data and metadata onto the model draft section of the OEP.
+- Create data and metadata on the OEP.
+
+## How can I upload data to the OEP without programming skills?
 
 :oep-icon-text: [Upload tables as csv](../tutorials/99_other/wizard.md) (tutorial):
 
 This tutorial teaches you step-by-step on how to upload data from a .csv file to the OEP using the csv upload wizard.
 
-## How can I publish data to the OEP as a beginner interested in programming?
+## How can I upload data to the OEP as a beginner interested in programming?
 
 :oep-icon-text: , </i>:oep-icon-code: [Beginners Guide: How to upload data to the OpenEnergy Platform and initiate the review process](../tutorials/99_other/beginners_guide.md) (tutorial):
 
-This tutorial (also available as PDF) describes how to upload data to the OEP using Python and the REST-API. Several steps are needed to contribute actively to the platform. Some of these you only need to do once to get started. The steps described in this document are the following: Register at the OpenEnergy Platform, Register at GitHub and get invited to the OpenEnergy Platform Group, Create data table(s) and upload data table(s), create metadata and upload metadata to GitHub, initiate data review, create scenario factsheets.
+This course (also available as PDF) describes how to upload data to the OEP using Python and the REST-API. Several steps are needed to contribute actively to the platform. Some of these you only need to do once to get started. The steps described in this document are the following: Register at the OpenEnergy Platform, Register at GitHub and get invited to the OpenEnergy Platform Group, Create data table(s) and upload data table(s), create metadata and upload metadata to GitHub, initiate data review, create scenario factsheets.
 
-## How can I publish data to the OEP with programming skills?
+## How can I upload data to the OEP with programming skills?
 
 :oep-icon-code: [How to upload data (and metadata) using the API](../tutorials/01_api/02_api_upload.ipynb) (tutorial:
 
@@ -51,13 +59,42 @@ This jupyter notebook tutorial provides you a quick way of having all the code a
 
 This jupyter notebook tutorial provides you a quick way of having all the code at hand for using the OEP Client to upload your data and metadata to the OEP. You will use the command line.
 
+## What does 'publish' mean?
+
+To publish something generally means to make it accessable to people. In this case specifically it stands for:
+- Move table from the model draft to a target data theme on the OEP.
+For data to be publish onto the OEP it must have an Open Data license.
+:oep-icon-text: For further information see [SPDX-Datalicense list](https://spdx.org/licenses/)
+
+## How can I publish my uploaded table?
+
+ :oep-icon-text: [Tutorial to the publishing process]((../tutorials/99_other/publish.md)
+  :oep-icon-code: [Tutorial to the publishing process using the API](../tutorials/03_api/03_api_publish.ipynb)
+Comment: all tutorials and chapters will be linked here.
+Note: As long as the data is published, it cannot be changed. It is possible to 'unpublish' data.
+
+## What does 'embargo' mean?
+
+Generally an 'embargo period' means a period during which access to academic publication is not allowed to users who have not paid for access. In this case it stands for:
+- A period of time during which data cannot be accessed without permission. Von wem?
+- The OEP Dataviewer is blocked, as well as the API-based data access.
+
+## How can I apply an embargo period?
+
+Comment: all tutorials and chapters will be linked here.
+Note: The embargo can be set during data upload and data publication. Only embargo periods of 6 months or 1 year are possible. Embargoed data still requires an open license in order to be published.
+
+
+
+
+
 ---
 
 ## About this course
 
 :oep-logo-sirop:
 
-- Authors: Hannah Förster, Anne Siemons
+- Authors: Hannah Förster, Anne Siemons, Vismaya Jochem
 - Copyright: [Öko-Institut](https://www.oeko.de) (2022)
 - Contact: [email protected]
 - License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)

docs/tutorials/01_api/03_api_publish.ipynb

@@ -0,0 +1,234 @@
+{
+  "cells": [
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "# API Tutorial 03 - Publish data to the Open Energy Platform using the API.\n",
+        "\n",
+        "<!-- keep img below title and without align=\"left\"  -->\n",
+        "<img src=\"https://raw.githubusercontent.com/OpenEnergyPlatform/academy/develop/docs/data/img/OEP_logo_2_no_text.svg\" alt=\"OpenEnergy Platform\" height=\"75\" width=\"75\"/>\n"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Learnings\n",
+        "\n",
+        "This tutorials will enable you to:\n",
+        "\n",
+        "- [Publish an existing table on the OEP](#create-table)\n",
+        "- [Optionally apply an embargo period](#upload-data)\n",
+        "- [Undo the table publishing](#upload-metadata)\n",
+        "\n",
+        "\n",
+        "## Requirements\n",
+        "\n",
+        "- have your OEP API token at hand. You find this in settings tab in your profile page. (If you don't have a token see [Things you only need to do once](https://openenergyplatform.github.io/academy/tutorials/99_other/beginners_guide/#1-things-you-only-need-to-do-once).)\n",
+        "- Your tabular data is available on the open energy platform and got oemetadata including an open license (according to the SPDX data license list) in the licenses field.\n",
+        "  - we use the tutorial_example_table_publish from model_draft on the OEP, make sure it exists beforehand. If it is missing try to recreate it using this [script]() TODO.\n",
+        "- Python\n",
+        "- Python packages:\n",
+        "  - `requests` (See the packages [documentation](https://requests.readthedocs.io/en/latest/) for more help)\n",
+        "  - jupyter notebook, e.g. `notebook` or `jupyterlab`\n",
+        "- Skills:\n",
+        "  - use [python](https://wiki.python.org/moin/BeginnersGuide) with [jupyter notebooks](https://docs.jupyter.org/en/latest/)\n",
+        "  - _some_ understanding of [JSON encoded data](https://www.w3schools.com/js/js_json_datatypes.asp)\n",
+        "- Internet: obviously, but also make sure your network settings allow https traffic from python\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Feedback\n",
+        "\n",
+        "You can provide feedback on this tutorial in this [GitHub issue](https://github.com/OpenEnergyPlatform/academy/issues/TBD)\n"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Setup\n",
+        "\n",
+        "This part of the code sets up all you need to execute the further code below. It imports the required packages into python, sets up the test table and asks you for the OEP API token so that you can identify with the OEP, which is mandatory if you want to upload to the OEP. The setup is a necessary step.\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 5,
+      "metadata": {
+        "metadata": {}
+      },
+      "outputs": [
+        {
+          "name": "stdout",
+          "output_type": "stream",
+          "text": [
+            "https://openenergyplatform.org/api/v0/schema/model_draft/tables/tutorial_example_table/move_publish/demand/\n",
+            "https://openenergyplatform.org/api/v0/schema/demand/tables/tutorial_example_table/move_publish/model_draft/\n"
+          ]
+        }
+      ],
+      "source": [
+        "# install required packages with: pip install requests\n",
+        "# import required packages\n",
+        "import json\n",
+        "from getpass import getpass\n",
+        "from os import environ\n",
+        "\n",
+        "import requests as req\n",
+        "\n",
+        "topic = \"model_draft\"\n",
+        "table = f\"tutorial_example_table\"\n",
+        "\n",
+        "publish_in_topic = \"demand\"\n",
+        "token = environ.get(\"OEP_API_TOKEN\") or getpass(\"Enter your OEP API token:\")\n",
+        "\n",
+        "# for read/write, we need to add authorization header\n",
+        "auth_headers = {\"Authorization\": \"Token %s\" % token}\n",
+        "table_publish_api_url = f\"https://openenergyplatform.org/api/v0/schema/{topic}/tables/{table}/move_publish/{publish_in_topic}/\"\n",
+        "table_undo_publish_api_url = f\"https://openenergyplatform.org/api/v0/schema/{publish_in_topic}/tables/{table}/move_publish/{topic}/\"\n",
+        "\n",
+        "print(table_publish_api_url)\n",
+        "print(table_undo_publish_api_url)"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Optional: Setup an embargo period \n",
+        "\n",
+        "This part of the code will add an embargo period of either 6 months or 1 year on the published tabular dataset. Since it is also possible to apply an embargo on data that is not yet published, during table creation, this section code also shows how to remove an embargo period that is already active. \n",
+        "\n",
+        "Note: It is not possible to set a custom embargo period like embargo\": {\"start\": \"2024-08-27\", \"end\": \"2024-02-27\"} We might consider implementing such a possibility later on."
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 4,
+      "metadata": {
+        "metadata": {}
+      },
+      "outputs": [],
+      "source": [
+        "embargo = {\n",
+        "    \"embargo\": {\"duration\": \"6_months\"},\n",
+        "    # \"embargo\": None,\n",
+        "}"
+      ]
+    },
+    {
+      "attachments": {},
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Publish a table\n",
+        "\n",
+        "This part of the code deletes the table (including the metadata) that you have uploaded to the OEP.\n",
+        "\n",
+        "To be able to execute this part you will need to have executed the [setup](#setup) and you will need to have [created a table](#create-table).\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": 3,
+      "metadata": {},
+      "outputs": [
+        {
+          "ename": "NameError",
+          "evalue": "name 'table_undo_publish_api_url' is not defined",
+          "output_type": "error",
+          "traceback": [
+            "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+            "\u001b[0;31mNameError\u001b[0m                                 Traceback (most recent call last)",
+            "Cell \u001b[0;32mIn[3], line 2\u001b[0m\n\u001b[1;32m      1\u001b[0m \u001b[38;5;66;03m# Deleting tables needs authentication headers\u001b[39;00m\n\u001b[0;32m----> 2\u001b[0m res \u001b[38;5;241m=\u001b[39m req\u001b[38;5;241m.\u001b[39mpost(\u001b[43mtable_undo_publish_api_url\u001b[49m, headers\u001b[38;5;241m=\u001b[39mauth_headers)\n\u001b[1;32m      4\u001b[0m \u001b[38;5;66;03m# raise Exception if request fails\u001b[39;00m\n\u001b[1;32m      5\u001b[0m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;129;01mnot\u001b[39;00m res\u001b[38;5;241m.\u001b[39mok:\n",
+            "\u001b[0;31mNameError\u001b[0m: name 'table_undo_publish_api_url' is not defined"
+          ]
+        }
+      ],
+      "source": [
+        "# Deleting tables needs authentication headers\n",
+        "res = req.post(table_publish_api_url, json={\"query\": embargo}, headers=auth_headers)\n",
+        "\n",
+        "# raise Exception if request fails\n",
+        "if not res.ok:\n",
+        "    raise Exception(res.text)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## Revoke a published table\n",
+        "\n",
+        "This part of the code deletes the table (including the metadata) that you have uploaded to the OEP.\n",
+        "\n",
+        "To be able to execute this part you will need to have executed the [setup](#setup) and you will need to have [created a table](#create-table).\n"
+      ]
+    },
+    {
+      "cell_type": "code",
+      "execution_count": null,
+      "metadata": {},
+      "outputs": [],
+      "source": [
+        "# Deleting tables needs authentication headers\n",
+        "res = req.post(table_undo_publish_api_url, headers=auth_headers)\n",
+        "\n",
+        "# raise Exception if request fails\n",
+        "if not res.ok:\n",
+        "    raise Exception(res.text)"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": [
+        "## About this tutorial\n",
+        "\n",
+        ":oep-logo-sirop:\n",
+        "\n",
+        "- Author: Jonas Huber\n",
+        "- Copyright: [Reiner-Lemoine-Institut](https://www.reiner-lemoine-institut.de/) (2024)\n",
+        "- License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)\n",
+        "- Attribution: Reiner-Lemoine-Institut (2024): API Tutorial 03 - Publish data to the Open Energy Platform using the API.\n",
+        "- Last update :oep-auto-lastupdate:\n"
+      ]
+    },
+    {
+      "cell_type": "markdown",
+      "metadata": {},
+      "source": []
+    }
+  ],
+  "metadata": {
+    "kernelspec": {
+      "display_name": "Python 3",
+      "language": "python",
+      "name": "python3"
+    },
+    "language_info": {
+      "codemirror_mode": {
+        "name": "ipython",
+        "version": 3
+      },
+      "file_extension": ".py",
+      "mimetype": "text/x-python",
+      "name": "python",
+      "nbconvert_exporter": "python",
+      "pygments_lexer": "ipython3",
+      "version": "3.10.12"
+    },
+    "orig_nbformat": 4
+  },
+  "nbformat": 4,
+  "nbformat_minor": 2
+}

docs/tutorials/99_other/publish.md

@@ -0,0 +1,56 @@
+# Tutorial - How to Publish data and metadata on the OEP
+
+<!-- keep img below title and without align="left"  -->
+<img src="https://raw.githubusercontent.com/OpenEnergyPlatform/academy/develop/docs/data/img/OEP_logo_2_no_text.svg" alt="OpenEnergy Platform" height="75" width="75" />
+
+## Learnings
+
+This tutorials will enable you to:
+
+- Publish your data and metadata 
+	- with an embargo
+	- to a topic
+
+## Requirements
+
+- OEP Account
+- a table or dataset with metadata uploaded to the [OpenEnergyPlatform](https://openenergyplatform.org/) (OEP)
+
+
+## Introduction
+
+This tutorial will teach you step-by-step how to publish your data and metadata using the OEP wizard. If you want to use the API, please have a look at this tutorial. For more general information und the piblishing process and the empbargo period, see [this course](https://openenergyplatform.github.io/academy/courses/04_upload/). Comment: Links will be implemented once they are avalaible.
+
+
+## Choosing a table to be published
+If you want to publish a table (including [data and] metadata) you have to go to your [profile page](https://openenergyplatform.org/user/profile/959/settings) on the OEP.
+Now select the tab [`Tables`](https://openenergyplatform.org/user/profile/959/tables). Listed are data tables that you have uploaded or where you got group permissions assinged to; First are listed all tables you have already published, if you have not published any tables yet, this field will show 'You have never published a table.'.
+Beneath are listed all of your tables in the [model draft](https://openenergyplatform.org/dataedit/view/model_draft) section. 
+From this tables you can select the table you want to publish. Note: Only tables with an valid id and license in their metadata can be published on the OEP.
+
+## Choosing an embargo and a topic
+Once you selected the table you want to publish, a pop-up field `Publish Table` will open. Next to the fields is an `i` icon giving more information if you hover your mouse over it.
+First you can apply an embargo; in a drop down menu you can select between none, and a 6 months or 1 year embargo period. 
+Note: If you selct `None` there will be no embargo periode applied, even if you selected one while uploading the table.
+Second you have to `Select data topic` from one of the 12 available topics; scenario, boundaries, climate, demand, economy, enviromente, grid, openstreetmap, policy, refernce, society and supply.
+Note: You can only choose one data topic. With tags it is possible to mark the content of you data even more precisely.
+
+## Finishing the process
+Once you click `Confirm` in the pop-up window the table will be moved from the model draft to the data topic you selected. 
+Now it can be seen in the [Database](https://openenergyplatform.org/dataedit/schemas) under the selcted topic and on the profile page it is now visible as [Published datatables](https://openenergyplatform.org/user/profile/959/tables).
+
+
+
+## Feedback
+
+You can provide feedback on this tutorial here: https://github.com/OpenEnergyPlatform/academy/issues/189
+
+## About this tutorial
+
+:oep-logo-sirop:
+
+- Author: Vismaya Jochem
+- Copyright: Reiner Lemoine Institut
+- License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/deed.en)
+- Attribution: Open Energy Academy - OEMetaData Tutorial © [Reiner Lemoine Institut](https://reiner-lemoine-institut.de/)
+- Last update: :oep-auto-lastupdate:

docs/tutorials/99_other/wizard.md

@@ -32,6 +32,8 @@ We are logged in on the open energy platform
 - The first column must always be a numerical type named `id`. The wizard has this as a default.
 - We don't have to use all the columns from the csv file and we also don't have to use the same names, these can be mapped later. In this example, we use the following columns:
   ![](https://openenergyplatform.org/media/image/2021/02/tutorial_upload_img3.png)
+- Beneath the columns of the table is the option to `Apply Embargo`. The default is set to `None` but in the drop down menu the options 6 Months and 1 Year are also given. Comment: Here a link to the course explaining embargo will be inserted.
+- Next to all tabs is a small `i`. If you hover your mouse over it you will get information about the correponding field.
 - Click on `Create table`.
 - If successful, you are redirected to the upload section.