1.AiiDA data types and verdi commands.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*You can execute the active code cell's by pressing Ctrl+Enter*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Load your aiida profile\n",
    "from aiida import load_profile\n",
    "\n",
    "load_profile();"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to AiiDA"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note**: All commands starting with a `!` are bash commands. Python notebooks provide this handy way of running shell/bash commands inside a python environment. \n",
    "\n",
    "*Also read through the comments*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# example\n",
    "!ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Verdi Commands"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this part of the tutorial, you will learn some basics about the AiiDA framework. Get familiar with some useful `verdi` commands.\n",
    "\n",
    "The command-line utility `verdi` is one of the most common ways to interact with AiiDA. Verdi with its subcommands enables a variety of operations such as inspecting the status of ongoing or terminated calculations, showing the details of calculations, computers, codes, or data structures, access the input and the output of a calculation, etc. \n",
    "\n",
    "Similar to the bash shell, verdi command support Tab completion. Try right now to type verdi in a terminal of the AiiDA container and tap Tab twice to have a list of subcommands. **Whenever you need the explanation of a command type verdi help or add `-h` flag if you are using any of the verdi subcommands**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Check if everything needed for AiiDA is available and running\n",
    "!verdi status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# The daemon details\n",
    "!verdi daemon status"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Looking for calculations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We have prepared some data in the database for this part. In the later tutorials, you will learn to create such data using AiiDA-fleur. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List all calculations currently running\n",
    "!verdi process list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since you do not have any running processes, the previous command should yield an old calculation. To see all the calculations in the database use the following."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# List all calculations in the Postgres database\n",
    "!verdi process list -a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You must have noticed a column named `PK`. This is the **Primary Key** by which a particular node is referenced. There is also a `UUID` associated with each node, since these are very long and not so user-friendly `PK` is used."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look at a FINISHED inpgen calculation in more detail with `PK = 79`, what nodes went in? what output nodes were produced?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi process show 79"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The input file for inpgen for this calculation (with default parameters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi calcjob inputcat 79"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Find the `PK` of a `FleurCalculation` node and check the convergence using the following command"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# grep for the distance of that calculation in the out.xml file\n",
    "!verdi calcjob outputcat <pk> | grep distance"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Look at the files retrieved from the remote computer/cluster, we store the last charge density (cdn1) too, this way any calculation can be continued from what we have 'locally' in the repository"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi calcjob outputls <pk>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Codes and computers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A code represents (in the database) the actual executable used to run the calculation."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The command prints information on the plugin used to interface the code to AiiDA, the remote machine on which the code is executed, the path of its executable, etc. To have a list of all available codes type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi code list"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " Find the `pk` of such a node in the graph and type"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi code show 7"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Similarly, the list of computers on which AiiDA can submit calculations is accessible by means of the command"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi computer list -a"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "`-a` shows all computers, also the one imported in your database but that you did not configure, i.e., to which you\n",
    "don’t have access). Details about each computer can be obtained by the comma"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi computer show localhost"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "For a list of all the `verdi` commands check the [AiiDA documentation](https://aiida-core.readthedocs.io/en/latest/working_with_aiida/index.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## AiiDA data types\n",
    "\n",
    "There are a number of data types distributed with AiiDA. We summarize here the most common, and some useful features/functionalities to work with them."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Most common datatypes\n",
    "\n",
    "\n",
    "Here follows a short summary of common datatypes already provided with AiiDA. This list is not\n",
    "complete, see also inside the module `aiida.orm.nodes.data` for the list of all available plugins."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**NOTE**: To inspect, create and manage data nodes use the `verdi` command `verdi data`"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Base types\n",
    "\n",
    "In addition to the base datatypes such as `Int`, `Float`, `Str`, etc,. AiiDA also provides datatypes such as `Dict`, `StructureData`, `RemoteData`, etc. Uses and examples of these datatypes are illustrated in this notebook.\n",
    "\n",
    "**The different datatypes can be accessed through the `DataFactory()` function by passing an entry point to it as an argument** (you will see them in the examples used in this notebook).\n",
    "\n",
    "A list of all the data entry points can be obtain running the command ``verdi plugin list aiida.data``."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Dict\n",
    "`Dict` type is used to store a dictionary of python base types in the database.\n",
    "It can store any dictionary where elements can be a base python type (strings, floats,\n",
    "integers, booleans, None type,datetime objects, etc.) and lists or dictionaries of them, at\n",
    "any depth level (e.g. a dictionary where a value is a list of dictionaries of\n",
    "strings and floats).\n",
    "  "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aiida.orm import Dict\n",
    "\n",
    "options=Dict(dict={\n",
    "    'resources'  : {\"num_machines\": 1},\n",
    "    'max_wallclock_seconds':  60*60,\n",
    "})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can access all the methods associated with the options object using the `dot` operator (`options.<tab>`).\n",
    "\n",
    "you can also read the documentation associated with any object using `?`. Run the following cell to see it in action"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "options.get_dict?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print options.get_dict()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Aiida created a database storable dictionary for us. It is not stored in the database yet, for that we use the following method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "options.store()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This object is stored in the database and we can use `PK` to refer to this object from now on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!verdi data dict show <pk>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### StructureData\n",
    "\n",
    "To store a crystal structure to be used by atomistic codes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from aiida.plugins import DataFactory\n",
    "\n",
    "StructureData = DataFactory('structure')\n",
    "\n",
    "alat = 5.4 #angstrom\n",
    "cell = [[alat/2, alat/2, 0.],\n",
    "        [alat/2, 0., alat/2],\n",
    "        [0., alat/2, alat/2],\n",
    "       ]\n",
    "\n",
    "s = StructureData(cell=cell)\n",
    "s.append_atom(position=(0.,0.,0.), symbols='Si')\n",
    "s.append_atom(position=(alat/4.,alat/4.,alat/4.),symbols=\"Si\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can access and inspect the structure sites with the command"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.sites"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you make a mistake, start over from `structure = StructureData(cell=the_cell)`, or equivalently use\n",
    "`s.clear_kinds()` to remove all kinds (atomic species) and sites. Once you are done with it store the StructureData to the database."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.store()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `pk` method to see the `PK` of this StructureData"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "s.pk"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Alternatively, AiiDA structures can also be converted directly from ASE structures using"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from ase.spacegroup import crystal\n",
    "\n",
    "ase_structure = crystal('Si', \n",
    "                        [(0,0,0)], \n",
    "                        spacegroup=227,\n",
    "                        cellpar=[alat, alat, alat, 90, 90, 90],\n",
    "                        primitive_cell=True,\n",
    "                       )\n",
    "structure=StructureData(ase=ase_structure)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "structure.store()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A structure can also be read from a cif file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#replace the '/path/to/cif_file'\n",
    "\n",
    "!verdi data cif import \"/path/to/cif_file\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### FolderData\n",
    "\n",
    "To store a set of files/folders (with possibly a folder/subfolder structure)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### RemoteData\n",
    "\n",
    "This basically represents a \"symbolic link\" to a specific folder on\n",
    "  a remote computer.\n",
    "  Its main use is to allow users to persist the provenance when e.g. a calculation\n",
    "  produces data in a raw/scratch folder, and the whole folder needs to be provided\n",
    "  to restart/continue."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note: There are many more derived datatypes available in AiiDA, for a full list see the [AiiDA Documentation](https://aiida-core.readthedocs.io/en/latest/datatypes/index.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Code\n",
    "\n",
    "A code represents (in the database) the actual executable used to run the calculation. \n",
    "Note that in AiiDA the object `Code` in the database is meant to represent a specific executable, i.e. a given compiled version of a code. Every calculation in AiiDA is linked to a code, installed on a specific computer. This means that if you install `fleur` and `inpgen` on two computers A and B, you will need to have two different `codes` in the database (although the source of the code is the same, the binary file is different)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modify the following code cell using the information gathered in the <a href=\"#Codes-and-computers\">Codes and computers</a> section"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "codename = 'somecode@somecomputer'\n",
    "from aiida.orm import Code\n",
    "code = Code.get_from_string(codename)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Task"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Create and store `StructureData` for FCC `Fe`. *Hint*: The period length of FCC iron is about 3.645 Å."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "kernel_info": {
   "name": "python2"
  },
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.16"
  },
  "nteract": {
   "version": "0.14.5"
  },
  "toc": {
   "base_numbering": 1,
   "nav_menu": {},
   "number_sections": false,
   "sideBar": true,
   "skip_h1_title": false,
   "title_cell": "Table of Contents",
   "title_sidebar": "Contents",
   "toc_cell": false,
   "toc_position": {},
   "toc_section_display": true,
   "toc_window_display": false
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}