workshop/MIT QuHack.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "ed102cae",
   "metadata": {},
   "source": [
    "# Covalent QuHack Workshop\n",
    "\n",
    "\n",
    "The goal of this workshop is to showcase how one can install & set up covalent as well as how to use the plugin ecosystem to interact with cloud compute resources. We will run through several examples of workflows using covalent locally, a remote deployment of covalent on AWS, and workflows that utilize AWS plugins. Lastly we will cover an example using Qiskit & Pennylane with Covalent to evaluate Quantum Circuits.\n",
    "\n",
    "The structure is as follows:\n",
    "1. How to use covalent locally \n",
    "2. How to use AWS executor plugins\n",
    "3. Host covalent on AWS\n",
    "4. Use executor plugins on hosted covalent\n",
    "5. Example workflow using Qiskit & Pennylane\n",
    "\n",
    "\n",
    "Some useful links include:\n",
    "- https://github.com/AgnostiqHQ/covalent Covalent Github Repo\n",
    "- https://covalent.readthedocs.io/en/latest/getting_started/quick_start/index.html Getting Started Guide"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0f8b42ad",
   "metadata": {},
   "source": [
    "## Getting Started with Covalent\n",
    "\n",
    "To install covalent install it with pip:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "63d50a84",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install covalent --no-cache"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed3b5d59",
   "metadata": {},
   "source": [
    "To start covalent run the following in your terminal:\n",
    "\n",
    "**$ covalent start** \n",
    "\n",
    "> Note: This may not work if run in the notebook so be sure to use your shell in the same virtualenv as your notebook.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6f851db2",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Say we have a simple script as follows\n",
    "def add(x, y):\n",
    "   return x + y\n",
    "\n",
    "def multiply(x, y):\n",
    "   return x*y\n",
    "\n",
    "def divide(x, y):\n",
    "   return x/y\n",
    "\n",
    "def workflow(x, y):\n",
    "   r1 = add(x, y)\n",
    "   r2 = [multiply(r1, y) for _ in range(4)]\n",
    "   r3 = [divide(x, value) for value in r2]\n",
    "   return r1,r2,r3\n",
    "\n",
    "print(workflow(1,2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8d8df005",
   "metadata": {},
   "outputs": [],
   "source": [
    "# With no code changes we add decorators to covert to a covalent workflow\n",
    "\n",
    "import covalent as ct\n",
    "\n",
    "@ct.electron\n",
    "def add(x, y):\n",
    "   return x + y\n",
    "\n",
    "@ct.electron\n",
    "def multiply(x, y):\n",
    "   return x*y\n",
    "\n",
    "@ct.electron\n",
    "def divide(x, y):\n",
    "   return x/y\n",
    "\n",
    "@ct.lattice\n",
    "def workflow(x, y):\n",
    "   r1 = add(x, y)\n",
    "   r2 = [multiply(r1, y) for _ in range(4)]\n",
    "   r3 = [divide(x, value) for value in r2]\n",
    "   return r1,r2,r3\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95bb84b4",
   "metadata": {},
   "outputs": [],
   "source": [
    "dispatch_id = ct.dispatch(workflow)(1, 2)\n",
    "result = ct.get_result(dispatch_id, wait=True)\n",
    "print(result.result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dbf0f02",
   "metadata": {},
   "source": [
    "##  AWS Account Setup \n",
    "\n",
    "In order to interact with AWS services from your machine you must:\n",
    "\n",
    "1. Have an AWS Account (Free): https://aws.amazon.com/free/\n",
    "2. Install the AWS CLI: https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html\n",
    "\n",
    "Furthermore, if you don't already have one you will need to create an account. Once you are logged in you will want to go to IAM and then to Manage Security Credentials to obtain an `aws_access_key_id` and `aws_secret_access_key`. At which point you will need to store your credentials locally by using the cli:\n",
    "\n",
    "```bash\n",
    "$ aws configure\n",
    "```\n",
    "\n",
    "Alternatively you can go to `~/.aws/credentials` and create a section:\n",
    "\n",
    "```\n",
    "[default]\n",
    "aws_access_key_id=<ACCESS_KEY>\n",
    "aws_secret_access_key=<SECRET_ACCESS_KEY>\n",
    "```\n",
    "\n",
    "## Using AWS Executor Plugins in Covalent (AWS Batch)\n",
    "\n",
    "It is recommended to read our documentation of [AWS Executor Plugins](https://covalent.readthedocs.io/en/latest/api/executors/awsplugins.html)\n",
    "\n",
    "### We will now show a toy example of training an SVM ML model using a covalent workflow (without executors)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4c245bf9",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install numpy==1.23.2 scikit-learn==1.1.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7551c93",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Train SVM w/out Executor\n",
    "\n",
    "from numpy.random import permutation\n",
    "from sklearn import svm, datasets\n",
    "import covalent as ct\n",
    "\n",
    "@ct.electron\n",
    "def train_svm(data, C, gamma):\n",
    "    X, y = data\n",
    "    clf = svm.SVC(C=C, gamma=gamma)\n",
    "    clf.fit(X[90:], y[90:])\n",
    "    return clf\n",
    "\n",
    "@ct.electron\n",
    "def load_data():\n",
    "    iris = datasets.load_iris()\n",
    "    perm = permutation(iris.target.size)\n",
    "    iris.data = iris.data[perm]\n",
    "    iris.target = iris.target[perm]\n",
    "    return iris.data, iris.target\n",
    "\n",
    "@ct.electron\n",
    "def score_svm(data, clf):\n",
    "    X_test, y_test = data\n",
    "    return clf.score(\n",
    "      X_test[:90],\n",
    "    y_test[:90]\n",
    "    )\n",
    "\n",
    "@ct.lattice\n",
    "def run_experiment(C=1.0, gamma=0.7):\n",
    "    data = load_data()\n",
    "    clf = train_svm(\n",
    "      data=data,\n",
    "      C=C,\n",
    "      gamma=gamma\n",
    "    )\n",
    "    score = score_svm(\n",
    "      data=data,\n",
    "    clf=clf\n",
    "    )\n",
    "    return score\n",
    "\n",
    "# Dispatch the workflow\n",
    "dispatch_id = ct.dispatch(run_experiment)(\n",
    "  C=1.0,\n",
    "  gamma=0.7\n",
    ")\n",
    "\n",
    "# Wait for our result and get result value\n",
    "result = ct.get_result(dispatch_id=dispatch_id, wait=True).result\n",
    "\n",
    "# Get Mean Accuracy of our model\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "915b8bd1",
   "metadata": {},
   "source": [
    "### We will now use the AWS Batch executor plugin\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "191095c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install covalent-awsbatch-plugin --no-cache # Note: we need to restart covalent after installing plugins"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b76cad58",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Executor (AWS Batch)\n",
    "\n",
    "import covalent as ct\n",
    "from covalent.executor import AWSBatchExecutor\n",
    "\n",
    "deps_pip = ct.DepsPip(\n",
    "  packages=[\"numpy==1.23.2\", \"scikit-learn==1.1.2\"]\n",
    ")\n",
    "\n",
    "executor = AWSBatchExecutor(\n",
    "    region='us-east-1',\n",
    "    s3_bucket_name='<s3_bucket_name>',\n",
    "    batch_queue='<batch_queue>',\n",
    "    batch_job_role_name='<batch_job_role_name>',\n",
    "    batch_job_log_group_name='<batch_job_log_group_name>',\n",
    "    batch_execution_role_name='<batch_execution_role_name>',\n",
    "    vcpu=2,\n",
    "    memory=3.75,\n",
    "    time_limit=300\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ed6ed4ba",
   "metadata": {},
   "source": [
    "To use any executor plugins we add the `executor` key to the `@electron` decorator, for example:\n",
    "\n",
    "```python\n",
    "@ct.electron(\n",
    "    executor=executor\n",
    ")\n",
    "def train_svm(data, C, gamma):\n",
    "    X, y = data\n",
    "    clf = svm.SVC(C=C, gamma=gamma)\n",
    "    clf.fit(X[90:], y[90:])\n",
    "    return clf\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "9c18a501",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Use executor plugin to train our SVM model.\n",
    "from numpy.random import permutation\n",
    "from sklearn import svm, datasets\n",
    "import covalent as ct\n",
    "\n",
    "@ct.electron(\n",
    "    executor=executor,\n",
    "    deps_pip=deps_pip\n",
    ")\n",
    "def train_svm(data, C, gamma):\n",
    "    X, y = data\n",
    "    clf = svm.SVC(C=C, gamma=gamma)\n",
    "    clf.fit(X[90:], y[90:])\n",
    "    return clf\n",
    "\n",
    "@ct.electron\n",
    "def load_data():\n",
    "    iris = datasets.load_iris()\n",
    "    perm = permutation(iris.target.size)\n",
    "    iris.data = iris.data[perm]\n",
    "    iris.target = iris.target[perm]\n",
    "    return iris.data, iris.target\n",
    "\n",
    "@ct.electron\n",
    "def score_svm(data, clf):\n",
    "    X_test, y_test = data\n",
    "    return clf.score(\n",
    "      X_test[:90],\n",
    "    y_test[:90]\n",
    "    )\n",
    "\n",
    "@ct.lattice\n",
    "def run_experiment(C=1.0, gamma=0.7):\n",
    "    data = load_data()\n",
    "    clf = train_svm(\n",
    "      data=data,\n",
    "      C=C,\n",
    "      gamma=gamma\n",
    "    )\n",
    "    score = score_svm(\n",
    "      data=data,\n",
    "    clf=clf\n",
    "    )\n",
    "    return score\n",
    "\n",
    "# Dispatch the workflow\n",
    "dispatch_id = ct.dispatch(run_experiment)(\n",
    "  C=1.0,\n",
    "  gamma=0.7\n",
    ")\n",
    "\n",
    "# Wait for our result and get result value\n",
    "result = ct.get_result(dispatch_id=dispatch_id, wait=True).result\n",
    "\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "062e2152",
   "metadata": {},
   "source": [
    "## Using Self Hosted Version of Covalent\n",
    "\n",
    "In order to deploy the self-hosted instance of covalent we will require Terraform in order to programatically create AWS Resources such as VMs (EC2) instances, and other resources required for using the AWS Executors.\n",
    "\n",
    "The installation guide for Terraform is the following: \n",
    "- https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli#install-terraform\n",
    "\n",
    "We will now run through an example deployment of Covalent to AWS, the following repository contains all of the instructions for going about this:\n",
    "- https://github.com/AgnostiqHQ/covalent-terraform"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c1c45588",
   "metadata": {},
   "outputs": [],
   "source": [
    "import covalent as ct\n",
    "\n",
    "ct.set_config(\"dispatcher.address\", \"<URL_FROM_TERRAFORM>\")\n",
    "ct.set_config(\"dispatcher.port\", \"48008\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a80d8d88",
   "metadata": {},
   "source": [
    "### Important! Since now covalent is now running remotely we will need to install all dependencies required for each task\n",
    "\n",
    "```python\n",
    "@ct.electron(deps_pip=ct.DepsPip(packages=[\"numpy==1.23.2\"]))\n",
    "def my_task():\n",
    "    return np.random.rand(3,2)\n",
    "```\n",
    "\n",
    "### Furthermore, executor plugins on remote covalent are pre-configured so they can be referred to simply as:\n",
    "\n",
    "```python\n",
    "@ct.electron(executor=\"awsbatch\")\n",
    "def my_batch_task():\n",
    "    pass\n",
    "```\n",
    "\n",
    "The following are supported executors on hosted covalent:\n",
    "\n",
    "- AWS Batch used as `\"awsbatch\"`\n",
    "- AWS Lambda used as `\"awslambda\"`\n",
    "- AWS Braket used as `\"braket\"`\n",
    "\n",
    "#### To use Amazon Braket in a new account you must visit\n",
    "- https://console.aws.amazon.com/braket/home?#/permissions?tab=executionRoles\n",
    "To create a service-linked role"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8c48ea82",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Import Executor (AWS Batch) - Note: we are using aliased executors\n",
    "\n",
    "import covalent as ct\n",
    "from covalent.executor import AWSBatchExecutor\n",
    "\n",
    "deps_pip = ct.DepsPip(\n",
    "  packages=[\"numpy==1.23.2\", \"scikit-learn==1.1.2\"]\n",
    ")\n",
    "\n",
    "# As before we use the executor and install dependencies\n",
    "@ct.electron(\n",
    "    executor=\"awsbatch\",\n",
    "    deps_pip=deps_pip\n",
    ")\n",
    "def train_svm(data, C, gamma):\n",
    "    X, y = data\n",
    "    clf = svm.SVC(C=C, gamma=gamma)\n",
    "    clf.fit(X[90:], y[90:])\n",
    "    return clf\n",
    "\n",
    "# IMPORTANT now since covalent is running remotely we must install the dependencies each task.\n",
    "@ct.electron(\n",
    "    deps_pip=deps_pip\n",
    ")\n",
    "def load_data():\n",
    "    iris = datasets.load_iris()\n",
    "    perm = permutation(iris.target.size)\n",
    "    iris.data = iris.data[perm]\n",
    "    iris.target = iris.target[perm]\n",
    "    return iris.data, iris.target\n",
    "\n",
    "@ct.electron(\n",
    "    deps_pip=deps_pip\n",
    ")\n",
    "def score_svm(data, clf):\n",
    "    X_test, y_test = data\n",
    "    return clf.score(\n",
    "      X_test[:90],\n",
    "    y_test[:90]\n",
    "    )\n",
    "\n",
    "@ct.lattice\n",
    "def run_experiment(C=1.0, gamma=0.7):\n",
    "    data = load_data()\n",
    "    clf = train_svm(\n",
    "      data=data,\n",
    "      C=C,\n",
    "      gamma=gamma\n",
    "    )\n",
    "    score = score_svm(\n",
    "      data=data,\n",
    "    clf=clf\n",
    "    )\n",
    "    return score\n",
    "\n",
    "# Dispatch the workflow\n",
    "dispatch_id = ct.dispatch(run_experiment)(\n",
    "  C=1.0,\n",
    "  gamma=0.7\n",
    ")\n",
    "\n",
    "# Wait for our result and get result value\n",
    "result = ct.get_result(dispatch_id=dispatch_id, wait=True).result\n",
    "\n",
    "print(result)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec07be18",
   "metadata": {},
   "source": [
    "### To revert to local covalent you can use the following"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "39022f9c",
   "metadata": {},
   "outputs": [],
   "source": [
    "import covalent as ct\n",
    "\n",
    "ct.set_config(\"dispatcher.address\", \"localhost\")\n",
    "ct.set_config(\"dispatcher.port\", \"48008\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d749258",
   "metadata": {},
   "source": [
    "\n",
    "## Quiskit & Pennylane Quantum Circuit Example\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3f7aee4d",
   "metadata": {},
   "source": [
    "Install qiskit & pennylane as well as the pennylane braket plugin:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6fd13841",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install qiskit pennylane amazon-braket-pennylane-plugin"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b771684b",
   "metadata": {},
   "outputs": [],
   "source": [
    "\"\"\"\n",
    "Compare expectation value estimates for 'M = I ⊗ Z' on the final state of the circuit,\n",
    "     ┌────┐\n",
    "q_0: ┤ Rz ├──■────────\n",
    "     └────┘┌─┴─┐┌────┐\n",
    "q_1: ──────┤ X ├┤ Ry ├\n",
    "           └───┘└────┘\n",
    "for arbitrary Z- and Y-rotation angles, as implemented in qiskit versus pennylane.\n",
    "\"\"\"\n",
    "from math import pi as PI\n",
    "from typing import List, Tuple\n",
    "\n",
    "import covalent as ct\n",
    "import pennylane as qml\n",
    "from qiskit import Aer, QuantumCircuit\n",
    "from qiskit.opflow import CircuitStateFn, I, Z\n",
    "from qiskit.opflow.state_fns import StateFn\n",
    "from qiskit.opflow.expectations import PauliExpectation\n",
    "from qiskit.opflow.converters import CircuitSampler\n",
    "\n",
    "SHOTS = 10_000\n",
    "\n",
    "# use the local braket backend for pennylane\n",
    "QML_BRAKET_DEVICE = qml.device(\"braket.local.qubit\", wires=2, shots=SHOTS)\n",
    "\n",
    "# NOTE: use the Aer simulator instead of Quantinuum backend for now\n",
    "QISKIT_BACKEND = Aer.get_backend(\"aer_simulator_statevector\", shots=SHOTS)\n",
    "\n",
    "\n",
    "\n",
    "@ct.electron(deps_pip=ct.DepsPip(packages=[\"qiskit\"]))\n",
    "def get_expval_from_qiskit(phi_z: float, phi_y: float) -> float:\n",
    "    \"\"\"qiskit implementation of the simulation\"\"\"\n",
    "    qiskit_circuit = QuantumCircuit(2)\n",
    "    qiskit_circuit.rz(phi_z, 0)\n",
    "    qiskit_circuit.cnot(0, 1)\n",
    "    qiskit_circuit.ry(phi_y, 1)\n",
    "\n",
    "    psi = CircuitStateFn(qiskit_circuit)\n",
    "    op = Z ^ I\n",
    "    measureable_expr = StateFn(op, is_measurement=True).compose(psi)\n",
    "    expectation = PauliExpectation().convert(measureable_expr)\n",
    "    sampler = CircuitSampler(QISKIT_BACKEND).convert(expectation)\n",
    "    return sampler.eval().real\n",
    "\n",
    "\n",
    "@ct.electron(deps_pip=ct.DepsPip(packages=[\"pennylane\"]))\n",
    "def get_expval_from_pennylane(phi_z: float, phi_y: float) -> float:\n",
    "    \"\"\"pennylane implementation of the simulation\"\"\"\n",
    "    @qml.qnode(device=QML_BRAKET_DEVICE)\n",
    "    def _circuit():\n",
    "        qml.RZ(phi_z, wires=0)\n",
    "        qml.CNOT(wires=[0, 1])\n",
    "        qml.RY(phi_y, wires=1)\n",
    "        return qml.expval(qml.PauliZ(1))\n",
    "\n",
    "    return float(_circuit())\n",
    "\n",
    "\n",
    "@ct.electron\n",
    "def mse(expvals_1: List[float], expvals_2: List[float]) -> float:\n",
    "    \"\"\"calculates MSE between two lists of floats\"\"\"\n",
    "    assert len(expvals_1) == len(expvals_2)\n",
    "    squared_error_sum = 0.0\n",
    "    for e1, e2 in zip(expvals_1, expvals_2):\n",
    "        squared_error_sum += (e1 - e2)**2\n",
    "    return squared_error_sum / len(expvals_1)\n",
    "\n",
    "\n",
    "@ct.lattice\n",
    "def sweep_angles(angles: List[Tuple[float, float]]) -> float:\n",
    "    \"\"\"\n",
    "    Get exp. values from qiskit and pennylane functions implementing the same\n",
    "    quantum circuit and compare the outputs based on MSE\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    angles : List[Tuple[float, float]]\n",
    "        pairs of angle parameters (phi_z, phi_y) for the quantum circuit\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    float\n",
    "        mean squared error between qiskit and pennylane circuit results (should be ~0)\n",
    "    \"\"\"\n",
    "    expvals_qiskit = []\n",
    "    expvals_pennylane = []\n",
    "    for phi_z, phi_y in angles:\n",
    "        expvals_qiskit.append(get_expval_from_qiskit(phi_z, phi_y))\n",
    "        expvals_pennylane.append(get_expval_from_pennylane(phi_z, phi_y))\n",
    "\n",
    "    return mse(expvals_qiskit, expvals_pennylane)\n",
    "\n",
    "\n",
    "if __name__ == \"__main__\":\n",
    "    inputs = [(PI, PI / 2), (PI / 3, PI / 4), (PI / 5, PI / 6), (PI / 7, PI / 8)]\n",
    "    dispatch_id = ct.dispatch(sweep_angles)(inputs)\n",
    "    mean_squared_error = ct.get_result(dispatch_id, wait=True).result\n",
    "    print(f\"MSE: {mean_squared_error:.4f}\")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}