deploy a saved keras model to aws.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Deploy your pre-trained keras model to AWS\n",
    "adapted from https://aws.amazon.com/blogs/machine-learning/deploy-trained-keras-or-tensorflow-models-using-amazon-sagemaker/"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 0. Upload your model to SageMaker\n",
    "This is a manual step. \n",
    "\n",
    "(If you don't have a model, deel free to use my model below - simply uncomment and run the `wget` statement)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#! wget https://raw.githubusercontent.com/liampearson/Youtube/master/Keras%20to%20aws%20SageMaker/models/model.h5"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. input the model names\n",
    "complete either:\n",
    "- MODEL_LOCATION.  or\n",
    "- both of JSON and WEIGHTS_LOCATION"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "\n",
    "# if your model is saved as only a .h5 file\n",
    "MODEL_LOCATION ='model.h5'\n",
    "\n",
    "# or if your model is saved as 2 files: model as a .json file, and weights as a .h5 file\n",
    "JSON_LOCATION = ''\n",
    "WEIGHTS_LOCATION = ''"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Load Your Model\n",
    "Simply run the cell below; the model will be loaded based on how you defined the above"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "if MODEL_LOCATION!='': #if your model is saved as a .h5 file only\n",
    "    from keras.models import load_model\n",
    "    model = load_model(MODEL_LOCATION) #load the model\n",
    "    print(\"loaded model from MODEL_LOCATION\")\n",
    "    \n",
    "elif JSON_LOCATION!='': # you have your model saved as a JSON file AND weights\n",
    "#adapted from https://machinelearningmastery.com/save-load-keras-deep-learning-models/\n",
    "    from keras.models import model_from_json\n",
    "    json_file = open(JSON_LOCATION, 'r')\n",
    "    loaded_model_json = json_file.read()\n",
    "    json_file.close()\n",
    "    \n",
    "    model = model_from_json(loaded_model_json)\n",
    "    # load weights into new model\n",
    "    model.load_weights(WEIGHTS_LOCATION)\n",
    "    print(\"loaded model from JSON_LOCATION and WEIGHTS_LOCATION\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Convert the Keras Model to the format AWS wants\n",
    "- Converts to a Protobuff file\n",
    "- Saves it in a certain aws file structure\n",
    "- Tarballs this file and zips it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def convert_h5_to_aws(loaded_model):\n",
    "    \"\"\"\n",
    "    given a pre-trained keras model, this function converts it to a TF protobuf format\n",
    "    and saves it in the file structure which aws expects\n",
    "    \"\"\"  \n",
    "    from tensorflow.python.saved_model import builder\n",
    "    from tensorflow.python.saved_model.signature_def_utils import predict_signature_def\n",
    "    from tensorflow.python.saved_model import tag_constants\n",
    "    \n",
    "    # This is the file structure which AWS expects. Cannot be changed. \n",
    "    model_version = '1'\n",
    "    export_dir = 'export/Servo/' + model_version\n",
    "    \n",
    "    # Build the Protocol Buffer SavedModel at 'export_dir'\n",
    "    builder = builder.SavedModelBuilder(export_dir)\n",
    "    \n",
    "    # Create prediction signature to be used by TensorFlow Serving Predict API\n",
    "    signature = predict_signature_def(\n",
    "        inputs={\"inputs\": loaded_model.input}, outputs={\"score\": loaded_model.output})\n",
    "    \n",
    "    from keras import backend as K\n",
    "    with K.get_session() as sess:\n",
    "        # Save the meta graph and variables\n",
    "        builder.add_meta_graph_and_variables(\n",
    "            sess=sess, tags=[tag_constants.SERVING], signature_def_map={\"serving_default\": signature})\n",
    "        builder.save()\n",
    "    \n",
    "    #create a tarball/tar file and zip it\n",
    "    import tarfile\n",
    "    with tarfile.open('model.tar.gz', mode='w:gz') as archive:\n",
    "        archive.add('export', recursive=True)\n",
    "        \n",
    "convert_h5_to_aws(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Move the tarball (tar.gz) to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sagemaker\n",
    "\n",
    "sagemaker_session = sagemaker.Session()\n",
    "inputs = sagemaker_session.upload_data(path='model.tar.gz', key_prefix='model')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is the name of the bucket which SageMaker made in S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# where did it upload to?\n",
    "print(\"Bucket name is:\")\n",
    "sagemaker_session.default_bucket()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Create a SageMaker Model\n",
    "First, create an empty train.py file (TensorFlowModel expects this at its 'entry point', but can be empty)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!touch train.py #create an empty python file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import boto3, re\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "# the (default) IAM role you created when creating this notebook\n",
    "role = get_execution_role()\n",
    "\n",
    "# Create a Sagemaker model (see AWS console>SageMaker>Models)\n",
    "from sagemaker.tensorflow.model import TensorFlowModel\n",
    "sagemaker_model = TensorFlowModel(model_data = 's3://' + sagemaker_session.default_bucket() + '/model/model.tar.gz',\n",
    "                                  role = role,\n",
    "                                  framework_version = '1.12',\n",
    "                                  entry_point = 'train.py')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 6a) Host the SageMaker model and\n",
    "## 6b) Create an Endpoint to access the model "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Deploy the model. This can take ~10 minutes\n",
    "\n",
    "Ignore the message `update_endpoint is a no-op in sagemaker>=2`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deploy a SageMaker to an endpoint\n",
    "predictor = sagemaker_model.deploy(initial_instance_count=1,\n",
    "                                   instance_type='ml.m4.xlarge')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What is our endpoint called?\n",
    "#endpoint = predictor.endpoint\n",
    "#endpoint"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Success! You have deployed a keras model into AWS\n",
    "# ---------------------\n",
    "### 7. Confirm its working correctly by making a prediction\n",
    "Now, we want to use our endpoint/model. Create a predictor which uses the endpoint\n",
    "\n",
    "This step depends on what inputs your model is expecting. I simply used the iris dataset and so can feed it 4 inputs of which it will give me 3 probabilities - 1 for each iris type. \n",
    "\n",
    "Before deploying to aws, I got the predictions of my model - __locally__ - so that I could compare the local vs aws results (they should be the same). \n",
    "\n",
    "Locally, with the input below we get the following predictions:\n",
    "expected predictions:\n",
    "\n",
    "- 0.99930930\n",
    "- 0.00069067377\n",
    "- 0.00000000000000015728773"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'predictions': [[0.999309, 0.000690674, 1.57288e-16]]}"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Create a predictor which uses this new endpoint\n",
    "import sagemaker\n",
    "from sagemaker.tensorflow.model import TensorFlowModel\n",
    "\n",
    "endpoint = '' #get endpoint name from SageMaker > endpoints\n",
    "\n",
    "predictor=sagemaker.tensorflow.model.TensorFlowPredictor(endpoint, sagemaker_session)\n",
    "# .predict send the data to our endpoint\n",
    "data = np.asarray([[5. , 3.5, 1.3, 0.3]]) #<-- update this to have inputs for your model\n",
    "predictor.predict(data)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Cleanup!\n",
    "\n",
    "else you will incur extra charges\n",
    "\n",
    "https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-cleanup.html\n",
    "\n",
    "- Stop Notebook\n",
    "- delete endpoints\n",
    "- delete models\n",
    "- delete S3 bucket\n",
    "- delete cloudwatch groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}