notebooks/106-auto-device/106-auto-device.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "190e8e4c-461f-4521-ae7f-3491fa827ab7",
   "metadata": {
    "tags": []
   },
   "source": [
    "# Automatic Device Selection with OpenVINO™\n",
    "\n",
    "The [Auto device](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_AUTO.html) (or AUTO in short) selects the most suitable device for inference by considering the model precision, power efficiency and processing capability of the available [compute devices](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_Supported_Devices.html). The model precision (such as `FP32`, `FP16`, `INT8`, etc.) is the first consideration to filter out the devices that cannot run the network efficiently.\n",
    "\n",
    "Next, if dedicated accelerators are available, these devices are preferred (for example, integrated and discrete [GPU](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_GPU.html#doxid-openvino-docs-o-v-u-g-supported-plugins-g-p-u) or [VPU](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_VPU.html)). [CPU](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_CPU.html) is used as the default \"fallback device\". Keep in mind that AUTO makes this selection only once, during the loading of a model. \n",
    "\n",
    "When using accelerator devices such as GPUs, loading models to these devices may take a long time. To address this challenge for applications that require fast first inference response, AUTO starts inference immediately on the CPU and then transparently shifts inference to the GPU, once it is ready. This dramatically reduces the time to execute first inference.\n",
    "\n",
    "\n",
    "<center><img src=\"https://user-images.githubusercontent.com/15709723/161451847-759e2bdb-70bc-463d-9818-400c0ccf3c16.png\" width=580 alt=\"Auto Device Selection logic\"></center>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "22e8054d-1f4a-4337-bd2e-9e00d9e33820",
   "metadata": {},
   "source": [
    "## Download and convert the model\n",
    "\n",
    "This tutorial uses the [bvlc_googlenet](https://github.com/BVLC/caffe/tree/master/models/bvlc_googlenet) model. The bvlc_googlenet model is the first of the [Inception](https://github.com/tensorflow/tpu/tree/master/models/experimental/inception) family of models designed to perform image classification. Like other Inception models, bvlc_googlenet was pre-trained on the [ImageNet](https://image-net.org/) data set. For more details about this family of models, see the [research paper](https://arxiv.org/abs/1512.00567)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e58e0582-a77f-4621-ba92-429a9c5f5e56",
   "metadata": {},
   "outputs": [],
   "source": [
    "import sys\n",
    "\n",
    "from pathlib import Path\n",
    "from openvino.tools import mo\n",
    "from openvino.runtime import serialize\n",
    "from IPython.display import Markdown, display\n",
    "\n",
    "sys.path.append(\"../utils\")\n",
    "\n",
    "import notebook_utils as utils\n",
    "\n",
    "base_model_dir = Path(\"./model\").expanduser()\n",
    "\n",
    "model_name = \"bvlc_googlenet\"\n",
    "caffemodel_name = f'{model_name}.caffemodel'\n",
    "prototxt_name = f'{model_name}.prototxt'\n",
    "\n",
    "caffemodel_path = base_model_dir / caffemodel_name\n",
    "prototxt_path = base_model_dir / prototxt_name\n",
    "\n",
    "if not caffemodel_path.exists() or not prototxt_path.exists():\n",
    "    caffemodel_url = \"https://storage.openvinotoolkit.org/repositories/open_model_zoo/public/2022.1/googlenet-v1/bvlc_googlenet.caffemodel\"\n",
    "    prototxt_url = \"https://raw.githubusercontent.com/BVLC/caffe/88c96189bcbf3853b93e2b65c7b5e4948f9d5f67/models/bvlc_googlenet/deploy.prototxt\"\n",
    "\n",
    "    utils.download_file(caffemodel_url, caffemodel_name, base_model_dir)\n",
    "    utils.download_file(prototxt_url, prototxt_name, base_model_dir)\n",
    "else:\n",
    "    print(f'{caffemodel_name} and {prototxt_name} already downloaded to {base_model_dir}')\n",
    "\n",
    "# postprocessing of model\n",
    "text = prototxt_path.read_text()\n",
    "text = text.replace('dim: 10', 'dim: 1')\n",
    "res = prototxt_path.write_text(text)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcfc461c",
   "metadata": {},
   "source": [
    "## Import modules and create Core"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "967c128a",
   "metadata": {},
   "outputs": [],
   "source": [
    "import cv2\n",
    "import matplotlib.pyplot as plt\n",
    "import numpy as np\n",
    "from openvino.runtime import Core, CompiledModel, AsyncInferQueue, InferRequest\n",
    "import sys\n",
    "import time\n",
    "\n",
    "ie = Core()\n",
    "\n",
    "if \"GPU\" not in ie.available_devices:\n",
    "    display(Markdown('<div class=\"alert alert-block alert-danger\"><b>Warning: </b> A GPU device is not available. This notebook requires GPU device to have meaningful results. </div>'))"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "id": "ee513ee2",
   "metadata": {},
   "source": [
    "## Convert the model to OpenVINO IR format\n",
    "\n",
    "Use Model Optimizer to convert the Caffe model to OpenVINO IR with `FP16` precision. The models are saved to the `model/ir_model/` directory. For more information about Model Optimizer, see the [Model Optimizer Developer Guide](https://docs.openvino.ai/latest/openvino_docs_MO_DG_Deep_Learning_Model_Optimizer_DevGuide.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ed2789f7",
   "metadata": {},
   "outputs": [],
   "source": [
    "ir_model_path = base_model_dir / 'ir_model' / f'{model_name}.xml'\n",
    "model = None\n",
    "\n",
    "if not ir_model_path.exists():\n",
    "    model = mo.convert_model(input_model=base_model_dir / caffemodel_name,\n",
    "                             input_proto=base_model_dir / prototxt_name,\n",
    "                             input_shape=[1, 3, 224, 224],\n",
    "                             layout=\"NCHW\",\n",
    "                             mean_values=[104.0,117.0,123.0],\n",
    "                             output=\"prob\",\n",
    "                             compress_to_fp16=True)\n",
    "    serialize(model, str(ir_model_path))\n",
    "    print(\"IR model saved to {}\".format(ir_model_path))\n",
    "else:\n",
    "    print(\"Read IR model from {}\".format(ir_model_path))\n",
    "    model = ie.read_model(ir_model_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "740bfdd8",
   "metadata": {},
   "source": [
    "## (1) Simplify selection logic\n",
    "### Default behavior of Core::compile_model API without device_name\n",
    "By default, `compile_model` API will select **AUTO** as `device_name` if no device is specified."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "73d7aedb",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set LOG_LEVEL to LOG_INFO.\n",
    "ie.set_property(\"AUTO\", {\"LOG_LEVEL\":\"LOG_INFO\"})\n",
    "\n",
    "# Load the model onto the target device.\n",
    "compiled_model = ie.compile_model(model=model)\n",
    "\n",
    "if isinstance(compiled_model, CompiledModel):\n",
    "    print(\"Successfully compiled model without a device_name.\")   "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "66bfa571-8241-4623-9e9e-01bf2d0f89d4",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deleted model will wait until compiling on the selected device is complete.\n",
    "del compiled_model\n",
    "print(\"Deleted compiled_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "85b48949",
   "metadata": {},
   "source": [
    "### Explicitly pass AUTO as device_name to Core::compile_model API\n",
    "It is optional, but passing AUTO explicitly as `device_name` may improve readability of your code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0814073d",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Set LOG_LEVEL to LOG_NONE.\n",
    "ie.set_property(\"AUTO\", {\"LOG_LEVEL\":\"LOG_NONE\"})\n",
    "\n",
    "compiled_model = ie.compile_model(model=model, device_name=\"AUTO\")\n",
    "\n",
    "if isinstance(compiled_model, CompiledModel):\n",
    "    print(\"Successfully compiled model using AUTO.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "58e765f5-8e31-4eb4-bed8-9fba30bb29e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deleted model will wait until compiling on the selected device is complete.\n",
    "del compiled_model\n",
    "print(\"Deleted compiled_model\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "705ce668",
   "metadata": {},
   "source": [
    "## (2) Improve the first inference latency\n",
    "One of the benefits of using AUTO device selection is reducing FIL (first inference latency). FIL is the model compilation time combined with the first inference execution time. Using the CPU device explicitly will produce the shortest first inference latency, as the OpenVINO graph representation loads quickly on CPU, using just-in-time (JIT) compilation. The challenge is with GPU devices since OpenCL graph complication to GPU-optimized kernels takes a few seconds to complete. This initialization time may be intolerable for some applications. To avoid this delay, the AUTO uses CPU transparently as the first inference device until GPU is ready.\n",
    "### Load an Image"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "dc1cfeaf",
   "metadata": {},
   "outputs": [],
   "source": [
    "# For demonstration purposes, load the model to CPU and get inputs for buffer preparation.\n",
    "compiled_model = ie.compile_model(model=model, device_name=\"CPU\")\n",
    "\n",
    "input_layer_ir = next(iter(compiled_model.inputs))\n",
    "\n",
    "# Read image in BGR format.\n",
    "image = cv2.imread(\"../data/image/coco.jpg\")\n",
    "\n",
    "# N, C, H, W = batch size, number of channels, height, width.\n",
    "N, C, H, W = input_layer_ir.shape\n",
    "\n",
    "# Resize image to the input size expected by the model.\n",
    "resized_image = cv2.resize(image, (W, H))\n",
    "\n",
    "# Reshape to match the input shape expected by the model.\n",
    "input_image = np.expand_dims(resized_image.transpose(2, 0, 1), 0)\n",
    "\n",
    "plt.imshow(cv2.cvtColor(image, cv2.COLOR_BGR2RGB))\n",
    "\n",
    "del compiled_model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f9fcaba9",
   "metadata": {},
   "source": [
    "### Load the model to GPU device and perform inference"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "175b43dd",
   "metadata": {},
   "outputs": [],
   "source": [
    "if \"GPU\" not in ie.available_devices:\n",
    "    print(f\"A GPU device is not available. Available devices are: {ie.available_devices}\")\n",
    "else :       \n",
    "    # Start time.\n",
    "    gpu_load_start_time = time.perf_counter()\n",
    "    compiled_model = ie.compile_model(model=model, device_name=\"GPU\")  # load to GPU\n",
    "\n",
    "    # Get input and output nodes.\n",
    "    input_layer = compiled_model.input(0)\n",
    "    output_layer = compiled_model.output(0)\n",
    "\n",
    "    # Execute the first inference.\n",
    "    results = compiled_model([input_image])[output_layer]\n",
    "\n",
    "    # Measure time to the first inference.\n",
    "    gpu_fil_end_time = time.perf_counter()\n",
    "    gpu_fil_span = gpu_fil_end_time - gpu_load_start_time\n",
    "    print(f\"Time to load model on GPU device and get first inference: {gpu_fil_end_time-gpu_load_start_time:.2f} seconds.\")\n",
    "    del compiled_model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09dfac5d",
   "metadata": {},
   "source": [
    "### Load the model using AUTO device and do inference\n",
    "When GPU is the best available device, the first few inferences will be executed on CPU until GPU is ready."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c6a52b3",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Start time.\n",
    "auto_load_start_time = time.perf_counter()\n",
    "compiled_model = ie.compile_model(model=model)  # The device_name is AUTO by default.\n",
    "\n",
    "# Get input and output nodes.\n",
    "input_layer = compiled_model.input(0)\n",
    "output_layer = compiled_model.output(0)\n",
    "\n",
    "# Execute the first inference.\n",
    "results = compiled_model([input_image])[output_layer]\n",
    "\n",
    "\n",
    "# Measure time to the first inference.\n",
    "auto_fil_end_time = time.perf_counter()\n",
    "auto_fil_span = auto_fil_end_time - auto_load_start_time\n",
    "print(f\"Time to load model using AUTO device and get first inference: {auto_fil_end_time-auto_load_start_time:.2f} seconds.\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "06673210-5af9-4209-acc3-7b2fed1eb165",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Deleted model will wait for compiling on the selected device to complete.\n",
    "del compiled_model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0a2de05c-6b75-4bc2-814a-08aa4924c2a7",
   "metadata": {
    "tags": []
   },
   "source": [
    "## (3) Achieve different performance for different targets\n",
    "It is an advantage to define **performance hints** when using Automatic Device Selection. By specifying a **THROUGHPUT** or **LATENCY** hint, AUTO optimizes the performance based on the desired metric. The **THROUGHPUT** hint delivers higher frame per second (FPS) performance than the **LATENCY** hint, which delivers lower latency. The performance hints do not require any device-specific settings and they are completely portable between devices – meaning AUTO can configure the performance hint on whichever device is being used.\n",
    "\n",
    "For more information, refer to the [Performance Hints](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_AUTO.html#performance-hints) section of [Automatic Device Selection](https://docs.openvino.ai/latest/openvino_docs_OV_UG_supported_plugins_AUTO.html) article.\n",
    "\n",
    "### Class and callback definition"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "37bc1059-5a3c-4d11-a340-09c224a9d5e5",
   "metadata": {},
   "outputs": [],
   "source": [
    "class PerformanceMetrics:\n",
    "    \"\"\"\n",
    "    Record the latest performance metrics (fps and latency), update the metrics in each @interval seconds\n",
    "    :member: fps: Frames per second, indicates the average number of inferences executed each second during the last @interval seconds.\n",
    "    :member: latency: Average latency of inferences executed in the last @interval seconds.\n",
    "    :member: start_time: Record the start timestamp of onging @interval seconds duration.\n",
    "    :member: latency_list: Record the latency of each inference execution over @interval seconds duration.\n",
    "    :member: interval: The metrics will be updated every @interval seconds\n",
    "    \"\"\"\n",
    "    def __init__(self, interval):\n",
    "        \"\"\"\n",
    "        Create and initilize one instance of class PerformanceMetrics.\n",
    "        :param: interval: The metrics will be updated every @interval seconds\n",
    "        :returns:\n",
    "            Instance of PerformanceMetrics\n",
    "        \"\"\"\n",
    "        self.fps = 0\n",
    "        self.latency = 0\n",
    "        \n",
    "        self.start_time = time.perf_counter()\n",
    "        self.latency_list = []\n",
    "        self.interval = interval\n",
    "        \n",
    "    def update(self, infer_request: InferRequest) -> bool:\n",
    "        \"\"\"\n",
    "        Update the metrics if current ongoing @interval seconds duration is expired. Record the latency only if it is not expired.\n",
    "        :param: infer_request: InferRequest returned from inference callback, which includes the result of inference request.\n",
    "        :returns:\n",
    "            True, if metrics are updated.\n",
    "            False, if @interval seconds duration is not expired and metrics are not updated.\n",
    "        \"\"\"\n",
    "        self.latency_list.append(infer_request.latency)\n",
    "        exec_time = time.perf_counter() - self.start_time\n",
    "        if exec_time >= self.interval:\n",
    "            # Update the performance metrics.\n",
    "            self.start_time = time.perf_counter()\n",
    "            self.fps = len(self.latency_list) / exec_time\n",
    "            self.latency = sum(self.latency_list) / len(self.latency_list)\n",
    "            print(f\"throughput: {self.fps: .2f}fps, latency: {self.latency: .2f}ms, time interval:{exec_time: .2f}s\")\n",
    "            sys.stdout.flush()\n",
    "            self.latency_list = []\n",
    "            return True\n",
    "        else :\n",
    "            return False\n",
    "\n",
    "\n",
    "class InferContext:\n",
    "    \"\"\"\n",
    "    Inference context. Record and update peforamnce metrics via @metrics, set @feed_inference to False once @remaining_update_num <=0\n",
    "    :member: metrics: instance of class PerformanceMetrics \n",
    "    :member: remaining_update_num: the remaining times for peforamnce metrics updating.\n",
    "    :member: feed_inference: if feed inference request is required or not.\n",
    "    \"\"\"\n",
    "    def __init__(self, update_interval, num):\n",
    "        \"\"\"\n",
    "        Create and initilize one instance of class InferContext.\n",
    "        :param: update_interval: The performance metrics will be updated every @update_interval seconds. This parameter will be passed to class PerformanceMetrics directly.\n",
    "        :param: num: The number of times performance metrics are updated.\n",
    "        :returns:\n",
    "            Instance of InferContext.\n",
    "        \"\"\"\n",
    "        self.metrics = PerformanceMetrics(update_interval)\n",
    "        self.remaining_update_num = num\n",
    "        self.feed_inference = True\n",
    "        \n",
    "    def update(self, infer_request: InferRequest):\n",
    "        \"\"\"\n",
    "        Update the context. Set @feed_inference to False if the number of remaining performance metric updates (@remaining_update_num) reaches 0\n",
    "        :param: infer_request: InferRequest returned from inference callback, which includes the result of inference request.\n",
    "        :returns: None\n",
    "        \"\"\"\n",
    "        if self.remaining_update_num <= 0 :\n",
    "            self.feed_inference = False\n",
    "            \n",
    "        if self.metrics.update(infer_request) :\n",
    "            self.remaining_update_num = self.remaining_update_num - 1\n",
    "            if self.remaining_update_num <= 0 :\n",
    "                self.feed_inference = False\n",
    "\n",
    "\n",
    "def completion_callback(infer_request: InferRequest, context) -> None:\n",
    "    \"\"\"\n",
    "    callback for the inference request, pass the @infer_request to @context for updating\n",
    "    :param: infer_request: InferRequest returned for the callback, which includes the result of inference request.\n",
    "    :param: context: user data which is passed as the second parameter to AsyncInferQueue:start_async()\n",
    "    :returns: None\n",
    "    \"\"\"\n",
    "    context.update(infer_request)\n",
    "\n",
    "\n",
    "# Performance metrics update interval (seconds) and number of times.\n",
    "metrics_update_interval = 10\n",
    "metrics_update_num = 6"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4fbaebca-d7a3-43bd-a3d3-aeab1e177171",
   "metadata": {},
   "source": [
    "### Inference with THROUGHPUT hint\n",
    "\n",
    "Loop for inference and update the FPS/Latency every @metrics_update_interval seconds."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94b8e571",
   "metadata": {},
   "outputs": [],
   "source": [
    "THROUGHPUT_hint_context = InferContext(metrics_update_interval, metrics_update_num)\n",
    "\n",
    "print(\"Compiling Model for AUTO device with THROUGHPUT hint\")\n",
    "sys.stdout.flush()\n",
    "\n",
    "compiled_model = ie.compile_model(model=model, config={\"PERFORMANCE_HINT\":\"THROUGHPUT\"})\n",
    "\n",
    "infer_queue = AsyncInferQueue(compiled_model, 0)  # Setting to 0 will query optimal number by default.\n",
    "infer_queue.set_callback(completion_callback)\n",
    "\n",
    "print(f\"Start inference, {metrics_update_num: .0f} groups of FPS/latency will be measured over {metrics_update_interval: .0f}s intervals\")\n",
    "sys.stdout.flush()\n",
    "\n",
    "while THROUGHPUT_hint_context.feed_inference:\n",
    "    infer_queue.start_async({input_layer_ir.any_name: input_image}, THROUGHPUT_hint_context)\n",
    "    \n",
    "infer_queue.wait_all()\n",
    "\n",
    "# Take the FPS and latency of the latest period.\n",
    "THROUGHPUT_hint_fps = THROUGHPUT_hint_context.metrics.fps\n",
    "THROUGHPUT_hint_latency = THROUGHPUT_hint_context.metrics.latency\n",
    "\n",
    "print(\"Done\")\n",
    "\n",
    "del compiled_model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a63d446b-0c7c-4f2d-baac-21d5ea92089d",
   "metadata": {},
   "source": [
    "### Inference with LATENCY hint\n",
    "\n",
    "Loop for inference and update the FPS/Latency for each @metrics_update_interval seconds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "42e0396c-d7cb-4176-844c-1d5d94ae788f",
   "metadata": {},
   "outputs": [],
   "source": [
    "LATENCY_hint_context = InferContext(metrics_update_interval, metrics_update_num)\n",
    "\n",
    "print(\"Compiling Model for AUTO Device with LATENCY hint\")\n",
    "sys.stdout.flush()\n",
    "\n",
    "compiled_model = ie.compile_model(model=model, config={\"PERFORMANCE_HINT\":\"LATENCY\"})\n",
    "\n",
    "# Setting to 0 will query optimal number by default.\n",
    "infer_queue = AsyncInferQueue(compiled_model, 0)\n",
    "infer_queue.set_callback(completion_callback)\n",
    "\n",
    "print(f\"Start inference, {metrics_update_num: .0f} groups fps/latency will be out with {metrics_update_interval: .0f}s interval\")\n",
    "sys.stdout.flush()\n",
    "\n",
    "while LATENCY_hint_context.feed_inference:\n",
    "    infer_queue.start_async({input_layer_ir.any_name: input_image}, LATENCY_hint_context)\n",
    "    \n",
    "infer_queue.wait_all()\n",
    "\n",
    "# Take the FPS and latency of the latest period.\n",
    "LATENCY_hint_fps = LATENCY_hint_context.metrics.fps\n",
    "LATENCY_hint_latency = LATENCY_hint_context.metrics.latency\n",
    "\n",
    "print(\"Done\")\n",
    "\n",
    "del compiled_model"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9f1907e4-e71b-471f-891a-66161c03bc85",
   "metadata": {},
   "source": [
    "### Difference in FPS and latency\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e3eb5f0a-a47a-4cc3-886e-347b187867a1",
   "metadata": {},
   "outputs": [],
   "source": [
    "TPUT = 0\n",
    "LAT = 1\n",
    "labels = [\"THROUGHPUT hint\", \"LATENCY hint\"]\n",
    "\n",
    "fig1, ax1 = plt.subplots(1, 1) \n",
    "fig1.patch.set_visible(False)\n",
    "ax1.axis('tight') \n",
    "ax1.axis('off') \n",
    "\n",
    "cell_text = []\n",
    "cell_text.append(['%.2f%s' % (THROUGHPUT_hint_fps,\" FPS\"), '%.2f%s' % (THROUGHPUT_hint_latency, \" ms\")])\n",
    "cell_text.append(['%.2f%s' % (LATENCY_hint_fps,\" FPS\"), '%.2f%s' % (LATENCY_hint_latency, \" ms\")])\n",
    "\n",
    "table = ax1.table(cellText=cell_text, colLabels=[\"FPS (Higher is better)\", \"Latency (Lower is better)\"], rowLabels=labels,  \n",
    "                  rowColours=[\"deepskyblue\"] * 2, colColours=[\"deepskyblue\"] * 2,\n",
    "                  cellLoc='center', loc='upper left')\n",
    "table.auto_set_font_size(False)\n",
    "table.set_fontsize(18)\n",
    "table.auto_set_column_width(0)\n",
    "table.auto_set_column_width(1)\n",
    "table.scale(1, 3)\n",
    "\n",
    "fig1.tight_layout()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "86f5261e-581e-4b91-a8fb-9715981d450e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Output the difference.\n",
    "width = 0.4\n",
    "fontsize = 14\n",
    "\n",
    "plt.rc('font', size=fontsize)\n",
    "fig, ax = plt.subplots(1,2, figsize=(10, 8))\n",
    "\n",
    "rects1 = ax[0].bar([0], THROUGHPUT_hint_fps, width, label=labels[TPUT], color='#557f2d')\n",
    "rects2 = ax[0].bar([width], LATENCY_hint_fps, width, label=labels[LAT])\n",
    "ax[0].set_ylabel(\"frames per second\")\n",
    "ax[0].set_xticks([width / 2]) \n",
    "ax[0].set_xticklabels([\"FPS\"])\n",
    "ax[0].set_xlabel(\"Higher is better\")\n",
    "\n",
    "rects1 = ax[1].bar([0], THROUGHPUT_hint_latency, width, label=labels[TPUT], color='#557f2d')\n",
    "rects2 = ax[1].bar([width], LATENCY_hint_latency, width, label=labels[LAT])\n",
    "ax[1].set_ylabel(\"milliseconds\")\n",
    "ax[1].set_xticks([width / 2])\n",
    "ax[1].set_xticklabels([\"Latency (ms)\"])\n",
    "ax[1].set_xlabel(\"Lower is better\")\n",
    "\n",
    "fig.suptitle('Performance Hints')\n",
    "fig.legend(labels, fontsize=fontsize)\n",
    "fig.tight_layout()\n",
    "\n",
    "plt.show()"
   ]
  }
 ],
 "metadata": {
  "interpreter": {
   "hash": "ae617ccb002f72b3ab6d0069d721eac67ac2a969e83c083c4321cfcab0437cd1"
  },
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.10"
  },
  "toc-autonumbering": false,
  "toc-showmarkdowntxt": false,
  "toc-showtags": false
 },
 "nbformat": 4,
 "nbformat_minor": 5
}