03b_Image_Display_with_Firefly.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img align=\"left\" src = https://project.lsst.org/sites/default/files/Rubin-O-Logo_0.png width=250 style=\"padding: 10px\"> \n",
    "<br>\n",
    "<b>Displaying images using Firefly</b> <br>\n",
    "Last verified to run on 2022-05-31 with LSST Science Pipelines release w_2022_22<br>\n",
    "Contact authors: Jeff Carlin <br>\n",
    "Target audience: All DP0 delegates. <br>\n",
    "Container Size: medium <br>\n",
    "Questions welcome at <a href=\"https://community.lsst.org/c/support/dp0\">community.lsst.org/c/support/dp0</a> <br>\n",
    "Find DP0 documentation and resources at <a href=\"https://dp0-1.lsst.io\">dp0-1.lsst.io</a> <br>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Credit:** This tutorial is based in part on the `Firefly.ipynb` notebook that is available under your home directory, in `notebooks/notebook-demo/`.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Learning Objectives\n",
    "\n",
    "This notebook is a counterpart to tuturial notebook `03_Image_Display_and_Manipulation.ipynb`, and is intended to demonstrate the [Firefly](https://pipelines.lsst.io/v/daily/modules/lsst.display.firefly/index.html) interactive interface for viewing image data. As in the complementary \"Image display\" notebook, we will use the Butler to access images. We will also use the `lsst.afw.display` library to display images, but in this case with a different \"back end\" -- the interactive Firefly display tool.\n",
    "\n",
    "### Set Up"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# What version of the Stack are we using?\n",
    "! echo $IMAGE_DESCRIPTION\n",
    "! eups list -s | grep lsst_distrib"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 1. Load the LSST Science Pipelines\n",
    "\n",
    "First, we load the `lsst.afw.display` library to gain access to the image visualization routines we'd like to use, and the `lsst.daf.butler` library, which is used to access data products."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load lsst.afw.display to gain access to image visualization routines.\n",
    "import lsst.afw.display as afwDisplay\n",
    "# load the Butler, which provides programmatic access to LSST data products.\n",
    "from lsst.daf.butler import Butler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let the kernel know that we're happy not to have some useful warnings printed during this tutorial."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# prevent some helpful but ancillary warning messages from printing\n",
    "#   during some LSST DM Release calls\n",
    "import warnings\n",
    "warnings.simplefilter(\"ignore\", category=UserWarning)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### 2. Load the Data to Visualize\n",
    "\n",
    "To display an image, we must first load some data. These data have been processed with the LSST Science Pipelines, and are organized in a structure that enables us to access them through the `Butler`. For more information on the `Butler`, see [lsst.daf.butler](https://pipelines.lsst.io/modules/lsst.daf.butler/index.html).\n",
    "\n",
    "The DP0.1 data set contains simulated images from the LSST DESC Data Challenge 2 (DC2). These data are available in an S3 bucket `s3://butler-us-central1-dp01`. We access a single image from a specific visit (`192350`) and detector (`175`).\n",
    "\n",
    "Once we define a string that contains the data directory, we start the `Butler` instance using the `lsst.daf.butler` library and its `Butler` class. The `Butler` object is initialized with a string containing the data directory we wish to access. Running the cell may take a few moments.\n",
    "\n",
    "With the `Butler` instance now generated using our data directory, we can retrieve the desired calibrated exposure by telling the butler which filter (\"band\"), CCD (\"detector\"), and visit we wish to view. To do this, we define a dictionary with the required information."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataId = {'visit': 192350, 'detector': 175, 'band': 'i'}\n",
    "# Note: because the combination of visit+detector already uniquely identifies\n",
    "# the exposure, specifying \"band\" above is unnecessary.\n",
    "\n",
    "repo = 's3://butler-us-central1-dp01'\n",
    "collection = '2.2i/runs/DP0.1'\n",
    "butler = Butler(repo, collections=collection)\n",
    "\n",
    "# Retrieve the data using the `butler` instance and its function `get()`\n",
    "calexp = butler.get('calexp', **dataId)\n",
    "\n",
    "# Note: This will trigger a warning from CFITSIO in w_2022_22. This warning can be safely ignored and will be corrected in the future."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Display\n",
    "\n",
    "To display the `calexp` you will use the LSST `afwDisplay` framework. It provides a uniform API for multiple display backends, including DS9, matplotlib, and LSST’s Firefly viewer. The default backend is `ds9`, but since we are working remotely on `JupyterLab` we would prefer to use the web-based Firefly display. A [user guide](https://pipelines.lsst.io/v/daily/modules/lsst.display.firefly/index.html)  for `lsst.display.firefly` is available on the [pipelines.lsst.io](https://pipelines.lsst.io/v/daily) site.\n",
    "\n",
    "Now, we create a Firefly display."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "afwDisplay.setDefaultBackend('firefly')\n",
    "afw_display = afwDisplay.Display(frame=1)\n",
    "afw_display.getClient().show_lab_tab()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the Science Platform Notebook aspect, a Firefly viewer tab appears. You may wish to drag it to the right side of the JupyterLab area to create two side by side panes, one with the notebook and one with the display."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Display the calexp (calibrated exposure)\n",
    "\n",
    "We can now build the display and use the `mtv` method to view the `calexp` with Firefly. First we display an image with mask planes and then overplot some sources."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "afw_display.mtv(calexp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As soon as you execute the command a single calibrated HSC exposure for the `{'band': 'i', 'detector': 175, 'visit': 192350}` data ID should appear in the Firefly `JupyterLab` tab.\n",
    "\n",
    "Notice that the image is overlaid with colorful regions. These are mask regions. Each color reflects a different mask bit that correspond to detections and different types of detector artifacts. You’ll learn how to interpret these colors later, but first you’ll likely want to adjust the image display."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Improving the image display\n",
    "\n",
    "The display framework gives you control over the image display to help bring out image details. \n",
    "For example, to make masked regions semi-transparent, so that underlying image features are visible, try:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "afw_display.setMaskTransparency(80)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The setMaskTransparency method’s argument can range from 0 (fully opaque) to 100 (fully transparent).\n",
    "\n",
    "You can also control the colorbar scaling algorithm with the display’s scale method. Try an asinh stretch with explicit minimum (black) and maximum (white) values:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "afw_display.scale(\"asinh\", -5, 20)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can also use an automatic algorithm like `zscale` (or `minmax`) to select the white and black thresholds:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "afw_display.scale(\"asinh\", \"zscale\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Interpreting displayed mask colors\n",
    "\n",
    "The display framework renders each plane of the mask in a different color (plane being a different bit in the mask). To interpret these colors you can get a dictionary of mask planes from the `calexp` and query the display for the colors it rendered each mask plane with. For example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "mask = calexp.getMask()\n",
    "for maskName, maskBit in mask.getMaskPlaneDict().items():\n",
    "    print('{}: {}'.format(maskName, afw_display.getMaskPlaneColor(maskName)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In the Firefly viewer tab, the <a href=\"https://irsa.ipac.caltech.edu/onlinehelp/finderchart/finderchart/img/layers.png\">overlays button</a> on the toolbar gives you detailed control over the mask planes, such as turning individual planes on and off, changing the color and adjusting the transparency. Mask transparency and colors can also be set using `afw.display` commands, for individual planes or for all."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting sources on the display\n",
    "\n",
    "The measurements for all sources detected in the `calexp` image by the LSST science pipelines are in a dataset type `src`. Let's use the Butler to get the `src` catalog corresponding to the same `dataId` as the image we displayed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "src = butler.get('src', **dataId)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The returned object, `src`, is a `lsst.afw.table.SourceTable` object. `SourceTables` are explored more elsewhere, but you can do some simple investigations using common python functions. For example, to check the length of the object:\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "len(src)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can view an HTML rendering of the `src` table by getting an `astropy.table.Table` version of it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "src.asAstropy()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we’ll overplot sources from the `src` table onto the image display using the Display’s `dot` method for plotting markers. `Display.dot` plots markers individually, so you’ll need to iterate over rows in the `SourceTable`.  It is more efficient to send a batch of updates to the display, so we enclose the loop in a `display.Buffering` context, like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "with afw_display.Buffering():\n",
    "    for record in src:\n",
    "        afw_display.dot('o', record.getX(), record.getY(), size=20, ctype='orange')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Clearing markers\n",
    "\n",
    "`Display.dot` always adds new markers to the display. To clear the display of all markers, use the erase method:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "afw_display.erase()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Display a coadd image\n",
    "\n",
    "\"Coadds\" are the images made by combining multiple overlapping `calexp`s to create a deeper image. Let's look at the one of them (in fact, the same coadd that was examined in the related \"Image Display\" notebook). The Butler dataset type is called a `deepCoadd_calexp`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataId_coadd = {'tract': 4431, 'patch': 17, 'band': 'i'}\n",
    "\n",
    "# Retrieve the data using the `butler` instance and its function `get()`\n",
    "coadd_calexp = butler.get('deepCoadd_calexp', **dataId_coadd)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will also grab the measurements for sources in this coadd (called `deepCoadd_forced_src` because it is \"forced\" photometry at the positions of all detected objects), and the \"reference table\" that contains merged information about all detected objects from all filters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "forced_src = butler.get('deepCoadd_forced_src', **dataId_coadd)\n",
    "refTable = butler.get('deepCoadd_ref', **dataId_coadd)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display the coadd_calexp image in Firefly:\n",
    "afw_display.mtv(coadd_calexp)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Plotting sources on the display\n",
    "\n",
    "As we did for the `calexp`, let's display sources on top of the `deepCoadd_calexp` image. We will apply three selection criteria:\n",
    "\n",
    "1. Spatially select objects that are near the bottom left corner of the image (at X<1000 and Y<1000, where X and Y are the pixel values. (This is just so that the overplotting will go faster.)\n",
    "\n",
    "2. Use the `detect_isPrimary` flag to select non-duplicate objects that have been deblended (see [this section of the pipelines tutorial](https://pipelines.lsst.io/getting-started/multiband-analysis.html#the-go-to-flag-detect-isprimary) for details).\n",
    "\n",
    "3. Use one of the flags from the reference table that tells you whether a source was measured to be \"extended\" (like a galaxy). The `base_ClassificationExtendedness_value` is set to 1 for extended sources (galaxies) and 0 for point sources (stars). Again, see the tutorial section linked above for more about this flag and an example of its usage.\n",
    "\n",
    "Note that if you want to see all of the columns in the `forced_src` or `refTable` catalogs, you can use the method \"forced_src.schema\" or \"refTable.schema\"."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Note: because this \"patch\" is part of a larger \"tract,\"\n",
    "#   the (X, Y) coordinates of the lower-left corner are not (0, 0).\n",
    "#   We need to get the coordinates of that corner and subtract them off.\n",
    "xy0 = coadd_calexp.getXY0()\n",
    "xPos = refTable[\"base_SdssCentroid_x\"] - xy0.getX()\n",
    "yPos = refTable[\"base_SdssCentroid_y\"] - xy0.getY()\n",
    "\n",
    "boxSelect = (xPos < 1000) & (yPos < 1000)\n",
    "\n",
    "isPrimary = refTable['detect_isPrimary']\n",
    "\n",
    "isStellar = refTable['base_ClassificationExtendedness_value'] < 1."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display stars with orange circles:\n",
    "\n",
    "with afw_display.Buffering():\n",
    "    for record in forced_src[boxSelect & isPrimary & isStellar]:\n",
    "        afw_display.dot('o', record.getX(), record.getY(), size=20, ctype='orange')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Display galaxies (really, just anything that is \"extended\") with blue circles:\n",
    "\n",
    "with afw_display.Buffering():\n",
    "    for record in forced_src[boxSelect & isPrimary & ~isStellar]:\n",
    "        afw_display.dot('o', record.getX(), record.getY(), size=20, ctype='blue')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This tutorial was a basic introduction to displaying and manipulating images with Firefly. Enjoy exploring the DP0 images and catalogs!\n",
    "\n",
    "### Additional Documentation\n",
    "\n",
    "If you'd like some more information on `lsst.afw.display`, please have a look at the following websites:\n",
    "\n",
    "* [Firefly user guide](https://pipelines.lsst.io/v/daily/modules/lsst.display.firefly/index.html)\n",
    "* [Getting Started on Image Display (pipelines.lsst.io)](https://pipelines.lsst.io/getting-started/display.html)\n",
    "\n",
    "* [afw.display Doxygen website](http://doxygen.lsst.codes/stack/doxygen/x_masterDoxyDoc/namespacelsst_1_1afw_1_1display.html)  \n",
    "* [afw.display GitHub website](https://github.com/lsst/afw/tree/master/python/lsst/afw/display)  \n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "LSST",
   "language": "python",
   "name": "lsst"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}