Part 0.1: Regex.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    },
    "colab": {
      "name": "Part 0.1: Regex.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "include_colab_link": true
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/gulabpatel/NLP_Basics/blob/main/Part%200.1%3A%20Regex.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_AFHDSmhpKAk"
      },
      "source": [
        "Regex 101 : https://regex101.com/\n",
        "\n",
        "Code walkthrough video : https://www.youtube.com/watch?v=sHw5hLYFaIw\n",
        "\n"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "LRbmziO9eYpl"
      },
      "source": [
        "import re"
      ],
      "execution_count": 1,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "gZIvwPkCeYps"
      },
      "source": [
        "<h3>Extract phone numbers</h3>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "AmH1OxJneYpx",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "89091b32-48e1-4274-e9a3-4e5bdc4a9dce"
      },
      "source": [
        "text='''\n",
        "Elon musk's phone number is 9991116666, call him if you have any questions on dodgecoin. Tesla's revenue is 40 billion\n",
        "Tesla's CFO number (999)-333-7777\n",
        "'''\n",
        "pattern = '\\(\\d{3}\\)-\\d{3}-\\d{4}|\\d{10}'\n",
        "\n",
        "matches = re.findall(pattern, text)\n",
        "matches"
      ],
      "execution_count": 2,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['9991116666', '(999)-333-7777']"
            ]
          },
          "metadata": {},
          "execution_count": 2
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "AHd-6M07eYp0"
      },
      "source": [
        "<h3>Extract Note Titles</h3>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "nnMbBQ9ceYp1"
      },
      "source": [
        "<img src='https://github.com/codebasics/py/blob/master/Advanced/regex/tesla_report_notes.jpg?raw=1' />"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "FveIZw0JeYp1"
      },
      "source": [
        "text = '''\n",
        "Note 1 - Overview\n",
        "Tesla, Inc. (“Tesla”, the “Company”, “we”, “us” or “our”) was incorporated in the State of Delaware on July 1, 2003. We design, develop, manufacture and sell high-performance fully electric vehicles and design, manufacture, install and sell solar energy generation and energy storage\n",
        "products. Our Chief Executive Officer, as the chief operating decision maker (“CODM”), organizes our company, manages resource allocations and measures performance among two operating and reportable segments: (i) automotive and (ii) energy generation and storage.\n",
        "Beginning in the first quarter of 2021, there has been a trend in many parts of the world of increasing availability and administration of vaccines\n",
        "against COVID-19, as well as an easing of restrictions on social, business, travel and government activities and functions. On the other hand, infection\n",
        "rates and regulations continue to fluctuate in various regions and there are ongoing global impacts resulting from the pandemic, including challenges\n",
        "and increases in costs for logistics and supply chains, such as increased port congestion, intermittent supplier delays and a shortfall of semiconductor\n",
        "supply. We have also previously been affected by temporary manufacturing closures, employment and compensation adjustments and impediments to\n",
        "administrative activities supporting our product deliveries and deployments.\n",
        "Note 2 - Summary of Significant Accounting Policies\n",
        "Unaudited Interim Financial Statements\n",
        "The consolidated balance sheet as of September 30, 2021, the consolidated statements of operations, the consolidated statements of\n",
        "comprehensive income, the consolidated statements of redeemable noncontrolling interests and equity for the three and nine months ended September\n",
        "30, 2021 and 2020 and the consolidated statements of cash flows for the nine months ended September 30, 2021 and 2020, as well as other information\n",
        "disclosed in the accompanying notes, are unaudited. The consolidated balance sheet as of December 31, 2020 was derived from the audited\n",
        "consolidated financial statements as of that date. The interim consolidated financial statements and the accompanying notes should be read in\n",
        "conjunction with the annual consolidated financial statements and the accompanying notes contained in our Annual Report on Form 10-K for the year\n",
        "ended December 31, 2020.\n",
        "'''"
      ],
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "wrnVrskUeYp6",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "73caaee5-c6a6-4b06-e85c-4e75e6b7918b"
      },
      "source": [
        "pattern = 'Note \\d - ([^\\n]*)'\n",
        "matches = re.findall(pattern, text)\n",
        "matches"
      ],
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Overview', 'Summary of Significant Accounting Policies']"
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Kdr0-VjeeYp-"
      },
      "source": [
        "<h3>Extract financial periods from a company's financial reporting</h3>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "_WG4tdUHeYqA",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ad2efdbd-4d07-4d12-edc3-63eac78f5632"
      },
      "source": [
        "text = '''\n",
        "The gross cost of operating lease vehicles in FY2021 Q1 was $4.85 billion.\n",
        "In previous quarter i.e. FY2020 Q4 it was $3 billion. \n",
        "'''\n",
        "\n",
        "pattern = 'FY\\d{4} Q[1-4]'\n",
        "\n",
        "matches = re.findall(pattern, text)\n",
        "matches"
      ],
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['FY2021 Q1', 'FY2020 Q4']"
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "xJMJY72QeYqD"
      },
      "source": [
        "**Case insensitive pattern match using flags**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "WqXIQ3LweYqE",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "8a48fc64-116e-47f4-d489-38f6174d7746"
      },
      "source": [
        "text = '''\n",
        "The gross cost of operating lease vehicles in FY2021 Q1 was $4.85 billion.\n",
        "In previous quarter i.e. fy2020 Q4 it was $3 billion. \n",
        "'''\n",
        "\n",
        "pattern = 'FY\\d{4} Q[1-4]'\n",
        "\n",
        "matches = re.findall(pattern, text, flags=re.IGNORECASE)\n",
        "matches"
      ],
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['FY2021 Q1', 'fy2020 Q4']"
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "f702JPQReYqE"
      },
      "source": [
        "<h3>Extract only financial numbers</h3>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "QqpAM7YbeYqH",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "ef0ac42d-281c-4957-bc7c-f2d3b2333032"
      },
      "source": [
        "text = '''\n",
        "Tesla's gross cost of operating lease vehicles in FY2021 Q1 was $4.85 billion. \n",
        "In previous quarter i.e. FY2020 Q4 it was $3 billion.\n",
        "'''\n",
        "\n",
        "pattern = '\\$([0-9\\.]+)'\n",
        "matches = re.findall(pattern, text)\n",
        "matches"
      ],
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['4.85', '3']"
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Vlc2IGjIeYqI"
      },
      "source": [
        "<h3>Extract periods and financial numbers both</h3>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "7JNFtBzIeYqI",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "032917a3-d6d5-4738-ec24-535f3924ae39"
      },
      "source": [
        "text = '''\n",
        "Tesla's gross cost of operating lease vehicles in FY2021 Q1 was $4.85 billion. \n",
        "In previous quarter i.e. FY2020 Q4 it was $3 billion.\n",
        "'''\n",
        "pattern = 'FY(\\d{4} Q[1-4])[^\\$]+\\$([\\d\\.]+)'\n",
        "\n",
        "matches = re.findall(pattern, text)\n",
        "matches"
      ],
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "[('2021 Q1', '4.85'), ('2020 Q4', '3')]"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "diUCezFAeYqJ"
      },
      "source": [
        "<h3>re.search</h3>"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "d-PygifHeYqK",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "a78cb2c3-41b7-4395-c8f6-7135e90a9102"
      },
      "source": [
        "text = '''\n",
        "Tesla's gross cost of operating lease vehicles in FY2021 Q1 ljh lsj a 123 was $4.85 billion. Same number for FY2020 Q4 was $8 billion\n",
        "'''\n",
        "pattern = 'FY(\\d{4} Q[1-4])[^\\$]+\\$([0-9\\.]+)'\n",
        "\n",
        "matches = re.search(pattern, text)\n",
        "matches"
      ],
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "<re.Match object; span=(51, 84), match='FY2021 Q1 ljh lsj a 123 was $4.85'>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "3Cnt36L3eYqL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "6dd5a6a6-205b-48de-91f7-b25db4c2f848"
      },
      "source": [
        "matches.groups()"
      ],
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "('2021 Q1', '4.85')"
            ]
          },
          "metadata": {},
          "execution_count": 10
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "ofzsG38deYp4"
      },
      "source": [
        "**1. Extract all twitter handles from following text. Twitter handle is the text that appears after https://twitter.com/ and is a single word. Also it contains only alpha numeric characters i.e. A-Z a-z , o to 9 and underscore _**"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "scrolled": true,
        "id": "R664a3zxeYp7",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "3fece103-4fc0-457d-b7f6-a412897f4b96"
      },
      "source": [
        "text = '''\n",
        "Follow our leader Elon musk on twitter here: https://twitter.com/elonmusk, more information \n",
        "on Tesla's products can be found at https://www.tesla.com/. Also here are leading influencers \n",
        "for tesla related news,\n",
        "https://twitter.com/teslarati\n",
        "https://twitter.com/dummy_tesla\n",
        "https://twitter.com/dummy_2_tesla\n",
        "'''\n",
        "pattern = 'https://twitter\\.com/([a-zA-Z0-9_]+)'\n",
        "\n",
        "re.findall(pattern, text)"
      ],
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['elonmusk', 'teslarati', 'dummy_tesla', 'dummy_2_tesla']"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qatSy43XeYp-"
      },
      "source": [
        "**2. Extract Concentration Risk Types. It will be a text that appears after \"Concentration Risk:\", In below example, your regex should extract these two strings**\n",
        "\n",
        "(1) Credit Risk\n",
        "\n",
        "(2) Supply Rish"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "IEmhbBh_eYp_",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "196b18ee-f695-4c7e-d553-2de36d7f1cf2"
      },
      "source": [
        "text = '''\n",
        "Concentration of Risk: Credit Risk\n",
        "Financial instruments that potentially subject us to a concentration of credit risk consist of cash, cash equivalents, marketable securities,\n",
        "restricted cash, accounts receivable, convertible note hedges, and interest rate swaps. Our cash balances are primarily invested in money market funds\n",
        "or on deposit at high credit quality financial institutions in the U.S. These deposits are typically in excess of insured limits. As of September 30, 2021\n",
        "and December 31, 2020, no entity represented 10% or more of our total accounts receivable balance. The risk of concentration for our convertible note\n",
        "hedges and interest rate swaps is mitigated by transacting with several highly-rated multinational banks.\n",
        "Concentration of Risk: Supply Risk\n",
        "We are dependent on our suppliers, including single source suppliers, and the inability of these suppliers to deliver necessary components of our\n",
        "products in a timely manner at prices, quality levels and volumes acceptable to us, or our inability to efficiently manage these components from these\n",
        "suppliers, could have a material adverse effect on our business, prospects, financial condition and operating results.\n",
        "'''\n",
        "pattern = 'Concentration of Risk: ([^\\n]*)'\n",
        "\n",
        "re.findall(pattern, text)"
      ],
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['Credit Risk', 'Supply Risk']"
            ]
          },
          "metadata": {},
          "execution_count": 12
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "VicDW8OoeYqA"
      },
      "source": [
        "**3. Companies in europe reports their financial numbers of semi annual basis and you can have a document like this. To exatract quarterly and semin annual period you can use a regex as shown below**\n",
        "\n",
        "Hint: you need to use (?:) here to match everything enclosed"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "HZIjTLYweYqB",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "b5a82cac-cfd7-4e0c-f870-a4def5a8f90a"
      },
      "source": [
        "text = '''\n",
        "Tesla's gross cost of operating lease vehicles in FY2021 Q1 was $4.85 billion.\n",
        "BMW's gross cost of operating vehicles in FY2021 S1 was $8 billion.\n",
        "'''\n",
        "\n",
        "pattern = 'FY(\\d{4} (?:Q[1-4]|S[1-2]))'\n",
        "matches = re.findall(pattern, text)\n",
        "matches"
      ],
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "['2021 Q1', '2021 S1']"
            ]
          },
          "metadata": {},
          "execution_count": 14
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "Hb_YF4yFneKu"
      },
      "source": [
        ""
      ],
      "execution_count": null,
      "outputs": []
    }
  ]
}