api-entities/sources/search-sources.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "[![GitHub Repository](https://img.shields.io/badge/GitHub-Repository-181717?style=for-the-badge&logo=GitHub&link=https://github.com/Mearman/openalex-docs/tree/main)](https://github.com/Mearman/openalex-docs/tree/main)[![Open in GitHub](https://img.shields.io/badge/Open%20in-GitHub-181717?style=for-the-badge&logo=github&link=https://github.com/Mearman/openalex-docs/blob/main/api-entities/sources/search-sources.ipynb)](https://github.com/Mearman/openalex-docs/blob/main/api-entities/sources/search-sources.ipynb)[![Open in Colab](https://img.shields.io/badge/Open%20in-Colab-F9AB00?style=for-the-badge&logo=Google%20Colab&link=https://colab.research.google.com/github/Mearman/openalex-docs/blob/main/api-entities/sources/search-sources.ipynb)](https://colab.research.google.com/github/Mearman/openalex-docs/blob/main/api-entities/sources/search-sources.ipynb)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "%pip install --upgrade \"git+https://github.com/Mearman/openalex-python-pydantic-v1.git\"\n",
        "%pip install pandasai"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "import json\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "from openalex_api import Configuration, ApiClient, AutocompleteApi, AuthorsApi, ConceptsApi, FundersApi, InstitutionsApi, PublishersApi, SourcesApi, WorksApi\n",
        "\n",
        "configuration = Configuration(host=\"https://api.openalex.org\")\n",
        "autocomplete_api = AutocompleteApi(ApiClient(configuration))\n",
        "authors_api = AuthorsApi(ApiClient(configuration))\n",
        "concepts_api = ConceptsApi(ApiClient(configuration))\n",
        "funders_api = FundersApi(ApiClient(configuration))\n",
        "institutions_api = InstitutionsApi(ApiClient(configuration))\n",
        "publishers_api = PublishersApi(ApiClient(configuration))\n",
        "sources_api = SourcesApi(ApiClient(configuration))\n",
        "works_api = WorksApi(ApiClient(configuration))\n",
        "\n",
        "from pandasai import SmartDataframe\n",
        "from pandasai.llm import OpenAI"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title  { run: \"auto\", display-mode: \"form\" }\n",
        "openapi_token = \"\" # @param {type:\"string\"}"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Search sources\n",
        "\n",
        "The best way to search for sources is to use the `search` query parameter, which searches across [`display_name`](./source-object.md#display_name), [`alternate_titles`](./source-object.md#alternate_titles), and [`abbreviated_title`](./source-object.md#abbreviated_title). Example:\n",
        "\n",
        "* Search for the abbreviated version of the Journal of the American Chemical Society \"`jacs`\":\n",
        "  [https://api.openalex.org/sources?search=jacs](https://api.openalex.org/sources?search=jacs)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/sources?search=jacs\n",
        "search=\"jacs\" # @param {type: \"string\"}\n",
        "\n",
        "response = sources_api.get_sources(\n",
        "\tsearch=search\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response.results)\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "numeric_df = df[['id', 'display_name'] +\n",
        "\t[col for col in df.columns if df[col].dtype in ['int64', 'float64'] and col != 'relevance_score']]\n",
        "display(numeric_df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "try:\n",
        "\tprompt = \"Visualize this data\" # @param {type:\"string\"}\n",
        "\tSmartDataframe(\n",
        "\t\tnumeric_df,\n",
        "\t\tconfig={\"llm\": (OpenAI(api_token=openapi_token))},\n",
        "\t).chat(prompt)\n",
        "except:\n",
        "\tif not openapi_token:\n",
        "\t\tprint(\"Error: openapi_token not set\")\n",
        "\telse:\n",
        "\t\tprint(\"Error when creating SmartDataframe\")"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "{% hint style=\"info\" %}\n",
        "You can read more about search [here](./../../how-to-use-the-api/get-lists-of-entities/search-entities.ipynb). It will show you how relevance score is calculated, how words are stemmed to improve search results, and how to do complex boolean searches.\n",
        "{% endhint %}\n",
        "\n",
        "## Search a specific field\n",
        "\n",
        "You can also use search as a [filter](./../../how-to-use-the-api/get-lists-of-entities/filter-entity-lists.ipynb), allowing you to fine-tune the fields you're searching over. To do this, you append `.search` to the end of the property you are filtering for:\n",
        "\n",
        "* Get sources with \"nature\" in the title:\n",
        "  [https://api.openalex.org/sources?filter=display_name.search:nature](https://api.openalex.org/sources?filter=display_name.search:nature)"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/sources?filter=display_name.search:nature\n",
        "filter=\"display_name.search:nature\" # @param {type: \"string\"}\n",
        "\n",
        "response = sources_api.get_sources(\n",
        "\tfilter=filter\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response.results)\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "numeric_df = df[['id', 'display_name'] +\n",
        "\t[col for col in df.columns if df[col].dtype in ['int64', 'float64'] and col != 'relevance_score']]\n",
        "display(numeric_df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "try:\n",
        "\tprompt = \"Visualize this data\" # @param {type:\"string\"}\n",
        "\tSmartDataframe(\n",
        "\t\tnumeric_df,\n",
        "\t\tconfig={\"llm\": (OpenAI(api_token=openapi_token))},\n",
        "\t).chat(prompt)\n",
        "except:\n",
        "\tif not openapi_token:\n",
        "\t\tprint(\"Error: openapi_token not set\")\n",
        "\telse:\n",
        "\t\tprint(\"Error when creating SmartDataframe\")"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The following fields can be searched as a filter within sources:\n",
        "\n",
        "| Search filter                                                  | Field that is searched                          |\n",
        "| -------------------------------------------------------------- | ----------------------------------------------- |\n",
        "| [`display_name.search`](./filter-sources.md#display_name.search) | [`display_name`](./source-object.md#display_name) |\n",
        "\n",
        "You can also use the filter `default.search`, which works the same as using the [`search` parameter](./README.md#search-sources).\n",
        "\n",
        "## Autocomplete sources\n",
        "\n",
        "You can autocomplete sources to create a very fast type-ahead style search function:\n",
        "\n",
        "* Autocomplete sources with \"neuro\" in the display_name:\n",
        "  https://api.openalex.org/autocomplete/sources?q=neuro"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "# @title { run: \"auto\", vertical-output: false }\n",
        "# https://api.openalex.org/autocomplete/sources?q=neuro\n",
        "q=\"neuro\" # @param {type: \"string\"}\n",
        "\n",
        "response = autocomplete_api.get_autocomplete_sources(\n",
        "\tq=q\n",
        ")\n",
        "\n",
        "df = pd.DataFrame(response.results)\n",
        "display(df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "numeric_df = df[['id', 'display_name'] +\n",
        "\t[col for col in df.columns if df[col].dtype in ['int64', 'float64'] and col != 'relevance_score']]\n",
        "display(numeric_df)"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "code",
      "metadata": {},
      "source": [
        "try:\n",
        "\tprompt = \"Visualize this data\" # @param {type:\"string\"}\n",
        "\tSmartDataframe(\n",
        "\t\tnumeric_df,\n",
        "\t\tconfig={\"llm\": (OpenAI(api_token=openapi_token))},\n",
        "\t).chat(prompt)\n",
        "except:\n",
        "\tif not openapi_token:\n",
        "\t\tprint(\"Error: openapi_token not set\")\n",
        "\telse:\n",
        "\t\tprint(\"Error when creating SmartDataframe\")"
      ],
      "outputs": [],
      "execution_count": null
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "This returns a list of sources with the publisher set as the hint:\n",
        "\n",
        "<pre class=\"language-json\"><code class=\"lang-json\">{ \n",
        "  \"results\": [\n",
        "    {\n",
        "        \"id\": \"https://openalex.org/S5555990\",\n",
        "        \"display_name\": \"The Journal of Neuroscience\",\n",
        "        \"hint\": \"Society for Neuroscience\",\n",
        "        \"cited_by_count\": 4274712,\n",
        "        \"works_count\": 40376,\n",
        "        \"entity_type\": \"source\",\n",
        "        \"external_id\": \"0270-6474\"\n",
        "    },\n",
        "    // more results\n",
        "<strong>  ]\n",
        "</strong><strong>}\n",
        "</strong></code></pre>\n",
        "\n",
        "{% hint style=\"info\" %}\n",
        "Read more in the [autocomplete page](./../../how-to-use-the-api/get-lists-of-entities/autocomplete-entities.ipynb) in the API guide.\n",
        "{% endhint %}"
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": "Python 3",
      "language": "python",
      "name": "python3"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}