Skip to content

Commit

Permalink
docs: ElasticsearchCache in cache integrations documentation (#20790)
Browse files Browse the repository at this point in the history
The package for LangChain integrations with Elasticsearch
https://github.com/langchain-ai/langchain-elastic is going to contain a
LLM cache integration in the next release (see
langchain-ai/langchain-elastic#14). This is the
documentation contribution on the page dedicated to cache integrations
  • Loading branch information
giacbrd authored Apr 26, 2024
1 parent 8c085fc commit 61f14f0
Showing 1 changed file with 151 additions and 12 deletions.
163 changes: 151 additions & 12 deletions docs/docs/integrations/llms/llm_caching.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -1428,6 +1428,15 @@
},
{
"cell_type": "code",
"execution_count": 82,
"id": "14ca942820e8140c",
"metadata": {
"ExecuteTime": {
"end_time": "2024-03-12T00:12:57.462226Z",
"start_time": "2024-03-12T00:12:55.166201Z"
},
"collapsed": false
},
"outputs": [
{
"name": "stdout",
Expand All @@ -1439,7 +1448,9 @@
},
{
"data": {
"text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'"
"text/plain": [
"'\\n\\nWhy was the math book sad? Because it had too many problems.'"
]
},
"execution_count": 82,
"metadata": {},
Expand All @@ -1450,16 +1461,7 @@
"%%time\n",
"# The first time, it is not yet in cache, so it should take longer\n",
"llm(\"Tell me a joke\")"
],
"metadata": {
"collapsed": false,
"ExecuteTime": {
"end_time": "2024-03-12T00:12:57.462226Z",
"start_time": "2024-03-12T00:12:55.166201Z"
}
},
"id": "14ca942820e8140c",
"execution_count": 82
]
},
{
"cell_type": "code",
Expand All @@ -1482,7 +1484,9 @@
},
{
"data": {
"text/plain": "'\\n\\nWhy was the math book sad? Because it had too many problems.'"
"text/plain": [
"'\\n\\nWhy was the math book sad? Because it had too many problems.'"
]
},
"execution_count": 83,
"metadata": {},
Expand All @@ -1495,6 +1499,141 @@
"llm(\"Tell me a joke\")"
]
},
{
"cell_type": "markdown",
"id": "306ff47b",
"metadata": {},
"source": [
"## `Elasticsearch` Cache\n",
"A caching layer for LLMs that uses Elasticsearch.\n",
"\n",
"First install the LangChain integration with Elasticsearch."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "9ee5cd3e",
"metadata": {},
"outputs": [],
"source": [
"%pip install -U langchain-elasticsearch"
]
},
{
"cell_type": "markdown",
"id": "9e70b0a0",
"metadata": {},
"source": [
"Use the class `ElasticsearchCache`.\n",
"\n",
"Simple example:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1762c9c1",
"metadata": {},
"outputs": [],
"source": [
"from elasticsearch import Elasticsearch\n",
"from langchain.globals import set_llm_cache\n",
"from langchain_elasticsearch import ElasticsearchCache\n",
"\n",
"es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n",
"set_llm_cache(\n",
" ElasticsearchCache(\n",
" es_connection=es_client,\n",
" index_name=\"llm-chat-cache\",\n",
" metadata={\"project\": \"my_chatgpt_project\"},\n",
" )\n",
")"
]
},
{
"cell_type": "markdown",
"id": "d4fac5d6",
"metadata": {},
"source": [
"The `index_name` parameter can also accept aliases. This allows to use the \n",
"[ILM: Manage the index lifecycle](https://www.elastic.co/guide/en/elasticsearch/reference/current/index-lifecycle-management.html)\n",
"that we suggest to consider for managing retention and controlling cache growth.\n",
"\n",
"Look at the class docstring for all parameters."
]
},
{
"cell_type": "markdown",
"id": "eaf9dfd7",
"metadata": {},
"source": [
"### Index the generated text\n",
"\n",
"The cached data won't be searchable by default.\n",
"The developer can customize the building of the Elasticsearch document in order to add indexed text fields,\n",
"where to put, for example, the text generated by the LLM.\n",
"\n",
"This can be done by subclassing end overriding methods.\n",
"The new cache class can be applied also to a pre-existing cache index:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5104c2c0",
"metadata": {},
"outputs": [],
"source": [
"import json\n",
"from typing import Any, Dict, List\n",
"\n",
"from elasticsearch import Elasticsearch\n",
"from langchain.globals import set_llm_cache\n",
"from langchain_core.caches import RETURN_VAL_TYPE\n",
"from langchain_elasticsearch import ElasticsearchCache\n",
"\n",
"\n",
"class SearchableElasticsearchCache(ElasticsearchCache):\n",
" @property\n",
" def mapping(self) -> Dict[str, Any]:\n",
" mapping = super().mapping\n",
" mapping[\"mappings\"][\"properties\"][\"parsed_llm_output\"] = {\n",
" \"type\": \"text\",\n",
" \"analyzer\": \"english\",\n",
" }\n",
" return mapping\n",
"\n",
" def build_document(\n",
" self, prompt: str, llm_string: str, return_val: RETURN_VAL_TYPE\n",
" ) -> Dict[str, Any]:\n",
" body = super().build_document(prompt, llm_string, return_val)\n",
" body[\"parsed_llm_output\"] = self._parse_output(body[\"llm_output\"])\n",
" return body\n",
"\n",
" @staticmethod\n",
" def _parse_output(data: List[str]) -> List[str]:\n",
" return [\n",
" json.loads(output)[\"kwargs\"][\"message\"][\"kwargs\"][\"content\"]\n",
" for output in data\n",
" ]\n",
"\n",
"\n",
"es_client = Elasticsearch(hosts=\"http://localhost:9200\")\n",
"set_llm_cache(\n",
" SearchableElasticsearchCache(es_connection=es_client, index_name=\"llm-chat-cache\")\n",
")"
]
},
{
"cell_type": "markdown",
"id": "db0dea73",
"metadata": {},
"source": [
"When overriding the mapping and the document building, \n",
"please only make additive modifications, keeping the base mapping intact."
]
},
{
"cell_type": "markdown",
"id": "0c69d84d",
Expand Down

0 comments on commit 61f14f0

Please sign in to comment.