community: init signature revision for Cassandra LLM cache classes + …

…small maintenance (#17765) This PR improves on the `CassandraCache` and `CassandraSemanticCache` classes, mainly in the constructor signature, and also introduces several minor improvements around these classes. ### Init signature A (sigh) breaking change is tentatively introduced to the constructor. To me, the advantages outweigh the possible discomfort: the new syntax places the DB-connection objects `session` and `keyspace` later in the param list, so that they can be given a default value. This is what enables the pattern of _not_ specifying them, provided one has previously initialized the Cassandra connection through the versatile utility method `cassio.init(...)`. In this way, a much less unwieldy instantiation can be done, such as `CassandraCache()` and `CassandraSemanticCache(embedding=xyz)`, everything else falling back to defaults. A downside is that, compared to the earlier signature, this might turn out to be breaking for those doing positional instantiation. As a way to mitigate this problem, this PR typechecks its first argument trying to detect the legacy usage. (And to make this point less tricky in the future, most arguments are left to be keyword-only). If this is considered too harsh, I'd like guidance on how to further smoothen this transition. **Our plan is to make the pattern of optional session/keyspace a standard across all Cassandra classes**, so that a repeatable strategy would be ideal. A possibility would be to keep positional arguments for legacy reasons but issue a deprecation warning if any of them is actually used, to later remove them with 0.2 - please advise on this point. ### Other changes - class docstrings: enriched, completely moved to class level, added note on `cassio.init(...)` pattern, added tiny sample usage code. - semantic cache: revised terminology to never mention "distance" (it is in fact a similarity!). Kept the legacy constructor param with a deprecation warning if used. - `llm_caching` notebook: uniform flow with the Cassandra and Astra DB separate cases; better and Cassandra-first description; all imports made explicit and from community where appropriate. - cache integration tests moved to community (incl. the imported tools), env var bugfix for `CASSANDRA_CONTACT_POINTS`. --------- Co-authored-by: Erick Friis <[email protected]>
langchain-ai · Jun 20, 2024 · 63108eb · 63108eb
1 parent ef7353a
commit 63108eb
Show file tree

Hide file tree

Showing 4 changed files with 272 additions and 96 deletions.
diff --git a/docs/docs/integrations/llms/llm_caching.ipynb b/docs/docs/integrations/llms/llm_caching.ipynb
@@ -916,92 +916,183 @@
    "source": [
     "## `Cassandra` caches\n",
     "\n",
-    "You can use Cassandra / Astra DB through CQL for caching LLM responses, choosing from the exact-match `CassandraCache` or the (vector-similarity-based) `CassandraSemanticCache`.\n",
+    "> [Apache Cassandra®](https://cassandra.apache.org/) is a NoSQL, row-oriented, highly scalable and highly available database. Starting with version 5.0, the database ships with [vector search capabilities](https://cassandra.apache.org/doc/trunk/cassandra/vector-search/overview.html).\n",
     "\n",
-    "Let's see both in action in the following cells."
+    "You can use Cassandra for caching LLM responses, choosing from the exact-match `CassandraCache` or the (vector-similarity-based) `CassandraSemanticCache`.\n",
+    "\n",
+    "Let's see both in action. The next cells guide you through the (little) required setup, and the following cells showcase the two available cache classes."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6cf6acb4-1bc4-4c4b-9325-2420c17e5e2b",
+   "metadata": {},
+   "source": [
+    "### Required dependency"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "id": "fe842b0d-fd3d-47dd-bc6a-975997c9707f",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "%pip install --upgrade --quiet \"cassio>=0.1.4\""
    ]
   },
   {
    "cell_type": "markdown",
    "id": "a4a6725d",
    "metadata": {},
    "source": [
-    "#### Connect to the DB\n",
+    "### Connect to the DB\n",
+    "\n",
+    "The Cassandra caches shown in this page can be used with Cassandra as well as other derived databases, such as Astra DB, which use the CQL (Cassandra Query Language) protocol.\n",
+    "\n",
+    "> DataStax [Astra DB](https://docs.datastax.com/en/astra-serverless/docs/vector-search/quickstart.html) is a managed serverless database built on Cassandra, offering the same interface and strengths.\n",
+    "\n",
+    "Depending on whether you connect to a Cassandra cluster or to Astra DB through CQL, you will provide different parameters when instantiating the cache (through initialization of a CassIO connection)."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "15735abe-2567-43ce-aa91-f253b33b5a88",
+   "metadata": {},
+   "source": [
+    "#### Connecting to a Cassandra cluster\n",
     "\n",
-    "First you need to establish a `Session` to the DB and to specify a _keyspace_ for the cache table(s). The following gets you connected to Astra DB through CQL (see e.g. [here](https://cassio.org/start_here/#vector-database) for more backends and connection options)."
+    "You first need to create a `cassandra.cluster.Session` object, as described in the [Cassandra driver documentation](https://docs.datastax.com/en/developer/python-driver/latest/api/cassandra/cluster/#module-cassandra.cluster). The details vary (e.g. with network settings and authentication), but this might be something like:"
    ]
   },
   {
    "cell_type": "code",
    "execution_count": 1,
-   "id": "cc53ce1b",
+   "id": "e4b898a5-fe0e-4f11-a87b-7979652322a7",
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from cassandra.cluster import Cluster\n",
+    "\n",
+    "cluster = Cluster([\"127.0.0.1\"])\n",
+    "session = cluster.connect()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "6435198e-8713-4045-906b-879613bf5083",
+   "metadata": {},
+   "source": [
+    "You can now set the session, along with your desired keyspace name, as a global CassIO parameter:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "id": "992267dc-0d19-45e0-9a13-ccbb6348d804",
    "metadata": {},
    "outputs": [
     {
-     "name": "stdout",
+     "name": "stdin",
      "output_type": "stream",
      "text": [
-      "\n",
-      "Keyspace name? my_keyspace\n",
-      "\n",
-      "Astra DB Token (\"AstraCS:...\") ········\n",
-      "Full path to your Secure Connect Bundle? /path/to/secure-connect-databasename.zip\n"
+      "CASSANDRA_KEYSPACE =  demo_keyspace\n"
+     ]
+    }
+   ],
+   "source": [
+    "import cassio\n",
+    "\n",
+    "CASSANDRA_KEYSPACE = input(\"CASSANDRA_KEYSPACE = \")\n",
+    "\n",
+    "cassio.init(session=session, keyspace=CASSANDRA_KEYSPACE)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "id": "2cc7ba29-8f84-4fbf-aaf7-3daa1be7e7b0",
+   "metadata": {},
+   "source": [
+    "#### Connecting to Astra DB through CQL\n",
+    "\n",
+    "In this case you initialize CassIO with the following connection parameters:\n",
+    "\n",
+    "- the Database ID, e.g. `01234567-89ab-cdef-0123-456789abcdef`\n",
+    "- the Token, e.g. `AstraCS:6gBhNmsk135....` (it must be a \"Database Administrator\" token)\n",
+    "- Optionally a Keyspace name (if omitted, the default one for the database will be used)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 12,
+   "id": "ead97077-cc79-4f5c-940c-91eb21650466",
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdin",
+     "output_type": "stream",
+     "text": [
+      "ASTRA_DB_ID =  01234567-89ab-cdef-0123-456789abcdef\n",
+      "ASTRA_DB_APPLICATION_TOKEN =  ········\n",
+      "ASTRA_DB_KEYSPACE (optional, can be left empty) =  my_keyspace\n"
      ]
     }
    ],
    "source": [
     "import getpass\n",
     "\n",
-    "keyspace = input(\"\\nKeyspace name? \")\n",
-    "ASTRA_DB_APPLICATION_TOKEN = getpass.getpass('\\nAstra DB Token (\"AstraCS:...\") ')\n",
-    "ASTRA_DB_SECURE_BUNDLE_PATH = input(\"Full path to your Secure Connect Bundle? \")"
+    "ASTRA_DB_ID = input(\"ASTRA_DB_ID = \")\n",
+    "ASTRA_DB_APPLICATION_TOKEN = getpass.getpass(\"ASTRA_DB_APPLICATION_TOKEN = \")\n",
+    "\n",
+    "desired_keyspace = input(\"ASTRA_DB_KEYSPACE (optional, can be left empty) = \")\n",
+    "if desired_keyspace:\n",
+    "    ASTRA_DB_KEYSPACE = desired_keyspace\n",
+    "else:\n",
+    "    ASTRA_DB_KEYSPACE = None"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 2,
-   "id": "4617f485",
+   "execution_count": 13,
+   "id": "cc53ce1b",
    "metadata": {},
    "outputs": [],
    "source": [
-    "from cassandra.auth import PlainTextAuthProvider\n",
-    "from cassandra.cluster import Cluster\n",
+    "import cassio\n",
     "\n",
-    "cluster = Cluster(\n",
-    "    cloud={\n",
-    "        \"secure_connect_bundle\": ASTRA_DB_SECURE_BUNDLE_PATH,\n",
-    "    },\n",
-    "    auth_provider=PlainTextAuthProvider(\"token\", ASTRA_DB_APPLICATION_TOKEN),\n",
-    ")\n",
-    "session = cluster.connect()"
+    "cassio.init(\n",
+    "    database_id=ASTRA_DB_ID,\n",
+    "    token=ASTRA_DB_APPLICATION_TOKEN,\n",
+    "    keyspace=ASTRA_DB_KEYSPACE,\n",
+    ")"
    ]
   },
   {
    "cell_type": "markdown",
    "id": "8665664a",
    "metadata": {},
    "source": [
-    "### Exact cache\n",
+    "### Cassandra: Exact cache\n",
     "\n",
     "This will avoid invoking the LLM when the supplied prompt is _exactly_ the same as one encountered already:"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 6,
+   "execution_count": 3,
    "id": "00a5e66f",
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.cache import CassandraCache\n",
-    "from langchain.globals import set_llm_cache\n",
+    "from langchain_community.cache import CassandraCache\n",
+    "from langchain_core.globals import set_llm_cache\n",
     "\n",
-    "set_llm_cache(CassandraCache(session=session, keyspace=keyspace))"
+    "set_llm_cache(CassandraCache())"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 11,
+   "execution_count": 9,
    "id": "956a5145",
    "metadata": {},
    "outputs": [
@@ -1011,9 +1102,9 @@
      "text": [
       "\n",
       "\n",
-      "The Moon always shows the same side because it is tidally locked to Earth.\n",
-      "CPU times: user 41.7 ms, sys: 153 µs, total: 41.8 ms\n",
-      "Wall time: 1.96 s\n"
+      "The Moon is tidally locked with the Earth, which means that its rotation on its own axis is synchronized with its orbit around the Earth. This results in the Moon always showing the same side to the Earth. This is because the gravitational forces between the Earth and the Moon have caused the Moon's rotation to slow down over time, until it reached a point where it takes the same amount of time for the Moon to rotate on its axis as it does to orbit around the Earth. This phenomenon is common among satellites in close orbits around their parent planets and is known as tidal locking.\n",
+      "CPU times: user 92.5 ms, sys: 8.89 ms, total: 101 ms\n",
+      "Wall time: 1.98 s\n"
      ]
     }
    ],
@@ -1025,7 +1116,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 12,
+   "execution_count": 10,
    "id": "158f0151",
    "metadata": {},
    "outputs": [
@@ -1035,9 +1126,9 @@
      "text": [
       "\n",
       "\n",
-      "The Moon always shows the same side because it is tidally locked to Earth.\n",
-      "CPU times: user 4.09 ms, sys: 0 ns, total: 4.09 ms\n",
-      "Wall time: 119 ms\n"
+      "The Moon is tidally locked with the Earth, which means that its rotation on its own axis is synchronized with its orbit around the Earth. This results in the Moon always showing the same side to the Earth. This is because the gravitational forces between the Earth and the Moon have caused the Moon's rotation to slow down over time, until it reached a point where it takes the same amount of time for the Moon to rotate on its axis as it does to orbit around the Earth. This phenomenon is common among satellites in close orbits around their parent planets and is known as tidal locking.\n",
+      "CPU times: user 5.51 ms, sys: 0 ns, total: 5.51 ms\n",
+      "Wall time: 5.78 ms\n"
      ]
     }
    ],
@@ -1052,14 +1143,14 @@
    "id": "8fc4d017",
    "metadata": {},
    "source": [
-    "### Semantic cache\n",
+    "### Cassandra: Semantic cache\n",
     "\n",
     "This cache will do a semantic similarity search and return a hit if it finds a cached entry that is similar enough, For this, you need to provide an `Embeddings` instance of your choice."
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 13,
+   "execution_count": 14,
    "id": "b9ad3f54",
    "metadata": {},
    "outputs": [],
@@ -1071,26 +1162,25 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 14,
+   "execution_count": 17,
    "id": "4623f95e",
    "metadata": {},
    "outputs": [],
    "source": [
-    "from langchain.cache import CassandraSemanticCache\n",
+    "from langchain_community.cache import CassandraSemanticCache\n",
+    "from langchain_core.globals import set_llm_cache\n",
     "\n",
     "set_llm_cache(\n",
     "    CassandraSemanticCache(\n",
-    "        session=session,\n",
-    "        keyspace=keyspace,\n",
     "        embedding=embedding,\n",
-    "        table_name=\"cass_sem_cache\",\n",
+    "        table_name=\"my_semantic_cache\",\n",
     "    )\n",
     ")"
    ]
   },
   {
    "cell_type": "code",
-   "execution_count": 15,
+   "execution_count": 19,
    "id": "1a8e577b",
    "metadata": {},
    "outputs": [
@@ -1100,9 +1190,9 @@
      "text": [
       "\n",
       "\n",
-      "The Moon always shows the same side because it is tidally locked with Earth. This means that the same side of the Moon always faces Earth.\n",
-      "CPU times: user 21.3 ms, sys: 177 µs, total: 21.4 ms\n",
-      "Wall time: 3.09 s\n"
+      "The Moon is always showing the same side because of a phenomenon called synchronous rotation. This means that the Moon rotates on its axis at the same rate that it orbits around the Earth, which takes approximately 27.3 days. This results in the same side of the Moon always facing the Earth. This is due to the gravitational forces between the Earth and the Moon, which have caused the Moon's rotation to gradually slow down and become synchronized with its orbit. This is a common occurrence among many moons in our solar system.\n",
+      "CPU times: user 49.5 ms, sys: 7.38 ms, total: 56.9 ms\n",
+      "Wall time: 2.55 s\n"
      ]
     }
    ],
@@ -1114,7 +1204,7 @@
   },
   {
    "cell_type": "code",
-   "execution_count": 16,
+   "execution_count": 20,
    "id": "f7abddfd",
    "metadata": {},
    "outputs": [
@@ -1124,9 +1214,9 @@
      "text": [
       "\n",
       "\n",
-      "The Moon always shows the same side because it is tidally locked with Earth. This means that the same side of the Moon always faces Earth.\n",
-      "CPU times: user 10.9 ms, sys: 17 µs, total: 10.9 ms\n",
-      "Wall time: 461 ms\n"
+      "The Moon is always showing the same side because of a phenomenon called synchronous rotation. This means that the Moon rotates on its axis at the same rate that it orbits around the Earth, which takes approximately 27.3 days. This results in the same side of the Moon always facing the Earth. This is due to the gravitational forces between the Earth and the Moon, which have caused the Moon's rotation to gradually slow down and become synchronized with its orbit. This is a common occurrence among many moons in our solar system.\n",
+      "CPU times: user 21.2 ms, sys: 3.38 ms, total: 24.6 ms\n",
+      "Wall time: 532 ms\n"
      ]
     }
    ],