examples/notebooks/langchain/Langchain-Ollama-Prompt-memory.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "766aaa81-96e6-42dc-b29d-8216d2a7feec",
   "metadata": {},
   "source": [
    "## Example of querying the Ollama Inference server with Mistral-7b, Langchain and a custom Prompt"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e65a9bb1-b06e-4569-b1cf-c7cdcd65b5d9",
   "metadata": {},
   "source": [
    "### Set the Inference server url (replace with your own address)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6b28afc8-7a57-4c81-a0de-3fdb6270830d",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "inference_server_url = \"http://ollama.ic-shared-llm.svc.cluster.local:11434\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0022f3fb-ee50-40f2-b276-b8194668e49e",
   "metadata": {},
   "source": [
    "## Preparation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a16ed2e6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "!pip install -q langchain==0.1.14"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60bb3f0f-40b5-49a6-b493-5e361db0113e",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Imports\n",
    "from langchain.chains import ConversationChain\n",
    "from langchain.memory import ConversationBufferMemory\n",
    "from langchain_community.llms import Ollama\n",
    "from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler\n",
    "from langchain.prompts import PromptTemplate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "472b2f3f-ac23-4531-984b-6e8357233992",
   "metadata": {},
   "source": [
    "#### Create the LLM instance"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "01baa2b8-529d-455d-ad39-ef4a96dbaf97",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# LLM definition\n",
    "llm = Ollama(\n",
    "    base_url=inference_server_url,\n",
    "    model=\"mistral\",\n",
    "    top_p=0.92,\n",
    "    temperature=0.01,\n",
    "    num_predict=512,\n",
    "    repeat_penalty=1.03,\n",
    "    callbacks=[StreamingStdOutCallbackHandler()]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d211bd1f-3e55-41ce-b3bc-0ee5db689c66",
   "metadata": {},
   "source": [
    "#### Create the Prompt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "fdd8bc4c-b353-4a51-a8b7-6cb348e19623",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "template=\"\"\"<s>[INST] <<SYS>>\n",
    "You are a helpful, respectful and honest assistant. Always be as helpful as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.\n",
    "\n",
    "If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.\n",
    "<</SYS>>\n",
    "\n",
    "Current conversation:\n",
    "{history}\n",
    "Human: {input}\n",
    "AI:\n",
    "[/INST]\n",
    "\"\"\"\n",
    "PROMPT = PromptTemplate(input_variables=[\"history\", \"input\"], template=template)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "17e36542-81a4-4eee-bcf8-f0dd5d9e3096",
   "metadata": {},
   "source": [
    "#### And finally some memory for the conversation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c0005da7-1386-42a4-9d20-456c5dcca2be",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "memory=ConversationBufferMemory()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c240fd72-f910-4ba4-82dc-c5ebf278452b",
   "metadata": {
    "tags": []
   },
   "source": [
    "## First example, fully verbose mode"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "af5cb39d-1639-4713-8423-4ea4925ef6d1",
   "metadata": {},
   "source": [
    "#### Create the Chain using the different components"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3d7998b5-4373-4b70-bd81-9617f7ff6bf1",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Verbose mode is intentionally set to True so you can see the prompt and the history from the buffer memory\n",
    "conversation = ConversationChain(llm=llm,\n",
    "                                 prompt=PROMPT,\n",
    "                                 verbose=True,\n",
    "                                 memory=memory\n",
    "                                )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "969484f2-1f55-4021-a995-1d529cf9cc54",
   "metadata": {},
   "source": [
    "#### Let's talk..."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ee09847d-9ff7-4181-ad11-96186bf0a322",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "first_input = \"Describe Paris in 100 words or less.\"\n",
    "conversation.predict(input=first_input);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "60ed6c44-a016-49bd-8623-f231c5bba5b6",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This is a follow-up question without context to showcase the conversation memory\n",
    "second_input = \"Is there a river?\"\n",
    "conversation.predict(input=second_input);"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fc7dd8b1-c5a1-41fd-8c7f-61e221360753",
   "metadata": {},
   "source": [
    "### Second example, no verbosity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "842e0ab2-a061-4ebd-af71-7305c54af940",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Verbose mode is intentionally set to True so you can see the prompt and the history from the buffer memory\n",
    "conversation2 = ConversationChain(llm=llm,\n",
    "                                 prompt=PROMPT,\n",
    "                                 verbose=False,\n",
    "                                 memory=memory\n",
    "                                )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b60769ba-dd7a-4ac7-911a-3e0abe9e1732",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# Let's clear the previous conversation first\n",
    "memory.clear()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e9113c36-db08-464c-a6a7-e8fdf1068c3b",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "first_input = \"Can you describe London in 100 words?\"\n",
    "conversation2.predict(input=first_input);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d618b39d-0e45-4147-b7c7-63f9f195c805",
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# This is a follow-up question without context to showcase the conversation memory\n",
    "second_input = \"Is there a river?\"\n",
    "conversation2.predict(input=second_input);"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.9",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}