Effectively use LLamaCPP with Langchain - ChatModel, JSON Mode & Function Calling Support

[samuelint]

https://github.com/samuelint/langchain-llamacpp-chat-model
https://medium.com/@samuelint/langchain-llamacpp-%EF%B8%8F-048604447ca3

I have developed an integration between LLamaCPP and LangChain that enables the use of a ChatModel, JSON Mode, and Function Calling.
This integration allows you to create LangGraph agents that run entirely locally using LlamaCPP!!

import os
from langchain_core.pydantic_v1 import BaseModel, Field
from llama_cpp import Llama
from langchain_llamacpp_chat_model import LlamaChatModel
from langchain_core.tools import tool

model_path = os.path.join(
    os.path.expanduser("~/.cache/lm-studio/models"),
    "lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF/Meta-Llama-3-8B-Instruct-Q4_K_M.gguf",
)

llama = Llama(
    model_path=model_path,
    _gpu_layers=-1,
    chat_format="chatml-function-calling", # https://llama-cpp-python.readthedocs.io/en/latest/#function-calling
)
chat_model = LlamaChatModel(llama=llama)

result = chat_model.invoke("Tell me a joke about cats")
print(result.content)  # Why was the cat sitting on the computer? Because it wanted to keep an eye on the mouse!