python-lint (#14689)

# Description: _python-lint_

This agent writes Python code that is formatted and linted using
`black`, `ruff`, and `mypy`, but does not execute the code. It writes
the code to a temporary file and then runs the linters. Once these
checks pass, the code is returned.

# Dependencies

- black
- ruff
- mypy

# Demo

The functionality can be seen here:
https://huggingface.co/spaces/joshuasundance/langchain-streamlit-demo

templates/python-lint/.gitignore

@@ -0,0 +1 @@
+__pycache__

templates/python-lint/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2023 LangChain, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

templates/python-lint/README.md

@@ -0,0 +1,74 @@
+# python-lint
+
+This agent specializes in generating high-quality Python code with a focus on proper formatting and linting. It uses `black`, `ruff`, and `mypy` to ensure the code meets standard quality checks.
+
+This streamlines the coding process by integrating and responding to these checks, resulting in reliable and consistent code output.
+
+It cannot actually execute the code it writes, as code execution may introduce additional dependencies and potential security vulnerabilities.
+This makes the agent both a secure and efficient solution for code generation tasks.
+
+You can use it to generate Python code directly, or network it with planning and execution agents.
+
+## Environment Setup
+
+- Install `black`, `ruff`, and `mypy`: `pip install -U black ruff mypy`
+- Set `OPENAI_API_KEY` environment variable.
+
+## Usage
+
+To use this package, you should first have the LangChain CLI installed:
+
+```shell
+pip install -U langchain-cli
+```
+
+To create a new LangChain project and install this as the only package, you can do:
+
+```shell
+langchain app new my-app --package python-lint
+```
+
+If you want to add this to an existing project, you can just run:
+
+```shell
+langchain app add python-lint
+```
+
+And add the following code to your `server.py` file:
+```python
+from python_lint import agent_executor as python_lint_agent
+
+add_routes(app, python_lint_agent, path="/python-lint")
+```
+
+(Optional) Let's now configure LangSmith. 
+LangSmith will help us trace, monitor and debug LangChain applications. 
+LangSmith is currently in private beta, you can sign up [here](https://smith.langchain.com/). 
+If you don't have access, you can skip this section
+
+
+```shell
+export LANGCHAIN_TRACING_V2=true
+export LANGCHAIN_API_KEY=<your-api-key>
+export LANGCHAIN_PROJECT=<your-project>  # if not specified, defaults to "default"
+```
+
+If you are inside this directory, then you can spin up a LangServe instance directly by:
+
+```shell
+langchain serve
+```
+
+This will start the FastAPI app with a server is running locally at 
+[http://localhost:8000](http://localhost:8000)
+
+We can see all templates at [http://127.0.0.1:8000/docs](http://127.0.0.1:8000/docs)
+We can access the playground at [http://127.0.0.1:8000/python-lint/playground](http://127.0.0.1:8000/python-lint/playground)  
+
+We can access the template from code with:
+
+```python
+from langserve.client import RemoteRunnable
+
+runnable = RemoteRunnable("http://localhost:8000/python-lint")
+```

templates/python-lint/pyproject.toml

@@ -0,0 +1,33 @@
+[tool.poetry]
+name = "python-lint"
+version = "0.0.1"
+description = "Python code-writing agent whose work is checked by black, ruff, and mypy."
+authors = ["Joshua Sundance Bailey"]
+readme = "README.md"
+
+[tool.poetry.dependencies]
+ruff = ">=0.1.8"
+black = ">=23.12.0"
+mypy = ">=1.7.1"
+python = ">=3.8.1,<4.0"
+langchain = ">=0.0.313, <0.1"
+openai = ">=1.3.9"
+
+[tool.poetry.group.dev.dependencies]
+langchain-cli = ">=0.0.4"
+fastapi = "^0.104.0"
+sse-starlette = "^1.6.5"
+
+[tool.langserve]
+export_module = "python_lint"
+export_attr = "agent_executor"
+
+[tool.templates-hub]
+use-case = "code-generation"
+author = "Joshua Sundance Bailey"
+integrations = ["OpenAI"]
+tags = ["python", "agent"]
+
+[build-system]
+requires = ["poetry-core"]
+build-backend = "poetry.core.masonry.api"

templates/python-lint/python_lint/__init__.py

@@ -0,0 +1,3 @@
+from python_lint.agent_executor import agent_executor
+
+__all__ = ["agent_executor"]

templates/python-lint/python_lint/agent_executor.py

@@ -0,0 +1,216 @@
+import os
+import re
+import subprocess  # nosec
+import tempfile
+
+from langchain.agents import AgentType, initialize_agent
+from langchain.agents.tools import Tool
+from langchain.chat_models import ChatOpenAI
+from langchain.llms.base import BaseLLM
+from langchain.prompts import ChatPromptTemplate
+from langchain.pydantic_v1 import BaseModel, Field, ValidationError, validator
+from langchain.schema.runnable import ConfigurableField, Runnable
+
+
+def strip_python_markdown_tags(text: str) -> str:
+    pat = re.compile(r"```python\n(.*)```", re.DOTALL)
+    code = pat.match(text)
+    if code:
+        return code.group(1)
+    else:
+        return text
+
+
+def format_black(filepath: str):
+    """Format a file with black."""
+    subprocess.run(  # nosec
+        f"black {filepath}",
+        stderr=subprocess.STDOUT,
+        text=True,
+        shell=True,
+        timeout=3,
+        check=False,
+    )
+
+
+def format_ruff(filepath: str):
+    """Run ruff format on a file."""
+    subprocess.run(  # nosec
+        f"ruff check --fix {filepath}",
+        shell=True,
+        text=True,
+        timeout=3,
+        universal_newlines=True,
+        check=False,
+    )
+
+    subprocess.run(  # nosec
+        f"ruff format {filepath}",
+        stderr=subprocess.STDOUT,
+        shell=True,
+        timeout=3,
+        text=True,
+        check=False,
+    )
+
+
+def check_ruff(filepath: str):
+    """Run ruff check on a file."""
+    subprocess.check_output(  # nosec
+        f"ruff check {filepath}",
+        stderr=subprocess.STDOUT,
+        shell=True,
+        timeout=3,
+        text=True,
+    )
+
+
+def check_mypy(filepath: str, strict: bool = True, follow_imports: str = "skip"):
+    """Run mypy on a file."""
+    cmd = (
+        f"mypy {'--strict' if strict else ''} "
+        f"--follow-imports={follow_imports} {filepath}"
+    )
+
+    subprocess.check_output(  # nosec
+        cmd,
+        stderr=subprocess.STDOUT,
+        shell=True,
+        text=True,
+        timeout=3,
+    )
+
+
+class PythonCode(BaseModel):
+    code: str = Field(
+        description="Python code conforming to "
+        "ruff, black, and *strict* mypy standards.",
+    )
+
+    @validator("code")
+    @classmethod
+    def check_code(cls, v: str) -> str:
+        v = strip_python_markdown_tags(v).strip()
+        try:
+            with tempfile.NamedTemporaryFile(mode="w", delete=False) as temp_file:
+                temp_file.write(v)
+                temp_file_path = temp_file.name
+
+            try:
+                # format with black and ruff
+                format_black(temp_file_path)
+                format_ruff(temp_file_path)
+            except subprocess.CalledProcessError:
+                pass
+
+            # update `v` with formatted code
+            with open(temp_file_path, "r") as temp_file:
+                v = temp_file.read()
+
+            # check
+            complaints = dict(ruff=None, mypy=None)
+
+            try:
+                check_ruff(temp_file_path)
+            except subprocess.CalledProcessError as e:
+                complaints["ruff"] = e.output
+
+            try:
+                check_mypy(temp_file_path)
+            except subprocess.CalledProcessError as e:
+                complaints["mypy"] = e.output
+
+            # raise ValueError if ruff or mypy had complaints
+            if any(complaints.values()):
+                code_str = f"```{temp_file_path}\n{v}```"
+                error_messages = [
+                    f"```{key}\n{value}```"
+                    for key, value in complaints.items()
+                    if value
+                ]
+                raise ValueError("\n\n".join([code_str] + error_messages))
+
+        finally:
+            os.remove(temp_file_path)
+        return v
+
+
+def check_code(code: str) -> str:
+    try:
+        code_obj = PythonCode(code=code)
+        return (
+            f"# LGTM\n"
+            f"# use the `submit` tool to submit this code:\n\n"
+            f"```python\n{code_obj.code}\n```"
+        )
+    except ValidationError as e:
+        return e.errors()[0]["msg"]
+
+
+prompt = ChatPromptTemplate.from_messages(
+    [
+        (
+            "system",
+            "You are a world class Python coder who uses "
+            "black, ruff, and *strict* mypy for all of your code. "
+            "Provide complete, end-to-end Python code "
+            "to meet the user's description/requirements. "
+            "Always `check` your code. When you're done, "
+            "you must ALWAYS use the `submit` tool.",
+        ),
+        (
+            "human",
+            ": {input}",
+        ),
+    ],
+)
+
+check_code_tool = Tool.from_function(
+    check_code,
+    name="check-code",
+    description="Always check your code before submitting it!",
+)
+
+submit_code_tool = Tool.from_function(
+    strip_python_markdown_tags,
+    name="submit-code",
+    description="THIS TOOL is the most important. "
+    "use it to submit your code to the user who requested it... "
+    "but be sure to `check` it first!",
+    return_direct=True,
+)
+
+tools = [check_code_tool, submit_code_tool]
+
+
+def get_agent_executor(
+    llm: BaseLLM,
+    agent_type: AgentType = AgentType.OPENAI_FUNCTIONS,
+) -> Runnable:
+    _agent_executor = initialize_agent(
+        tools,
+        llm,
+        agent=agent_type,
+        verbose=True,
+        handle_parsing_errors=True,
+        prompt=prompt,
+    )
+    return _agent_executor | (lambda output: output["output"])
+
+
+class Instruction(BaseModel):
+    __root__: str
+
+
+agent_executor = (
+    get_agent_executor(ChatOpenAI(model_name="gpt-4-1106-preview", temperature=0.0))
+    .configurable_alternatives(
+        ConfigurableField("model_name"),
+        default_key="gpt4turbo",
+        gpt4=get_agent_executor(ChatOpenAI(model_name="gpt-4", temperature=0.0)),
+        gpt35t=get_agent_executor(
+            ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.0),
+        ),
+    )
+    .with_types(input_type=Instruction, output_type=str)
+)