docs: developer guide (#158)

Co-authored-by: Adam Dąbrowski <[email protected]>

README.md

@@ -38,7 +38,8 @@ The RAI framework aims to:
 # Table of Contents
 
 - [Features](#features)
-- [Quick Start](#installation)
+- [Quick Start](#quick-start)
+- [Developer guide](./docs/developer_guide.md)
 - [Usage examples (demos)](#planned-demos)
 - [Further documentation](#further-documentation)
 - [ROSCon 2024 Talk](#roscon-2024)
@@ -117,7 +118,7 @@ If you do not have a key, see how to generate one [here](https://platform.openai
 export OPENAI_API_KEY=""
 ```
 
-#### Congratulations, your installation is now complete!
+#### Congratulations, your installation is now complete! Head to [Running example](./docs/developer_guide.md)
 
 # Running RAI
 

docs/developer_guide.md

@@ -1,8 +1,88 @@
 # Developer Guide
 
-Developer Guide is coming soon, here are some resources:
+RAI is a problem-agnostic framework designed to provide a general solution for various tasks. It is easily extendable, allowing developers to adapt and integrate new functionalities and tools to meet specific needs.
+
+## Try out RAI
+
+```python
+from rai import ROS2Agent
+
+agent = ROS2Agent(vendor='openai') # openai or bedrock
+print(agent("What topics, services, and actions are available?"))
+print(agent("Please describe the interfaces of two of the existing topics."))
+print(agent("Please publish 'Hello RAI' to /chatter topic only once")) # make sure to listen first ros2 topic echo /chatter
+```
+
+## Adjusting RAI for Your Robot
+
+### 1. Create Your Robot whoami configuration package [Optional]
+
+The whoami package is used for configuring your robot within the RAI ecosystem. It helps define the robot's identity, self-understanding, ethical code, and documentation. For instructions see [configure RAI for your robot](create_robots_whoami.md).
+
+> [!NOTE]  
+> While RAI's agents can be started without a whoami configuration package, we strongly recommend integrating it for a more complete setup.
+
+### 2. Implement new tools specific to your robot
+
+To extend RAI with tools specific to your robot, implement new tool functions as shown below. These functions should be decorated with @tool and should interact with your robot's API.
+
+```python
+from langchain.tools import tool, BaseTool
+from langchain.pydantic_v1 import BaseModel, Field
+from typing import Type
+from myrobot import robot
+
+# decorator based api, for basic tools
+@tool
+def pick_up_object(bbox_3d):
+    """Tool used for picking up objects"""
+    return robot.pick_up_object(bbox_3d)
+
+@tool
+def scan_object():
+    """Tool used for examining previously picked object"""
+    return robot.scan_object()
+
+# class based api, useful when tools use objects like ROS 2 Node
+class SayToolInput(BaseModel):
+    text: str = Field(..., description="Text to be said.")
+
+class SayTool(BaseTool):
+    name: str = "say"
+    description: str = "Tool used for speaking in e.g. human-robot conversation"
+    robot: type[Robot]
+    args_schema: Type[SayToolInput] = SayToolInput
+
+    def _run(self, text: str):
+        return self.robot.speak(text)
+
+def state_retriever():
+    """
+    State retriever used for feeding state information to agent every iteration.
+    The state can consist of robot's logs, as well as any information that might
+    be useful to the robot's operations and agent's reasoning.
+    """
+    return {"temperature": 30, "battery_state": 33, "logs": [...]}
+
+```
+
+### 3. Run the agent with new tools
+
+Once you have implemented your tools, you can run the agent with these new tools as follows:
+
+```python
+from rai.agents.state_based import create_state_based_agent
+from langchain_openai import ChatOpenAI
+from myrobot import robot
+
+llm = ChatOpenAI(model='gpt-4o') # initialize your vendor of choice
+tools = [pick_up_object, scan_object, SayTool(robot=robot)]
+agent = create_state_based_agent(llm=llm, state_retriever=state_retriever, tools=tools)
+agent.invoke({"messages": ["Please pick up an object and scan it."]})
+```
+
+Additional resources:
 
-- How to [configure RAI for your robot](create_robots_whoami.md).
 - [Beta demos](demos.md).
 - [Multimodal Messages](multimodal_messages.md) definition.
 - Available ROS 2 packages: [ros packages](ros_packages.md).

src/rai/rai/__init__.py

@@ -12,3 +12,6 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 #
+from rai.apps.high_level_api import ROS2Agent
+
+__all__ = ["ROS2Agent"]

src/rai/rai/apps/high_level_api.py

@@ -0,0 +1,59 @@
+# Copyright (C) 2024 Robotec.AI
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#         http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+from typing import List, Literal
+
+from langchain_aws import ChatBedrock
+from langchain_core.messages import BaseMessage, HumanMessage
+from langchain_openai import ChatOpenAI
+
+from rai.agents.conversational_agent import create_conversational_agent
+from rai.config.models import BEDROCK_MULTIMODAL, OPENAI_MULTIMODAL
+from rai.tools.ros.cli import (
+    Ros2ActionTool,
+    Ros2InterfaceTool,
+    Ros2ServiceTool,
+    Ros2TopicTool,
+)
+
+
+class Agent:
+    def __init__(self, vendor: Literal["openai", "bedrock"]):
+        self.vendor = vendor
+        self.history: List[BaseMessage] = []
+        if vendor == "openai":
+            self.llm = ChatOpenAI(**OPENAI_MULTIMODAL)
+        else:
+            self.llm = ChatBedrock(**BEDROCK_MULTIMODAL)
+
+
+class ROS2Agent(Agent):
+    def __init__(self, vendor: Literal["openai", "bedrock"]):
+        super().__init__(vendor)
+        self.tools = [
+            Ros2TopicTool(),
+            Ros2InterfaceTool(),
+            Ros2ServiceTool(),
+            Ros2ActionTool(),
+        ]
+        self.agent = create_conversational_agent(
+            self.llm, self.tools, "You are a ROS2 expert."
+        )
+
+    def __call__(self, message: str):
+        self.history.append(HumanMessage(content=message))
+        response = self.agent.invoke({"messages": self.history})
+        output = response["messages"][-1].content
+        return output

src/rai/rai/tools/ros/cli.py

@@ -58,7 +58,7 @@ class Ros2TopicTool(BaseTool):
     def _run(self, command: str):
         """Executes the specified ROS2 topic command."""
         result = subprocess.run(
-            f"ros2 topic {command}", shell=True, capture_output=True
+            f"ros2 topic {command}", shell=True, capture_output=True, timeout=2
         )
         return result
 
@@ -95,7 +95,7 @@ class Ros2InterfaceTool(BaseTool):
 
     def _run(self, command: str):
         command = f"ros2 interface {command}"
-        result = subprocess.run(command, shell=True, capture_output=True)
+        result = subprocess.run(command, shell=True, capture_output=True, timeout=2)
         return result
 
 
@@ -129,5 +129,39 @@ class Ros2ServiceTool(BaseTool):
 
     def _run(self, command: str):
         command = f"ros2 service {command}"
+        result = subprocess.run(command, shell=True, capture_output=True, timeout=2)
+        return result
+
+
+class Ros2ActionToolInput(BaseModel):
+    """Input for the ros2_action tool."""
+
+    command: str = Field(..., description="The command to run")
+
+
+class Ros2ActionTool(BaseTool):
+    name: str = "Ros2ActionTool"
+
+    description: str = """
+    usage: ros2 action [-h] Call `ros2 action <command> -h` for more detailed usage. ...
+
+    Various action related sub-commands
+
+    options:
+      -h, --help            show this help message and exit
+
+    Commands:
+      info       Print information about an action
+      list       Output a list of action names
+      send_goal  Send an action goal
+      type       Print a action's type
+
+      Call `ros2 action <command> -h` for more detailed usage.
+    """
+
+    args_schema: Type[Ros2ActionToolInput] = Ros2ActionToolInput
+
+    def _run(self, command: str):
+        command = f"ros2 action {command}"
         result = subprocess.run(command, shell=True, capture_output=True)
         return result

src/rai_hmi/rai_hmi/streamlit_hmi_node.py

@@ -31,12 +31,12 @@
 from std_msgs.msg import String
 from std_srvs.srv import Trigger
 
+from rai.agents.conversational_agent import State as ConversationState
+from rai.agents.conversational_agent import create_conversational_agent
 from rai.extensions.navigator import RaiNavigator
 from rai.messages import HumanMultimodalMessage, ToolMultimodalMessage
 from rai.node import RaiBaseNode
 from rai.tools.ros.native import GetCameraImage, Ros2GetTopicsNamesAndTypesTool
-from rai_hmi.agent import State as ConversationState
-from rai_hmi.agent import create_conversational_agent
 from rai_hmi.task import Task
 from rai_interfaces.srv import VectorStoreRetrieval