Merge branch 'concept_docs' into eugene/concepts_runnables

docs/docs/concepts/agents.mdx

@@ -0,0 +1,40 @@
+# Agents
+
+By themselves, language models can't take actions - they just output text.
+A big use case for LangChain is creating **agents**.
+Agents are systems that use an LLM as a reasoning engine to determine which actions to take and what the inputs to those actions should be.
+The results of those actions can then be fed back into the agent and it determine whether more actions are needed, or whether it is okay to finish.
+
+[LangGraph](https://github.com/langchain-ai/langgraph) is an extension of LangChain specifically aimed at creating highly controllable and customizable agents.
+Please check out that documentation for a more in depth overview of agent concepts.
+
+There is a legacy `agent` concept in LangChain that we are moving towards deprecating: `AgentExecutor`.
+AgentExecutor was essentially a runtime for agents.
+It was a great place to get started, however, it was not flexible enough as you started to have more customized agents.
+In order to solve that we built LangGraph to be this flexible, highly-controllable runtime.
+
+If you are still using AgentExecutor, do not fear: we still have a guide on [how to use AgentExecutor](/docs/how_to/agent_executor).
+It is recommended, however, that you start to transition to LangGraph.
+In order to assist in this, we have put together a [transition guide on how to do so](/docs/how_to/migrate_agent).
+
+## ReAct agents
+<span data-heading-keywords="react,react agent"></span>
+
+One popular architecture for building agents is [**ReAct**](https://arxiv.org/abs/2210.03629).
+ReAct combines reasoning and acting in an iterative process - in fact the name "ReAct" stands for "Reason" and "Act".
+
+The general flow looks like this:
+
+- The model will "think" about what step to take in response to an input and any previous observations.
+- The model will then choose an action from available tools (or choose to respond to the user).
+- The model will generate arguments to that tool.
+- The agent runtime (executor) will parse out the chosen tool and call it with the generated arguments.
+- The executor will return the results of the tool call back to the model as an observation.
+- This process repeats until the agent chooses to respond.
+
+There are general prompting based implementations that do not require any model-specific features, but the most
+reliable implementations use features like [tool calling](/docs/how_to/tool_calling/) to reliably format outputs
+and reduce variance.
+
+Please see the [LangGraph documentation](https://langchain-ai.github.io/langgraph/) for more information,
+or [this how-to guide](/docs/how_to/migrate_agent/) for specific information on migrating to LangGraph.

docs/docs/concepts/callbacks.mdx

@@ -1,4 +1,17 @@
 # Callbacks
 
-PLACEHOLDER TO BE REPLACED BY ACTUAL DOCUMENTATION
-USED TO MAKE SURE THAT WE DO NOT FORGET TO ADD LINKS LATER
+The lowest level way to stream outputs from LLMs in LangChain is via the [callbacks](/docs/concepts/#callbacks) system. You can pass a
+callback handler that handles the [`on_llm_new_token`](https://python.langchain.com/api_reference/langchain/callbacks/langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.html#langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.on_llm_new_token) event into LangChain components. When that component is invoked, any
+[LLM](/docs/concepts/#llms) or [chat model](/docs/concepts/#chat-models) contained in the component calls
+the callback with the generated token. Within the callback, you could pipe the tokens into some other destination, e.g. a HTTP response.
+You can also handle the [`on_llm_end`](https://python.langchain.com/api_reference/langchain/callbacks/langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.html#langchain.callbacks.streaming_aiter.AsyncIteratorCallbackHandler.on_llm_end) event to perform any necessary cleanup.
+
+You can see [this how-to section](/docs/how_to/#callbacks) for more specifics on using callbacks.
+
+Callbacks were the first technique for streaming introduced in LangChain. While powerful and generalizable,
+they can be unwieldy for developers. For example:
+
+- You need to explicitly initialize and manage some aggregator or other stream to collect results.
+- The execution order isn't explicitly guaranteed, and you could theoretically have a callback run after the `.invoke()` method finishes.
+- Providers would often make you pass an additional parameter to stream outputs instead of returning them all at once.
+- You would often ignore the result of the actual model call in favor of callback results.

docs/docs/concepts/chat_models.mdx

@@ -40,88 +40,6 @@ Please see the [tool calling section](/docs/concepts/#functiontool-calling) for
 
 For specifics on how to use chat models, see the [relevant how-to guides here](/docs/how_to/#chat-models).
 
-* Follows the Runnable Interface: efficient batching, async, streaming etc.
-
-### Messages
-
-Some language models take a list of messages as input and return a message.
-There are a few different types of messages.
-All messages have a `role`, `content`, and `response_metadata` property.
-
-The `role` describes WHO is saying the message. The standard roles are "user", "assistant", "system", and "tool".
-LangChain has different message classes for different roles.
-
-The `content` property describes the content of the message.
-This can be a few different things:
-
-- A string (most models deal with this type of content)
-- A List of dictionaries (this is used for multimodal input, where the dictionary contains information about that input type and that input location)
-
-Optionally, messages can have a `name` property which allows for differentiating between multiple speakers with the same role.
-For example, if there are two users in the chat history it can be useful to differentiate between them. Not all models support this.
-
-#### HumanMessage
-
-This represents a message with role "user".
-
-#### AIMessage
-
-This represents a message with role "assistant". In addition to the `content` property, these messages also have:
-
-**`response_metadata`**
-
-The `response_metadata` property contains additional metadata about the response. The data here is often specific to each model provider.
-This is where information like log-probs and token usage may be stored.
-
-**`tool_calls`**
-
-These represent a decision from a language model to call a tool. They are included as part of an `AIMessage` output.
-They can be accessed from there with the `.tool_calls` property.
-
-This property returns a list of `ToolCall`s. A `ToolCall` is a dictionary with the following arguments:
-
-- `name`: The name of the tool that should be called.
-- `args`: The arguments to that tool.
-- `id`: The id of that tool call.
-
-#### SystemMessage
-
-This represents a message with role "system", which tells the model how to behave. Not every model provider supports this.
-
-#### ToolMessage
-
-This represents a message with role "tool", which contains the result of calling a tool. In addition to `role` and `content`, this message has:
-
-- a `tool_call_id` field which conveys the id of the call to the tool that was called to produce this result.
-- an `artifact` field which can be used to pass along arbitrary artifacts of the tool execution which are useful to track but which should not be sent to the model.
-
-With most chat models, a `ToolMessage` can only appear in the chat history after an `AIMessage` that has a populated `tool_calls` field.
-
-#### (Legacy) FunctionMessage
-
-This is a legacy message type, corresponding to OpenAI's legacy function-calling API. `ToolMessage` should be used instead to correspond to the updated tool-calling API.
-
-This represents the result of a function call. In addition to `role` and `content`, this message has a `name` parameter which conveys the name of the function that was called to produce this result.
-
-
-## System Message
-
-## User Message
-
-## AI Message
-
-### Usage Metadata
-
-## Tool Messages
-
-### Artifacts
-
-## Chat Model Context Length
-
-## Tokenization
-
-## Tool-calling
-
 ## Cache
 
 * Cache is available, but should be exercised with caution.

docs/docs/concepts/embedding_models.mdx

@@ -0,0 +1,15 @@
+# Embedding models
+<span data-heading-keywords="embedding,embeddings"></span>
+
+Embedding models create a vector representation of a piece of text. You can think of a vector as an array of numbers that captures the semantic meaning of the text.
+By representing the text in this way, you can perform mathematical operations that allow you to do things like search for other pieces of text that are most similar in meaning.
+These natural language search capabilities underpin many types of [context retrieval](/docs/concepts/#retrieval),
+where we provide an LLM with the relevant data it needs to effectively respond to a query.
+
+![](/img/embeddings.png)
+
+The `Embeddings` class is a class designed for interfacing with text embedding models. There are many different embedding model providers (OpenAI, Cohere, Hugging Face, etc) and local models, and this class is designed to provide a standard interface for all of them.
+
+The base Embeddings class in LangChain provides two methods: one for embedding documents and one for embedding a query. The former takes as input multiple texts, while the latter takes a single text. The reason for having these as two separate methods is that some embedding providers have different embedding methods for documents (to be searched over) vs queries (the search query itself).
+
+For specifics on how to use embedding models, see the [relevant how-to guides here](/docs/how_to/#embedding-models).