Skip to content

Commit

Permalink
add docs
Browse files Browse the repository at this point in the history
  • Loading branch information
Yunnglin committed Dec 26, 2024
1 parent 79b8adb commit 187348e
Show file tree
Hide file tree
Showing 8 changed files with 287 additions and 22 deletions.
97 changes: 97 additions & 0 deletions docs/docs/integrations/chat/modelscope_endpoint.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,97 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# ModelScope Chat Endpoint\n",
"\n",
"ModelScope ([Home](https://www.modelscope.cn/) | [GitHub](https://github.com/modelscope/modelscope)) is built upon the notion of “Model-as-a-Service” (MaaS). It seeks to bring together most advanced machine learning models from the AI community, and streamlines the process of leveraging AI models in real-world applications. The core ModelScope library open-sourced in this repository provides the interfaces and implementations that allow developers to perform model inference, training and evaluation.\n",
"\n",
"This example goes over how to use LangChain to interact with ModelScope Chat Endpoint.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"Generate your sdk token from: https://modelscope.cn/my/myaccesstoken"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"MODELSCOPE_SDK_TOKEN\"] = \"YOUR_SDK_TOKEN\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Available models: https://modelscope.cn/docs/model-service/API-Inference/intro"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"AIMessage(content='Certainly! Sorting an array is a common task in programming, and Python provides several ways to do it. Below is a simple example using the built-in `sorted()` function and the `sort()` method for lists. I\\'ll also include a basic implementation of the Bubble Sort algorithm for educational purposes.\\n\\n### Using the Built-in `sorted()` Function\\n\\nThe `sorted()` function returns a new sorted list from the elements of any iterable.\\n\\n```python\\ndef sort_array(arr):\\n return sorted(arr)\\n\\n# Example usage\\narray = [5, 2, 9, 1, 5, 6]\\nsorted_array = sort_array(array)\\nprint(\"Sorted array:\", sorted_array)\\n```\\n\\n### Using the List\\'s `sort()` Method\\n\\nThe `sort()` method sorts the list in place and returns `None`.\\n\\n```python\\ndef sort_array_in_place(arr):\\n arr.sort()\\n\\n# Example usage\\narray = [5, 2, 9, 1, 5, 6]\\nsort_array_in_place(array)\\nprint(\"Sorted array:\", array)\\n```\\n\\n### Implementing Bubble Sort\\n\\nBubble Sort is a simple sorting algorithm that repeatedly steps through the list, compares adjacent elements, and swaps them if they are in the wrong order. The pass through the list is repeated until the list is sorted.\\n\\n```python\\ndef bubble_sort(arr):\\n n = len(arr)\\n for i in range(n):\\n # Track if any swap was made\\n swapped = False\\n for j in range(0, n-i-1):\\n if arr[j] > arr[j+1]:\\n # Swap if the element found is greater than the next element\\n arr[j], arr[j+1] = arr[j+1], arr[j]\\n swapped = True\\n # If no two elements were swapped by inner loop, then break\\n if not swapped:\\n break\\n\\n# Example usage\\narray = [5, 2, 9, 1, 5, 6]\\nbubble_sort(array)\\nprint(\"Sorted array:\", array)\\n```\\n\\nThese examples demonstrate different ways to sort an array in Python. The built-in methods are highly optimized and should be preferred for most use cases. The Bubble Sort implementation is more educational and shows how a basic sorting algorithm works.', additional_kwargs={}, response_metadata={'token_usage': {'completion_tokens': 466, 'prompt_tokens': 26, 'total_tokens': 492, 'completion_tokens_details': None, 'prompt_tokens_details': None}, 'model_name': 'Qwen/Qwen2.5-Coder-32B-Instruct', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-f05a1ba2-7be9-480b-b7aa-7eb3aeaf3c23-0')"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_community.chat_models import ModelScopeChatEndpoint\n",
"\n",
"llm = ModelScopeChatEndpoint(model=\"Qwen/Qwen2.5-Coder-32B-Instruct\")\n",
"\n",
"llm.invoke(\"write a python program to sort an array\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"for chunk in llm.stream(\"write a python program to sort an array\"):\n",
" print(chunk)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "langchain",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.16"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
168 changes: 168 additions & 0 deletions docs/docs/integrations/llms/modelscope_endpoint.ipynb
Original file line number Diff line number Diff line change
@@ -0,0 +1,168 @@
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# ModelScope Endpoint\n",
"\n",
"ModelScope ([Home](https://www.modelscope.cn/) | [GitHub](https://github.com/modelscope/modelscope)) is built upon the notion of “Model-as-a-Service” (MaaS). It seeks to bring together most advanced machine learning models from the AI community, and streamlines the process of leveraging AI models in real-world applications. The core ModelScope library open-sourced in this repository provides the interfaces and implementations that allow developers to perform model inference, training and evaluation.\n",
"\n",
"This example goes over how to use LangChain to interact with ModelScope Chat Endpoint.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"Generate your sdk token from: https://modelscope.cn/my/myaccesstoken"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"\n",
"os.environ[\"MODELSCOPE_SDK_TOKEN\"] = \"bc34909d-af9a-42e5-b483-ce4dd246f368\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Available models: https://modelscope.cn/docs/model-service/API-Inference/intro"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"'Certainly! Sorting an array can be done in Python using various methods. One of the simplest ways is to use Python\\'s built-in sorting functions. Below are examples of how you can sort an array using both the `sort()` method and the `sorted()` function.\\n\\n### Using the `sort()` Method\\nThe `sort()` method sorts the list in place and modifies the original list.\\n\\n```python\\n# Define the array\\narray = [5, 2, 9, 1, 5, 6]\\n\\n# Sort the array in ascending order\\narray.sort()\\n\\n# Print the sorted array\\nprint(\"Sorted array (ascending):\", array)\\n\\n# Sort the array in descending order\\narray.sort(reverse=True)\\n\\n# Print the sorted array\\nprint(\"Sorted array (descending):\", array)\\n```\\n\\n### Using the `sorted()` Function\\nThe `sorted()` function returns a new list that is sorted, leaving the original list unchanged.\\n\\n```python\\n# Define the array\\narray = [5, 2, 9, 1, 5, 6]\\n\\n# Get a new sorted array in ascending order\\nsorted_array_asc = sorted(array)\\n\\n# Print the sorted array\\nprint(\"Sorted array (ascending):\", sorted_array_asc)\\n\\n# Get a new sorted array in descending order\\nsorted_array_desc = sorted(array, reverse=True)\\n\\n# Print the sorted array\\nprint(\"Sorted array (descending):\", sorted_array_desc)\\n```\\n\\n### Custom Sorting\\nYou can also sort based on custom criteria using a key function.\\n\\n```python\\n# Define the array of tuples\\narray_of_tuples = [(1, \\'one\\'), (3, \\'three\\'), (2, \\'two\\')]\\n\\n# Sort the array of tuples based on the second element of each tuple\\nsorted_by_second_element = sorted(array_of_tuples, key=lambda x: x[1])\\n\\n# Print the sorted array\\nprint(\"Sorted array by second element:\", sorted_by_second_element)\\n```\\n\\nThese examples demonstrate how to sort arrays in Python using different methods. You can choose the one that best fits your needs.'"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"from langchain_community.llms import ModelScopeEndpoint\n",
"\n",
"llm = ModelScopeEndpoint(model=\"Qwen/Qwen2.5-Coder-32B-Instruct\")\n",
"\n",
"llm.invoke(\"write a python program to sort an array\")"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Certainly! Sorting an array can be done in Python using various methods. One of the simplest ways is to use Python's built-in sorting functions. However, if you want to implement a sorting algorithm yourself, I'll show you how to do it using the Bubble Sort algorithm as an example.\n",
"\n",
"Here's a Python program that sorts an array using both the built-in method and the Bubble Sort algorithm:\n",
"\n",
"### Using Built-in Method\n",
"\n",
"Python provides a built-in method called `sort()` for lists, or you can use the `sorted()` function which returns a new sorted list.\n",
"\n",
"```python\n",
"# Using built-in sort() method\n",
"def sort_array_builtin(arr):\n",
" arr.sort()\n",
" return arr\n",
"\n",
"# Using built-in sorted() function\n",
"def sort_array_sorted(arr):\n",
" return sorted(arr)\n",
"\n",
"# Example usage\n",
"array = [64, 34, 25, 12, 22, 11, 90]\n",
"print(\"Original array:\", array)\n",
"\n",
"sorted_array_builtin = sort_array_builtin(array.copy())\n",
"print(\"Sorted array using sort():\", sorted_array_builtin)\n",
"\n",
"sorted_array_sorted = sort_array_sorted(array.copy())\n",
"print(\"Sorted array using sorted():\", sorted_array_sorted)\n",
"```\n",
"\n",
"### Using Bubble Sort Algorithm\n",
"\n",
"Bubble Sort is a simple sorting algorithm that repeatedly steps through the list, compares adjacent elements, and swaps them if they are in the wrong order. The pass through the list is repeated until the list is sorted.\n",
"\n",
"```python\n",
"# Using Bubble Sort algorithm\n",
"def bubble_sort(arr):\n",
" n = len(arr)\n",
" for i in range(n):\n",
" # Track if any swapping happens\n",
" swapped = False\n",
" for j in range(0, n-i-1):\n",
" if arr[j] > arr[j+1]:\n",
" # Swap if the element found is greater than the next element\n",
" arr[j], arr[j+1] = arr[j+1], arr[j]\n",
" swapped = True\n",
" # If no two elements were swapped by inner loop, then break\n",
" if not swapped:\n",
" break\n",
" return arr\n",
"\n",
"# Example usage\n",
"array = [64, 34, 25, 12, 22, 11, 90]\n",
"print(\"Original array:\", array)\n",
"\n",
"sorted_array_bubble = bubble_sort(array.copy())\n",
"print(\"Sorted array using Bubble Sort:\", sorted_array_bubble)\n",
"```\n",
"\n",
"### Explanation\n",
"\n",
"- **Built-in Methods**: These are highly optimized and should be preferred for most use cases.\n",
"- **Bubble Sort**: This is a simple algorithm but not very efficient for large datasets (O(n^2) time complexity). It's useful for educational purposes to understand basic sorting concepts.\n",
"\n",
"You can choose either method based on your needs. For practical applications, the built-in methods are recommended due to their efficiency and simplicity."
]
}
],
"source": [
"for chunk in llm.stream(\"write a python program to sort an array\"):\n",
" print(chunk, end=\"\", flush=True)"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "langchain",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.10.16"
}
},
"nbformat": 4,
"nbformat_minor": 2
}
6 changes: 3 additions & 3 deletions libs/community/langchain_community/chat_models/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,7 +123,7 @@
ChatMLX,
)
from langchain_community.chat_models.modelscope_endpoint import (
ModelscopeChatEndpoint,
ModelScopeChatEndpoint,
)
from langchain_community.chat_models.moonshot import (
MoonshotChat,
Expand Down Expand Up @@ -254,7 +254,7 @@
"JinaChat",
"LlamaEdgeChatService",
"MiniMaxChat",
"ModelscopeChatEndpoint",
"ModelScopeChatEndpoint",
"MoonshotChat",
"PaiEasChatEndpoint",
"PromptLayerChatOpenAI",
Expand Down Expand Up @@ -320,7 +320,7 @@
"JinaChat": "langchain_community.chat_models.jinachat",
"LlamaEdgeChatService": "langchain_community.chat_models.llama_edge",
"MiniMaxChat": "langchain_community.chat_models.minimax",
"ModelscopeChatEndpoint": "langchain_community.chat_models.modelscope_endpoint",
"ModelScopeChatEndpoint": "langchain_community.chat_models.modelscope_endpoint",
"MoonshotChat": "langchain_community.chat_models.moonshot",
"PaiEasChatEndpoint": "langchain_community.chat_models.pai_eas_endpoint",
"PromptLayerChatOpenAI": "langchain_community.chat_models.promptlayer_openai",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,11 @@
from langchain_community.chat_models import ChatOpenAI
from langchain_community.llms.modelscope_endpoint import (
MODELSCOPE_SERVICE_URL_BASE,
ModelscopeCommon,
ModelScopeCommon,
)


class ModelscopeChatEndpoint(ModelscopeCommon, ChatOpenAI): # type: ignore[misc, override, override]
class ModelScopeChatEndpoint(ModelScopeCommon, ChatOpenAI): # type: ignore[misc, override, override]
"""Modelscope chat model inference api integration. To use, must have a modelscope account and a modelscope sdk token.
Refer to https://modelscope.cn/docs/model-service/API-Inference/intro for more details.
Expand Down
8 changes: 4 additions & 4 deletions libs/community/langchain_community/llms/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -369,9 +369,9 @@ def _import_modal() -> Type[BaseLLM]:


def _import_modelscope_endpoint() -> Type[BaseLLM]:
from langchain_community.llms.modelscope_endpoint import ModelscopeEndpoint
from langchain_community.llms.modelscope_endpoint import ModelScopeEndpoint

return ModelscopeEndpoint
return ModelScopeEndpoint


def _import_mosaicml() -> Type[BaseLLM]:
Expand Down Expand Up @@ -791,7 +791,7 @@ def __getattr__(name: str) -> Any:
return _import_mlx_pipeline()
elif name == "Modal":
return _import_modal()
elif name == "ModelscopeEndpoint":
elif name == "ModelScopeEndpoint":
return _import_modelscope_endpoint()
elif name == "MosaicML":
return _import_mosaicml()
Expand Down Expand Up @@ -955,7 +955,7 @@ def __getattr__(name: str) -> Any:
"MlflowAIGateway",
"MLXPipeline",
"Modal",
"ModelscopeEndpoint",
"ModelScopeEndpoint",
"MosaicML",
"NIBittensorLLM",
"NLPCloud",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,7 @@ def _convert_chunk_to_str(chunk: str) -> str:
return text


class ModelscopeClient(BaseModel):
class ModelScopeClient(BaseModel):
"""An API client that talks to the Modelscope api inference server."""

api_key: SecretStr
Expand Down Expand Up @@ -102,7 +102,7 @@ async def astream(self, request: Any) -> AsyncIterator[str]:
yield text


class ModelscopeCommon(BaseModel):
class ModelScopeCommon(BaseModel):
"""Common parameters for Modelscope LLMs."""

client: Any
Expand Down Expand Up @@ -159,15 +159,15 @@ def validate_environment(cls, values: Dict) -> Dict:
)
)

values["client"] = ModelscopeClient(
values["client"] = ModelScopeClient(
api_key=values["modelscope_sdk_token"],
base_url=values["base_url"],
timeout=values["timeout"],
)
return values


class ModelscopeEndpoint(ModelscopeCommon, LLM):
class ModelScopeEndpoint(ModelScopeCommon, LLM):
"""Modelscope model inference API endpoint.
To use, you should have a modelscope account and the environment variable ``MODELSCOPE_SDK_TOKEN`` set with your
Expand Down
Loading

0 comments on commit 187348e

Please sign in to comment.