-
Notifications
You must be signed in to change notification settings - Fork 2.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[Feature Request]: RAGFlow API proposal #1102
Comments
This is exactly what I am looking for... |
I am wondering with flaskrest,flaskrestplus will helps a lot. or flask_restx |
for me: Also even after this change I could not call the create dataset endpoint. |
Hi jeremi, thanks for your question. I would like to inform you that we have introduced a newly proposed API endpoint - http://<host_address>/api/v1/. The previous URL you mentioned is now deprecated . If you want to create a dataset, you can use http://<host_address>/api/v1/dataset by POST request. |
I tried it by building the latest main, and it does not work; I get a 404 with some HTML as a returned value. If I invert API and version number, I get a JSON response, but with a 404 in the body:
|
Could you share your screenshot for the input and output? |
### What problem does this PR solve? Add retrieval api on a specific knowledge base ![ragflow](https://github.com/user-attachments/assets/dc30a4c3-03c5-4d34-bb7c-60b8830f1225) #1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality)
i have the same question |
Same issue. If I post
I've checked the code and I guess it happens because login is required to make a requests. So my question is, how do I login? |
First, |
### What problem does this PR solve? Complete implementation of dataset SDK. #1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Feiue <[email protected]> Co-authored-by: Kevin Hu <[email protected]>
### What problem does this PR solve? Complete DataSet SDK implementation #1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Feiue <[email protected]>
### What problem does this PR solve? SDK for Assistant #1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) Co-authored-by: Feiue <[email protected]>
### What problem does this PR solve? SDK for session #1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Feiue <[email protected]> Co-authored-by: Kevin Hu <[email protected]>
### What problem does this PR solve? Includes SDK for creating, updating sessions, getting sessions, listing sessions, and dialogues #1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: liuhua <[email protected]>
这部分创建知识库API的代码 #1106 看上去和 https://github.com/infiniflow/ragflow/blob/main/api/apps/kb_app.py#L39 应用中的创建知识库API的方法内容差距挺大,而且 API的路径也完全不同,我理解这出自不同开发人员,而且知识库和数据集确实是一一对应的,但创建知识库就是创建知识库,用 /api/v1/dataset 这样的url会很难受啊 我之前因为急需要用 retrieval api 所以也提了个 PR #1763 但其实方法的内容主要都是从 https://github.com/infiniflow/ragflow/blob/main/api/apps/chunk_app.py#L256 的retrieval_test方法复制过来的 这样的开发方式让我对代码的健壮性比较担忧,屎山都是你一铲子我一铲子堆起来的,趁屎堆不大就该早点解决 所以能不能对 API 部分重构一下,我初步的设想:
大家如果有更好的想法可以提出来我们一起讨论,有好的方案我可以来开发这部分内容 |
IMHO 依我拙见,可以基于RAGFLOW已有实现的可以对外提供的resources[dataset,agent,dialog,conversation,tenant,user]来RESTFUI,或者openAPI标准更佳:- ) |
@Valdanitooooo @yangboz Thank you guys comments on RAGFlow API. |
Good point. We're gona spend more time on this. |
@yangboz @JinHai-CN @KevinHuSh In order to not disrupt existing functionality, I am attempting to refactor the API in a new directory. The most ideal scenario is for the Web APP API, Server API, and SDK API to all use the same code. I hope everything goes smoothly. |
Hint: APIs to Web/SDK/developers are somewhat different. |
What about choosing a relative naive one to file a pull request? |
I won't do everything at once. I hope to only complete the API for the dataset as a starting point, and then everyone can discuss together what defects and issues are. If the solution is mature, then assign the API of each resource to different people to complete. |
### What problem does this PR solve? discuss:#1102 #### Completed 1. Integrate API Flask to generate Swagger API documentation, through http://ragflow_host:ragflow_port/v1/docs visit 2. Refactored http_token_auth ``` class AuthUser: def __init__(self, tenant_id, token): self.id = tenant_id self.token = token def get_token(self): return self.token @http_token_auth.verify_token def verify_token(token: str) -> Union[AuthUser, None]: try: objs = APIToken.query(token=token) if objs: api_token = objs[0] user = AuthUser(api_token.tenant_id, api_token.token) return user except Exception as e: server_error_response(e) return None # resources api @manager.auth_required(http_token_auth) def get_all_datasets(query_data): .... ``` 3. Refactored the Datasets (Knowledgebase) API to extract the implementation logic into the api/apps/services directory ![image](https://github.com/user-attachments/assets/ad1f16f1-b0ce-4301-855f-6e162163f99a) 4. Python SDK, I only added get_all_datasets as an attempt, Just to verify that SDK API and Server API can use the same method. ``` from ragflow.ragflow import RAGFLow ragflow = RAGFLow('<ACCESS_KEY>', 'http://127.0.0.1:9380') ragflow.get_all_datasets() ``` 5. Request parameter validation, as an attempt, may not be necessary as this feature is already present at the data model layer. This is mainly easier to test the API in Swagger Docs service ``` class UpdateDatasetReq(Schema): kb_id = fields.String(required=True) name = fields.String(validate=validators.Length(min=1, max=128)) description = fields.String(allow_none=True) permission = fields.String(validate=validators.OneOf(['me', 'team'])) embd_id = fields.String(validate=validators.Length(min=1, max=128)) language = fields.String(validate=validators.OneOf(['Chinese', 'English'])) parser_id = fields.String(validate=validators.OneOf([parser_type.value for parser_type in ParserType])) parser_config = fields.Dict() avatar = fields.String() ``` #### TODO 1. Simultaneously supporting multiple authentication methods, so that the Web API can use the same method as the Server API, but perhaps this feature is not important. I tried using this method, but it was not successful. It only allows token authentication when not logged in, but cannot skip token authentication when logged in 😢 ``` def http_basic_auth_required(func): @wraps(func) def decorated_view(*args, **kwargs): if 'Authorization' in flask_request.headers: # If the request header contains a token, skip username and password verification return func(*args, **kwargs) if flask_request.method in EXEMPT_METHODS or current_app.config.get("LOGIN_DISABLED"): pass elif not current_user.is_authenticated: return current_app.login_manager.unauthorized() if callable(getattr(current_app, "ensure_sync", None)): return current_app.ensure_sync(func)(*args, **kwargs) return func(*args, **kwargs) return decorated_view ``` 2. Refactoring the SDK API using the same method as the Server API is feasible and constructive, but it still requires time I see some differences between the Web and SDK APIs, such as the key_mapping handling of the returned results. Until I figure it out, I cannot modify these codes to avoid causing more problems ``` for kb in kbs: key_mapping = { "chunk_num": "chunk_count", "doc_num": "document_count", "parser_id": "parse_method", "embd_id": "embedding_model" } renamed_data = {} for key, value in kb.items(): new_key = key_mapping.get(key, key) renamed_data[new_key] = value renamed_list.append(renamed_data) return get_json_result(data=renamed_list) ``` ### Type of change - [x] Refactoring
### What problem does this PR solve? #1102 ### Type of change - [x] Performance Improvement --------- Co-authored-by: Kevin Hu <[email protected]>
It actually is a valid URL. As you can se in the documentation of ragflow_api, Not only do you require the token to use this endpoint, but also a login is needed, just check the dataset_api.py file: The decorator If you compare this code with another used to implement a method that just requires a valid token, lets say, Get conversation history, we can notice how the Or like in the case of the Get answer method, just isn't there at all. So, anyone knows how to log in? |
The ragflow_api may have some issues. You can look |
I used RAGFlow as a knowledge base management tool and refactored some APIs that need to be used in my application.
swagger docs: http://your_ragflow_address/v1/docs
sdk usage
|
### What problem does this PR solve? #1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality)
I request the interface of login, and give the payload data(account and password), and successfully acquire the response. |
### What problem does this PR solve? Add retrieval api on a specific knowledge base ![ragflow](https://github.com/user-attachments/assets/dc30a4c3-03c5-4d34-bb7c-60b8830f1225) infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality)
### What problem does this PR solve? Complete implementation of dataset SDK. infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Feiue <[email protected]> Co-authored-by: Kevin Hu <[email protected]>
### What problem does this PR solve? Complete DataSet SDK implementation infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Feiue <[email protected]>
### What problem does this PR solve? SDK for Assistant infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) Co-authored-by: Feiue <[email protected]>
### What problem does this PR solve? SDK for session infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: Feiue <[email protected]> Co-authored-by: Kevin Hu <[email protected]>
### What problem does this PR solve? Includes SDK for creating, updating sessions, getting sessions, listing sessions, and dialogues infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality) --------- Co-authored-by: liuhua <[email protected]>
### What problem does this PR solve? infiniflow#1102 ### Type of change - [x] Performance Improvement --------- Co-authored-by: Kevin Hu <[email protected]>
### What problem does this PR solve? infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality)
…nfiniflow#3121) ### What problem does this PR solve? feat: Delete http_api_reference.md from api folder infiniflow#1102 ### Type of change - [x] New Feature (non-breaking change which adds functionality)
Is there an existing issue for the same feature request?
Describe the feature you'd like
RAGFlow's API interfaces are not enough and theAPI are not RESTful style. The goal of this issue is to propose the RESTFul APIs which covers most functions of RAGFlow.
Knowledge base
Content management in knowledge base
AI assistant management
Conversation management
File management
Related issues: #345 #717
The text was updated successfully, but these errors were encountered: