Merge remote-tracking branch 'origin/dev-minor' into update-docs

.github/workflows/integration-test-workflow-debian.yml

@@ -117,8 +117,44 @@ jobs:
         poetry run python tests/integration/harness_sdk.py test_user_search_and_rag
         poetry run python tests/integration/harness_sdk.py test_user_password_management
         poetry run python tests/integration/harness_sdk.py test_user_profile_management
+        poetry run python tests/integration/harness_sdk.py test_user_overview
         poetry run python tests/integration/harness_sdk.py test_user_logout
 
+    - name: Run Collections
+      working-directory: ./py
+      run: |
+        poetry run python tests/integration/harness_sdk.py test_user_creates_collection
+        poetry run python tests/integration/harness_sdk.py test_user_updates_collection
+        poetry run python tests/integration/harness_sdk.py test_user_lists_collections
+        poetry run python tests/integration/harness_sdk.py test_user_collection_document_management
+        poetry run python tests/integration/harness_sdk.py test_user_removes_document_from_collection
+        poetry run python tests/integration/harness_sdk.py test_user_lists_documents_in_collection
+        poetry run python tests/integration/harness_sdk.py test_pagination_and_filtering
+        poetry run python tests/integration/harness_sdk.py test_advanced_collection_management
+        poetry run python tests/integration/harness_sdk.py test_user_gets_collection_details
+        poetry run python tests/integration/harness_sdk.py test_user_adds_user_to_collection
+        poetry run python tests/integration/harness_sdk.py test_user_removes_user_from_collection
+        poetry run python tests/integration/harness_sdk.py test_user_lists_users_in_collection
+        poetry run python tests/integration/harness_sdk.py test_user_gets_collections_for_user
+        poetry run python tests/integration/harness_sdk.py test_user_gets_collections_for_document
+        poetry run python tests/integration/harness_sdk.py test_user_permissions
+
+    - name: Run SDK GraphRAG
+      working-directory: ./py
+      run: |
+        poetry run python tests/integration/harness_sdk.py test_remove_all_files_and_ingest_sample_file_sdk
+        poetry run python tests/integration/harness_sdk.py test_kg_create_graph_sample_file_sdk
+        poetry run python tests/integration/harness_sdk.py test_kg_enrich_graph_sample_file_sdk
+        poetry run python tests/integration/harness_sdk.py test_kg_search_sample_file_sdk
+
+    - name: Run CLI GraphRAG
+      working-directory: ./py
+      run: |
+        poetry run python tests/integration/harness_sdk.py test_remove_all_files_and_ingest_sample_file_sdk
+        poetry run python tests/integration/harness_cli.py test_kg_create_graph_sample_file_cli
+        poetry run python tests/integration/harness_cli.py test_kg_enrich_graph_sample_file_cli
+        poetry run python tests/integration/harness_cli.py test_kg_search_sample_file_cli
+
     - name: Stop R2R server
       if: always()
       run: ps aux | grep "r2r serve" | awk '{print $2}' | xargs kill || true

docs/documentation/deployment/introduction.mdx

@@ -27,7 +27,7 @@ For detailed, step-by-step instructions on deploying R2R in various environments
 
 - [Local Deployment](/documentation/installation/overview)
 - [Azure Deployment](/documentation/deployment/azure)
-- [SciPhi Cloud](/sciphi-cloud/introduction/)
+- [SciPhi Cloud](/documentation/deployment/sciphi/)
 
 Choose the guide that best fits your infrastructure and scaling needs. Each guide provides specific instructions for setting up R2R in that environment, including necessary configurations and best practices.
 

docs/documentation/deployment/sciphi.mdx

@@ -0,0 +1,35 @@
+---
+title: 'SciPhi Enterprise'
+description: 'Fully managed R2R for enterprises'
+icon: 'building'
+---
+
+# SciPhi Enterprise: Fully Managed R2R for Your Organization
+
+SciPhi offers a fully managed, enterprise-grade solution for deploying and scaling R2R (RAG to Riches) within your organization. Our SciPhi Enterprise offering provides all the benefits of R2R, including multimodal ingestion, hybrid search, GraphRAG, user management, and observability, in a hassle-free, scalable environment tailored to your business needs.
+
+## Why SciPhi Enterprise?
+
+- **Fully Managed**: We handle the infrastructure, deployment, scaling, updates, and maintenance of R2R, so your team can focus on building RAG applications.
+- **Scalable**: Seamlessly scale your R2R deployment to handle growing user bases, document collections, and query volumes.
+- **Secure**: Benefit from enterprise-level security, compliance, and data privacy measures.
+- **Customizable**: Tailor your R2R deployment to your organization's specific requirements and integrate with your existing systems and workflows.
+- **Expert Support**: Get direct access to the R2R team for guidance, troubleshooting, and best practices, ensuring your success with the platform.
+
+## Key Features
+
+- All the powerful features of R2R, including multimodal ingestion, hybrid search, GraphRAG, user management, and observability
+- Fully managed deployment and scaling across your choice of cloud provider or on-premises environment
+- Enterprise-grade security and compliance measures to protect your data and meet regulatory requirements
+- Customizable configuration and integration options to fit your organization's unique needs
+- Dedicated support and success management from the R2R team to ensure you get the most value from the platform
+
+## Getting Started
+
+Getting started with SciPhi Enterprise is easy:
+
+1. **Contact Us**: Reach out to our team at [[email protected]](mailto:[email protected]) to discuss your organization's RAG application needs and learn more about SciPhi Enterprise.
+2. **Discovery**: Our experts will work with you to understand your requirements, existing systems, and goals for R2R within your organization.
+3. **Deployment**: We'll handle the deployment and configuration of R2R in your environment, whether cloud-based or on-premises, and integrate with your existing systems and workflows.
+4. **Onboarding**: Our team will provide training and support to help your developers and users get up and running with R2R quickly and effectively.
+5. **Ongoing Support**: With SciPhi Enterprise, you'll have ongoing access to our team for support, updates, and guidance as you scale and evolve your RAG applications.

docs/documentation/js-sdk/auth.mdx

@@ -3,6 +3,11 @@ title: 'Authentication'
 description: 'Manage users in R2R with built-in authentication'
 ---
 
+<Note>
+Occasionally this SDK documentation falls out of date, cross-check with the automatcially generated <a href="/api-reference/introduction"> API Reference documentation </a> for the latest parameters.
+</Note>
+
+
 ## User Authentication and Management
 
 R2R provides a comprehensive set of user authentication and management features, allowing you to implement secure and feature-rich authentication systems in your applications.

docs/documentation/js-sdk/collections.mdx

@@ -0,0 +1,230 @@
+---
+title: 'Collection Management'
+description: 'Manage collections in the R2R TypeScript client'
+---
+
+<Note>
+Occasionally this SDK documentation falls out of date, cross-check with the automatically generated <a href="/api-reference/introduction"> API Reference documentation </a> for the latest parameters.
+</Note>
+
+A collection in R2R is a logical grouping of users and documents that allows for efficient access control and organization. Collections enable you to manage permissions and access to documents at a collection level, rather than individually.
+
+## Overview
+
+The R2R TypeScript client provides methods to manage collections, including:
+
+- Creating and deleting collections
+- Updating collection metadata
+- Listing collections
+- Adding and removing users from collections
+- Assigning and removing documents from collections
+- Retrieving users and documents in a collection
+
+## Usage
+
+### Creating a Collection
+
+To create a new collection, use the `createCollection` method:
+
+````typescript
+async createCollection(
+  name: string,
+  description?: string
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+const newCollection = await r2rClient.createCollection('My Collection', 'A sample collection');
+````
+
+### Updating a Collection
+
+To update a collection's name or description, use the `updateCollection` method:
+
+````typescript
+async updateCollection(
+  collectionId: string,
+  name?: string,
+  description?: string
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+await r2rClient.updateCollection('collection_id', 'Updated Name', 'Updated description');
+````
+
+### Deleting a Collection
+
+To delete a collection, use the `deleteCollection` method:
+
+````typescript
+async deleteCollection(collectionId: string): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+await r2rClient.deleteCollection('collection_id');
+````
+
+### Listing Collections
+
+To list all collections, use the `listCollections` method:
+
+````typescript
+async listCollections(
+  offset?: number,
+  limit?: number
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+const collections = await r2rClient.listCollections(0, 10);
+````
+
+### Adding a User to a Collection
+
+To add a user to a collection, use the `addUserToCollection` method:
+
+````typescript
+async addUserToCollection(
+  userId: string,
+  collectionId: string
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+await r2rClient.addUserToCollection('user_id', 'collection_id');
+````
+
+### Removing a User from a Collection
+
+To remove a user from a collection, use the `removeUserFromCollection` method:
+
+````typescript
+async removeUserFromCollection(
+  userId: string,
+  collectionId: string
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+await r2rClient.removeUserFromCollection('user_id', 'collection_id');
+````
+
+### Getting Users in a Collection
+
+To get all users in a collection, use the `getUsersInCollection` method:
+
+````typescript
+async getUsersInCollection(
+  collectionId: string,
+  offset?: number,
+  limit?: number
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+const users = await r2rClient.getUsersInCollection('collection_id', 0, 10);
+````
+
+### Getting Collections for a User
+
+To get all collections a user belongs to, use the `getCollectionsForUser` method:
+
+````typescript
+async getCollectionsForUser(
+  userId: string,
+  offset?: number,
+  limit?: number
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+const userCollections = await r2rClient.getCollectionsForUser('user_id', 0, 10);
+````
+
+### Assigning a Document to a Collection
+
+To assign a document to a collection, use the `assignDocumentToCollection` method:
+
+````typescript
+async assignDocumentToCollection(
+  document_id: string,
+  collection_id: string
+): Promise<any>
+````
+
+Example:
+````typescript
+await r2rClient.assignDocumentToCollection('document_id', 'collection_id');
+````
+
+### Removing a Document from a Collection
+
+To remove a document from a collection, use the `removeDocumentFromCollection` method:
+
+````typescript
+async removeDocumentFromCollection(
+  document_id: string,
+  collection_id: string
+): Promise<any>
+````
+
+Example:
+````typescript
+await r2rClient.removeDocumentFromCollection('document_id', 'collection_id');
+````
+
+### Getting Collections for a Document
+
+To get all collections a document is assigned to, use the `getDocumentCollections` method:
+
+````typescript
+async getDocumentCollections(
+  documentId: string,
+  offset?: number,
+  limit?: number
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+const documentCollections = await r2rClient.getDocumentCollections('document_id', 0, 10);
+````
+
+### Getting Documents in a Collection
+
+To get all documents in a collection, use the `getDocumentsInCollection` method:
+
+````typescript
+async getDocumentsInCollection(
+  collectionId: string,
+  offset?: number,
+  limit?: number
+): Promise<Record<string, any>>
+````
+
+Example:
+````typescript
+const collectionDocuments = await r2rClient.getDocumentsInCollection('collection_id', 0, 10);
+````
+
+## Security Best Practices
+
+When implementing collection permissions, consider the following security best practices:
+
+1. Always use HTTPS in production to encrypt data in transit.
+2. Implement the principle of least privilege by assigning the minimum necessary permissions to users and collections.
+3. Regularly audit collection memberships and document assignments.
+4. Ensure that only authorized users (e.g., admins) can perform collection management operations.
+5. Implement comprehensive logging for all collection-related actions.
+6. Consider implementing additional access controls or custom roles within your application logic for more fine-grained permissions.
+
+For more advanced use cases or custom implementations, refer to the R2R documentation or reach out to the community for support.

docs/documentation/js-sdk/ingestion.mdx

@@ -3,6 +3,11 @@ title: 'Ingestion'
 description: 'Ingesting files with R2R.'
 ---
 
+<Note>
+Occasionally this SDK documentation falls out of date, cross-check with the automatcially generated <a href="/api-reference/introduction"> API Reference documentation </a> for the latest parameters.
+</Note>
+
+
 ## Document Ingestion and Management
 
 ### Ingest Files
@@ -180,6 +185,51 @@ const ingestResponse = await client.ingestFiles(files, {
   </Expandable>
 </ParamField>
 
+### Ingest Chunks
+
+Ingest pre-parsed text chunks into your R2R system:
+
+````javascript
+const chunks = [
+  {
+    text: "Aristotle was a Greek philosopher...",
+  },
+  {
+    text: "He was born in 384 BC in Stagira...",
+  }
+];
+
+const ingestResponse = await client.ingestChunks(chunks, {
+  metadata: { title: "Aristotle", source: "wikipedia" }
+});
+````
+
+<AccordionGroup>
+  <Accordion title="Response">
+    <ResponseField name="response" type="object">
+      The response from the R2R system after ingesting the chunks.
+      ```bash
+      {'message': 'Ingest chunks task queued successfully.', 'task_id': '8f27dfca-606d-422d-b73f-2d9e138661c3', 'document_id': 'd4391abf-8a4e-5d9d-80fd-232ef6fd8527'}
+      ```
+    </ResponseField>
+  </Accordion>
+</AccordionGroup>
+
+<ParamField path="chunks" type="Array<{ text: string; metadata?: Record<string, any> }>" required>
+  An array of chunk objects to ingest. Each object should contain at least a "text" property with the chunk text. An optional "metadata" property can contain a dictionary of metadata for the chunk.
+</ParamField>
+
+<ParamField path="options" type="object">
+  <ParamField path="document_id" type="string">
+    An optional document ID to assign to the ingested chunks. If not provided, a new document ID will be generated.
+  </ParamField>
+
+  <ParamField path="metadata" type="Record<string, any>">
+    An optional metadata object for the document.
+  </ParamField>
+</ParamField>
+
+
 ### Update Files
 
 Update existing documents: