feat: mongo-db adaptor #1427
feat: mongo-db adaptor #1427
Conversation
Added an adaptor which connects to mongo db atlas. Allowing you to store agent data in the cloud. If you have the appropriate tier you can also take advantage of their vector search functionaility.
📝 WalkthroughWalkthroughThis pull request introduces MongoDB integration by adding configuration entries and creating a comprehensive MongoDB database adapter. The changes span multiple files, primarily focusing on setting up MongoDB connection settings in the Changes
Possibly related PRs
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (4)
packages/adapter-mongodb/src/index.ts (2)
20-20: Specify a concrete type for 'database' instead of 'any'Using
anydefeats the purpose of TypeScript's type safety. Specify a more precise type fordatabaseto improve type safety and code maintainability.For example:
import { Db } from 'mongodb'; private database: Db;
589-624: Use a standard library for Levenshtein distance calculationImplementing custom algorithms can introduce unnecessary complexity and potential bugs. Using a well-tested library for Levenshtein distance can improve code readability and maintainability.
For example, you can use the
fast-levenshteinpackage:import levenshtein from 'fast-levenshtein'; private calculateLevenshteinDistanceOptimized(str1: string, str2: string): number { return levenshtein.get(str1, str2); }.env.example (2)
413-414: Enhance MongoDB configuration documentation.Add more descriptive comments to guide users:
-MONGODB_CONNECTION_STRING= #mongodb connection string -MONGODB_DATABASE= #name of the database in mongoDB atlas +MONGODB_CONNECTION_STRING= # MongoDB Atlas connection string (format: mongodb+srv://<username>:<password>@<cluster>.mongodb.net) +MONGODB_DATABASE= # Name of the database in MongoDB Atlas (e.g., agent_data)
413-413: Add security note for MongoDB connection string.The connection string contains sensitive credentials. Consider adding a comment about security best practices.
-MONGODB_CONNECTION_STRING= #mongodb connection string +# WARNING: Connection string contains credentials. Never commit actual values to version control. +MONGODB_CONNECTION_STRING= # MongoDB Atlas connection string
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (14)
.env.example(2 hunks).idea/.gitignore(1 hunks).idea/eliza.iml(1 hunks).idea/inspectionProfiles/Project_Default.xml(1 hunks).idea/modules.xml(1 hunks).idea/prettier.xml(1 hunks).idea/vcs.xml(1 hunks)agent/src/index.ts(1 hunks)packages/adapter-mongodb/.npmignore(1 hunks)packages/adapter-mongodb/eslint.config.mjs(1 hunks)packages/adapter-mongodb/package.json(1 hunks)packages/adapter-mongodb/src/index.ts(1 hunks)packages/adapter-mongodb/tsconfig.json(1 hunks)packages/adapter-mongodb/tsup.config.ts(1 hunks)
✅ Files skipped from review due to trivial changes (10)
- .idea/prettier.xml
- .idea/eliza.iml
- packages/adapter-mongodb/.npmignore
- packages/adapter-mongodb/eslint.config.mjs
- .idea/modules.xml
- packages/adapter-mongodb/tsconfig.json
- packages/adapter-mongodb/tsup.config.ts
- .idea/.gitignore
- .idea/vcs.xml
- packages/adapter-mongodb/package.json
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: integration-tests
🔇 Additional comments (1)
packages/adapter-mongodb/src/index.ts (1)
66-66:⚠️ Potential issueCorrect 'expireAfterSeconds' value in TTL index
Setting
expireAfterSeconds: 0in the TTL index may not function as intended, as MongoDB requires a positive integer value. To ensure documents expire correctly based on theexpiresAtfield, you can omitexpireAfterSecondsor set it to a positive integer.Apply this diff to fix the issue:
{ key: { expiresAt: 1 }, options: { - expireAfterSeconds: 0 } }Likely invalid or redundant comment.
| <component name="InspectionProjectProfileManager"> | ||
| <profile version="1.0"> | ||
| <option name="myName" value="Project Default" /> | ||
| <inspection_tool class="Eslint" enabled="true" level="WARNING" enabled_by_default="true" /> | ||
| </profile> | ||
| </component> |
There was a problem hiding this comment.
💡 Codebase verification
Remove tracked .idea files from git repository
The .idea directory is correctly ignored in .gitignore, but previously committed files need to be removed from git tracking. Run:
git rm -r --cached .idea/
git commit -m "Remove .idea directory from git"🔗 Analysis chain
IDE configuration files should be excluded from version control.
The .idea directory contains IDE-specific settings that should be kept local.
Let's check if .idea is properly ignored:
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check if .idea is in .gitignore
if ! grep -q "^\.idea" .gitignore; then
echo "Warning: .idea directory is not ignored"
echo "Current .gitignore contents:"
cat .gitignore
fi
Length of output: 30
Script:
#!/bin/bash
# Check if .gitignore exists and show its contents
if [ -f ".gitignore" ]; then
echo ".gitignore exists. Contents:"
cat .gitignore
else
echo ".gitignore does not exist"
fi
# Check for .idea files in git status
git ls-files .idea/
Length of output: 938
| if (process.env.POSTGRES_URL) { | ||
| if (process.env.MONGODB_CONNECTION_STRING) { | ||
| elizaLogger.log("Initializing database on MongoDB Atlas"); | ||
| const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING, { |
There was a problem hiding this comment.
Import 'MongoClient' from 'mongodb'
The MongoClient class is used but not imported, which will cause a runtime error. Please import MongoClient at the beginning of the file.
Add the following import statement at the top of the file:
import { MongoClient } from 'mongodb';| retryReads: true | ||
| }); | ||
|
|
||
| const dbName = process.env.MONGODB_DATABASE_NAME || 'CumulusAiAgent'; // Default database name |
There was a problem hiding this comment.
Correct the environment variable name
The code uses process.env.MONGODB_DATABASE_NAME, but the environment variable is defined as MONGODB_DATABASE in the .env.example file. To avoid confusion and potential errors, please use process.env.MONGODB_DATABASE instead.
Apply this diff to fix the issue:
-const dbName = process.env.MONGODB_DATABASE_NAME || 'CumulusAiAgent'; // Default database name
+const dbName = process.env.MONGODB_DATABASE || 'CumulusAiAgent'; // Default database name📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const dbName = process.env.MONGODB_DATABASE_NAME || 'CumulusAiAgent'; // Default database name | |
| const dbName = process.env.MONGODB_DATABASE || 'CumulusAiAgent'; // Default database name |
| if (process.env.MONGODB_CONNECTION_STRING) { | ||
| elizaLogger.log("Initializing database on MongoDB Atlas"); | ||
| const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING, { | ||
| maxPoolSize: 100, | ||
| minPoolSize: 5, | ||
| maxIdleTimeMS: 60000, | ||
| connectTimeoutMS: 10000, | ||
| serverSelectionTimeoutMS: 5000, | ||
| socketTimeoutMS: 45000, | ||
| compressors: ['zlib'], | ||
| retryWrites: true, | ||
| retryReads: true | ||
| }); | ||
|
|
||
| const dbName = process.env.MONGODB_DATABASE_NAME || 'CumulusAiAgent'; // Default database name | ||
| const db = new MongoDBDatabaseAdapter(client, dbName); | ||
|
|
||
| // Test the connection | ||
| db.init() | ||
| .then(() => { | ||
| elizaLogger.success( | ||
| "Successfully connected to MongoDB Atlas" | ||
| ); | ||
| }) | ||
| .catch((error) => { | ||
| elizaLogger.error("Failed to connect to MongoDB Atlas:", error); | ||
| throw error; // Re-throw to handle it in the calling code | ||
| }); | ||
|
|
||
| return db; |
There was a problem hiding this comment.
Declare 'initializeDatabase' as an async function and await 'db.init()'
The initializeDatabase function performs asynchronous operations but is not declared as async. Additionally, db.init() is not awaited before returning db, which may lead to the database not being fully initialized when used. Please declare the function as async and await the db.init() call to ensure proper initialization.
Apply this diff to fix the issue:
-function initializeDatabase(dataDir: string) {
+async function initializeDatabase(dataDir: string) {
if (process.env.MONGODB_CONNECTION_STRING) {
elizaLogger.log("Initializing database on MongoDB Atlas");
const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING, {
maxPoolSize: 100,
minPoolSize: 5,
maxIdleTimeMS: 60000,
connectTimeoutMS: 10000,
serverSelectionTimeoutMS: 5000,
socketTimeoutMS: 45000,
compressors: ['zlib'],
retryWrites: true,
retryReads: true
});
const dbName = process.env.MONGODB_DATABASE || 'CumulusAiAgent'; // Default database name
const db = new MongoDBDatabaseAdapter(client, dbName);
- // Test the connection
- db.init()
- .then(() => {
- elizaLogger.success(
- "Successfully connected to MongoDB Atlas"
- );
- })
- .catch((error) => {
- elizaLogger.error("Failed to connect to MongoDB Atlas:", error);
- throw error; // Re-throw to handle it in the calling code
- });
+ // Test the connection
+ try {
+ await db.init();
+ elizaLogger.success("Successfully connected to MongoDB Atlas");
+ } catch (error) {
+ elizaLogger.error("Failed to connect to MongoDB Atlas:", error);
+ throw error; // Re-throw to handle it in the calling code
+ }
return db;
} else if (process.env.POSTGRES_URL) {
// Existing PostgreSQL logic📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (process.env.MONGODB_CONNECTION_STRING) { | |
| elizaLogger.log("Initializing database on MongoDB Atlas"); | |
| const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING, { | |
| maxPoolSize: 100, | |
| minPoolSize: 5, | |
| maxIdleTimeMS: 60000, | |
| connectTimeoutMS: 10000, | |
| serverSelectionTimeoutMS: 5000, | |
| socketTimeoutMS: 45000, | |
| compressors: ['zlib'], | |
| retryWrites: true, | |
| retryReads: true | |
| }); | |
| const dbName = process.env.MONGODB_DATABASE_NAME || 'CumulusAiAgent'; // Default database name | |
| const db = new MongoDBDatabaseAdapter(client, dbName); | |
| // Test the connection | |
| db.init() | |
| .then(() => { | |
| elizaLogger.success( | |
| "Successfully connected to MongoDB Atlas" | |
| ); | |
| }) | |
| .catch((error) => { | |
| elizaLogger.error("Failed to connect to MongoDB Atlas:", error); | |
| throw error; // Re-throw to handle it in the calling code | |
| }); | |
| return db; | |
| if (process.env.MONGODB_CONNECTION_STRING) { | |
| elizaLogger.log("Initializing database on MongoDB Atlas"); | |
| const client = new MongoClient(process.env.MONGODB_CONNECTION_STRING, { | |
| maxPoolSize: 100, | |
| minPoolSize: 5, | |
| maxIdleTimeMS: 60000, | |
| connectTimeoutMS: 10000, | |
| serverSelectionTimeoutMS: 5000, | |
| socketTimeoutMS: 45000, | |
| compressors: ['zlib'], | |
| retryWrites: true, | |
| retryReads: true | |
| }); | |
| const dbName = process.env.MONGODB_DATABASE_NAME || 'CumulusAiAgent'; // Default database name | |
| const db = new MongoDBDatabaseAdapter(client, dbName); | |
| // Test the connection | |
| try { | |
| await db.init(); | |
| elizaLogger.success("Successfully connected to MongoDB Atlas"); | |
| } catch (error) { | |
| elizaLogger.error("Failed to connect to MongoDB Atlas:", error); | |
| throw error; // Re-throw to handle it in the calling code | |
| } | |
| return db; |
| private cosineSimilarity(a: number[], b: number[]): number { | ||
| const dotProduct = a.reduce((sum, val, i) => sum + val * b[i], 0); | ||
| const magnitudeA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0)); | ||
| const magnitudeB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0)); | ||
| return dotProduct / (magnitudeA * magnitudeB); | ||
| } |
There was a problem hiding this comment.
Handle potential division by zero in 'cosineSimilarity'
If either input vector has zero magnitude, the calculation dotProduct / (magnitudeA * magnitudeB) will result in a division by zero, causing NaN or Infinity. Add a check to prevent division by zero and handle zero-magnitude vectors appropriately.
Apply this diff to fix the issue:
const magnitudeA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0));
const magnitudeB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0));
+ if (magnitudeA === 0 || magnitudeB === 0) {
+ return 0; // or handle the case as needed
+ }
return dotProduct / (magnitudeA * magnitudeB);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| private cosineSimilarity(a: number[], b: number[]): number { | |
| const dotProduct = a.reduce((sum, val, i) => sum + val * b[i], 0); | |
| const magnitudeA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0)); | |
| const magnitudeB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0)); | |
| return dotProduct / (magnitudeA * magnitudeB); | |
| } | |
| private cosineSimilarity(a: number[], b: number[]): number { | |
| const dotProduct = a.reduce((sum, val, i) => sum + val * b[i], 0); | |
| const magnitudeA = Math.sqrt(a.reduce((sum, val) => sum + val * val, 0)); | |
| const magnitudeB = Math.sqrt(b.reduce((sum, val) => sum + val * val, 0)); | |
| if (magnitudeA === 0 || magnitudeB === 0) { | |
| return 0; // or handle the case as needed | |
| } | |
| return dotProduct / (magnitudeA * magnitudeB); | |
| } |
Added an adaptor which connects to mongo db atlas. Allowing you to store agent data in the cloud. If you have the appropriate tier you can also take advantage of their vector search functionality.
It should have all the same functionality as the other database adaptors.
Relates to:
Adding the option of using MongoDB in the cloud to store you agents data
Risks
Low, adds additional options for database usage
Background
What does this PR do?
Adds the option to store your agents data in MongoDB Atlas, and take advantage of their vector search. I have also updated the .env.example to include the additional variables for the mongo connection. And added to the agent index with functionality to initialise this database if required.
What kind of change is this?
Adding an additional feature
Why are we doing this? Any context or related work?
In my day job we exclusively use mongoDB for our database needs, for commercials and performance reasons. As we want to develop AI Agents, we need to have the mongoDB adaptor. IN the sprit of OpenSource I wanted to make sure it was available for the whole community to use as well.
Documentation changes needed?
A small additional input to the documentation is required to make users aware of the additional functionality and how it can be actioned.
Similar to the Postgres set up just adding in the appropriate connection string to the .env file will action he change to start using MongoDB as the database.
Testing
Where should a reviewer start?
The reviewer will need to have a least a free account with mongoDB atlas, then add the connection string and a Database name to the .env file. After that the agent should run as expected, saving memories as before. The reviewer can verify the saved data inside the MongoDB interface.
Detailed testing steps
Summary by CodeRabbit
Configuration
.env.exampleNew Package
@ai16z/adapter-mongodbpackageDevelopment Tools