Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DX-1033: Namespace Support #4

Merged
merged 2 commits into from
Jul 1, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified bun.lockb
Binary file not shown.
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,6 @@
"vitest": "latest"
},
"dependencies": {
"@upstash/vector": "^1.0.7"
"@upstash/vector": "^1.1.3"
}
}
20 changes: 18 additions & 2 deletions readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,9 @@

> [!NOTE]
> **This project is in the Experimental Stage.**
>
>
> We declare this project experimental to set clear expectations for your usage. There could be known or unknown bugs, the API could evolve, or the project could be discontinued if it does not find community adoption. While we cannot provide professional support for experimental projects, we’d be happy to hear from you if you see value in this project!


Semantic Cache is a tool for caching natural text based on semantic similarity. It's ideal for any task that involves querying or retrieving information based on meaning, such as natural language classification or caching AI responses. Two pieces of text can be similar but not identical (e.g., "great places to check out in Spain" vs. "best places to visit in Spain"). Traditional caching doesn't recognize this semantic similarity and misses opportunities for reuse.

Semantic Cache allows you to:
Expand Down Expand Up @@ -87,6 +86,23 @@ runDemo();

The `minProximity` parameter ranges from `0` to `1`. It lets you define the minimum relevance score to determine a cache hit. The higher this number, the more similar your user input must be to the cached content to be a hit. In practice, a score of 0.95 indicates a very high similarity, while a score of 0.75 already indicates a low similarity. For example, a value of 1.00, the highest possible, would only accept an _exact_ match of your user query and cache content as a cache hit.

### Namespace Support

You can seperate your data into partitions with namespaces.

```typescript
import { SemanticCache } from "@upstash/semantic-cache";
import { Index } from "@upstash/vector";

// 👇 your vector database
const index = new Index();

// 👇 your semantic cache
const semanticCache = new SemanticCache({ index, minProximity: 0.95, namespace: "user1" });

await semanticCache.set("Capital of Turkey", "Ankara");
```

## Examples

The following examples demonstrate how you can utilize Semantic Cache in various use cases:
Expand Down
31 changes: 31 additions & 0 deletions src/semantic-cache.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -80,4 +80,35 @@ describe("semantic-cache", () => {

expect(firstResult).toBeUndefined();
});

test("should work with namespaces", async () => {
const cache1 = new SemanticCache({
index: new Index(),
minProximity: PROXIMITY_THRESHOLD,
namespace: "cache1",
});

const cache2 = new SemanticCache({
index: new Index(),
minProximity: PROXIMITY_THRESHOLD,
namespace: "cache2",
});

const cachedValue1 = "water";
const cachedValue2 = "H2O";

await cache1.set("best drink on a hot day", cachedValue1);
await cache2.set("chemical formula for water", cachedValue2);
await sleep(DELAY_MS);

const result1 = await cache1.get("what to drink when it's hot");
const result2 = await cache2.get("what is water's chemical formula");

expect(result1).toBe(cachedValue1);
expect(result2).toBe(cachedValue2);

// Cleanup
await cache1.flush();
await cache2.flush();
});
});
19 changes: 18 additions & 1 deletion src/semantic-cache.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,15 +11,27 @@ type SemanticCacheConfig = {
* Upstash serverless vector client
*/
index: Index;

/**
* Optional namespace for the cache
*/
namespace?: string;
};

export class SemanticCache {
private minProximity: number;
private index: Index;
private namespace?: string;

constructor(config: SemanticCacheConfig) {
this.minProximity = config.minProximity;
this.index = config.index;

if (config.namespace) {
this.index = config.index.namespace(config.namespace) as unknown as Index;
this.namespace = config.namespace;
} else {
this.index = config.index;
}
}

async get(key: string): Promise<string | undefined>;
Expand Down Expand Up @@ -86,6 +98,11 @@ export class SemanticCache {
}

async flush(): Promise<void> {
if (this.namespace) {
await this.index.reset({ namespace: this.namespace });
return;
}

await this.index.reset();
}
}
Loading