Skip to content

Latest commit

 

History

History
73 lines (44 loc) · 4.2 KB

README.md

File metadata and controls

73 lines (44 loc) · 4.2 KB
Raw

Example

Before running an example, MUST enter the specific example directory. For example, to run the map example, switch to the map directory, and then use python to run it.

git clone https://github.com/zilliztech/GPTCache.git
cd GPTCache

cd examples/map
python map.py

If the running example includes a model or complex third-party library (like: faiss, towhee), the first run may take some time as it needs to download the model runtime environment, model data, dependencies, etc. However, the subsequent runs will be significantly faster.

Basic example

How to use the map to cache data.

Sqlite + Faiss manage cache data

How to use the sqlite to store the scale data and the faiss to query the vector data.

Sqlite + Faiss + Towhee

On the basis of the above example, use towhee for embedding operation

Sqlite + Milvus + Towhee

How to use the sqlite to store the scale data and the Milvus or Zilliz Cloud to store the vector data.

PostgreSQL + Milvus

How to use the PostgreSQL to store the scale data and the Milvus or Zilliz Cloud to store the vector data.

Note: you can set with your own postgresql with the sql_url parameter, such as the pseudocode get_ss_data_manager("postgresql", "faiss", sql_url="postgresql+psycopg2://<username>:<password>@<host>:<port>/<database>")

And it is same as the mysql, mariadb, sql server and oracle database, more details refer to the engine documentation.

Mysql + Milvus

How to use the MySQL to store the scale data and the Milvus or Zilliz Cloud to store the vector data.

MariaDB + Milvus

How to use the MariaDB to store the scale data and the Milvus or Zilliz Cloud to store the vector data.

SQL Server + Milvus

How to use the SQL Server to store the scale data and the Milvus or Zilliz Cloud to store the vector data.

Oracle+Milvus

How to use the Oracle to store the scale data and the Milvus or Zilliz Cloud to store the vector data.

Benchmark

The benchmark script about the Sqlite + Faiss + Towhee

Test data source: Randomly scrape some information from the webpage (origin), and then let chatgpt produce corresponding data (similar).

  • threshold: answer evaluation threshold, A smaller value means higher consistency with the content in the cache, a lower cache hit rate, and a lower cache miss hit; a larger value means higher tolerance, a higher cache hit rate, and at the same time also have higher cache misses.
  • positive: effective cache hit, which means entering similar to search and get the same result as origin
  • negative: cache hit but the result is wrong, which means entering similar to search and get the different result as origin
  • fail count: cache miss

data file: mock_data.json similarity evaluation func: pair_evaluation (search distance)

threshold average time positive negative fail count
20 0.04s 455 27 517
50 0.09s 871 86 42
100 0.12s 905 93 1