Update embedding generation

.env.local.example

@@ -1,22 +1,7 @@
-# Clerk related environment variables
-NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_****
-CLERK_SECRET_KEY=sk_****
-NEXT_PUBLIC_CLERK_SIGN_IN_URL=/sign-in
-NEXT_PUBLIC_CLERK_SIGN_UP_URL=/sign-up
-NEXT_PUBLIC_CLERK_AFTER_SIGN_IN_URL=/
-NEXT_PUBLIC_CLERK_AFTER_SIGN_UP_URL=/
-
-# OpenAI related environment variables
-OPENAI_API_KEY=sk-****
-
-# Replicate related environment variables
-REPLICATE_API_TOKEN=r8_****
-
-# Pinecone related environment variables
-PINECONE_API_KEY=bb****
-PINECONE_ENVIRONMENT=us****
-PINECONE_INDEX=ai****
+OLLAMA_URL=http://localhost:11434 # default
+OLLAMA_MODEL=llama2
 
 # Supabase related environment variables
-SUPABASE_URL=https://****
-SUPABASE_PRIVATE_KEY=eyJ****
+SUPABASE_URL=http://localhost:54321 # default
+SUPABASE_PRIVATE_KEY=ey**
+SUPABASE_EMBEDDING_ENDPOINT=http://localhost:54321/functions/v1/embed #default

README.md

@@ -1,138 +1,81 @@
-# AI Getting Started
-
-[Live Demo (deployed on fly.io)](https://ai-getting-started.com/)
-
-[Join our community Discord: AI Stack Devs](https://discord.gg/PQUmTBTGmT)
-
-
-
-<img width="1305" alt="Screen Shot 2023-06-20 at 1 30 56 PM" src="https://github.com/a16z-infra/ai-getting-started/assets/3489963/bcc762d2-68f5-4c4e-8c49-14602bee4995">
+# Local AI Stack (WIP)
 
+This project was built on [AI Starter Kit](https://github.com/a16z-infra/ai-getting-started). The idea is to make it possible for anyone to run a simple AI app 100% locally without having to swipe a credit card 💳
 
 ## Stack
 
-- Auth: [Clerk](https://clerk.com/)
+- Inference: [Ollama](https://github.com/jmorganca/ollama)
+- VectorDB: [Supabase pgvector](https://supabase.com/docs/guides/database/extensions/pgvector)
+- LLM Orchestration: [Langchain.js](https://js.langchain.com/docs/)
 - App logic: [Next.js](https://nextjs.org/)
-- VectorDB: [Pinecone](https://www.pinecone.io/) / [Supabase pgvector](https://supabase.com/docs/guides/database/extensions/pgvector)
-- LLM Orchestration: [Langchain.js](https://js.langchain.com/docs/) 
-- Image Model: [Replicate](https://replicate.com/)
-- Text Model: [OpenAI](https://platform.openai.com/docs/models)
-- Text streaming: [ai sdk](https://github.com/vercel-labs/ai)
-- Deployment: [Fly](https://fly.io/)
 
-## Overview
-- 🚀 [Quickstart](#quickstart)
-- 💻 [Contribute to this repo](#how-to-contribute-to-this-repo)
-
-## Quickstart 
-The simplest way to try out this stack is to test it out locally and traverse through code files to understand how each component work. Here are the steps to get started. 
+## Quickstart
 
 ### 1. Fork and Clone repo
 
-Fork the repo to your Github account, then run the following command to clone the repo: 
-```
-git clone [email protected]:[YOUR_GITHUB_ACCOUNT_NAME]/ai-getting-started.git
-```
-
-### 2. Install dependencies 
+Fork the repo to your Github account, then run the following command to clone the repo:
 
 ```
-cd ai-getting-started
-npm install 
+git clone [email protected]:[YOUR_GITHUB_ACCOUNT_NAME]/local-ai-stack.git
 ```
 
-### 3. Fill out secrets 
+### 2. Install dependencies
 
 ```
-cp .env.local.example .env.local
+cd local-ai-stack
+npm install
 ```
 
-a. **Clerk Secrets**
+### 3. Install Ollama
 
-Go to https://dashboard.clerk.com/ -> "Add Application" -> Fill in Application name/select how your users should sign in -> Create Application
-Now you should see both `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` and `CLERK_SECRET_KEY` on the screen
-<img width="1011" alt="clerk" src="https://github.com/a16z-infra/ai-getting-started/assets/3489963/6ce72263-4e83-406d-838e-08a95ea79023">
+Instructions are [here](https://github.com/jmorganca/ollama#macos)
 
+### 4. Run Supabase locally
 
-b. **OpenAI API key**
+1. Install Supabase CLI
 
-Visit https://platform.openai.com/account/api-keys to get your OpenAI API key
+```
+brew install supabase/tap/supabase
+```
 
-c. **Replicate API key**
+2. Start Supabase
+   Under the local-ai-stack dir, run
 
-Visit https://replicate.com/account/api-tokens to get your Replicate API key
+```
+supabase start
+```
 
-> **_NOTE:_**  By default, this template uses Pinecone as vector store, but you can turn on Supabase pgvector easily. This means you only need to fill out either Pinecone API key _or_ Supabase API key. 
+3. Serve embedding generation function
 
-d. **Pinecone API key**
-- Create a Pinecone index by visiting https://app.pinecone.io/ and click on "Create Index"
-- Give it an index name (this will be the environment variable `PINECONE_INDEX`)
-- Fill in Dimension as `1536`
-- Once the index is successfully created, click on "API Keys" on the left side nav and create an API key: copy "Environment" value to `PINECONE_ENVIRONMENT` variable, and "Value" to `PINECONE_API_KEY`
+```
+supabase functions serve
+```
 
-e. **Supabase API key**
-- Create a Supabase instance [here](https://supabase.com/dashboard/projects); then go to Project Settings -> API 
-- `SUPABASE_URL` is the URL value under "Project URL"
-- `SUPABASE_PRIVATE_KEY` is the key starts with `ey` under Project API Keys
-- Now, you should enable pgvector on Supabase and create a schema. You can do this easily by clicking on "SQL editor" on the left hand side on supabase UI and then clicking on "+New Query". Copy paste [this code snippet](https://github.com/a16z-infra/ai-getting-started/blob/main/pgvector.sql) in the SQL editor and click "Run".
+### 5. Fill in secrets
 
-### 4. Generate embeddings 
+```
+cp .env.local.example .env.local
+```
 
-There are a few markdown files under `/blogs` directory as examples so you can do Q&A on them. To generate embeddings and store them in the vector database for future queries, you can run the following command: 
+The only thing you need to fill out is `SUPABASE_PRIVATE_KEY` -- you can find this by running `supabase status` and copy `anon key`
 
-#### If using Pinecone
-Run the following command to generate embeddings and store them in Pinecone: 
-```bash
-npm run generate-embeddings-pinecone
-```
-#### If using Supabase pgvector
-In `QAModel.tsx`, replace `/api/qa-pinecone` with `/api/qa-pg-vector`. Then run the following command to generate embeddings and store them in Supabase pgvector:
+### 6. Generate embeddings
 
 ```bash
-npm run generate-embeddings-supabase
+node src/scripts/indexBlogPGVectorWithEmbed.mjs
 ```
 
-
-### 5. Run app locally
+### 7. Run app locally
 
 Now you are ready to test out the app locally! To do this, simply run `npm run dev` under the project root.
 
 ### 6. Deploy the app
 
-#### Deploy to fly.io
-- Register an account on fly.io and then [install flyctl](https://fly.io/docs/hands-on/install-flyctl/)
-- Run `fly launch` under project root -- this will generate a `fly.toml` that includes all the configurations you will need 
-- Run `fly deploy --ha=false` to deploy the app -- the --ha flag makes sure fly only spins up one instance, which is included in the free plan. You also want to run `fly scale memory 512` to scale up the fly vm memory for this app. 
-- For any other non-localhost environment, the existing Clerk development instance should continue to work. You can upload the secrets to Fly by running `cat .env.local | fly secrets import`
-- If you are ready to deploy to production, you should create a prod environment under the [current Clerk instance](https://dashboard.clerk.com/). For more details on deploying a production app with Clerk, check out their documentation [here](https://clerk.com/docs/deployments/overview). **Note that you will likely need to manage your own domain and do domain verification as part of the process.**
-- Create a new file `.env.prod` locally and fill in all the production-environment secrets. Remember to update `NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY` and `CLERK_SECRET_KEY` by copying secrets from Clerk's production instance
--`cat .env.prod | fly secrets import` to upload secrets
-
-#### Other deployment options
-- [Netlify](https://www.netlify.com/)
-- [Vercel](https://vercel.com/)
-
-
-## How to contribute to this repo
-
-### Code contribution workflow
-You can fork this repo, make changes, and create a PR. Add **@ykhli or @timqian** as reviewers. 
-
-If you are new to contributing on github, here is a step-by-step guide: 
-
-1. Clcik on `Fork` on the top right of this page
-2. Work on your change and push it to your forked repo. Now when you navigate to the forked repo's UI, you should see something like the following:
-<img width="904" alt="pr-preview" src="https://github.com/a16z-infra/ai-getting-started/assets/3489963/631e5f45-39ec-4b54-b9d1-b963e279dcc6">
-
-
-3. Click on "Contribute" -> "Open Pull Request".
-4. Once you have a PR, you can add reviewers.
-
-### Other contributions
-Feel free to open feature requests, bug reports etc under Issues.
+If you want to the the local-only app to the next level, feel free to follow instructions on [AI Starter Kit](https://github.com/a16z-infra/ai-getting-started) for using Clerk, Pinecone/Supabase, OpenAI, Replicate and other cloud-based vendors.
 
 ## Refs
+
+- https://github.com/a16z-infra/ai-getting-started
 - https://js.langchain.com/docs/modules/indexes/vector_stores/integrations/pinecone
 - https://js.langchain.com/docs/modules/models/llms/integrations#replicate
 - https://js.langchain.com/docs/modules/chains/index_related_chains/retrieval_qa
-

next.config.js

@@ -8,26 +8,29 @@ const nextConfig = {
         port: "",
         pathname: "**",
       },
-			{
+      {
         protocol: "https",
         hostname: "tjzk.replicate.delivery",
         port: "",
         pathname: "**",
       },
-			{
+      {
         protocol: "https",
         hostname: "replicate.delivery",
         port: "",
         pathname: "**",
       },
-			{
+      {
         protocol: "https",
         hostname: "a16z.com",
         port: "",
         pathname: "**",
       },
     ],
   },
+  experimental: {
+    serverComponentsExternalPackages: ["sharp", "onnxruntime-node"],
+  },
 };
 
 module.exports = nextConfig;