python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
CMAKE_ARGS="-DLLAMA_CUDA=on" pip install llama-cpp-python --no-cache-dir --force-reinstall --upgrade # install llamacpp with cuda
Mixedbread Embedding Model - HuggingFace. Save this model in ai/embedding/models/
Llama3 8B Instruct Language Model - NousResearch/Meta-Llama-3-8B-Instruct. Save this model in ai/llm/models
Make sure you have necessary models specified in config.json in the respective folders and environment variables set in .env file. Then,
python3 app.py