Cookbook

cookbooks 📖 for Mesolitica products! This repository only applicable for commercial models from https://mesolitica.com, not open models from https://mesolitica.com/opensource

API key

Get your API key at https://app.nous.mesolitica.com/

Documentation

We hosted scalar documentation at https://llm-router.nous.mesolitica.com/scalar

Concurrency

All API support concurrency requests, it means,

1. If you have a batch of texts, you can fire multiple requests at the same without impacting the latency, same goes to Speech API.
2. For Websocket Speech API, you can connect multiple users from your side to do real-time audio streaming to the Websocket server.

MaLLaM 🌙

1. Python OpenAI compatible
2. Streaming Python OpenAI compatible
3. RAG
4. JSON format
5. Function call
6. System prompt

Evaluation

1. json-mode-eval,

Originally from https://huggingface.co/datasets/NousResearch/json-mode-eval, this test is to test how good the models able to convert human natural text into JSON output given the OpenAPI schema.

Retrieval

1. Python OpenAI compatible

Speech

1. Python OpenAI compatible
2. Streaming using AIOHTTP
3. Streaming using AIOHTTP with VAD

For streaming, make sure read our streaming documentation.

Streaming using Websocket

1. Microphone PyAudio -> Websocket Python

pyaudio-websocket.mov

2. RecordRTC JS -> Websocket FastAPI

recordrtc-fastapi.mov

3. RecordRTC JS -> Websocket ExpressJS

recordrtc.mov

Translation

1. Python Requests

Self-hosted Enterprise

If you are interested to self-host Mesolitica products in your virtual private network, Contact us at [email protected] or [email protected] to know more.

What do you get

1. ♾️ usage of all quota, MaLLaM 🌙, Speech, Translation and chatbots.
2. Up-to-date enterprise versioning including evolution of MaLLaM 🌙, Speech, Translation and chatbots.
3. Can be either on private cloud or on premise, Dashboard and all the APIs will be hosted using virtual private IP.

Hardware

1. If you do not have hardware accelerator, we partner with NVIDIA APAC to include hardware solution with support.
2. Bring your own hardware accelerator, we do not provide support.

Software

We only provide yearly licensing with starting specific amount.