Awesome Generative AI Meets Julia Programming Language

Comprehensive guide to generative AI projects and resources in/for/associated with Julia.

Julia is a high-level, high-performance dynamic language for technical computing.

Generative AI encompasses algorithms and models that leverage large-scale machine learning to generate new content (across many modalities), automate, understand, parse, extract and much more, adapting to a wide range of applications beyond mere content creation.

Contents

• Generative AI Projects and Julia
• Models
• API SDKs
  • Model Providers
  • Cloud Services Providers
  • Vector Databases
  • General-purpose DBMS with Vector Index Support
• Packages
• JLL Packages
• Benchmarks/Comparisons
• Applications/Products
• Waiting Room
• Archived Projects
• Tutorials/learning
• Noteworthy Mentions
  • Local Deployments
  • Generative AI - Previous Generation
• Must-Know Python Projects
• Other Awesome Lists

Generative AI Projects and Julia

• JuliaGenAI Organization - A GitHub organization and a community of Julia developers and researchers working on generative AI.

Models

Build, train, and deploy Large language models (and other modalities) in Julia.

• Flux.jl - Flux is a machine learning library for Julia that is flexible and allows building complex models. However, at the time of writing, I'm not aware of any Large Language Models (LLMs) that have been implemented and trained in Flux.
• Transformers.jl - Transformers.jl is a Julia package that provides a high-level API for using pre-trained transformer models. It also allows to download any models from Hugging Face hub with @hgf_str macro string.
• Pickle.jl - Great package for loading Pytorch weights into Julia (if you want to implement models yourself).
• BytePairEncoding.jl - Pure Julia implementation of Byte Pair Encoding (BPE) algorithm. It's used by Transformers.jl to tokenize text.
• Llama2.jl - Llama2.jl provides simple code for inference and training of llama2-based language models based on llama2.c. It supports loading quantized weights in GGUF format (q4_K_S variant). Other similar projects: LanguageModels.jl
• Llama.jl - Julia interface to llama.cpp, a C/C++ library for running language models locally. Supports a wide range of models.
• Whisper.jl - Julia interface to whisper.cpp, a high-performance inference in C/C++ of OpenAI's Whisper automatic speech recognition (ASR) model.

API SDKs

Model Providers

Access Generative AI models via official APIs.

• OpenAI.jl - A community-maintained Julia wrapper to the OpenAI API.

Cloud Services Providers

Access Generative AI models via SDKs of popular cloud service providers.

• GoogleCloud.jl - SDK for Google Cloud. There is an open PR to enable Vertex AI endpoints.
• GoogleGenAI.jl - Unofficial wrapper for the Google Gemini API.

Vector Databases

• Pinecone.jl - SDK for Pinecone.io vector database.

General-purpose DBMS with Vector Index Support

Name	Julia Client	Usage Examples
Elasticsearch	ElasticsearchClient.jl	GptSeachPlugin with Elasticsearch
OpenSearch	ElasticsearchClient.jl	GptSeachPlugin with Opensearch
PostgreSQL + pgvector	LibPQ.jl	pgvector examples for Julia

Packages

• ReplGPT.jl - Brings ChatGPT interface as a Julia REPL mode.
• HelpGPT.jl - Calls ChatGPT to explain any errors in Julia code.
• GenGPT3.jl - A Gen.jl generative function that wraps the OpenAI API.
• PromptingTools.jl - Helps with everyday applications of Large Language Models in Julia by wrapping coming APIs, re-using prompts via templates, and enabling easy transition between different model providers (eg, OpenAI, Ollama). (Disclaimer: I'm the author of this package.)
• LLMTextAnalysis.jl - Leverage Large Language Models to uncover, evaluate, and label themes/concepts/spectra in large document collections. (Disclaimer: I'm the author of this package.)
• AIHelpMe.jl - Enhanced AI code assistance by leveraging package documentation (Retrieval Augmented Generation). Comes pre-packaged for common Julia packages, but can be used for any package.
• AIHelpUI.jl - Web-based graphical interface for AIHelpMe.jl built on top of Stipple.jl.
• ProToPortal.jl - Web-based graphical interface for PromptingTools.jl with extra prompt templates and functionality (Julia code execution and auto-fixing, meta-prompting, auto-critic, speech-to-text entry). Similar to ChatGPT but geared towards Julia.

JLL Packages

JLLs are prebuilt libraries and executables to easily install and call non-Julia projects (eg, C/C++). Often they are the first step towards a Julia package with an idiomatic interface.

• llama_cpp_jll.jl - JLL package for llama.cpp, the best interface for quantized llama2-style models.

Benchmarks/Comparisons

• Julia LLM Leaderboard - Comparison of Julia language generation capabilities of various Large Language Models across a range of tasks. Visit if you want help choosing the right model for your application.
• HumanEval.jl - The Julia version of openai/human-eval. It rewrites the original Python problems into the Julia version and provides evaluation results with several latest LLMs.

Applications/Products

Applications and products that "work" with Julia language.

• GitHub Copilot - Excellent inline suggestions with the help of OpenAI models. It works extremely well with Julia language for repetitive tasks one line at a time, but larger code chunks are rarely correct.
• Codium.ai - Great IDE or VSCode plugin for code analysis, suggestion and generation of test suites. Although the tests are written more in the style of Pytest rather than idiomatic Julia. It has a free tier.
• Replit - Replit's REPL is powered by an in-house model that supports Julia language.
• Codeium - Free alternative to GitHub Copilot with extensions for most editors.
• Cursor - Alternative IDE based on VSCode with AI-powered code completion and suggestions. It works really well with Julia language.

Julia-affiliated applications and products using LLMs

• JuliaHub AskAI - AskAI is a JuliaHub's RAG (Retrieval Augmented Generation) application that allows users to ask questions about the Julia language and its ecosystem. It is free, but you need to be logged in to JuliaHub to use it.
• Genie UI Assistant - Genie UI Assistant is a GPT-4 powered UI builder helping Genie.jl's users create UIs faster using natural language.
• Comind - A social network, messaging, and LLM interface built in Julia.

Waiting Room

New projects that are still waiting to prove themselves and collect enough stars.

• FlashRank.jl - Fast and local document ranking with models that can run on any computer (CPU-based). Based on Python's FlashRank.
• SemanticCaches.jl - Smarter caching for GenAI applications with a tiny embedding model - reducing latency, one request at a time.
• Spehulak.jl - GenAI observability platform for debugging your LLM calls. Fully integrated with PromptingTools.jl.
• StreamCallbacks.jl - Unifies LLM streaming interfaces, simplifies SSE handling, and provides built-in sinks for streaming data. Easy to extend with custom logic.
• LLMCheatsheets.jl - Summarize GitHub repositories into AI-friendly cheatsheets with a single command.
• SwarmAgents.jl - Based on OpenAI's Swarm package for multi-agent systems with tight integration with PromptingTools.jl and additional features.
• ChromeDevToolsLite.jl - Browser automation using the Chrome DevTools Protocol (CDP). Ideal for Computer Use with LLMs. Inspired by Python's Playwright.
• OmniParserIconDetectors.jl - Lightweight Julia wrapper for Microsoft's OmniParser icon detection model with additional utilities for working with screenshots (eg, drawing detections, adding labels).

Unreleased, experimental but functional:

• Milvus.jl - A minimal and unofficial implementation of Milvus VectorDB client for Julia.
• Jjama3.jl - Hackable Llama3.1, Llama3.2 (text), and Qwen 2.5 (eg. base, Qwen2.5-Coder, and Qwen2.5-Math) in Julia.

Archived Projects

• GPTCodingTools.jl - Code generation tool for Julia language with useful prompt templates and self-healing features (ala OpenAI Code Interpreter). It does work, but development has been abandoned. (Disclaimer: I'm the author of this package.)

Tutorials/learning

• Tutorial for using LLMs with Transformers.jl - A brief tutorial on how to use Transformers.jl to access LLMs from HuggingFace Hub.
• Building a RAG Chatbot over DataFrames.jl Documentation - Hands-on Guide - A hands-on guide on how to build a RAG chatbot over DataFrames.jl documentation using only minimal dependencies.
• GenAI Mini-Tasks: Extracting Data from (.*)? Look No Further! - A tutorial on structured data extraction. A part of a larger series of tutorials on small tasks that can be done with GenAI.
• Tiny Binary RAG - An excellent deep-dive on semantic search (the "R" in RAG). It showcases that with 100 lines of Julia, you can search 15M chunks (~size of Wikipedia) in <20ms.

Noteworthy Mentions

Some of the below projects are not necessarily Julia-specific, but noteworthy mentions in the generative AI space and interesting for Julia developers.

Local Deployments

• Ollama - The best option for those looking to host a Large Language Model locally. Simply start the server and send the requests with HTTP.jl.
• LM Studio - A desktop app for hosting and interacting with LLMs locally. It's a great option for those who want to use LLMs without coding. It's free for personal use.

Generative AI - Previous Generation

• GenerativeModels.jl - Useful library to train more traditional generative models like VAEs. It's built on top of Flux.jl.

### Useful Utilities

• Stipple.jl - For building interactive data applications in pure Julia (part of Genie.jl ecosystem). Excellent for building web-based interfaces for GenAI applications.
• Taro.jl - Powerful parser for various types of documents (interop to Java). Very useful for building Retrieval-Augmented Generation (RAG) applications.

Must-Know Python Projects

Python is on the leading edge of the generative AI revolution. Fortunately, we have PythonCall.jl allowing us to easily call all the below Python packages.

• HuggingFace Transformers - The most popular library for accessing LLMs and other models. It can be mostly used via Transformers.jl (see above).
• LangChain - The best option for building applications on top of LLMs (eg, Chains, Agents). It has a lot of adapters for common models, databases, and other services.
• Llama Index - Similar to LangChain but with a focus on data-centered applications like RAG.
• Instructor - Simple yet powerful structured extraction framework on top of OpenAI API. Excellent to understand the power of function calling API together with Pydantic.
• Marvin - Powerful building blocks to quickly build AI applications and expose them via a production-ready API.
• Open Interpreter - Let LLMs run code on your computer (eg, Python, JavaScript, Shell, and more). An open-source local alternative to OpenAI Code Interpreter.

Other Awesome Lists

• Awesome Generative AI - Great list for all things generative AI. An inspiration for this list!