Pinned Loading

1. xlite-dev/lite.ai.toolkit xlite-dev/lite.ai.toolkit Public

   🛠 A lite C++ toolkit: contains 100+ Awesome AI models, support MNN, NCNN, TNN, ONNXRuntime and TensorRT. 🎉🎉

   C++ 4k 737

2. vllm-project/vllm vllm-project/vllm Public

   A high-throughput and memory-efficient inference and serving engine for LLMs

   Python 42.5k 6.4k

3. xlite-dev/Awesome-LLM-Inference xlite-dev/Awesome-LLM-Inference Public

   📖A curated list of Awesome LLM/VLM Inference Papers with codes: WINT8/4, FlashAttention, PagedAttention, MLA, Parallelism, etc. 🎉🎉

   3.7k 261

4. xlite-dev/CUDA-Learn-Notes xlite-dev/CUDA-Learn-Notes Public

   📚200+ Tensor/CUDA Cores Kernels, ⚡️flash-attn-mma, ⚡️hgemm with WMMA, MMA and CuTe (98%~100% TFLOPS of cuBLAS/FA2 🎉🎉).

   Cuda 3k 309

5. xlite-dev/ffpa-attn-mma xlite-dev/ffpa-attn-mma Public

   📚FFPA(Split-D): Yet another Faster Flash Prefill Attention with O(1) GPU SRAM complexity for headdim > 256, ~2x↑🎉vs SDPA EA.

   Cuda 154 6

6. xlite-dev/hgemm-mma xlite-dev/hgemm-mma Public

   ⚡️Write HGEMM from scratch using Tensor Cores with WMMA, MMA and CuTe API, Achieve Peak⚡️ Performance.

   Cuda 62 2