rai

active

CPU-only LLM inference engine in pure Rust — 4-bit quantized models, hand-written AVX2 kernels, speculative decoding, and a local HTTP/MCP server. No GPU, no Python runtime.

Code-only MCP server: no hosted endpoint, so you install and run it yourself. It is not probed, graded or counted in our reachability statistics.

Get it

Repository: github.com/Classevelabs/rai
License: Apache-2.0
Last update: 2026-09-19
GitHub stars: ★ 16
Homepage: classeve.com/public/rai

avx2 classeve cpu-inference cpu-only inference-engine llm-inference local-llm mcp-server no-gpu quantization rust

Listed in

github · source

Run this server as a hosted endpoint? Submit the endpoint and it moves to the remote directory, where it is probed and graded.