CLASSEVE
RoutePublic
Public / RAI

RAI.

A CPU-only LLM inference engine written in Rust — 4-bit quantized kernels, model conversion, speculative decoding, and local serving. No CUDA, ROCm, Metal, PyTorch, or GGML at runtime.

Free. On Windows, run the Setup from the latest release. On macOS and Linux, paste this into a terminal:

curl -fsSL https://github.com/Classevelabs/rai/releases/latest/download/install.sh | sh
Open the latest release on GitHub
The journey

Bring a model. Shrink it. Run it.

This is the whole product in one scene: hand RAI a checkpoint, watch the converter pack its weights to 4-bit, see the file shrink by the bit ratio, load it into the RAM you already have — and put your CPU to work.

BRING A CHECKPOINT
DAILY DRIVER
YOUR MACHINE
FP16 CHECKPOINT13.8 GB
.RAIMODEL · W413.8 GB×4.0 SMALLER
SYSTEM RAM · 16 GBFITS · HEADROOM
WEIGHTS · W4 · 3.5 GB KV CACHE @ 8K · 1.0 GB ENGINE · 0.4 GB
— quantize first · then this runs on the CPU you already have —THE MECHANISM · NOT A BENCHMARK
The thesis

Everything the usual stack needs, it doesn't.

RAI runs on the CPU you already have. Throw the switch to see both architectures side by side.

AIR-GAPPED · READY
COMPUTE UNITx86_64 · AVX2/FMA/F16C
the CPU you already have
MEMORY ENVELOPESYSTEM RAM
weights load where your files live
DATA PATHLOOPBACK ONLY
binds 127.0.0.1 · studio + http api
RUNTIME SURFACE1 BINARY
RAI·BIN
pure rust · free

GPU-SIDE ROWS DESCRIBE THE GENERIC ALTERNATIVE THIS ENGINE WAS WRITTEN TO AVOID — NOT ANY NAMED PRODUCT.

Mechanism

One pass, end to end. Run it.

Prompt to stream through quantized kernels, GQA, mixture-of-experts routing, a KV cache, and a sampler whose temperature, top-k, top-p and speculative-decode switches are live — the simulation samples for real.

RUN
KERNEL
SPEC·DECODE
TEMP0.80
TOP-K8
TOP-P0.90
STEP00
TOK/STEP0.0
MODEDRAFT+VERIFY

Press RUN PASS and watch one token travel the whole engine.

OUT— halted · press RUN PASS —MECHANISM SIMULATION · NOT HARDWARE NUMBERS
Envelope

Drag the model. Watch the RAM.

Parameters and context window — slide them and the memory envelope moves. Arithmetic from the 4-bit weight width, not measured benchmarks.

PARAMETERS2.9B
0.5B2B8B32B
CONTEXT8K TOK
1K16K32K
HOST RAM16 GB
PRESETS
RAM ENVELOPE2.57 GBHEADROOM OK
48163264
LOG SCALE · 1–80 GB · TICKS AT 4/8/16/32/64
WEIGHTS · 4-BIT1.42 GB
KV CACHE · FP160.75 GB
RUNTIME0.40 GB
GEOMETRY · L24 · KV-H8 · D128 · ILLUSTRATIVE GQA CONFIG

DERIVED ESTIMATES FROM STATED QUANT WIDTHS AND CONTEXT — ARITHMETIC, NOT MEASURED PERFORMANCE.

Fine print

The rules, demonstrated.

Loopback-only serving, weights that never leave the machine, and an accelerator bay that stays empty — press everything.

01BIND ADDRESS● BOUND

rai serve publishes Studio and its HTTP API. Choose what it listens on:

> listening on 127.0.0.1 — studio ready
02MODEL ASSETS● ON YOUR DISK

Weights never enter a repository and never leave the machine. Run the conversion:

UPSTREAM WEIGHTS
CONVERTER
.raimodel
model.q4.raimodel · local diskCLOUD SLOT — NO UPLOAD PATH
03ACCELERATOR BAY● EMPTY · BY DESIGN

No CUDA. No ROCm. No Metal. Lift the dust cover:

DUST COVERHOVER / FOCUS TO LIFT
PCIE · NOT REQUIREDGPU GOES HERE — IN THE OTHER GUY’S MACHINE
FAQ

Common questions

What is RAI?
RAI is a free LLM inference engine written in pure Rust that runs entirely on the CPU. It includes 4-bit quantized kernels, the .raimodel format, speculative decoding, and a local HTTP server with a built-in chat UI (Studio).
Can I run a language model without a GPU?
Yes — that is the point of RAI. It performs inference on a standard x86-64 CPU with no graphics card, using AVX2-oriented quantized kernels.
Does RAI need CUDA, PyTorch, or Python?
No. There is no CUDA, ROCm, Metal, PyTorch, or GGML at runtime. RAI is a self-contained Rust workspace with no Python dependency.
What hardware does RAI need?
A 64-bit Intel or AMD processor from 2011 or later, or an Apple Silicon Mac; processors with AVX2, FMA, and F16C take the optimized path. It runs on Windows 10 and 11, macOS, and Linux with glibc 2.28 or later. No graphics card.
How do I install RAI?
On Windows, run the Setup from the GitHub releases page or paste one line into PowerShell. On macOS and Linux, paste one line into a terminal; Linux also has .deb and .rpm packages. Then open RAI Studio: it offers a starter model and opens the chat in your browser.
Is RAI available on crates.io?
Yes: classeve-rai-infer (the engine and CLI) and classeve-rai-compress (the quantization toolkit). cargo install classeve-rai-infer --locked installs the CLI. The installers, packages and archives for Windows, macOS and Linux are on the GitHub releases page.
How does RAI compare to llama.cpp?
RAI goes from checkpoint to chat in one pure-Rust binary: model conversion without Python, CPU-only 4-bit kernels, and a local chat server you can read, audit and build on. llama.cpp is the broad C/C++ default, with GPU offload. RAI is free on GitHub.