New: Kindi 0.3.0 is out, with CPU support, AMD graphics and more.

_Kindi 0.3.0 / Local inference

The world's fastestsoftware inference
engine
for
low-cost GPUs.

Kindi 0.3.0 runs modern 2B to 27B language models on old and inexpensive NVIDIA cards, on mini PCs with Radeon 780M graphics, and on the CPU, with no data sent to the cloud.

Find your setup Get Kindi on GitHub

Free software · AGPL-3.0 · Linux x86-64 · Commercial licensing available

TESTED CONFIGURATIONS2B–27B MODELS

GeForce GTX 108027B model · 65,536-token context

Quadro K4202B model · 1 GB of VRAM

Radeon 780M mini PC27B model · 64 GB RAM tested

2B–27BSupported model sizes

NVIDIA / AMD / CPUSupported compute backends

Local inferenceYour model requests stay on your machine

IN THE PRESS

Coverage and reviews.

Articles about Kindi, its engineering and its use on older hardware.

Inference Cafe

Review · Kindi 0.2.0 · October 26, 2026

“Kindi 0.2.0 is one of the more interesting local-AI projects of the year because it attacks a problem most vendors ignore: the huge installed base of old, low-VRAM NVIDIA hardware.”

From ‘Kindi: Modern Local AI on a $15 GPU.’
Read the review →

Nicheloom

Launch analysis · The engineering behind Kindi

“It also needs deep knowledge of quantized model formats, memory-constrained scheduling, and benchmarking against llama.cpp. Experienced systems and GPU programmers are required.”

Nicheloom’s assessment of the expertise needed to build a project like Kindi, in its ‘What it would actually take’ analysis.
Read the analysis →

YesPress

Profile · On Kindi and older hardware · October 25, 2026

“A specification can describe a limit without exhausting the possibilities.”

From ‘Amir Husain and the computers that refuse to retire,’ in the passage discussing Kindi.
Read the profile →

FROM THE COMMUNITY

People putting
Kindi to work.

and now you added Nemotron. Speechless…

@goldbrick16

you are democratizing the access to AI

@TheRojecas

this is a great project, thank you so much for all the work you have put in it!

@FroggyTWrite

This is amazing. The speed at which you've been able to do all of this is very inspiring.

@graciebigthumbs

mind blowing. actual cyberpunk.

@goldbrick16

And the manual is really good, especially on the use cases, on what works and not. And the benchmarks comparing different cards.

@aegontargaryen1390

1 / 6

HARDWARE COMPATIBILITY

Will Kindi work on my card?

Find your graphics card. See the models, recommended setup, driver requirements and community reports.

How compatibility is labeled, and privacy

TESTED BY US: we measured it. REPORTED BY A USER: a user's report, not verified by us. SHOULD WORK: a prediction from the installer's rules; the speed is an estimate. Card data: NVIDIA's CUDA GPU list and Wikipedia's lists of NVIDIA GPUs (CC BY-SA 4.0).

Privacy: each check loads a 1-pixel image from this site. Its address carries what you typed, the card that matched and the answer. It also carries your screen width class, language and time zone, and random anonymous ids for this visit and this browser. We do not ask for or record your name or e-mail. Like every web request, it also appears in the server log with your IP address and browser.

Installation.

On Linux with Docker, one installer discovers your cards and offers recommended configurations. Inspect the steps first with the dry run.

curl -fsSL https://amirhusain.com/kindi/install.sh | bash

curl -fsSL https://amirhusain.com/kindi/install.sh | bash -s -- --dry-run

The recommended setup for each computer

Your computer	Recommended
No GPU	2B on the CPU
Quadro K420 (1 GB or 2 GB)	2B
One Quadro K1200 4 GB	9B
Two Quadro K1200 4 GB	27B split over both cards
GeForce GTX 1080 8 GB	27B
GeForce RTX 3060 / 3080 (9.5 GB or more)	27B with Tensor Cores and the 2B draft model
AMD Radeon 780M	27B
Two or more GPUs that do not split one model	the best model on every GPU, one server each

Use it from your own code

# any OpenAI client; use the model id that the installer shows
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8093/v1", api_key="none")
r = client.chat.completions.create(model="bonsai2-27b",
        messages=[{"role": "user", "content": "Write a bash one-liner that counts .txt files"}])
print(r.choices[0].message.content)

Kindi on GitHub Release v0.3.0

The dry run shows each step and changes nothing on your computer. install.sh sha256: 4079515dd373384cd4e68e8a747ba9e667bf36d81ac3132f87886273ae96d38a. Kepler cards need the NVIDIA 470 driver. For GeForce RTX 30 cards, the installer offers to unlock full-rate 16-bit arithmetic with the excellent gpu-unlock module by Zagitta, and asks first.

BENCHMARKS

Performance Stats

One rule for each comparison: the same hardware, the same model file and the same context for both engines. Read speed measures prompt processing; write speed measures token generation. These benchmarks were carried out on our test systems. With the Bonsai models, Kindi writes faster than llama.cpp on every NVIDIA card we tested. With Nemotron (experimental), llama.cpp is faster on the GTX 1080 and the Radeon 780M.

Kindillama.cpp

Write speed at a short context, the recommended model of each setup (tok/s)

K420, 2B

22.9

1.65

K1200, 9B

23.4

5.9

2× K1200, 27B

12.6

1.69

K6000, 27B

20.4

2.07

GTX 1080, 27B

29.4

19.8

RTX 3080, 27B

63.2

52.2

Radeon 780M, 27B

10.6

10.3

Ryzen 9 CPU, 2B

64.8

23.6

Seconds for each cached request, Nemotron on a GTX 1080 (lower is better)

Agent, turn 2

0.67

1.29

Agent, turn 3

0.86

1.3

Regenerate an answer

0.43

4.93

Document, question 2

0.7

3.16

Document, question 3

0.42

3.1

Two agents, A turn 2

0.58

1.4

Two agents, B turn 2

0.53

1.44

The prompt cache: Nemotron on a GTX 1080 gets a third question about a 9K-token document. Kindi answers it in 10.71 seconds with the cache off and in 0.42 seconds with the cache on. The chart compares the two engines with both caches on.

All speeds are in tokens per second, with greedy output. Kindi uses the exact head, but the K420 write figure uses the fast head, at 2,000 tokens of context. For the RTX 3080 at 380 W and the conditions of each run, read the performance chapter of the manual. The manual (PDF) · docs/BENCHMARKS.md

Explore the full benchmark tables →

Show Click a column head to sort.

Hardware	Model	Context (tokens)	Kindi read	llama.cpp read	Kindi write	llama.cpp write	Write, Kindi ÷ llama.cpp
Quadro K420 1 GB	2B	512 (read), 2,000, fast head (write)	33.3	1.2	22.9	1.65	13.9×
Quadro K420 1 GB	2B	8,192	29.7	crashed	18.6	crashed	–
Quadro K420 1 GB	2B	24,000	24.2	–	15.3	–	–
Quadro K1200	2B	512	195	26.7	81.0	20.8	3.9×
Quadro K1200	2B	8,192	179	28.1	73.3	21.9	3.3×
Quadro K1200	2B	24,000	152	26.5	58.8	18.6	3.2×
Quadro K1200	4B	512	77.4	11.1	39.2	8.7	4.5×
Quadro K1200	4B	8,192	70.2	10.8	34.3	8.1	4.2×
Quadro K1200	4B	24,000	58.9	10.2	26.4	6.9	3.8×
Quadro K1200	9B	512	41.7	5.8	23.4	5.9	4.0×
Quadro K1200	9B	8,192	39.2	5.7	21.5	5.6	3.8×
Quadro K1200	9B	24,000	35.6	not finished	18.1	not finished	–
2× Quadro K1200	27B	512 (read), short (write)	19.5	1.63	12.6	1.69	7.5×
Quadro K6000	27B	512	36.2	7.0	20.4	2.07	9.9×
Quadro K6000	27B	8,192	30.2	6.9	16.4	2.15	7.6×
Quadro K6000	27B	24,000	26.2	did not finish in 1 hour	13.1	did not finish in 1 hour	–
Quadro K6000	Nemotron	512	188	90.7	20.1	10.7	1.9×
Quadro K6000	Nemotron	8,192	182	33.8	19.1	9.8	1.9×
Quadro K6000	Nemotron	24,000	164	15.6	17.4	8.56	2.0×
GeForce GTX 1080	27B	512	257	99.0	29.4	19.8	1.5×
GeForce GTX 1080	27B	8,192	229	96.5	26.9	18.0	1.5×
GeForce GTX 1080	27B	32,768	171	84.8	20.7	14.4	1.4×
GeForce GTX 1080	27B	49,152	146	78.6	18.1	12.7	1.4×
GeForce GTX 1080	27B	64,000	129	out of memory	16.3	out of memory	–
GeForce RTX 3080, 190 W	27B	512 (read), 1K (write)	1,093	936	63.2	52.2	1.2×
GeForce RTX 3080, 190 W	27B	3,000 (read), 32K (write)	1,029	995	56.3	41.0	1.4×
GeForce RTX 3080, 190 W	27B	12,000 (read), 64K (write)	944	926	49.8	28.0	1.8×
GeForce RTX 3080, 190 W	27B	22,000 (read), 90K (write)	855	855	44.0	23.5	1.9×
Radeon 780M	27B	512	104	60.5	10.6	10.3	1.0×
Radeon 780M	27B	8,192	95.6	58.2	9.99	8.52	1.2×
Radeon 780M	27B	24,000	80.8	54.5	8.75	6.13	1.4×
Radeon 780M	Nemotron	512	186	670	19.1	24.0	0.8×
Radeon 780M	Nemotron	8,192	392	599	20.0	21.4	0.9×
Radeon 780M	Nemotron	24,000	239	477	17.0	19.1	0.9×
CPU: Ryzen 9 7940HS	2B	about 2,000 (read), 256 out	90.4	60.5	64.8	23.6	2.7×
CPU: Ryzen 9 7940HS	4B	about 2,000 (read), 256 out	37.8	25.7	29.1	16.5	1.8×
CPU: Core i7-8700	2B	about 2,000 (read), 256 out	42.5	33.5	28.9	18.1	1.6×
CPU: Core i7-8700	4B	about 2,000 (read), 256 out	17.0	13.0	12.6	7.75	1.6×
Quadro K1200	Nemotron	512	66.8	17.6	10.4	8.27	1.3×
2× Quadro K1200	Nemotron	512	123	15.3	17.2	7.44	2.3×
GeForce GTX 1080	Nemotron	512	743	1,088	55.1	56.4	1.0×

KINDI BRIDGE

An agent workspace.

Text-based user interface with an agent workspace, command prompt environment and live performance data.

Use the menus to create and switch agents, send instructions, manage schedules, browse files and inspect system resources.

Layouts, scheduling and other features

Multiple layouts: Choose side panels, telemetry above the workspace or telemetry below it. Zoom a pane or refit the layout when the terminal size changes.
Agent workspace: Create named agents with a mission and working folder. Switch between conversations and queue follow-up instructions for a busy agent.
Command prompt: Use the command pane alongside the selected agent's conversation and performance panels.
Live performance data: Monitor hardware vitals and inference telemetry. Select which GPU to display in the vitals panel.
Built-in scheduling: Send an instruction once at a chosen time or repeat it at a fixed interval. Inspect, pause or delete pending instructions and view delivery history.
Re-prompt reminders: Schedule “check until done” prompts with a task, interval, maximum number of checks and optional deadline. Checks stop on model-confirmed completion, the configured limit or a delivery error.
Files and local references: Browse, preview and edit project files. Attach reference folders for local keyword search, with document extraction and semantic search available through the optional knowledge package.
System tools: Inspect processes, listening ports, resource usage and available cron or systemd timers.

Watch the recorded agent demonstration

kindi-bridge — vanguard (GTX 1080 · 27B) 19.4 tok/s

Kindi Bridge in a terminal on vanguard. Left: the agent writes a summary in six points, with times, of a GPU failure that it found in five log files. Right: hardware vitals for the GeForce GTX 1080 and the Quadro K420, and inference telemetry for bonsai2-27b: generating at 19.4 tok/s, 3,506 new prompt tokens and 34,124 tokens reused from the cache.

About this live capture

Kindi Bridge on vanguard: one frame of a live recording, not edited. The agent read five log files from a computer with a failed GPU job. Now it writes a summary in six points, with times. The Kindi server runs the 27B (bonsai2-27b) on a GeForce GTX 1080. At this moment, the server writes 19.4 tok/s. For this answer, it reused 34,124 prompt tokens from its cache and read 3,506 new tokens.

What you get, and how it works

OpenAI-compatiblekindi serve speaks /v1/chat/completions with streaming and tool calls.
Exact by defaultGreedy output matches llama.cpp token for token on the same model file.
Prompt cacheKindi reads the long system prompt of an agent one time. Kindi answers a third question about a long document in 0.42 s; llama.cpp takes 3.10 s.
Two cards as oneTensor parallelism puts half of each matrix on each card. Two low-profile K1200s hold the 27B.
Speculative decodingOn the RTX 3080, the 2B drafts tokens for the 27B. Greedy output stays the same.
Ternary tablesFour ternary weights are one byte that indexes a table of partial sums, built like the Cantor set.

Read the details

HARDWARE & MODELS

Hardware and models.

We tested Kindi 0.3.0 on these setups. Open a setup for its models, speed and use. The drawings are to one scale.

NVIDIA Quadro K420Kepler · 2014 · 1 or 2 GB · 41 W from the slot

Models: 2B on the 1 GB card; 2B or 4B on the 2 GB card
Speed: The 2B writes 22.9 tok/s with the fast head, at 2,000 tokens of context
Good for: Short text decisions and small agent tasks on an old office computer

NVIDIA Quadro K1200Maxwell · 2015 · 4 GB · 45 W from the slot

Models: 2B, 4B, 9B or Nemotron
Speed: At 512 tokens: 81.0 tok/s with the 2B, 39.2 with the 4B, 23.4 with the 9B
Good for: Agent work with the 4B, a 9B model in a small office computer

Two Quadro K1200 cardsTensor parallel · 8 GB in total · 90 W

Models: 27B or Nemotron, one model on both cards
Speed: The 27B writes 12.6 tok/s at a short context
Good for: A 27B model in a US$222 office computer

NVIDIA Quadro K6000Kepler · 2013 · 12 GB · 225 W

Models: 27B with 65,536 tokens of context (up to 156,672)
Speed: The 27B writes 20.4 tok/s at 512 tokens, about 10 times as fast as llama.cpp
Good for: The 27B with a long context on one used workstation card

NVIDIA GeForce GTX 1080Pascal · 2016 · 8 GB · 180 W

Models: 27B with 65,536 tokens of context, or Nemotron
Speed: The 27B writes 29.4 tok/s at 512 tokens and 16.3 at 64,000
Good for: The 27B on one card, long documents, software tasks

NVIDIA GeForce RTX 3080Ampere · 2020 · 10 GB · Tensor Cores

Models: 27B with the 2B as a drafter (about 85,000 tokens of context with the installer's settings), or Nemotron
Speed: The 27B writes 63.2 tok/s at a depth of 1K at 190 W; 99.9 tok/s on agent prompts with the 2B drafter
Good for: Fast agent work with the 27B, very long documents

AMD Radeon 780MRDNA3 · built into the Ryzen 7 7840HS / 9 7940HS · uses system memory

Models: 27B, Nemotron or 2B, and a second model on the CPU at the same time
Speed: The 27B reads 104 tok/s and writes 10.6 tok/s at 512 tokens
Good for: A small, quiet computer that does agent work

The CPUx86-64 with AVX2 · no graphics card

Models: 2B or 4B
Speed: The 2B writes 64.8 tok/s on a Ryzen 9 7940HS and 28.9 on a Core i7-8700
Good for: Short text jobs, or a second server next to the GPU servers

2BShort, clear jobs: decisions about a text, one-step terminal tasks.

4BTerminal work with several steps.

9BProgramming tasks on one 4 GB card.

27BProgramming, long documents and many steps. On the GTX 1080 it passed 63 of 65 agent tasks.

Which model for which job: from the 65 agent tasks of the Kindi benchmark. Nemotron 3.5 Bolt E4B (experimental) is a second model family for tool calls and agent work. The benchmark on GitHub

DEEP TECH WITH AMIR HUSAIN

Kindi videos.

Hardware demonstrations, release walkthroughs and practical examples.

▶ WATCH / 18:10

KINDI 0.3

27B on an AMD mini PC

AMD support, the CPU, prompt caching and Kindi Bridge.

▶ WATCH / 15:41

KINDI 0.2

Four NVIDIA families

Larger models, paired cards and long-context inference.

▶ WATCH / 19:18

THE FIELD MANUAL

The field manual and agent demos

Practical workflows, prompting and live Pi-agent demonstrations.

DOCUMENTATION

The field manual.

Cover of How To Get Work Done with Kindi and a $15 GPU, third edition for Kindi 0.3.0

How To Get Work Done with Kindi and a $15 GPU, third edition, for Kindi 0.3.0. Which hardware to use for which job, the full performance matrix, the prompt cache, two cards as one engine, power caps and the installer. Then how to work with Kindi and the pi agent: which jobs work, which to avoid, and how to break a big job into steps.

Download the manual (PDF)

Earlier editions: second edition (Kindi 0.2.0), first edition (Kindi 0.1.0).

Download and licensing.

Kindi is free software under AGPL-3.0. Commercial licenses are available for closed products and hosted services.

Read the license · Commercial licensing · Kindi 0.2 archive

Download Kindi

The world's fastestsoftware inference engine for low-cost GPUs.