Governed LLM Wiki
Karpathy's LLM Wiki pattern, governed for a team: workbench and authoritative spaces, human approval with four-eyes, permission-scoped agents, conflict handling, lint and a hash-chained audit log. Code, tests, ADRs and evaluation reports on GitHub.
All documents are fictional, invented for testing. The interactive part below runs the real Python package in your browser (Pyodide) with the deterministic backend; the LLM numbers come from local-GPU runs.
Demo video (99 s, every command ran live)
Try it: ask as different people
Governed (this system)
Loading Python in your browser...
Plain RAG over the same files (no permissions, no status)
Governance actions
Evaluation: 28 hand-written questions, Llama 3.1 8B on one GPU
| metric | plain RAG | governed |
|---|---|---|
| correct answers | 16 / 28 | 28 / 28 |
| restricted data leaked to a user without access | 4 | 0 |
| outdated or unflagged-conflict values | 3 | 0 |
| valid, correct citations | 12 / 17 | 19 / 19 |
| open conflict surfaced | 0 / 3 | 3 / 3 |
| permission questions refused without leaking | 0 / 4 | 4 / 4 |
Run twice at temperature 0: all 56 answers identical. Deterministic backend, first blind run: governed 27 / 28 (one router miss, fixed and unit-tested). Limits: 7 invented documents, 28 questions written by the author, rule-based fact extraction. It shows the mechanics work, not behaviour at scale.
Pavan Yadav Annappa