Karpathy's LLM Wiki: Structured Markdown as RAG Replacement

The most consequential claim embedded in Karpathy's LLM Wiki is deceptively simple: at personal and team scale, structured markdown files with an index are superior to vector databases and embedding pipelines. Traditional RAG treats every query as a cold start — retrieving document chunks, re-ranking them, and synthesizing an answer from scratch each time. LLM Wiki inverts this by having the LLM do the heavy synthesis work upfront during ingestion, producing cross-referenced markdown pages that accumulate knowledge over time. The index.md file replaces the vector index; the file system replaces the database.

This matters because the RAG ecosystem — vector databases like Pinecone, Weaviate, and Chroma, plus the surrounding tooling — has attracted billions in venture capital on the premise that retrieval-augmented generation is the canonical way to give LLMs access to private knowledge. Karpathy's argument is that for moderate-scale knowledge bases (his example: ~100 articles, ~400,000 words), expanding LLM context windows have made the embedding-retrieval-synthesis pipeline unnecessary overhead. The debate, as one analyst put it, centers on whether the industry has 'over-indexed on vector databases for problems that are fundamentally about structure, not similarity.' This is not a claim that RAG is useless at enterprise scale, but it is a sharp challenge to the assumption that RAG is the default starting point.

Perhaps the most underappreciated aspect of the release is not the wiki itself but how Karpathy chose to distribute it. Rather than publishing a traditional GitHub repository with installable code, he published a GitHub Gist — an 'idea file' describing the architecture, conventions, and workflow. His reasoning: in the era of LLM agents, sharing code is less valuable than sharing the idea, because 'the other person's agent customizes and builds it for your specific needs.'

This is a subtle but potentially significant shift in open-source culture. Traditional open-source assumes a shared codebase that contributors converge on. The idea file model assumes divergence by design — every user's agent produces a different implementation tailored to their tools, preferences, and data. The rapid proliferation of community implementations (obsidian-wiki, wiki-skills, LLM-wiki) within days is evidence that this distribution model works. It also raises questions about maintainability and interoperability: if every implementation is bespoke, how do improvements propagate? The idea file may represent a new primitive in how technical knowledge is shared — one level of abstraction above code.

While enthusiasm for LLM Wiki has been overwhelmingly positive, there is a structural risk that several analysts have flagged: error accumulation and knowledge drift. Unlike RAG, which re-derives answers from source documents on every query (providing a natural error-correction mechanism), LLM Wiki builds new knowledge on top of previously compiled knowledge. Once incorrect information is merged into the wiki, future ingestion and updates build on that error, potentially compounding it across dozens of interconnected pages.

Karpathy's architecture includes a 'Lint' operation — a health-check for contradictions, orphan pages, and missing cross-references — but this is a mitigation, not a solution. The lint operation itself relies on the LLM's judgment, which is the same judgment that introduced the error. At small scale with an attentive human operator, this is manageable. At larger scale, or in domains where subtle factual errors have consequences (medical research, legal analysis, financial compliance), the compounding nature of the wiki could become a liability. This tension — between the elegance of compiled knowledge and the safety of re-derived knowledge — is likely to define the next phase of the debate around LLM Wiki versus RAG.

Karpathy's follow-up posts emphasized four principles: Explicit (human-readable markdown), Yours (local files you own), File over app (data outlives any tool), and BYOAI (bring your own AI model). This philosophy directly challenges the SaaS model that dominates knowledge management — tools like Notion, Confluence, and Roam that store data in proprietary formats on remote servers. With LLM Wiki, the knowledge base is plain markdown on your local filesystem, readable by any text editor, searchable by any tool, and portable across any LLM provider.

This resonates with a growing movement in the developer community toward local-first software and data sovereignty. The choice of Obsidian as the IDE is deliberate: Obsidian stores everything as local markdown files with no vendor lock-in. Combined with the BYOAI principle — the wiki works with Claude, GPT, Gemini, or any future model — the architecture is designed to be maximally resilient to platform risk. For organizations concerned about sending proprietary knowledge to third-party vector database providers, the local-file approach offers a compelling alternative. The tradeoff is that collaboration, versioning, and access control must be handled separately, typically through git.

LLM Wiki's rapid traction reflects a broader inflection point in how practitioners think about LLM utility. The first wave of LLM tooling focused on code generation — Copilot, Cursor, Claude Code. The second wave focused on retrieval — RAG pipelines, vector databases, embedding models. Karpathy's framing suggests a third wave: knowledge orchestration, where the LLM's role shifts from answering questions to maintaining a living knowledge artifact.

In this framing, the LLM is not a search engine or a code generator but a knowledge worker — reading sources, synthesizing summaries, maintaining cross-references, resolving contradictions, and updating pages as new information arrives. Each ingested document triggers updates to 10-15 wiki pages, meaning the LLM is doing substantial editorial work, not just retrieval. This is closer to how a human research assistant operates: reading broadly, maintaining structured notes, and keeping a knowledge base current. The question is whether LLMs are reliable enough for this role at scale, or whether the pattern works best as a personal productivity tool for technically sophisticated users like Karpathy himself.