In this post, I describe a new architectural pattern for building knowledge databases for use by AI Agents, first introduced by Andrej Karpathy in April 2026—link to his article. We’ll discuss the differences between this new approach and traditional RAG architectures based on vector databases, exploring its trade-offs and possible use cases.

Why do we need RAG?

Before we dive deeper into LLM Wiki, let’s answer a simple question first: why do we even need RAG, and what is RAG in the first place?

Simply put, RAG (Retrieval-Augmented Generation) is an architectural pattern that enables us to provide an LLM with additional context from external data sources, thus reducing the risk of hallucinations, made-up sources, etc.

When a large language model is trained (or any ML model, to be precise), we must first gather enough data of good quality for the model to learn. How much exactly is “enough” and what constitutes “good quality” is, of course, a different question. Then the model is trained (in what we call pre-training), and at the end, we have something that is essentially a mathematical representation of the data it has been trained on. We are not going to go deeper into post-training methods or fine-tuning here, as that’s a topic for another post.

There are two main problems with this process that led to the creation of the RAG pattern:

1. The training dataset is finite

Training a large language model is expensive. Very expensive. There are techniques in traditional machine learning like “lifelong learning” or “continual learning,” but in the context of LLMs, they just don’t seamlessly apply. At some point, we must decide, “Okay, that’s enough data for our LLM, we stop gathering data now.” But time moves on; new data is generated every second, new events take place, and some facts become obsolete. Our model does not know any of that because, as such, it is just a frozen model.

One might say, “Yeah, but I can design tools for my agent like web_search and enable it to get any information from the Web!” That’s true, but it doesn’t solve the second problem.

2. Not everything is easily accessible online

A good analogy here is that of an iceberg. What we see above the water surface (public blogs, YouTube, Wikipedia, news articles, papers on Arxiv) is only a small fraction of a much larger structure. There are websites hidden behind paywalls and closed forums, but in our enterprise context, the most important are all kinds of Intranet websites—company SharePoints, Confluence Wikis, SVN repositories. Plus, there is a wealth of knowledge stored in PDF documents, PowerPoint presentations, or simply notes someone might have on their desktop.

This realization led to an architectural pattern in which we provide the LLM with additional information by adding it to its context window, so that it can answer the user based on verifiable data, rather than hallucinating non-existent sources and made-up information.

The Traditional Approach with Vector Databases

In the most popular way of building RAG systems, vector databases are used. There are multiple vector databases available, some of the most notable being Qdrant, pgvector, Pinecone, and Weaviate. They are a special type of database specifically designed for storing and querying data in the form of mathematical arrays, called “embeddings.”

Why do it this way? It is a consequence of how LLMs see the world—their data is stored in the form of vectors that sit within a multi-dimensional mathematical space. The number of dimensions in such a space can be very high, with 1536 being a standard industry value (such as with OpenAI’s embeddings). The reasoning behind this connects to a mathematical law called the “Johnson-Lindenstrauss Lemma.” You can read more about it here.

Storing data in the form of mathematical vectors enables us to perform similarity-based searches, instead of just relying on keyword or full-text searches. By performing relatively simple calculations (calculating the cosine distance, for example), we can check whether the data in the database is closely related to our query or has nothing in common with it.

Problems with Vector RAGs

While using vector stores to provide data to the LLM has solved many problems, it’s not a silver bullet. The main downsides to this approach are:

1. The need to choose the right chunking strategy

You can’t just throw an entire PDF into a vector database and hope for good results. The text must first be split into what we call “chunks,” and choosing the right size for these chunks has a massive influence on the final results. Chunks that are too big (whole pages or entire documents) will cause search precision to drop, while chunks that are too small (single sentences) will cause the information to lose its context.

2. The data in the vector store does not compound

Let’s say our database contains monthly financial reports from our company. We now ask an AI Agent to compile a summary for the last six months from these reports. The agent will turn our query into embeddings, perform a search on the vector database using similarity metrics, fetch relevant documents, produce a summary, and send it back to us.

Perfect. But the problem begins when we want to store this document in our database for future reference. In a traditional vector RAG, this document will simply become another swarm of points in a high-dimensional vector space. Asking the agent for the same summary again will cause the whole process of generating embeddings, calculating similarity metrics, fetching k-nearest results, and compiling a summary to happen all over again.

LLM Wiki

This is where the concept of LLM Wiki comes into play. Let’s look at why this idea is so powerful and how can we use it.

Inspiration - Wikipedia

We have all used Wikipedia. At school, teachers might have said not to use it as a valid source of information, but let’s be honest—we’ve always used it as such. The real power of Wikipedia, or any Wiki system, comes from links; the articles reference each other, building a web-like graph of knowledge.

Using LLM Wiki, we no longer fetch and stitch together isolated points from an abstract vector space—we traverse a graph. This is a huge difference, because a knowledge database built this way is readable not only by LLMs but also by humans.

Let’s now see how it’s built.

Three Layers

There are three distinct layers that together make up the building blocks of an LLM Wiki:

1. Raw Sources

These are the raw documents from which we build our knowledge database. They can be Markdown files, PDFs, Word documents, JIRA tickets—you name it. They will be ingested by an agent that will organize them based on a schema we define. One important thing: they are immutable. The agent reads information from them but never changes them.

2. Wiki Artifacts

These are the Wiki pages compiled by an agent from the raw sources. They can take the form of Markdown files, but not necessarily. In my provided demo application, we use xWiki as a database. In the language of Martin Kleppmann’s Designing Data-Intensive Applications, the whole LLM Wiki is a “derived-data system,” where the information is pulled from the original source (raw documents) and transformed into a Wiki.

3. Schema Definition

This is a single document outlining the definition of the database schema. It’s the entry point for the agent. Based on that document, it splits the compiled wiki pages into categories, links them together, updates the table of contents, and makes sure that the wiki is shaped exactly the way we want. It can be a single Markdown file, like WIKI.md, dropped into the root directory of our Wiki.

Basic Operations

Just like there are three layers, there are three basic operations:

1. Ingest

The ingestion process is split into several steps:

• Before being read by the LLM, the document can be normalized (for example, by turning PDF documents into Markdown files).
• After reading the provided document, the LLM checks the index file of the Wiki and looks for existing documents that might be related to the new one.
• Information from the source documents is stored in the wiki in a format defined by the schema file.
• All other relevant pages are updated with direct links to the new article.

2. Query

The question asked by the user is not turned into embeddings. Instead, the LLM checks the Wiki index, and just like during the ingestion process, it looks for pages that might be relevant to the query. Only then does it read the page and extract the information from it.

3. Lint

Everybody who has maintained documentation of any kind knows that it erodes with time. Links break, and information becomes outdated. Linting is an asynchronous process of keeping the Wiki verified and in good shape.

Open Knowledge Format

Designing and building a Wiki that works on a specific system is one thing, but making it portable to other systems is another. That’s why folks at Google have come up with a standard that helps make it easier to build Wiki systems that are compatible with one another—allowing knowledge to be exported from one system and imported into another. They named it the “Open Knowledge Format” (OKF).

By the time of writing this post, OKF is at version 0.2 and can be found here.

The OKF specification names two files that serve a particular purpose in the standard:

• index.md - This is the table of contents for the whole (or a part of the) Wiki. Every concept document that resides in the Wiki has an entry in the index.md file, where it sits under the right category and has a very short description of its contents. Every time the Wiki is queried by an agent, it reads the index.md file first, thus drastically reducing the number of documents it has to go through to find the needed information.
• log.md - An append-only log file that records the history of all changes made to the wiki. Every new ingest of a document, update to an existing one, or marking of documents as deprecated leaves a trace in this file.

Frontmatter

According to the OKF standard, every concept document in the Wiki consists of two parts: the YAML Frontmatter and the document body. The standard defines only one field as strictly required in the frontmatter—the type, which is a string identifying the kind of document.

Example

A very simple example of an OKF-conformant concept document might look like this:

---
type: Concept
title: Open Knowledge Format
description: A portable format for representing knowledge for humans and AI agents.
tags: [okf, knowledge, ai]
timestamp: 2026-09-05T18:14:00+02:00
---

# Open Knowledge Format

The Open Knowledge Format (OKF) represents curated knowledge as Markdown files with YAML metadata. It is designed to be portable, human-readable, and machine-readable.
OKF documents can describe concepts such as datasets, metrics, APIs, and operational procedures.

Code Demo

I presented the concept of LLM Wiki and Open Knowledge Format at the BRAVE AI Community Meetup in Wrocław on June 28, 2026. The presentation came with a short demo application showcasing a simple use case for the LLM Wiki: migrating legacy, SVN-like documents into xWiki. You can check out the code on my GitHub repo here.

Problems with LLM Wiki

As I mentioned at the very beginning of this article, LLM Wiki is not a silver bullet, nor is it a solution for all problems when providing external data to an LLM. As Thomas Sowell famously said, There are no solutions, only trade-offs.

1. Cold Start and Database Seeding

If you start with an empty database, it might take a while before the graph of links between pages starts to be useful and provide tangible value to you. On the other hand, if you try to seed the database with hundreds of documents at once, the structure it generates might be far from optimal and require tedious linting afterward. Because of that, the seeding process must be carefully planned and executed.

2. Cost of Knowledge Compilation

Generating embeddings from a query and searching a vector database is fairly cheap. Forcing an LLM to read through and reason about a large number of interconnected documents can be much more expensive computationally.

3. Scaling and Context Window Limits

As the database grows, the LLM might be forced to read through more and more documents before presenting the user with an answer. If the system is poorly designed, it might hit the model’s context window limits quickly, effectively obliterating the primary reason for building a RAG in the first place—reducing hallucinated sources and made-up information.

Summary

The LLM Wiki is an intriguing concept and an entirely new way of building knowledge systems that we share with AI agents. While it doesn’t solve every issue, there are many use cases—especially in the world of enterprise software development—where it might provide significantly more value than traditional RAGs built with vector databases.