VERA: Retrieval You Can Verify

Most retrieval systems fail in the same quiet way. You ask a question, the system pulls a few passages that look relevant, a language model writes a confident paragraph on top of them, and you have no way to tell whether the answer came from your documents or from the model’s imagination. It reads well. That is the problem. A fluent wrong answer is more dangerous than an obvious one, because nobody checks it.

VERA, Verified Enhanced Retrieval Architecture, is my answer to that failure mode. It is a design I wrote in the trenches of real document work: contracts, compliance files, scanned paperwork, operational handovers. The goal was never to make a chatbot that sounds smart. It was to build a system whose answers you can trace back to a source, and whose pipeline refuses to hand you something it cannot stand behind.

This is the Architecture series, so I will give you the blueprint, not the story. Here is how the thing is built and why each part is shaped the way it is.

The core bet: retrieval is the product, generation is the garnish

The dominant mental model of RAG treats the language model as the engine and retrieval as a feeder pipe. VERA inverts that. The retrieval layer is the product. The model’s job is to phrase what retrieval already found, and then to be audited on whether it stayed inside those bounds.

That inversion drives every decision below. If retrieval is the product, then retrieval has to do more than vector-match. It has to filter by structured rules, rank by classical relevance, re-rank by meaning, and pull in neighboring context that a naive search would miss. And the output has to carry enough metadata that a second agent can later check the answer against the facts that produced it.

A fluent wrong answer is more dangerous than an obvious one, because nobody checks it.

Ingestion: turn every document into a structured, labeled chunk

Garbage in, confident garbage out. The hardest and most underrated work in any retrieval system happens before a single query is typed.

The ingestion stage takes everything: clean digital files, semi-structured forms, and scanned paper that needs OCR. For the scanned material, multimodal models or specialized parsing tools extract the text. Everything then gets normalized into Markdown, which keeps headings, lists, and tables intact while staying lightweight to process.

Then comes chunking. Documents are split into semantically coherent pieces, roughly 500 tokens each, using overlapping windows, sentence-boundary detection, or heading-based segmentation. Chunk size is a real tradeoff. Too large and your retrieval is blunt, dragging in irrelevant text. Too small and you sever the context a passage needs to make sense. The overlap exists so that a fact sitting on a chunk boundary is not lost between two halves.

Each chunk is then handed to a language model that writes structured metadata for it:

• Categories: domain labels that let rule-based filters narrow the search space fast.
• Keywords: significant terms, surfaced with TF-IDF, that feed keyword retrieval.
• Atomic facts: short, self-contained statements that summarize what the chunk actually asserts. These become the unit of verification later.
• Custom entities: domain-specific names, codes, and identifiers pulled with named-entity recognition against a custom dictionary.
• Cross-relations: links to other chunks the passage refers to, depends on, explains, or contradicts.

That last item is what turns a pile of chunks into a graph. A chunk is no longer an isolated string. It is a node with typed edges to its neighbors. Here is the shape of a single labeled chunk:

{
  "chunk_id": "chunk_001",
  "content": "Details about shipment schedules and logistics...",
  "metadata": {
    "categories": ["Logistics", "Scheduling"],
    "keywords": ["vessel", "ETA", "port"],
    "atomic_facts": [
      "Vessel ABC is expected to arrive at Port XYZ on 2024-10-15."
    ],
    "cross_relations": ["chunk_002", "chunk_005"],
    "custom_entities": ["Vessel ABC", "Port XYZ"]
  }
}

The atomic facts are the quiet heroes here. When the system later claims something, it can point at the specific fact that justifies it. Traceability is not bolted on at the end. It is baked into the data model from the first pass.

The knowledge base: three stores, one job each

VERA does not pick a single database religion. Different retrieval methods want different storage, so the architecture uses three stores side by side, each doing what it is best at.

• A document store (a NoSQL engine such as Elasticsearch or MongoDB) holds the chunks and their metadata, and serves the inverted indexes that classical keyword search needs.
• A graph database (such as Neo4j) holds the cross-relations as nodes and typed edges: “refers to,” “explains,” “contradicts.” This is what lets a query traverse from one fact to the connected facts around it.
• A vector database (such as Weaviate, FAISS, or Pinecone) holds the embeddings and serves semantic similarity search, organized with an index like HNSW for speed at scale.

The three are not redundant. They answer different questions. The document store answers “which chunks contain these words.” The vector store answers “which chunks mean something similar.” The graph answers “what is connected to this.” A system that only has one of the three is blind in the other two directions.

Indexes get maintained, not just built once. When documents change, the affected indexes and embeddings are refreshed so the knowledge base stays current. Stale retrieval is its own kind of hallucination: confidently citing a version of reality that no longer holds.

Hybrid search: four passes that each correct the last

This is the engine room. A query does not hit one retriever. It runs a cascade, and each pass is there to cover a specific weakness of the pass before it.

Pass one, query understanding. The language model reads the query for intent, generates a vector embedding of it, and, crucially, suggests likely keywords that would surface a good answer. This last move matters more than it looks. If a user asks about a televised event, the model can propose the aliases and synonyms a keyword index would actually match on, lifting the relevance of the classical retrieval that follows. The query is being prepared for both worlds, the keyword world and the semantic world, at the same time.

Pass two, rule-based filtering. Before any expensive scoring, metadata filters slash the search space. Match on category. Require certain keywords. Constrain to specific custom entities. This is cheap, fast, and it keeps the heavier passes from drowning in obviously irrelevant chunks.

Pass three, classical relevance with TF-IDF and BM25. On the filtered set, TF-IDF identifies which terms carry weight, and BM25 ranks chunks while accounting for term-frequency saturation and document length. People are quick to call keyword search old-fashioned. It is also precise, explainable, and unbeatable when a user types an exact identifier, a clause number, or a proper name. Dropping it to go “pure vector” throws away precision you cannot get back.

Pass four, semantic re-ranking. Now the embeddings earn their keep. The system computes similarity (cosine distance) between the query embedding and the embeddings of the BM25-ranked shortlist, then re-orders by meaning and keeps the top N. This catches the cases keywords miss: the passage that answers the question perfectly while sharing almost no words with it.

The ordering is deliberate. Cheap filters first, classical ranking second, expensive semantic comparison last and only on a shortlist. You get the recall of semantic search without paying to embed-compare the entire corpus on every query.

Then one more move, the one most systems skip.

Contextual assembly via the graph. Once the top chunks are chosen, VERA walks the graph to pull in their connected neighbors: the preceding clause, the related schedule, the regulation a passage depends on. A single chunk often answers a question only partway. Its neighbors hold the qualifier that changes the answer. By assembling a coherent context instead of a ranked list of isolated fragments, the system gives the generation stage something it can actually reason over.

The agent pipeline: retrieve, generate, then verify

Here is where VERA stops being a search engine and becomes something that can be trusted. The retrieved context does not go straight to a model that writes an answer and calls it done. It passes through three specialized agents, in order.

The Retriever Agent receives the assembled chunks and validates them. It applies domain rules the search engine does not encode: date ranges, required entities, exclusions. It is the second opinion on relevance, and it can drop a chunk that scored well but does not belong.

The Answer Generation Agent synthesizes the validated context into a response. When the output is a structured document, it fills a template, dropping extracted data into fixed sections. This is where the system writes the report, the summary, the handover. Templates are not a limitation here. They are how you guarantee that every report has every section it is required to have.

The Quality Verification Agent is the part that makes the whole architecture worth building. It takes the draft answer, the original query, and the metadata and atomic facts of the source chunks, and it checks the answer against them. Does each claim trace to a fact? Is the answer internally consistent? Does it comply with the domain rules? If it finds a gap, an error, or an unsupported claim, it does not pass the answer through. It sends work back: it asks the Retriever Agent for more, or the Answer Generation Agent for a correction, and it loops until the answer clears the bar.

The cheapest place to catch a hallucination is before the user ever sees it.

That verification loop is the difference between a system that sounds reliable and one that is. It is also the most expensive part to run, which is exactly why so many RAG systems leave it out and ship the draft. VERA treats it as non-negotiable. An answer that cannot be verified against the source is not an answer. It is a guess with good grammar.

Where the language model lives, and where it does not

It is worth being precise about the model’s role, because the instinct is to let it do everything.

The model is used in four bounded places: understanding and reformulating the query, generating metadata during ingestion, synthesizing the answer, and assisting verification. In every one of those, it operates on top of structured, retrieved material. It is never the source of truth. The source of truth is the document store, the graph, and the atomic facts inside them. The model phrases, extracts, and checks. It does not invent the underlying facts, and the verification agent exists specifically to catch it when it tries.

That boundary is the whole philosophy in one line. Use the model for language. Use the architecture for truth.

What it is good for

Two business shapes justify a system this involved.

Question answering with traceable reasoning. External, for customers and end users who need accurate answers to complex questions, with the ability to surface the source behind each one. Internal, for staff in finance, legal, healthcare, or operations who need to dig into dense material and see the atomic facts a conclusion rests on. In both, the value is not just the answer. It is the citation underneath it.

Report generation with low fabrication risk. This is where the architecture pays for itself. Compiling a risk assessment, an operational handover, a compliance filing: tasks that are tedious, high-stakes, and punishing when a detail is wrong. Template-based generation keeps every report complete and consistent. Automated aggregation pulls the data from the right stores. And the verification agent cross-checks every figure and date against the source before the document is finalized. The output is a draft a human can sign, not a draft a human has to re-derive from scratch.

The common thread across both is the same property: every claim has a receipt.

The blueprint in one breath

Strip away the detail and VERA is four commitments stacked in order.

1. Structure the data before you query it. Chunk, label, extract atomic facts, and link chunks into a graph at ingestion time. Retrieval quality is decided here, not at query time.
2. Search in layers, not in one shot. Filter by rules, rank by keywords, re-rank by meaning, then enrich with graph neighbors. Each pass fixes the blind spot of the last.
3. Separate the roles in generation. Retrieve, generate, and verify are three jobs done by three agents, not one model doing all three on faith.
4. Make verification a gate, not a garnish. Nothing reaches the user that has not been checked against the facts that produced it.

None of these four pieces is exotic on its own. Hybrid search exists. Knowledge graphs exist. Agent pipelines exist. The architecture is in the ordering and the insistence: retrieval is the product, and an answer you cannot verify is not finished. Build it that way and you get a system that does the unglamorous thing most retrieval systems never do. It earns the trust it asks for.

---

Related reading: RAG in Easy Words - the plain-language primer on the retrieval pattern VERA hardens into something you can actually verify.