Why Better RAG Starts With Better Ingestion

The author rebuilt the front‑end of a RAG stack that previously sliced documents into 512‑token chunks, embedded them with a 1536‑dimensional model, and stored the vectors in PostgreSQL pgvector. By expanding the schema to include fields such as source_path, source_title, heading_path, effective_date, status, superseded_by, and content_hash, each chunk now carries the document’s identity, revision date, and lifecycle state. This change directly addressed a failure where two nearly identical policy sections—one from 2021 and a newer one—competed for retrieval, and the older chunk won by a margin of 0.008. Adding the effective date and status made the obsolete version invisible to the top‑5 retrieval, proving that missing provenance can be the weakest link in a RAG pipeline.

The redesign reflects a broader shift in the AI community: while many vendors (e.g., OpenAI, Cohere, Pinecone) pour resources into larger embedding models and sophisticated rerankers, they often overlook how documents are broken down and indexed. Structure‑aware chunking—splitting on markdown headings, preserving table integrity as markdown, and only falling back to token‑size splits when sections exceed limits—keeps semantic boundaries intact, which is essential for accurate similarity search. Embedding the enriched text (prefixed with “Document: … Section: … Effective: …”) supplies the vector model with the same contextual cues that a human would use, yielding a measurable boost without any model upgrade. This approach demonstrates that a well‑engineered ingestion layer can rival or surpass downstream model tricks.

The implications are immediate for any production RAG deployment. By tracking effective dates and supersession links, systems can automatically deprecate stale chunks, reducing hallucinations tied to outdated policies, regulations, or product specs. However, the solution introduces operational complexity: maintaining reliable date extraction across heterogeneous sources, handling edge cases like missing timestamps, and ensuring content_hash‑based re‑ingestion scales. Organizations should watch for emerging tooling that automates provenance capture and for vector‑DB vendors that expose native support for status‑based filtering, as these features will become differentiators in the competitive RAG market.