What RAG cannot see, and what KAG actually requires
In hindsight, switching from RAG to KAG looks like an obvious call. While building with RAG, it was not. RAG worked. For many teams and use cases, it still does. You chunk documents, embed them, retrieve on similarity, and stuff context into the prompt. The first time it answers something that would have taken a human twenty minutes to find, it feels like real progress.
The drift toward KAG was not a single incident. It was a slow accumulation of cases where the answer was almost right. Not hallucination-wrong. Those are obvious, and you can build evals around them. More like outdated-wrong. Source-confusion-wrong. The model retrieves the right document but misses the amendments. That kind of wrong is harder to catch, harder to explain to the user wondering why the chatbot is citing a two-year-old policy, and harder to fix, because retrieval did exactly what you asked. The problem is upstream of retrieval.
The shift KAG introduces is upstream. What the retrieval is working against matters more than how the retrieval works. If the knowledge base is a document dump, you get document retrieval. If knowledge is structured, relationships are explicit, sources have authority levels, and outdated content actually gets retired, the model has something to reason over instead of pattern-match against. Different class of output.
