Bigger Context Windows Are Making RAG Look Optional

Retrieval-augmented generation became the default answer to 'the model doesn't know my docs.' Chunk the corpus, embed it, pull the top-k, stuff the prompt. It works, and it still works when the corpus is huge. What changed in 2026 is that the cheap models started swallowing 128k–1M tokens without falling over, and a surprising number of internal chatbots discovered they did not need a vector database for the first pass.

The reason is not that embeddings got worse. It is that RAG was covering two different problems: recall over a warehouse of documents, and the model's inability to see the relevant ten pages at once. The second problem shrinks when you can paste the wiki, the RFC, and last week's incident notes into one request. Teams that measured this honestly found retrieval still wins on million-doc corpora and anything that changes hourly. It loses on the 200-page handbook everyone was chunking out of habit.

The trap is treating a context window like memory. Tokens in the prompt are a scratch pad for this call, not a filing cabinet. Long context also gets expensive and sloppy in the middle, which is why the better setups now do both: dump the working set when it fits, retrieve when it does not, and stop pretending every question needs a similarity search.