Our documents change daily and the index is always behind. Process at upload, or at query time?

Process at upload for anything derivable from the document itself, and leave only question-dependent work at query time. A corpus that changes daily does not reverse that split. It changes how you refresh: from a nightly rebuild of everything to an incremental reingest of what actually moved. The index falls behind because the refresh runs on a timer, not because the work happened too early.

Why this happens

A document is written once and read many times, so any work placed on the read path is paid again on every read. That asymmetry is the entire argument. Upload-time work is charged per document; query-time work is charged per request. If a page is read a thousand times between edits, parsing it at query time costs a thousand parses against one upload, plus the latency of doing it while the agent waits.

The second reason is that upload-time decisions are not reversible from the query side. Parsing, chunking, embedding, and the choice of index all fix what any later retrieval can see. Query-time stages can filter and reorder what those steps produced, but they cannot recover something the parser dropped or a boundary the chunker cut through the middle of. Chunk sizing in particular is a storage commitment: a query-time misconfiguration is a config change, and a chunking mistake is a backfill.

The payoff shows up in tokens. mem0’s State of AI Agent Memory 2026 reports well-structured retrieval answering queries in roughly 6,800 tokens against about 26,000 for loading full context, close to a 4x reduction. That saving exists only because there was structured context to retrieve against. A store that holds raw text and assembles meaning on the fly has nothing precise to select from, which is also why per-query cost appears to scale with knowledge base size in systems that skipped the upload path.

What happens when the corpus changes faster than you reindex

Freshness is a property of your refresh schedule, not of where the work happens. The gap that hurts is the time between a document changing in the source system and that change reaching the index, and every query landing on that document during the gap is answered from an outdated copy, confidently, with no signal that anything is wrong. A nightly batch leaves a window of up to 24 hours. An hourly batch leaves up to 60 minutes. A change-driven stream lands in seconds to minutes.

The failure gets attributed to the model more often than to the schedule. An agent acting on a snapshot that no longer matches the world produces exactly the behavior teams describe as agent drift, and no amount of query-time reranking fixes it, because the reranker is choosing among stale candidates. One production write-up tracked retrieval recall degrading from 0.92 to 0.74 as an index aged, with previously top-ranked documents sliding from second place to eighth.

The reflex fix, rebuilding more often, is the expensive one. A full reindex costs the same whether one document changed or ten thousand did, and the bill is real: a team re-embedding a 1TB corpus weekly reported spending 12,000 dollars a month on embedding calls alone. The cheaper move is to stop rebuilding on a timer and start reprocessing on a change event, which keeps the work at upload time while cutting both the staleness window and the spend.

Your options

Doing the work at query time is the default because it requires no pipeline. Store the file, let retrieval figure it out on demand. For a small corpus read occasionally, this is genuinely the right call and anything else is premature. It also wins outright when the underlying data changes faster than any pipeline could track. It stops being right the moment reads outnumber writes, because the repeated cost now sits on the request path and grows with traffic rather than with content.

An ingestion pipeline such as LlamaIndex or Unstructured moves parsing, chunking, embedding, and extraction to write time. Queries then read ready-made structure, which is faster and cheaper per request. The cost is real: you now own a pipeline, and changing the embedding model or the chunking strategy means backfilling everything already ingested. Budget for that before you build, and keep the raw source so a rebuild is reprocessing rather than re-collection.

Incremental reingest on change answers the daily-churn case specifically. Instead of rebuilding on a schedule, subscribe to a change feed, database log, or webhook and reprocess only what moved. This keeps the upload-time split intact while shrinking the staleness window to minutes. It depends on the source telling you what changed, and it handles local edits far better than corpus-wide ones: renaming an entity that appears everywhere still forces a broad pass.

Keeping the volatile data out of the index is not a fallback, it is the correct home for anything that moves by the hour. Prices, ticket status, inventory, and on-call rotation belong behind a tool call against the system of record rather than in an index that will be wrong before anyone reads it. The constraint is that you inherit the source’s latency and rate limits, and that a live call answers a point question well and a corpus-wide question badly.

A container that processes on write is the shape of this when the writes come from the agents themselves rather than a batch job. Content is parsed, linked, and embedded as it lands, so every write is already an incremental update and there is no scheduled rebuild to fall behind. It is the wrong home for data whose authoritative copy is changing by the minute somewhere else.

How to decide

How many times is a document read before it changes? This single ratio settles most of it. High read-to-write means upload-time processing pays back quickly. A ratio near one means you are paying to structure things nobody rereads, and query time is simply cheaper.

Can the source tell you what changed? If it emits webhooks or exposes a change log, incremental reingest is available and there is no reason to run a nightly full rebuild. If it cannot, you are choosing between a scheduled rebuild you can afford and moving that data out of the index and behind a live call.

What does a stale answer cost compared with a slow one? Reference documentation tolerates a day of lag. A support agent quoting last week’s pricing does not. Where staleness is the more expensive failure, do not index that data at all: fetch it live, and keep the index for the material that holds still.

What to do next

Measure the read-to-write ratio on your actual corpus, not the one you imagine. Most teams find a long tail of documents read constantly and edited almost never, which is exactly the population that should be fully processed at upload, and a small volatile set that should not be in the index at all.

Then check what your refresh is really doing. If it is a nightly full rebuild, you are paying for a whole corpus to fix a handful of documents and still living with a 24 hour staleness window. If you are considering a periodic consolidation job to tidy things up afterwards, read when agent memory needs sleep first: a background pass is worth having for genuinely cross-document work, but it is a poor substitute for a write path that never made the mess.

The last row in that table is ours. Wire parses, links, and embeds each write as it lands, so incremental is the only path and there is no scheduled rebuild to fall behind; the reasoning is written up in how it keeps agent queries efficient. That fits context the agents themselves produce and reread. It does not fit a system of record changing by the minute, where a live call is still the right answer.

Sources: State of AI Agent Memory 2026 (mem0) · The RAG freshness problem · RAG index staleness gap · Retrieval latency optimization for production RAG