Benchmark · July 6, 2026
The New Wire Architecture, Benchmarked
Same container, same 64 questions, same judge. We rebuilt Wire's platform from the ground up, including how retrieval works inside every container, and benchmarked it head to head against the current architecture before rollout.
What we tested
Both architectures answered every question through the same wire_search tool call with the same top-5 cutoff. Unlike our Wire vs RAG benchmark, which scored generated answers, this benchmark scores the retrieval step itself: every retrieved passage was judged for relevance against the expected answer, passage by passage, by an LLM judge at temperature zero. A re-judged sample agreed with the original verdicts 100% of the time. Both runs completed with zero errors and zero unjudged questions.
The architectures
Current architecture
Wire as it ships today: the platform measured in our earlier benchmark against standard RAG, serving semantic search over each container's processed entries.
Next-generation architecture
Wire rebuilt from the ground up. Every container runs as its own complete unit, with its data, knowledge graph, and search index together, served close to your agents. Retrieval was rebuilt along with it to draw on multiple signals instead of one. Same API, same tools, same response shape.
What changed under the hood
The headline numbers come from architectural decisions, not tuning. A few worth naming.
The vector index lives inside the container
On the new architecture the vector index moves into the container itself, embedded right next to the data it covers. Semantic search runs in-process: no network hop per query, and no separate index that can drift out of sync with the content it indexes. One container, one self-contained unit of data, graph, and search.
Isolation extends to compute
Wire containers have always been isolated at the data layer, never rows in someone else's table. On the new architecture, isolation reaches the compute layer too: workspaces can serve their containers on dedicated capacity, so a busy neighbor never slows your agents down.
Durable snapshots, fast wake
Idle containers suspend to durable snapshots and wake on the next request, which is where the 1.4 second cold start comes from. Snapshots mean a container can be reconstructed anywhere, byte for byte, including its search indexes, and placement is optimized for latency to your agents: containers serve from close to wherever the request comes from.
Nothing changes at the API
The same tools, the same request shapes, and the same response format sit on top of all of it. Agents and integrations built against Wire today keep working unchanged, they just get better evidence back.
Results: current vs next-generation architecture
Retrieval quality across all 64 questions, top 5 results per query.
| Current | Next generation | |
|---|---|---|
| Recall@5 | 78.1% | 89.1% |
| First result relevant (Hit@1) | 46.9% | 59.4% |
| Relevant result in top 3 (Hit@3) | 70.3% | 82.8% |
| Mean reciprocal rank (MRR@5) | 0.584 | 0.715 |
| Ranking quality (nDCG@5) | 0.622 | 0.751 |
| Precision@5 | 24.1% | 39.5% |
By question type
Recall@5 when the answer lives in one passage. The new architecture approaches saturation on this corpus.
Recall@5 when the answer requires evidence from multiple sources. Still the hardest tier, and still improved.
Faster and steadier to respond
Retrieval quality is only useful if the container answers when an agent calls. We measured tool-call responsiveness on both architectures from the same vantage point.
| Current | Next generation | |
|---|---|---|
| Warm tool response (typical) | ~0.4s, with spikes past 2s | ~0.3s, steady |
| Cold first response (idle container) | 3.7s | 1.4s |
Why consistency matters: agents call containers inside tool loops, where one slow response stalls the whole chain. The new architecture answers faster on average and, more importantly, without the latency spikes of the current one.
Existing containers come along, measured intact
The benchmark container did not start life on the new architecture. It was migrated from the current architecture, all 13,991 entries and 31,393 knowledge-graph relationships, and then benchmarked. The migrated data scored higher on the new architecture than the same data scored on the platform it came from.
That is the strongest parity check a migration can pass: nothing was lost, and everything got more findable. No integration changes are required, the same search tool returns better-ranked evidence.
Methodology notes
Both architectures were queried in a single session from a single vantage point, sequentially, so latency figures are directly comparable. Cold-start figures are the first request against an idle container.
Relevance was judged per retrieved passage against the expected answer by an LLM judge at temperature zero. The identical judge and prompts graded both architectures, so any judge bias applies equally to both columns: the deltas between architectures are robust even where absolute calibration might differ from a human grader. A stratified re-judging sample showed 100% agreement with the original verdicts.
Recall@5 here is answer-level: a question counts as recalled if at least one retrieved passage supports the expected answer. Precision@5, MRR@5, and nDCG@5 are computed from the per-passage verdicts with binary relevance. Raw per-question retrievals and verdicts are archived and auditable.
Frequently asked questions
About the benchmark and the new architecture.