In March I published how the memory layer works: ACT-R cognitive architecture, chunks competing on activation, spreading activation through associative links, decay for anything unused. That post describes the design. This is the post about what five months of production did to it — including the failure that taught me more than everything that worked.
March: one store, confidently wrong alone
The March setup was architecturally clean: a single ACT-R store, every memory a chunk with an activation level, retrieval as competition, decay handling forgetting automatically. It worked. Conversations gained continuity. Preferences stuck. The system felt like it remembered, because by every measurement it was being fed, it did.
That last clause is the whole story of this article.
The black hole
Sometime in early summer, a subtle ingestion bug landed. New memories were keyed on a hash of their slot values — but one memory type shipped without slots. No slots means identical hash for every record of that type. Every ingest of that kind merged into a single row. For weeks. Thousands of distinct memories collapsing into one black-hole record, its reference count climbing into the tens of thousands while the system reported itself healthy at every checkpoint.
The monitoring didn't fail because nobody wrote monitors. It failed because the monitors measured the system against itself. Ingest jobs completed successfully. Retrieval latency stayed fast. Store size looked plausible. Every telemetry signal said fine, because the system was executing its own wrong model of reality perfectly. A black hole is a perfectly valid database state. Nothing alerts on it unless something compares the store against ground truth.
What found it wasn't a dashboard. It was an audit: pulling real retrievals and asking whether the results made sense against what had actually been stored. The answers stopped making sense. Tracing backward from weird retrieval to merge logic to hash function took the fix from mystery to one-liner-class insight:
// the standing rule
Memory systems must be audited against reality, never against their own telemetry. Health checks verify that code ran. Only comparison against ground truth verifies that what ran was correct. If your system can be confidently wrong in a way no alert would catch — it already is.
The repair itself was almost boring by comparison: re-key ingestion on actual content instead of structural slots, backfill the collapsed history from raw logs, rebuild the index. What wasn't boring was the trust question. How do you know the repair worked? You can't take the store's word for it — that's the entity that lied. So the validation ran against independent evidence: replayed interactions, spot-check retrievals, counts reconciled against source records. Trust, once lost inside your own infrastructure, only comes back through external verification.
August: two systems, arguing productively
The architectural response to "one store was silently wrong" wasn't just fixing the bug. It was refusing to ever again have a single point of confident failure. The memory layer now runs two systems in parallel: the original ACT-R store and mem0, a second framework with different internals and different blind spots.
The division isn't redundant — it's complementary. ACT-R does cognitive modeling: activation dynamics, decay curves, associative spread, salience that shifts with context. mem0 handles fast operational recall with its own extraction pipeline, lighter-weight where full simulation is overkill. Each covers the other's weak angles.
The unexpected payoff: disagreement between them became a diagnostic instrument. When ACT-R's ranking and mem0's retrieval disagree about what matters to the current context, the disagreement itself carries information — stale associations, workload drift, a tuning regression in one side. Reconciliation went from afterthought to debugging tool. One system alone can't disagree with itself. Two can argue, and the argument surfaces problems neither would report.
The context-injection plugin
Parallel stores need a controlled way into conversations. Framework RAG hooks retrieve somewhere and hope prompt assembly notices — uncontrolled, unauditable, and impossible to budget. So the workshop built a custom context-injection plugin that sits directly in the response path:
- Query both stores before the model sees the message
- Score and merge candidates under an explicit per-turn retrieval budget
- Inject by role — recency buffer for conversational thread, declarative chunks for background, procedural hints where the task matches learned patterns
- Audit everything — any prompt can answer "why did this memory appear here?"
The custom route cost more code than a stock hook. It bought the thing that mattered during the black-hole repair: when confidence in the ACT-R side dropped, the merge layer shifted retrieval weight to mem0 with a config change. No outage, no scramble, no data lost — just traffic quietly rerouted inside a system designed to tolerate one leg being wrong.
What changed in five months, summarized
March's design was correct on paper and still is. But production added three things paper can't: a respect for silent failure modes, a second opinion that argues back, and an injection contract strict enough to audit. The memory doesn't just remember better now. It can prove it remembers — and when it's wrong, it says so out loud.