A Dozen Memory Systems a Month: Four Tests Before You Adopt One

This month Tencent open-sourced an agent memory system that runs entirely locally on SQLite. The r/LocalLLaMA thread gathered 272 upvotes and a reasonable architecture discussion. The most upvoted comment, at 77 points, was not about the architecture. It was a sigh:

"There are so many memory management systems. A dozen new ones pop up every day and they all claim to have solved the problem."

The fatigue is rational. Memory is the feature every agent stack promises and few demonstrate, and launch-week posts all sound identical. So instead of adding another voice to the pile, here are four tests, drawn from that discussion, that any memory system should pass before you adopt it. We grade our own product by the same tests, including where it loses points.

Test 1: recall you can verify

The sharpest problem in the thread: "how could an agent ever know what it should query for to find important but obscure bits of memories?" An agent that recalls well looks, from the outside, like an agent that never needed to recall. That invisibility is why vendor claims need public receipts: a public dataset, committed raw outputs, a scoring script you can run. Our number is 98.2% recall@5 on LongMemEval-S (500 questions, retrieval only), and re-running it costs about $10 with everything in the repo. Whatever tool you evaluate, ask for its equivalent. No receipts, no adoption.

Test 2: traceability over black boxes

One thread participant summarized Tencent's design in a way that doubles as a requirement: "Memory is readable and traceable, so the agent can point back to the original conversation logs instead of relying on a black box of embeddings." When your agent asserts something you will need the "according to what?" A memory layer should answer with a source: which conversation, which author, which document.

Test 3: deliberate forgetting

"Being able to forget is just as important as remembering in the first place, and having control over that is critical."

Forgetting is not a delete button bolted onto storage. It is decay for stale facts, supersede for corrected ones, and hard deletion with a tombstone for values that must never come back. If the system only accumulates, the silent-misleading failure mode we wrote about in "Your AI Notes Are Rotting" is a when, not an if.

Test 4: an exit

Sooner or later you will want out: pricing, a better tool, or the desire to self-host. Check before adopting. Can you export everything in an open format? Does the engine run on your own machine with the same features? A memory layer you cannot leave is a lock-in story waiting for its ending.

Grading ourselves by the same tests

Uteke is our engine, so weigh this accordingly: verifiable recall, pass (98.2% recall@5, raw outputs committed). Traceability, pass (author, source, timestamps, typed supersedes/contradicts links on every memory). Deliberate forgetting, pass (decay, supersede, rejected-value tombstones). Exit, pass (full export any time, Apache-2.0 engine runnable offline as a single binary; Uteke Cloud, the hosted free beta, uses the same engine with the same export path).

Where we honestly lose points: it is 0.x software with rough edges, the ecosystem is smaller than the big-name memory frameworks, and local embeddings trade some recall quality for privacy and zero cost. If those trade-offs bother you, the four tests above are still the right filter, applied to whoever you pick instead.

A dozen memory systems per month is not a signal to adopt faster. It is a signal to filter harder. The thread that complained about fatigue accidentally wrote the buyer's guide.

Sources: the r/LocalLLaMA thread (quotes via Reddit API), Uteke benchmarks, uteke.app.