Sign in Subscribe

MCP Memory Server: Local SQLite Plus Semantic Recall

MCP Memory Server: Local SQLite Plus Semantic Recall

If you have ever typed "MCP memory server" into a search box, you already know what comes back. A long list of memory servers that keep one JSON file on disk and call it persistent memory. Some wrap a notes app. Some wrap a cloud vector database and want an API key before your agent can remember anything at all. And the reference implementation everyone starts from, the official MCP memory server, stores a knowledge graph as flat JSON with no semantic search.

Meanwhile, one of the most upvoted MCP projects on Hacker News this month was a memory server built on SQLite FTS5. People clearly want memory that is local, file-based, and boring in the right ways. The missing piece in most of these setups is semantic recall that does not depend on a network call.

That is exactly the gap Uteke fills as an MCP server. This post walks through how it works, and why we made the choices we did.

The Model Context Protocol made it easy to plug tools into agents like Claude. What it did not standardize is what "memory" means. So the ecosystem converged on a few patterns, each with a cost:

  • Flat JSON or markdown files. Simple, greppable, and keyword-only. Ask for "the decision we made about auth retries" and you get nothing unless your query happens to match the stored words. Recall quality drops fast once you pass a few hundred entries.
  • Cloud vector databases. Good recall, but your agent now needs a network round trip, an API key, and a vendor account to remember yesterday. If the key expires, your agent has amnesia.
  • Homemade FTS5 wrappers. A step up, and genuinely useful. But full-text search alone still fails on paraphrase: "auth retry policy" will not match a memory stored as "we decided login failures should back off exponentially."

What we wanted for our own agents was hybrid recall, local files, and no API key. So we pointed the MCP layer at the engine we already had.

How It Works: Uteke as an MCP Server

Uteke is a memory engine written in Rust, distributed as a single binary. The core is fully offline: SQLite for storage, an embedded HNSW vector index, and an ONNX embedding model that runs on CPU. No config file, no API keys, no network.

There are two ways to use it, and both are true at the same time:

  1. Standalone engine. The uteke CLI and uteke serve HTTP API for direct use.
  2. MCP server mode. External agents, including Claude Desktop, connect to Uteke over the Model Context Protocol and treat it as their memory tool.

In MCP mode, the agent gets the usual memory verbs: store a memory, recall by meaning, search. Behind those verbs sits the same hybrid pipeline as the CLI: the query is embedded locally, matched against the HNSW index for semantic results, and run through SQLite FTS5 for keyword results, then merged. A paraphrased question finds the right memory. So does an exact identifier, which is exactly where pure vector search struggles.

Because the embedding model runs on your machine, recall works with the network cable unplugged. The MCP transport is local too. Your agent's memory never leaves the box unless you decide to expose the HTTP API yourself.

Why SQLite Is the Right Substrate

The Hacker News project that caught our attention picked SQLite FTS5 for the same reasons we did three releases ago:

  • One file. The entire memory store is a single SQLite database. Copy it, back it up with cp, put it in git if you want. No separate database server process to babysit.
  • Transactional writes. Memory writes either land or they do not. When an agent stores context mid-task and the process dies, you do not end up with half a memory.
  • FTS5 is built in. Full-text search with relevance ranking ships inside SQLite itself. There is nothing extra to install for the keyword half of hybrid search.
  • It scales honestly. SQLite handles tens of thousands of rows without breaking a sweat on a laptop. Agent memory workloads are nowhere near its ceiling.

The difference in our stack is that we added the semantic half on top: HNSW for approximate nearest neighbor search and a local ONNX embedder, both living in the same single binary. Hybrid beats either approach alone, and earlier posts on this blog cover the benchmark numbers behind that claim.

Decisions and Trade-offs

Being honest about what we gave up:

We chose a local ONNX model over a frontier embedding API. A hosted embedding model will edge out a small local one on some benchmarks. We accepted that gap in exchange for recall that works offline, with zero per-query cost and zero data egress. For agent memory, where you recall constantly, that trade usually wins.

MCP adds a protocol layer, not a cloud. Running Uteke as an MCP server means your agent speaks MCP to a local process. The memory itself stays in SQLite on your disk. If you never want an agent to talk to it, the CLI works fine alone. Both modes read the same store.

It is still 0.x. Uteke is at v0.16.0 with 236 stars and more than 50 releases, built by a small team that uses it daily. The API surface is stable for our own usage but we version it in the 0.x range for a reason. If you run it in production, pin your version and read the changelog.

License is Apache-2.0. You can embed it, fork it, ship it inside your product. We think memory for agents should not be proprietary infrastructure.

What's Next

The MCP surface is where most of our current work is going. The near-term roadmap, in rough priority order:

  • Richer tool metadata, so agents get better hints about when to store versus recall
  • Dream-cycle maintenance exposed through the MCP interface, so long-running agents can compact their own history (the uteke dream command already covers this in the CLI)
  • More examples for non-Claude MCP clients, since the protocol is client-agnostic by design

If you are evaluating memory servers for your agent stack, the honest pitch is simple: try the JSON-file one first, because it takes five minutes. Then try asking it something in different words than you stored. That failure is where Uteke starts.

Getting Started

The repo is at github.com/codecoradev/uteke, Apache-2.0, single Rust binary, current release v0.16.0. Documentation covers both the CLI and MCP server setup. If you hit a wall or find a recall case that should work and does not, open an issue. We read all of them.