Building a Multi-Agent Runtime With Durable Memory
DBB Software built the agent layer of a self-hosted AI platform: several AI agents per install, each running as an isolated process with its own memory, credentials and permissions, on top of an open-source agent runtime the platform runs unmodified.
Industry
Retail & E-Commerce
Service
Agentic AI Development
Team
1 Senior Full-stack Developer
Project State
March 2026 - Ongoing
Country
NDA
About the Client
The client is a venture-backed, remote-first technology studio. Their platform is a personal AI operating system that runs on the user's own hardware or a private hosted pod, giving each person long-term memory across every conversation and file, multiple collaborating AI agents with distinct roles and personalities, and direct connections to the accounts they already use.
The Client's Initial Request
The client engaged DBB Software to build the agent layer of the platform, where the product promise of a private, long-lived assistant had to survive contact with a runtime that was never designed for more than one person.
Several Agents Per Install
Let one person run a set of AI agents side by side, each with a distinct role, its own memory and its own credentials.
01
Shared Installs With Separated Data
Allow more than one person on a single pod, with each reaching their own files, mail and memory and no one else's.
02
Memory That Holds Over Months
Give each agent knowledge that accumulates from real conversations and mail, survives contradictions, and can be corrected by the person who owns it.
03
Work That Starts Without a User
Let agents act on a schedule and on incoming mail, on a machine with no public address.
04
Boundaries the Product Can Prove
Enforce what an agent may read, write and send in code rather than in instructions the model is asked to obey.
05
Solutions We Delivered
DBB Software built the agent layer around Nous Research's open-source Hermes agent runtime, unmodified. One decision governed the rest: build on the runtime, never inside it, because every fork becomes a permanent maintenance tax. Everything the product needed that the runtime has no concept of was layered around it rather than patched into it, across more than fifty architecture decision records.
1 Process Per User and Agent
The runtime is single-user by design, and the product needed several people on one pod, each reaching their own data. Passing a user id per request was audited and rejected: the runtime's API carries no user field, its tool hooks receive no user id, and one shared key grants the full toolset including a terminal. The runtime's own security documentation states the conclusion directly, that multi-user isolation belongs at the operating-system level.
A second design, one process per user with the personality injected per request, was rejected because it pools every chat and all memory into one process, and cannot be separated back out later.
What shipped treats the agent profile as a template and the running unit as an instance bound to one user-and-agent pair. Config, credentials, memory and wiki slice all bind at process start, which is what makes the pair the only key that gets tools, credentials and memory correct at the same time.
The cost was accepted in the open: 150 to 300 MB of RAM and a 10 to 30 second cold spawn per pair, paid down with lazy spawn on first message and a reaper that reclaims a process after 15 minutes idle. A 16 GB laptop holds four to six warm agents alongside a browser and an editor.
Deterministic Permission Boundaries
Every boundary in the agent layer is enforced in code, and none of it is prompt text.
Each process is mounted with only its own profile, the public wiki and that user's private slice. Mail is absent from the agent's filesystem entirely and is reachable only through a tool the pod scopes server-side.
A pre-tool-call guard is generated per profile. It denies wiki writes, reads outside the user's slice, reads of config and environment files, and shell commands containing those same patterns.
Tool access runs on per-toolkit allowlists. Gmail is pinned to 19 actions chosen from a live inventory, and the list is an include list that fails closed, so a vendor shipping new actions does not silently widen what an agent can reach.
On an agent's own mailbox a per-inbox gate is set to review-required by default. A rule that would auto-send still matches and still logs, and the send is withheld and surfaces to the user as an approve-and-send card.
An LLM-Maintained Memory Wiki
The memory system is a wiki an agent maintains, not a vector store, following the wiki-over-RAG pattern. It reached that shape after five architectures. The design everyone reaches for first, extracting facts and embedding them, was built and retired twice.
Nothing written in chat touches memory synchronously. Every 5 minutes a producer pulls new chat, mail and notes since a watermark, and a small model call returns a keep-or-skip verdict on each item.
Kept items are rendered to markdown deterministically by the shim and are never summarized by a model, then dropped into an inbox. Every 30 minutes an ingest run takes that inbox under a single-writer mutex and hands it to an agent.
Two things are stored separately. The verbatim source is immutable and checksummed, and it is captured before any agent run. The synthesized wiki pages are cross-linked and carry numbered citations back to that source, checked against what the current viewer is allowed to see.
Retrieval runs on two paths: a markdown scan at roughly 21 ms for search and typeahead, and a slower semantic query that runs an agent.
Everything carries a scope, and private content lives in a physically separate tree that pod-wide runs cannot reach.
Nightly Reconciliation and Verified Pages
Contradictions are resolved on a schedule rather than at write time, which is what lets the write path stay fast and deterministic.
A nightly maintenance pass deduplicates entries, merges duplicate pages, reconciles contradictory facts and rebuilds the index.
A deterministic guard sits on top. Once a person marks a page verified, an agent run cannot overwrite it; the agent's version becomes a proposal the person approves or rejects.
Model against model resolves in the nightly pass, and model against human always resolves to the human.
Task Handoff and Sub-Agent Delegation
An agent hands work to another agent through a single mechanism, a handoff tool it calls mid-turn.
The tool writes a task row into the pod's local database carrying title, description, size, parent id, owning agent and creating agent, and logs a handed-off entry on the parent's timeline.
The task lands as proposed rather than active, which is what happens to every agent-authored task. On a person's approval the pod opens a fresh conversation against the receiving agent and streams its events back onto the task record.
What moves between agents is the task text and a parent-child link. Conversation history, memory and files stay where they are, so the receiving agent starts cold and works from the description.
Inside a single agent the runtime's native delegation spawns real sub-agents, 50 iterations each and up to three concurrent, rendered in the chat as a nested tool-call tree.
Scheduled and Inbound Triggers
Agent runs start from five sources besides a user message: scheduled tasks with a cron expression polled every 60 seconds, runtime-native cron jobs per agent, the three memory timers, inbound mail polled at roughly the same 60-second interval, and a background-runs API for long detachable work. Inbound mail fires a deterministic rules engine rather than a full agent run.
Webhooks were dropped in favor of polling. A self-hosted pod then needs no public endpoint and no tunnel, and adds no infrastructure to the machine it runs on.
One agent does not trigger another without a person in between, because a handoff waits for approval. Sub-agent delegation inside a single agent is the one exception.
Failure Containment in the Agent Loop
Each failure mode has a defined path, and the task timeline records what happened.
A failed tool call returns to the loop as a tool result and the model decides what to do with it.
A model or connection failure gets one retry, and only at the connection level. An HTTP error is never retried, so a retry has no way to duplicate a side effect the first call already caused.
A crashed process is restarted by the supervisor, capped at five restarts per rolling hour, with a 90-second readiness timeout, a 5-second stop grace and periodic health checks. There is no drain on restart, so an in-flight turn is lost.
A runaway loop meets a 90-iteration ceiling on the turn, shown live in the chat header while the agent works.
Model Routing by Role
The pod stores a separate model for each role: chat, extraction, embedding, speech-to-text, text-to-speech, real-time audio and image. Cheap models are flagged as preferred for extraction, so background work stays off a frontier model, and memory ingest and bulk import can pin a model of their own again.
A per-message picker sits on top. When the chosen model shares the provider the agent is configured for, the message runs the full agent loop.
A runtime process binds one provider's credentials at spawn and its API takes no per-request provider, so a different provider is served by a direct call from the shim, with the pod injecting the key from the vault. The key is neither stored nor logged. The trade-off is stated to the user: that turn runs without tools, skills or long-term memory.
Users bring their own keys across nine providers: OpenAI, Anthropic, Google, xAI, DeepSeek, Moonshot, OpenRouter, Groq and Qwen, alongside the platform's hosted tier. Speech-to-text also runs locally, while language inference still needs a network.
A Benchmark Harness for Memory
Roughly 7,800 automated tests gate every merge, split across about 1,950 in the Go pod, 2,560 in the shim and 3,300 in the web and shared packages, alongside Playwright end-to-end journeys.
Memory needed something conventional tests cannot give it, because its output is not deterministic enough to assert on. A benchmark harness runs an iteration against a fixed dataset, snapshots the resulting wiki and emits machine-readable metrics: article counts, empty-frontmatter failures, category and size distribution, and planner skip reasons.
Those metrics build a scoreboard across iterations, which turns "did the memory get better?" from a matter of opinion into a number.
Results Achieved
Answers in 21 Milliseconds
Memory search returns in about 21 ms, fast enough to sit behind typeahead.
A Scoreboard for Nondeterministic Output
Memory changes are scored against a fixed dataset, so a rewrite is judged on numbers.
Optimized Task/Model Allocation
Extraction and background passes run on small models, keeping frontier calls for conversation.
Enforcement Outside the Model
Limits live in the filesystem mount and the tool allowlist, where a crafted prompt has no reach.
Source Survives Synthesis
Verbatim source is checksummed and stored before any agent run, so a failed synthesis loses no data.
Human-in-the-Loop Control
Once a person verifies a page, an agent can only propose a change to it.
Build Agents That Act Inside Real Accounts
DBB Software builds multi-agent systems with scoped credentials, durable memory and human approval on the actions that matter.
Contact Us
"Most of our work starts with a 30-minute call where someone describes a product they're trying to ship and one part of the engineering picture they can't get around.
If that's where you are, let's set one up; I'll tell you straight whether we're the right fit.”
Mina Morkos
Business Development Manager
Want a similar outcome for your team?
Ask our AI assistant — it can pull related case studies, talk through the approach, and put you in touch with the team if you want a deeper conversation.
