Agent Infrastructure

How to Create Everlasting Agentic Memory

Why storage is not memory, and what we built instead

Vince Mundy, CEO of Joxy· · 9 min read
TL;DR

Storage is not memory. A durable agent needs four things: consolidation (extract what future conversations should inherit), scope (each fact has one canonical home), supersession (current truth beats perfect recall), and controlled forgetting (resolution decays, lessons persist). Kat, our Meta Ads agent, keeps this memory outside the model: inspectable, correctable, and owned.

In this post
  1. Why we built memory this way
  2. Context is not durable memory
  3. Compaction is not long-term memory
  4. Consolidating conversations
  5. Memory needs scope
  6. Current truth over perfect recall
  7. Controlled forgetting
  8. Broad memory, narrow context
  9. Understanding, not evidence
  10. Memory should be owned
  11. How we know it works

The future of AI agents will not be won by the system that stores the most. It will be won by the system that knows what deserves to survive.

An agent that remembers everything will eventually remember nothing well. The important facts become buried beneath expired plans, outdated assumptions, repeated reports, and thousands of details that no longer matter.

That is because storage is not memory. A context window can temporarily hold information. A database can preserve it indefinitely. Neither guarantees that an agent will retrieve the right fact, understand whether it is still true, or know how it should influence the next decision.

For this article, we will use Kat, the Meta Ads agent we built on Joxy. Kat can analyze campaigns, diagnose performance problems, and recommend what to do next. But intelligence alone does not make her a long-term operating partner. Without durable memory, each new conversation begins with Kat reconstructing the client’s world: What does the client sell? What has already been tested? Which targets are current? Which recommendations were rejected? How does Vince prefer reports to be written?

Kat may be intelligent, but without continuity she is still a talented stranger.

Kat is one agent among many on Joxy. The same memory layer serves agents that work CRMs, inboxes, and spreadsheets; ads simply make the examples concrete.

We built Joxy’s memory system to change that. The goal is not to preserve every sentence. It is to ensure that meaningful work leaves the system more capable than before. A client decision should become client history. A correction should improve how Kat works with Vince. A technical failure should become an operational lesson.

“The model gives Kat intelligence. Memory gives that intelligence a history.”
A conversation with Kat Compaction compress the episode to continue the task Consolidation “Would a future conversation act differently for knowing this?” Episode resume this piece of work ages out with the task Client current truth CPA $40 → CPA $32 current Collaborator how Vince prefers to work Connection how external systems behave = the integration guide Org identity and authority One canonical home per fact · resolution decays with age, lessons persist Just-in-time retrieval assemble only what this task needs Reports stay behind as evidence; memory keeps the pointer The next conversation starts smarter Where facts are written and superseded Durable store Temporary or reference
The memory pipeline: conversations are compacted, consolidated into five scoped stores where truth can be superseded, then retrieved just in time for the next task.

Why we built memory this way

Models will continue to change. They may run in the cloud, on local devices, or across several providers. Kat might use one model today and another next year. But the client remains the same client. Its business model, targets, campaign history, previous experiments, and operational quirks should not disappear because the reasoning engine changed.

The model is replaceable intelligence. The accumulated understanding is not.

This is why we keep memory outside the model. We are not building chat history. We are building a persistent context layer that survives a closed thread, a new interface, a model upgrade, a team handoff, and years of accumulated work.

The conversation can end. The understanding should remain.

Context is not the same as durable memory

An agent’s context contains what it can see right now: the current conversation, instructions, documents, tool results, and recent decisions. But context is temporary. Anthropic describes context as a finite resource and recommends compaction, structured memory, and just-in-time retrieval rather than filling the window with everything an agent might possibly need.

Suppose Vince tells Kat: “The client’s target CPA is now $32 instead of $40.” At that moment, the information exists in Kat’s context. It becomes durable memory only when Kat can recover the current target in a future conversation without needing the original thread.

A larger context window can delay forgetting. It does not decide what deserves to survive.

Compaction is not long-term memory

Long-running agents need compaction: reducing a large conversation into a smaller representation so the agent can continue without carrying every previous message. Compaction answers one question: what does Kat need to retain to continue this task? Long-term memory answers another: what should Kat still know after this task has ended?

OpenAI’s guidance similarly separates short-term session management from state-based long-term memory: distill durable information, resolve changes, and make the resulting state available in future runs.

A thread summary may preserve that an audit is unfinished. But a lasting change to the client’s target, a repeated reporting preference, or a technical discovery should remain useful beyond that thread. This is the difference between preserving an episode and preserving knowledge.

“Compaction helps an agent continue. Memory helps an agent improve.”

Conversations must be consolidated into knowledge

Some details in a conversation matter only to the current task. Others should change how the agent behaves forever. We call this transition memory consolidation.

Consolidation in Joxy runs as an asynchronous step at natural boundaries: when a thread closes, when a task completes, or when compaction fires on a long session. A consolidation pass reads the compressed episode and asks one question of every candidate fact: would a future conversation act differently for knowing this? Facts that pass are written to the appropriate store with their provenance. Facts that fail stay in the episode and age out with it.

One conversation with Kat might reveal that a client’s performance target changed, that Vince prefers creative analysis to appear first, that a connection produces unreliable results under certain conditions, and that a report still needs finishing. Those facts should not be treated the same way. The unfinished report belongs to the current episode. The target change belongs to the continuing client state. The reporting preference should improve how Kat works with Vince across future clients. The technical discovery should prevent the same operational mistake from happening again; this is the same mechanism as the integration guide in our routing architecture, where every unblocked failure becomes documentation the next plan inherits.

Research on generative agents explored a related concept through reflection: individual experiences are synthesized into higher-level conclusions that guide later behavior. Our version is deliberately practical: temporary experience should be converted into reusable understanding.

This is where most homegrown memory systems fall short. They save the conversation but fail to extract what the next conversation should inherit.

Memory needs scope

Not every memory should be available everywhere. Something needed to resume one thread is different from a fact about a client, which is different from a preference about how Vince collaborates, which is different from an organization-wide approval rule.

We separate memory by the question it answers, and each fact carries its scope explicitly:

{
  "id": "mem_8a41",
  "scope": "client",
  "subject": "client_theramint",
  "fact": "Target CPA is $32",
  "source": "thread_5521, Vince, 2026-06-30",
  "valid_from": "2026-06-30",
  "superseded_by": null,
  "supersedes": "mem_7c02"
}

The scope field routes retrieval: episode for resuming work, client for current client state, collaborator for how a person prefers to work, connection for how an external system behaves (a Meta Ads connection today, a CRM connection tomorrow), org for identity and authority. This avoids one enormous memory pool in which every fact competes for attention. It also reduces contradictions, because each durable fact has one canonical owner even when search indexes provide several ways to find it.

“Retrieval can have many paths. Truth should have one home.”

Current truth matters more than perfect recall

Most discussions about AI memory focus on recall: can the agent retrieve something that was said before? But recall alone is not enough. Kat might perfectly remember that a client’s target was once $40 while failing to understand that it is now $32. That is not useful memory. It is accurately preserved staleness.

A long-term memory system needs supersession. When new information conflicts with an old fact, the new value becomes current truth and the old record is closed, not deleted:

{
  "id": "mem_7c02",
  "scope": "client",
  "subject": "client_theramint",
  "fact": "Target CPA is $40",
  "valid_from": "2026-02-14",
  "valid_until": "2026-06-30",
  "superseded_by": "mem_8a41"
}

The hard part is not writing this record. It is detecting the conflict in the first place: recognizing that “target CPA is $32” and “the client wants acquisitions under thirty-five dollars” describe the same fact about the same subject. Consolidation therefore resolves each candidate fact to a subject and attribute before writing, and when resolution is uncertain, the system keeps both records flagged for review rather than silently guessing.

Done right, Kat understands both what is true now and how the current state was reached. The old target no longer governs action, but the change still matters when interpreting past performance.

Everlasting memory requires controlled forgetting

An everlasting memory system cannot keep every detail at full resolution forever. Without forgetting, memory eventually becomes another transcript, only larger, older, and more confident.

We designed Joxy’s memory across time horizons. Recent work retains enough detail to continue the task. Current client state remains readily available while operationally relevant. Older activity is progressively compressed into decisions, outcomes, and meaningful changes. At the one-year horizon, the system preserves long-term baselines and important history while routine operational detail disappears.

That is not a one-year limit. A durable preference, an organizational rule, or a critical integration lesson may remain useful far longer. What changes over time is the resolution: recent memory preserves the event; older memory preserves what the event taught us.

Persistent memory should be broad, but active context should be narrow

Kat may eventually accumulate years of useful knowledge. That does not mean all of it should be loaded into every conversation.

When Kat starts work on a client, she receives the client’s current state, the relevant working preferences, and the operational context the task requires. She does not receive every previous conversation, every historical report, and every fact about every other account. Anthropic’s context-engineering guidance recommends the same: retrieve information when it becomes relevant rather than preloading everything.

The objective is not maximum retrieval. It is relevant retrieval.

“You want an archive behind the agent, not falling on top of it.”

Memory should preserve understanding without replacing evidence

A full Meta Ads audit contains charts, metrics, campaign-level findings, and detailed recommendations. That report should remain available. But Kat does not need the entire report inserted into every future conversation.

Instead, memory preserves the important conclusion and maintains a route back to the original artifact. The report remains the evidence. Memory carries forward the understanding. This keeps long-term memory compact without severing it from its source.

The accumulated understanding should be owned

Models will change. Interfaces will change. Agents will move between chat, email, dashboards, automated workflows, and private infrastructure. The accumulated understanding should survive all of it.

That is why memory lives outside the model and remains inspectable, correctable, portable, and removable. An organization should be able to see what its agent believes, correct something that is wrong, and govern which agents may access it. This matters most when consolidation gets something wrong: a misheard target or a preference attributed to the wrong person does not just sit in a log, it actively steers future behavior. A memory system without a correction path does not just forget badly. It remembers badly, at scale.

The future of agent memory cannot be an invisible black box that simply claims to “know you.” It must be governed context.

How we know it works

Memory quality is measurable, and we treat it that way. We watch four things:

1
Staleness incidents
Kat acting on a superseded fact.
2
Contradiction rate
Conflicting records for the same subject and attribute that consolidation failed to merge.
3
Retrieval relevance
Whether the context assembled for a task actually gets used by it.
4
Memory on vs. off
Task performance with memory enabled versus disabled, for the same request.

A memory system that cannot demonstrate the last one is a cost, not a capability.

The model gives Kat intelligence. Memory gives that intelligence a history. And the history, inspected, scoped, superseded, and owned, is what turns a talented stranger into a long-term operating partner.

VM
Vince Mundy
CEO, Joxy