Memory

Structured knowledge consolidation from conversations

Beyond chat history

Most agent frameworks store memory as freeform text — conversation logs, MEMORY.md files, or unstructured vector embeddings. When you ask “who’s Alice?” six months later, the system searches through raw conversation fragments and hopes for the best.

Unity takes a different approach. Every 50 messages, the MemoryManager runs a background extraction pass and pulls out structured data:

• Contact profiles — who people are, their roles, organizations, relationships
• Per-contact summaries — what you’ve been discussing with each person, sentiment, themes
• Response policies — how each person prefers to communicate (formal vs casual, preferred channel, timezone)
• Domain knowledge — project details, preferences, long-term facts
• Tasks — things committed to, deadlines, follow-ups

This is stored in typed, queryable tables — not freeform text. After a month of use, the system has a working model of your world.

This is continual learning at the agent layer, not model fine-tuning: Unity improves by updating durable structured state that future reasoning can query directly.

How consolidation works

The MemoryManager observes message events on the EventBus and runs extraction when the message count crosses a threshold:

1. Transcript chunk — the MemoryManager grabs the latest unconsolidated messages
2. Contact extraction — identifies people mentioned, creates or updates contact records with new details
3. Knowledge extraction — pulls out facts, project details, preferences, and stores them in the KnowledgeManager’s tables
4. Task extraction — identifies commitments, deadlines, and follow-ups
5. Summary updates — updates rolling per-contact summaries

Each extraction step uses its own LLM call with a specialized prompt. The outputs go directly into the respective managers’ structured storage — ContactManager for people, KnowledgeManager for facts, TaskScheduler for commitments.

Structured vs. freeform

The distinction matters in practice:

Capability	Freeform memory (MEMORY.md)	Structured consolidation (Unity)
“Who’s Alice?”	Search text for “Alice” mentions	Query Contacts table: name, role, org, relationship
”What did Alice say about the budget?”	Fuzzy search across all history	Join Contacts → Transcripts with contact filter
”How does Alice prefer to communicate?”	Hopefully mentioned somewhere	response_policy field on Alice’s contact record
”What’s overdue?”	Parse dates from text (unreliable)	Query Tasks with deadline < now filter
After 10,000 messages	Slow, noisy search	Same speed — structured tables don’t degrade

The EventBus connection

The MemoryManager subscribes to events via the EventBus (unity/events/event_bus.py). When enable_callbacks is set, it registers handlers for message events and ManagerMethod events. This means memory consolidation can run asynchronously without blocking the main conversation flow.

The EventBus itself is a typed pub/sub backbone using Pydantic event envelopes. Events carry lineage labels aligned with the tool loop hierarchy, so the MemoryManager can attribute events to specific operations.

Offline by design

The MemoryManager is explicitly offline — it runs periodically, not in the live conversation loop. Its methods return strings (not steerable handles) because there’s no user to steer them. This keeps the main conversation path fast: the user talks to the ConversationManager and Actor, while memory consolidation happens in the background.

Where to start reading

File	What’s there
unity/memory_manager/memory_manager.py	The consolidation pipeline
unity/events/event_bus.py	Typed pub/sub backbone
unity/contact_manager/base.py	Where contact extractions land
unity/knowledge_manager/base.py	Where knowledge extractions land