Memory Encoding & Retrieval Channels

Overview

Every memory in MetaMemory is stored with one semantic embedding plus a set of structured retrieval signals. Rather than relying on a single similarity search, retrieval runs five specialized channels in parallel, each keyed to a different signal:

• Semantic — What was discussed. An embedding generated by your provider (OpenAI, Gemini, Cohere, etc.), searched via pgvector.
• Temporal — When it happened. Timestamps and recency metadata let time-anchored queries resolve without depending on content similarity.
• Emotional — How it felt. Emotion tags (state, intensity, confidence) detected at encoding time act as retrieval and re-ranking signals.
• Keyword — Exact terms. Postgres full-text search catches names, identifiers, and phrasing that embeddings blur.
• Graph — Who and what it connects to. Extracted entities sync to Neo4j, letting retrieval follow relationships across memories.

Channel results are fused with Reciprocal Rank Fusion (RRF), which combines rankings rather than raw scores — a memory can surface through whichever channel makes it relevant to the query.

Semantic Embeddings

Generated asynchronously via your configured embedding provider. MetaMemory supports any provider through the BYOK architecture:

• OpenAI — text-embedding-3-small (1536d), text-embedding-3-large (3072d)
• Google Gemini — gemini-embedding-001
• Cohere — embed-v4.0
• Voyage AI, Mistral, Ollama, Qwen

Semantic embeddings are stored in pgvector for efficient approximate nearest neighbor search. They handle the "what was discussed" dimension — matching queries to memories based on content similarity.

Emotional Tagging

At encoding time, MetaMemory detects the emotional state of the interaction — confident, uncertain, confused, frustrated, insight, or breakthrough — along with an intensity and confidence score, and stores it as structured tags on the memory.

At retrieval time these tags feed the emotional channel and re-ranking: a query made in a frustrated context can preferentially surface memories from similar emotional situations, and longitudinal patterns (recurring frustration on a topic, growing confidence) become queryable signals.

Adaptive Channel Weights

The RRF fusion weights are not static. The channel weight learner adjusts the weight of each channel per query-type bucket based on retrieval-outcome feedback: when a channel keeps contributing memories that turn out to be useful, its weight rises for that query type; when it contributes noise, its weight falls.

This is the mechanism behind self-improving retrieval — in our corporate simulation benchmark, closing this feedback loop lifted recall@10 from 0.75 to 1.00 over 50 iterations while a static configuration stayed flat. Feed the loop by reporting which retrieved memories were actually useful via record_retrieval_outcome.

Two-Phase Storage

When a memory is created, work is split into two phases so writes stay fast:

1. Phase 1 (synchronous): The memory content, temporal metadata, and tags are written immediately and the API returns.
2. Phase 2 (asynchronous): The semantic embedding is generated via your embedding provider and stored to pgvector by a durable background worker. Entity extraction and graph sync to Neo4j also happen in this phase.

This means memory creation latency is determined by database write time, not embedding API latency. Semantic search becomes available as the async phase completes; keyword and temporal retrieval work immediately.

Search Strategies

Search requests accept a strategy: semantic, temporal, emotional, hybrid, fused, or auto. The fused strategy runs all channels with RRF fusion; auto lets the adaptive selector pick per query. Providing context metadata like taskType, domain, and emotionalTags gives the channels more signal to work with — anything not provided is inferred from the query text automatically.