How to Add Memory to Your AI Agent: Short-Term, Long-Term, and Vector Store Patterns

The agent you built from scratch can call tools, reason about results, and answer follow-up questions. But close the session and start a new one — it has no idea who you are.

That is the memory problem. An LLM's context window is working memory, not storage. Every conversation starts from zero unless you explicitly build persistence into your agent's architecture.

This tutorial walks through three memory patterns that solve this, each targeting a different need: short-term memory for keeping conversations coherent, long-term memory for remembering users across sessions, and vector store memory for semantically retrieving relevant past interactions. All examples use Python with practical, runnable code.

Why Your Agent Needs Memory

A context window is not memory. It is a scratchpad that gets wiped between API calls. Even within a single session, a long conversation will eventually exceed the token limit and start dropping earlier messages.

Without memory, your agent cannot:

• Remember user preferences — "I already told you I use TypeScript, not JavaScript"
• Build on past work — "Continue where we left off yesterday"
• Learn from interactions — making the same mistakes in every session

The fix is layered memory, the same way humans operate. You hold recent thoughts in working memory, store important facts long-term, and retrieve distant memories when something triggers them. Agent memory follows the same structure:

Memory Type	What It Stores	Persistence	Retrieval
Short-term	Recent messages in the current conversation	Session only	Sequential (last N messages)
Long-term	User preferences, facts, learned behaviors	Across sessions	Key-based lookup
Vector store	Embedded past interactions and documents	Across sessions	Semantic similarity search

Most production agents use at least two of these layers. Let's build each one.

Short-Term Memory: Keeping the Conversation Coherent

Short-term memory is the simplest pattern. You maintain a buffer of recent messages and pass them into each LLM call so the agent has conversational context.

The naive approach — appending every message to a list and sending the full history — works until it doesn't. Long conversations blow past token limits, increase latency, and inflate API costs.

The fix is message trimming: keep the most recent N messages (or N tokens) and discard the rest.

Implementation with LangGraph

LangGraph handles this with its MessagesState and a built-in message trimmer:

from langchain_openai import ChatOpenAI
from langgraph.graph import StateGraph, MessagesState
from langgraph.checkpoint.memory import MemorySaver
from langchain_core.messages import trim_messages

model = ChatOpenAI(model="gpt-4o-mini")

# Trim to the 10 most recent messages, always keeping the system message
trimmer = trim_messages(
    max_tokens=2048,
    strategy="last",
    token_counter=model,
    include_system=True,
)

def chatbot(state: MessagesState):
    trimmed = trimmer.invoke(state["messages"])
    response = model.invoke(trimmed)
    return {"messages": [response]}

# Build the graph with in-memory checkpointing
graph = StateGraph(MessagesState)
graph.add_node("chat", chatbot)
graph.set_entry_point("chat")

checkpointer = MemorySaver()
app = graph.compile(checkpointer=checkpointer)

# Each thread_id maintains its own conversation history
config = {"configurable": {"thread_id": "user-123"}}
result = app.invoke({"messages": [("user", "What is RAG?")]}, config)
print(result["messages"][-1].content)

The thread_id in the config isolates conversations. Same thread, same history. Different thread, fresh start.

When this breaks down: short-term memory only lives for the current session. Restart the process and the MemorySaver is empty. For anything beyond prototyping, you need persistence — which brings us to long-term memory.

A Simpler Alternative: Manual Buffer

If you are not using a framework, the pattern is straightforward:

from collections import deque

class ConversationBuffer:
    def __init__(self, max_messages: int = 20):
        self.messages = deque(maxlen=max_messages)

    def add(self, role: str, content: str):
        self.messages.append({"role": role, "content": content})

    def get_context(self) -> list[dict]:
        return list(self.messages)

Wrap your LLM calls with this buffer and you have basic short-term memory without any dependencies.

Long-Term Memory: Remembering Across Sessions

Long-term memory is where things get interesting. The agent needs to persist information — user preferences, extracted facts, past decisions — and retrieve it in future sessions.

Cognitive science breaks long-term memory into three subtypes, and this maps directly to agent design:

• Semantic memory — facts and knowledge: "This user prefers Python over JavaScript"
• Episodic memory — specific past experiences: "Last time this user asked about deployment, they were using AWS"
• Procedural memory — how to do things: "When this user asks for code, they want type hints included"

You don't need to implement all three. For most agents, semantic memory (storing user facts and preferences) covers 80% of use cases.

Implementation with Mem0

Mem0 is purpose-built for this. It extracts facts from conversations, deduplicates them, and makes them searchable — all behind a simple API:

from mem0 import Memory
from openai import OpenAI

memory = Memory()
client = OpenAI()

USER_ID = "user-123"

def chat_with_memory(user_message: str) -> str:
    # Retrieve relevant memories for context
    memories = memory.search(user_message, user_id=USER_ID)
    memory_context = "\n".join(
        f"- {m['memory']}" for m in memories
    )

    system_prompt = f"""You are a helpful assistant.
You know the following about this user:
{memory_context if memory_context else 'Nothing yet.'}"""

    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=[
            {"role": "system", "content": system_prompt},
            {"role": "user", "content": user_message},
        ],
    )

    assistant_reply = response.choices[0].message.content

    # Store new memories from this exchange
    memory.add(
        [
            {"role": "user", "content": user_message},
            {"role": "assistant", "content": assistant_reply},
        ],
        user_id=USER_ID,
    )

    return assistant_reply

# Session 1
chat_with_memory("I'm building a RAG pipeline in Python with pgvector.")

# Session 2 — days later, new process, but Mem0 remembers
chat_with_memory("What database should I use for my project?")
# Agent knows you're using pgvector and working in Python

Mem0 handles the hard parts: it extracts facts from conversation turns, resolves conflicts (if the user later says "Actually, I switched to Pinecone"), and stores everything in its own managed store. You can also self-host it with your own database backend.

Alternative: LangGraph with PostgresSaver

If you are already using LangGraph, its PostgresSaver checkpointer persists the full conversation state to Postgres. This gives you session continuity without a separate memory service:

from langgraph.checkpoint.postgres import PostgresSaver

checkpointer = PostgresSaver.from_conn_string(
    "postgresql://user:pass@localhost:5432/agents"
)
checkpointer.setup()  # Creates tables on first run

app = graph.compile(checkpointer=checkpointer)

This is not the same as Mem0's fact extraction — it replays the full message history rather than distilling it into structured memories. Both approaches have trade-offs. Checkpointing is simpler but uses more tokens. Fact extraction is leaner but adds complexity.

For agents that need to track how facts change over time, Zep builds a temporal knowledge graph where each fact has a timestamp. Useful when "the user's preferred framework" was Flask six months ago but FastAPI today.

Vector Store Memory: Semantic Search Over Past Interactions

Sometimes you need to retrieve information that does not match exact keywords. A user asks "How did we handle the authentication issue?" and the relevant memory is a conversation from three weeks ago about JWT token rotation.

Vector store memory solves this by embedding past interactions into vectors and retrieving them by semantic similarity rather than keyword match.

This is closely related to RAG, but instead of searching external documents, you are searching the agent's own past interactions.

Implementation with ChromaDB

ChromaDB is an open-source embedding database that runs locally with zero configuration:

import chromadb
from openai import OpenAI

client = OpenAI()
chroma = chromadb.PersistentClient(path="./agent_memory")
collection = chroma.get_or_create_collection("conversations")

def embed(text: str) -> list[float]:
    response = client.embeddings.create(
        model="text-embedding-3-small",
        input=text,
    )
    return response.data[0].embedding

def store_interaction(interaction_id: str, text: str):
    collection.add(
        ids=[interaction_id],
        embeddings=[embed(text)],
        documents=[text],
    )

def recall(query: str, top_k: int = 3) -> list[str]:
    results = collection.query(
        query_embeddings=[embed(query)],
        n_results=top_k,
    )
    return results["documents"][0]

# Store past interactions
store_interaction("conv-1", "User asked about JWT auth. We implemented refresh token rotation with Redis.")
store_interaction("conv-2", "User migrated their database from MySQL to PostgreSQL with pgvector.")
store_interaction("conv-3", "User deployed their agent to AWS Lambda behind API Gateway.")

# Later — semantic retrieval
results = recall("How did we handle authentication?")
# Returns the JWT auth conversation, even though "authentication" ≠ "JWT auth"

The key insight: you are not searching for exact words. The embedding captures meaning, so "authentication" finds "JWT refresh token rotation" because they are semantically related.

When to use vector memory vs. structured long-term memory:

• Vector memory excels when you do not know what questions will be asked. It is open-ended retrieval — good for "find me anything relevant to X."
• Structured memory (Mem0, Zep) excels when you know the shape of what you are storing — user preferences, facts, entity relationships.

Most production systems use both.

Putting It All Together: The Dual-Layer Architecture

The production pattern that has emerged across frameworks is a dual-layer architecture:

┌─────────────────────────────────────────────┐
│              Agent Runtime                  │
│                                             │
│  ┌─────────────────────────────────────┐    │
│  │  Hot Path (Context Window)          │    │
│  │  - System prompt                    │    │
│  │  - Recent messages (trimmed)        │    │
│  │  - Retrieved memories (injected)    │    │
│  └──────────────┬──────────────────────┘    │
│                 │ queries                    │
│  ┌──────────────▼──────────────────────┐    │
│  │  Cold Path (External Stores)        │    │
│  │  - Vector DB (semantic retrieval)   │    │
│  │  - Key-value store (user facts)     │    │
│  │  - Checkpoint DB (session state)    │    │
│  └─────────────────────────────────────┘    │
└─────────────────────────────────────────────┘

The hot path holds everything in the LLM's context window: the system prompt, trimmed recent messages, and any memories retrieved from external stores. The cold path holds everything else — queryable on demand but not consuming tokens until needed.

This is not theoretical. It is how LangGraph, CrewAI, and Letta all work under the hood. The only difference is where they draw the line between hot and cold.

Which Memory Framework Should You Pick?

Use Case	Best Fit	Why
Prototyping / learning	LangGraph + MemorySaver	Zero setup, in-memory, great docs
User personalization	Mem0	Automatic fact extraction, conflict resolution
Production checkpointing	LangGraph + PostgresSaver	Battle-tested, pause/resume, thread isolation
Temporal fact tracking	Zep + Graphiti	Knowledge graph with timestamps
Self-managing memory	Letta	Agent edits its own memory blocks
Semantic search over history	ChromaDB / pgvector	Simple, local, no managed service needed
All-in-one infrastructure	Redis	Short-term, long-term, caching in one store

There is no single right answer. Start with the simplest layer that solves your problem — usually short-term + one persistence layer — and add complexity only when you hit a real limitation.

Where to Go From Here

If you haven't built an agent yet, start with the from-scratch tutorial and add memory on top. Already have an agent? Pick one pattern from this post and wire it in — Mem0 for user preferences, ChromaDB for semantic recall, or PostgresSaver for session persistence.

For multi-agent systems where multiple agents share context, memory becomes even more critical. Each agent needs access to shared state without stepping on each other's context windows.

A few resources worth bookmarking:

• LangMem SDK — LangChain's dedicated library for semantic, episodic, and procedural memory extraction
• Mem0 documentation — full API reference and self-hosting guide
• Letta concepts guide — deep dive into MemGPT's self-editing memory architecture
• DeepLearning.AI course on agentic memory — free short course with LangGraph

Memory is what separates a demo agent from a useful one. The patterns are straightforward — the hard part is choosing where to start. Pick one, ship it, and iterate from there.