Why AI Forgets

A Desk That Can Only Hold So Many Papers

The context window is the limited desk space available for the current request. This simplified example removes older pages first, but real applications choose what to keep, summarize, or remove.

Your AI's Working Context

This is an illustrative 8,192-token window. The system prompt uses 35 example tokens here; real counts depend on the model and tokenizer.

Every Message Fills the Window

Watch the meter. Every message -- yours AND the AI's responses -- fills the same window. Long responses eat more space than your short prompts.

When the Desk Is Full

In this simplified application, older content—including the system prompt—was removed. Other applications may preserve instructions, summarize history, or remove different messages.

Not All Windows Are Equal

The same conversation that overflows a small window might barely register in a larger one.

Bigger windows hold more, but they're not infinite. And bigger windows cost more per token. The key insight: your conversation history, system prompt, and the AI's responses ALL share this space.