Skip to content
AI DemystifiedHow AI actually works
Lesson 02 of 22Foundationsbeginner4 min

Context Window

The Sticky Note: AI's limited working memory

What is the context window?

Imagine AI has a sticky notefor working memory. Everything it can “see” right now is on that note: messages, system instructions, and tool results. When the note is full, the application must choose a packing policy: truncate, summarize, retrieve selected history, or reject the oversized request.

Toy scale

This demo uses a 128-token whiteboard so you can fill it quickly. Replies and tool results are canned teaching text, not a live model. Published capacities, output reservations, token counting, and overflow behavior vary by model and API. Units here are plain tokens, not “K” (thousands).

Packing policy

Context limits do not prescribe which messages disappear. That is an application decision with different correctness tradeoffs.

request accepted

Everything fits; no packing intervention is required. Raw history: 12 tokens · packed request: 12 tokens.

Whiteboard · 128 token capacity

System Prompt
12 tokens
You are a helpful AI assistant. Be concise and friendly.
Context usage
12 packed128 max
Token breakdown
System Prompt12
Raw / packed12 / 12

Key Insight

A base model does not automatically carry your prior conversation into a new request. Products can store and retrieve history, and model weights retain learned statistical structure, but neither is the same as the current prompt. For a particular response, the model conditions on the request assembled now—plus any tools the application lets it call. Next: how it picks the next token.

See also: Unhobbling

Window size is one limit. What you put in is context engineering (write / select / compress / isolate). A full window of noise hobbles as much as a tiny one. Drill it in Unhobbling AI.

Check your understanding

1/5

What happens when the context window fills up?

Sources, scope & review statusVerified Jul 29, 2026

Claims this lesson makes

  • Stable conceptA context limit bounds the tokens available to a request and its generated output; the surrounding application decides what to include, compress, retrieve, or reject.

    Scope: The whiteboard is a packing-policy simulator, not a claim that every provider drops the oldest message.

  • Stable conceptInformation can be harder for a model to use even when it fits inside the advertised context window, and performance can depend on where relevant evidence appears.

    Scope: Lost in the Middle reports this behavior on evaluated long-context tasks and models; it is not a universal accuracy curve for every model or prompt.

Next content review: 2027-07-29. Source links establish the lesson's claims; interactive numbers remain labeled simulations unless explicitly identified as measured data.