Systems on Glass is an AI-automation side project of mine. Every episode is generated by Claude (Anthropic's AI, working through Claude Code in my own video studio): the script, the animations and 3D scenes, the edit, and the narration in an AI clone of my voice. I choose the topics, bring the stories from my own work, and review each episode before it goes up. Not affiliated with Anthropic.
Researchers gave a model twenty documents and a question; one document held the answer. With the answer first, it was right 75.8% of the time. With the answer in the middle, 53.8%: lower than the 56.1% it scored with no documents at all.
The short version
- Context is RAM. On every call the model sees one block of text: instructions, tools, files, the conversation, every tool result. Between calls it remembers nothing, so an agent sends it all again each turn. Files, notes and databases are the disk.
- Bigger RAM doesn't fix it. Models read the window unevenly (the start and end win; the middle gets lost), get less reliable as input grows (Chroma's "context rot" across 18 models), and you pay for every token on every step.
- The labs agree, in their own guides. Windows went to 1M (GPT-4.1), 2M (Grok 4 Fast) and 10M in Gemini 1.5's research tests, but finding one needle isn't reasoning over many. Google says question last; Anthropic says documents top, question bottom (up to 30% better in its tests); OpenAI says instructions at both ends.
- Four memory moves. Load late (keep pointers, not pages), compact (summarize and start fresh), write it down (notes and CLAUDE.md on disk), and fork (a sub-agent with a fresh window sends back one paragraph). Trim tool results: they're the biggest RAM hog.
- The rule. Load only what this step needs, put the key thing first or last, and write everything else to disk.
100K tokens × 50 steps = 5M tokens read (illustrative)
context editing: −84% tokens (100-turn web search) · LLMLingua: up to 20× shorter prompts
The compaction story in the video is true: the Claude Code session that makes this series ran out of context while building the caching episode, was summarized, and carried on from the summary. My studio also loads late on purpose: its skill is a short index, and each workflow file is read only when that kind of video is being made.
Further reading
- Lost in the Middle — Liu et al. (2023): the U-shaped curve; the numbers above are from its Table 6.
- Context Rot — Chroma (2025): 18 models, all less reliable as input grows.
- Effective context engineering for AI agents — Anthropic on the attention budget, compaction, notes and sub-agents.
- Managing context on the Claude Developer Platform — context editing and the memory tool, with the 84% and 39% figures.
- Long context prompting tips — Anthropic: documents at the top, the question at the end.
- GPT-4.1 prompting guide — OpenAI on 1M-token context and instructions at both ends.
- Gemini long context — Google on one needle versus many, and putting the query last.
- LLMLingua — Microsoft Research's prompt compression: up to 20× with little loss.
- MemGPT — Packer et al. (2023): an agent that pages memory like an operating system.
Next episode: Indexes, how you search billions of things without reading them. Previously: Caching. All episodes →