System rules, user messages, history, retrieved evidence, tool output, and future generation all compete for tokens. Allocate them deliberately, then see which information survives truncation and which evidence the model may still underuse.
The segmented 3D tape is the final context. Height represents retained tokens. Faded upper blocks are requested content dropped or compressed by the active policy.
The input and maximum generated output share one model context limit. Reserve generation space first; otherwise a full prompt can leave no room for the answer or trigger server-side truncation you did not design.
Long contexts can contain relevant evidence yet models may use it less reliably depending on position, distractors, instructions, and retrieval quality.
System rules and current user intent should not be silently displaced by verbose history, retrieved documents, or tool output.
Summaries save tokens but can erase quantities, uncertainty, chronology, and provenance. Keep original source references and structured facts.
Token counts vary by tokenizer and content. Add safety margin for serialization, tool schemas, citations, and unexpected outputs.
Keeps system and current retrieved evidence first, compresses durable memory, then drops older conversation and verbose tool output. Useful when policy and task evidence outrank conversational continuity.
Authority firstRetains system rules and the newest conversation, then fills remaining space with evidence, tools, and memory. This can preserve dialogue coherence while losing older but decisive facts.
Continuity firstScales flexible segments together after preserving system and output reserve. No category disappears suddenly, but every category may lose detail.
Balanced degradationDo not concatenate every available string. Each segment should earn tokens according to authority, relevance, recency, and the cost of omission.
Place policy, identity, tool constraints, and the current user request in explicit structured sections. Keep untrusted retrieved content distinguishable from instructions.
Retrieve a small candidate set, rerank for the current task, deduplicate overlapping passages, and include only evidence that can change the answer.
Store decisions, preferences, and unresolved commitments as structured memory. Keep recent turns verbatim when wording matters; compress older exchanges with links back to originals.
Log which segments were requested, retained, compressed, and dropped. Evaluate whether missing context caused errors instead of treating truncation as invisible infrastructure.
No. Retrieval controls relevance, permissions, latency, and attention competition. Larger capacity can reduce truncation while still benefiting from selective evidence.
Some models use relevant information less effectively when it appears between the beginning and end of long inputs. The effect varies by model, task, training, and context composition.
Follow the model's documented message format. More generally, preserve instruction authority and prevent untrusted content from being interpreted as policy.
Parse structured fields, remove irrelevant logs, cap repeated content, preserve errors and provenance, and treat tool output as potentially untrusted data.
Explore SuperPowers workflows that preserve useful computer state, turn explicit goals into visible work, and produce complete websites.
Explore SuperPowersComputer-use cacheWebsite-building agent