Context Windows
AI & ML: lesson 10 of 32
A hard limit on what the model can see at once.
Lesson 10 of 32 · 4 min
Context Windows
Step 1 of 12
window700 tokens
The context window is the most tokens a model can attend to in one call.
The Idea
The context window is the most tokens a model can attend to in one call. Instructions, history, retrieved passages and the reply all share it, and anything outside it simply does not exist for the model.
Real-World Example
An air-traffic controller's rack of paper flight strips. A new strip goes on only when one comes off, and a flight with no strip on that rack is, for that controller, not in the picture.
The Code
history = [("system", 20), ("user", 300), ("assistant", 250),
("user", 180), ("assistant", 400), ("user", 120)]
budget = 700
kept, used = [], 0
for role, size in reversed(history): # newest turns first
if used + size > budget:
break # everything older is dropped
kept.append((role, size)); used += size
print(list(reversed(kept)), used) # note: the system turn fell offYour turn
What does this print?
history = [("system", 50), ("user", 200), ("assistant", 300), ("user", 100)]
budget = 500
used = 0
for role, size in reversed(history):
if used + size > budget:
break
used += size
print(used)Mini quiz
1 / 3