Chunking and Reranking

Chunking and Reranking

Step 1 of 10

stageoffline

Everything the model can possibly say is decided here, at the cut — long before a prompt exists.

The Idea

Cutting documents into chunks happens once and decides everything afterwards. A cut that separates a definition from its subject leaves a passage that can never answer the question. So the cuts overlap, the first stage fetches generously, and a slower scorer reorders that shortlist.

Real-World Example

A library pulls thirty books off the shelves by catalogue keyword in a minute. A librarian then skims each index and hands you the three that actually address your question.

The Tradeoff

Overlap multiplies storage and lets near-duplicates crowd the shortlist, pushing out the one distinct passage you needed. Reranking adds a second model call to every query and only earns its latency once first-stage recall is already good. Measure recall at twenty before you buy a reranker.

Your turn

Put the steps in the right order.

1. 1Score each candidate against the question with the slower model↑↓
2. 2Cut the documents into overlapping chunks and embed each one↑↓
3. 3Send the top few reordered passages to the generator↑↓
4. 4Fetch a generous shortlist by vector similarity↑↓

Check order

Mini quiz

1 / 3

Overlapping chunks exist mainly to:

• AHalve the storage cost
• BSpeed up the embedding step
• CRemove the need for a reranker
• DStop a definition being cut away from its subject