Generative AI Engineer Associate · Domain 3: Application Development · 30% of the exam
Task 3.5: Select chunking strategy based on model & retrieval evaluation; Select an embedding model context length based on source documents, expected queries, and optimization strategy
Choosing chunk sizes from evaluation results, matching the embedding model's context length and dimension to chunk size, cost and latency.
Study it
Chunk size from evaluation; embedding context length and dimension
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
An evaluation compares chunking options at a fixed budget of about 5,200 tokens of retrieved context. 256-token chunks at k=20: recall 0.85, RetrievalSufficiency 76 percent. 512-token chunks at k=10: recall 0.83, sufficiency 81 percent. 1,024-token chunks at k=5: recall 0.74, sufficiency 70 percent. The goal is answers that have all the facts they need. Which option should the team choose?
A256-token chunks at k=20
B1,024-token chunks at k=5
C512-token chunks at k=10
D1,024-token chunks at k=10
Show the answer and why
A256-token chunks at k=20
Incorrect
Its recall is slightly higher, but many small chunks scatter related facts and its sufficiency is lower.
B1,024-token chunks at k=5
Incorrect
Larger chunks keep context, but with only five fitting the budget both recall and sufficiency drop.
C512-token chunks at k=10
Correct
It gives the highest RetrievalSufficiency inside the token budget, which is the measure of whether the context holds all needed facts.
D1,024-token chunks at k=10
Incorrect
Ten 1,024-token chunks need about 10,000 tokens, roughly twice the budget.
Chunk size and the number of retrieved chunks trade off against the context budget. Use evaluation metrics such as recall@k and the RetrievalSufficiency judge to choose the combination that best serves the goal.
A search box on a standard AI Search endpoint needs more queries per second. Tests show a 384-dimension embedding model retrieves as well as the current 768-dimension model on the team's evaluation set. What should the team expect from switching?
ANo change in QPS, since endpoint capacity depends only on how many vectors the index holds
BLower QPS, since smaller vectors need more candidates per query
CHigher QPS only after moving to a storage-optimized endpoint
DHigher QPS and lower query latency, since lower-dimension vectors need less compute
Show the answer and why
ANo change in QPS, since endpoint capacity depends only on how many vectors the index holds
Incorrect
Vector dimensionality is one of the factors that affect AI Search performance, alongside index size.
BLower QPS, since smaller vectors need more candidates per query
Incorrect
Lower-dimensional vectors reduce retrieval compute; the guidance is the opposite.
CHigher QPS only after moving to a storage-optimized endpoint
Incorrect
Storage-optimized endpoints trade latency for capacity; the dimension gain applies on standard endpoints too.
DHigher QPS and lower query latency, since lower-dimension vectors need less compute
Correct
Halving dimensionality from 768 to 384 typically improves QPS by about 1.5x and reduces latency by about 20 percent.
Choose the smallest embedding dimensionality that preserves retrieval quality. Lower dimensions mean faster queries and higher QPS, which matters most for latency-sensitive, high-throughput search.
Chunks are embedded with databricks-bge-large-en, and users ask short questions. The team wants to try the retrieval instruction that the model's authors suggest. How should it be applied?
APrefix it to each query only, before embedding
BPrefix it to every document chunk before indexing
CPrefix it to both the chunks and the queries
DAdd it to the system prompt of the generation model
Show the answer and why
APrefix it to each query only, before embedding
Correct
The BGE authors suggest the instruction "Represent this sentence for searching relevant passages:" for query embeddings; its impact depends on the domain.
BPrefix it to every document chunk before indexing
Incorrect
The suggested instruction is for query embeddings, not for the passages being searched.
CPrefix it to both the chunks and the queries
Incorrect
The suggestion covers the query side only; the indexed passages stay as they are.
DAdd it to the system prompt of the generation model
Incorrect
The instruction shapes how a query is embedded; the chat model does not compute embeddings.
An instruction on BGE query embeddings can improve retrieval in RAG applications, but the effect varies by domain, so measure it on the evaluation set before adopting it.
A team plans a storage-optimized AI Search endpoint. Which embedding dimension rule applies?
AIt must be a power of two, such as 512 or 1,024
BIt must be exactly 768
CIt must be divisible by 16
DIt must be below 256
Show the answer and why
AIt must be a power of two, such as 512 or 1,024
Incorrect
The documented rule is divisibility by 16.
BIt must be exactly 768
Incorrect
768 is a reference size in capacity figures, not a requirement.
CIt must be divisible by 16
Correct
For storage-optimized endpoints the embedding dimension must be evenly divisible by 16.
DIt must be below 256
Incorrect
No such upper bound applies to storage-optimized endpoints.
Storage-optimized endpoints have their own limitations, including this dimension rule; for precomputed embeddings you supply the dimension when you create the index.
Before fine-tuning an embedding model, a team compares retrieval with its current Databricks-hosted embedding model and a larger external embedding model. Both score about the same. What does Databricks suggest?
AFine-tune the current model at once on domain data
BSwitch to the larger external model for its extra capacity
CRaise the embedding dimensions to the maximum allowed
DLook beyond the embedding model; optimize other steps
Show the answer and why
AFine-tune the current model at once on domain data
Incorrect
Fine-tuning is a last resort when a stronger model clearly outperforms.
BSwitch to the larger external model for its extra capacity
Incorrect
With equal scores it adds cost without improving retrieval.
CRaise the embedding dimensions to the maximum allowed
Incorrect
More dimensions add cost and latency without a quality signal.
DLook beyond the embedding model; optimize other steps
Correct
If the larger model performs about the same, the issue is not the embedding model, so focus on other optimizations.
Step 8 of the retrieval quality guide: first diagnose whether you have an embedding problem; fine-tune only after other steps if a stronger model clearly wins.
A team has spent weeks testing chunk sizes of 300, 350 and 400 tokens, and each run moves retrieval recall by less than a point. The parsed text still has broken tables, and chunks carry no document or section context. What does the AI Search retrieval quality guide suggest?
AKeep testing in 10-token steps until one size wins
BImprove parsing and add metadata over size tuning
CRaise the overlap between chunks to 50 percent
DLower num_results so fewer chunks reach the model
Show the answer and why
AKeep testing in 10-token steps until one size wins
Incorrect
The guide warns against over-optimizing chunk size when other data preparation steps have more impact.
BImprove parsing and add metadata over size tuning
Correct
The guide names information extraction for metadata, high-quality parsing and semantic metadata as more impactful than fine-grained chunk size work.
CRaise the overlap between chunks to 50 percent
Incorrect
Heavy overlap repeats tokens in many chunks and does not fix broken tables or missing document context.
DLower num_results so fewer chunks reach the model
Incorrect
Fewer results tend to lower recall and leave the parsing and context problems in place.
The guide treats chunk size as a trade-off between localization and context to test, then advises spending effort on metadata extraction, ai_parse_document parsing and semantic metadata rather than over-optimizing chunk size.