AI-300 · Domain 5: Optimize generative AI systems and model performance · 13% of the exam
Task 5.1: Optimize retrieval-augmented generation (RAG) performance and accuracy
Similarity thresholds, chunk sizes and retrieval strategies, embedding model choice, hybrid search that combines semantic and keyword retrieval, and relevance metrics and A/B testing.
Try each one before opening the answer. Every option is explained, with the Microsoft documentation page that proves it.
Question 1 · choose 1
A RAG app runs vector queries with semantic ranking. For some questions, the right policy chunk never appears in the top results, and logs show it isn't among the first 50 vector matches either. The team wants to tune the semantic configuration's title and content fields. What should it do?
ATune the semantic configuration fields, since the ranker searches the whole index for better matches
BImprove first-stage recall, for example with a hybrid query; semantic ranking reorders only the top 50
CRaise k to 500 so that the semantic ranker scores 500 candidates per query
DEnable semantic answers, which compose a new answer when ranking finds no matching chunk
Show the answer and why
ATune the semantic configuration fields, since the ranker searches the whole index for better matches
Incorrect
Semantic ranking reranks the existing result set; it doesn't search the corpus again.
BImprove first-stage recall, for example with a hybrid query; semantic ranking reorders only the top 50
Correct
The semantic ranker can't rerun the query over the corpus; it reranks what the first stage returned.
CRaise k to 500 so that the semantic ranker scores 500 candidates per query
Incorrect
Even with more results, only the top 50 progress to semantic ranking.
DEnable semantic answers, which compose a new answer when ranking finds no matching chunk
Incorrect
Captions and answers are verbatim text from the index; nothing is composed.
Semantic ranking starts from the ranked result of the first-stage text, vector or hybrid query, and only the top 50 results progress to semantic ranking. It can't rerun the query over the entire corpus or create new information, so a chunk missing from the first-stage results must be recovered by improving recall, for example with hybrid search.
A team sets up chunking with the Text Split skill for RAG over long manuals. It wants Microsoft's suggested starting point, uses only GA features, and first tried maximumPageLength 2000 with pageOverlapLength 1200, which failed. Which settings fit?
AtextSplitMode pages, maximumPageLength about 2,000 characters and pageOverlapLength about 500
BtextSplitMode pages, maximumPageLength 2,000 characters and pageOverlapLength 1,200 characters
CtextSplitMode pages, maximumPageLength 512 tokens and pageOverlapLength 128 tokens
DtextSplitMode sentences, with each chunk set to one sentence and no overlap at all
Show the answer and why
AtextSplitMode pages, maximumPageLength about 2,000 characters and pageOverlapLength about 500
Correct
That's roughly 512 tokens with 25% overlap, and the overlap stays under half the page length.
BtextSplitMode pages, maximumPageLength 2,000 characters and pageOverlapLength 1,200 characters
Incorrect
pageOverlapLength must be less than half of maximumPageLength.
CtextSplitMode pages, maximumPageLength 512 tokens and pageOverlapLength 128 tokens
Incorrect
Token-based chunking is available only in the preview API.
DtextSplitMode sentences, with each chunk set to one sentence and no overlap at all
Incorrect
Single sentences rarely hold enough context, and the starting point is 512 tokens.
Microsoft suggests starting with chunks of 512 tokens (about 2,000 characters) and an initial overlap of 25% (128 tokens). In the GA Text Split skill, maximumPageLength counts characters, and token chunking is available in the latest preview API. pageOverlapLength, if set, must be less than half of maximumPageLength.
An index has a 1536-dimension vector field from text-embedding-ada-002. The team wants text-embedding-3-large at 3,072 dimensions without downtime, and queries must keep working during the switch. What should it do?
AChange the existing field's dimensions to 3,072 and reload the documents in place
BKeep the 1536 field and embed queries with text-embedding-3-large from now on
CAdd a new 3,072-dimension field with its own profile, embed into it, then point queries at it
DStore the new vectors in the same field, next to the old ones, until the reload finishes there
Show the answer and why
AChange the existing field's dimensions to 3,072 and reload the documents in place
Incorrect
Changing an existing field requires a drop and full index rebuild.
BKeep the 1536 field and embed queries with text-embedding-3-large from now on
Incorrect
Queries and documents must use the same embedding model and dimensions.
CAdd a new 3,072-dimension field with its own profile, embed into it, then point queries at it
Correct
New fields and new vector configurations don't need a rebuild, so the old field serves until the switch.
DStore the new vectors in the same field, next to the old ones, until the reload finishes there
Incorrect
Every vector in a field must have the field's dimensions.
Changing or deleting existing fields usually needs a drop and rebuild, but you can add new fields and new vector configurations assigned to new fields without rebuilding. All vectors in a field must match its dimensions, and the same embedding model should produce document and query vectors, so add a new field for the new model, populate it, then switch queries.
A request has one full-text query and one vector query that targets two vector fields. Which two statements about its ranking are true? (Choose TWO.)
ABM25 scores decide the final ranking because the text query runs first
BThe two vector fields are averaged into one vector before searching
CRRF scores range from 0 to 4, like reranker scores
DThree query executions run in parallel
EReciprocal Rank Fusion merges the result sets into one ranking
Show the answer and why
ABM25 scores decide the final ranking because the text query runs first
Incorrect
RRF fuses the rankings rather than keeping BM25 order.
BThe two vector fields are averaged into one vector before searching
Incorrect
Each field is searched as its own query.
CRRF scores range from 0 to 4, like reranker scores
Incorrect
RRF scores are small and bounded by the number of queries; 0-4 is the semantic reranker.
DThree query executions run in parallel
Correct
Text plus a vector query over two fields equals three executions.
EReciprocal Rank Fusion merges the result sets into one ranking
Correct
RRF is used whenever more than one query runs.
RRF is used whenever two or more queries execute in parallel. A full-text query plus one vector query targeting two vector fields is three query executions; RRF assigns each document a reciprocal rank score per result set and sums them. The RRF score's upper limit depends on the number of queries fused, while @search.rerankerScore from semantic ranking ranges from 0 to 4.
Which similarity metric should a vector profile use for Azure OpenAI embeddings?
Ahamming
Beuclidean
CdotProduct
Dcosine
Show the answer and why
Ahamming
Incorrect
Hamming is for binary data.
Beuclidean
Incorrect
Use the metric associated with the model; for Azure OpenAI it's cosine.
CdotProduct
Incorrect
Cosine is the documented choice here.
Dcosine
Correct
The guidance says cosine for Azure OpenAI embeddings.
vectorSearch.algorithms.metric should be cosine if you use Azure OpenAI embeddings; otherwise use the metric associated with your embedding model. Supported values are cosine, dotProduct, euclidean and hamming (for binary data).
Which textSplitMode does the Text Split skill use by default?
Apages, with chunks made of several sentences
Bsentences, one sentence per chunk
Ctokens, sized by a tokenizer
Dparagraphs, split on blank lines
Show the answer and why
Apages, with chunks made of several sentences
Correct
pages is the default mode.
Bsentences, one sentence per chunk
Incorrect
That's the other mode.
Ctokens, sized by a tokenizer
Incorrect
tokens isn't a textSplitMode value; the modes are pages and sentences.
Dparagraphs, split on blank lines
Incorrect
That isn't a Text Split mode.
The Text Split skill splits text by pages (the default), where chunks are made up of multiple sentences, or by sentences, where each chunk is a single sentence defined by the language's punctuation.
CA free plan with a monthly allowance, then a pay-as-you-go standard plan
DPer index, with a flat monthly fee
Show the answer and why
AAlways free on every tier
Incorrect
It's a premium feature billed by usage.
BOnly through provisioned throughput units
Incorrect
PTUs are for model deployments.
CA free plan with a monthly allowance, then a pay-as-you-go standard plan
Correct
Usage beyond the free allowance moves to the standard plan.
DPer index, with a flat monthly fee
Incorrect
Billing is by usage.
Semantic ranker is a premium feature billed by usage. It offers a free plan (the default) with a monthly free request allowance and a standard plan for pay-as-you-go pricing after the free allowance is used.
Must Azure AI Search and its Azure OpenAI embedding model be in the same region for integrated vectorization?
AYes, cross-region calls are blocked
BNo, though co-location can improve latency
CYes, unless the search service is on the free tier
DNo, and region never affects latency
Show the answer and why
AYes, cross-region calls are blocked
Incorrect
Cross-region works.
BNo, though co-location can improve latency
Correct
Same region is recommended, not required.
CYes, unless the search service is on the free tier
Incorrect
Tier doesn't change it.
DNo, and region never affects latency
Incorrect
Region can affect latency.
Integrated vectorization with Azure OpenAI embedding models doesn't require the resources to be in the same region, but using the same region can improve performance and reduce latency; check model and search regional availability.