Task 2.3: Optimize and operationalize generative AI systems

• ALower the temperature toward 0

  Correct

  Lower temperature makes sampling more focused and deterministic, so repeated runs agree more often.

• BRaise the presence penalty

  Incorrect

  Presence penalty discourages repeating tokens; it doesn't stabilize a classification.

• CRaise the maximum output tokens

  Incorrect

  A one-word label isn't limited by output length.

• DRaise top_p to 1 and temperature to 1.5

  Incorrect

  Both changes widen sampling and make labels vary more.

Temperature and top_p control randomness. For classification and extraction, use a low temperature (and change temperature or top_p, not both) so the model returns its most likely answer.

• AAsk for the reasoning steps to be written in the visible answer

  Incorrect

  Visible chain-of-thought adds output tokens on top of the hidden reasoning.

• BSend them with a lower reasoning_effort value

  Correct

  reasoning_effort controls how much the model thinks before answering; lower values spend fewer reasoning tokens.

• CAdd a few-shot block of hard examples to every request

  Incorrect

  Longer prompts add input tokens and latency to simple checks.

• DRaise max_output_tokens so reasoning finishes sooner

  Incorrect

  A higher cap allows more tokens; it doesn't make the model think less.

reasoning_effort sets how much a reasoning model thinks before it answers, so tune it per request type: low for simple checks, higher for hard planning.

• ARemove the review and raise the drafter's temperature for variety

  Incorrect

  That drops the quality check and adds randomness.

• BKeep looping until two consecutive drafts are identical

  Incorrect

  Drafts rarely repeat word for word, so this can loop for a long time.

• CStop when the review passes all criteria or after a few fixed rounds

  Correct

  A bounded loop with a clear pass condition keeps the benefit of critique while capping cost and latency.

• DLet the drafter judge its own draft in the same call, with no rubric

  Incorrect

  Without explicit acceptance criteria the checker can't make consistent pass or fail decisions.

Reflection loops pair a draft with a critique against explicit criteria, then revise. Give the loop a pass condition and an iteration cap; a rubric-style judge makes the pass condition measurable.

• AAdd print statements to the agent's instructions

  Incorrect

  Prompt agents have no code to instrument, and instructions don't produce telemetry.

• BExport conversations nightly and replay them in a playground

  Incorrect

  Replays after the fact don't capture the live spans, timings or token counts.

• CConnect an Application Insights resource to the Foundry project

  Correct

  Server-side tracing turns on automatically once Application Insights is connected, with no code changes, for agents hosted in Foundry.

• DTurn on diagnostic settings for the Foundry resource's audit logs

  Incorrect

  Audit logs record operations on the resource; they aren't per-run traces of model calls, tool calls and token counts.

Start with server-side tracing: connect Application Insights to the project and Foundry records traces using OpenTelemetry conventions.

• AAsk for shorter answers and lower the maximum output tokens

  Correct

  Generation size is a major part of latency; fewer generated tokens return faster.

• BSwitch to a larger model so the cached prefix goes further

  Incorrect

  Latency varies by model, and larger models are generally slower, not faster.

• CInsert the user's name and the time before the instruction block

  Incorrect

  A changing value at the start breaks the identical prefix that caching needs.

• DAdd more few-shot examples to the instruction block for accuracy

  Incorrect

  More prompt tokens add processing time rather than removing it.

• EKeep the instruction block identical and first in every request

  Correct

  Prompt caching reuses an identical prompt prefix, cutting latency and cost for long prompts.

Two levers dominate here: keep long static content at the start so it can be cached, and generate only what users need.

• AA model extracts facts into a schema; a rules engine applies thresholds

  Correct

  The model handles unstructured input; deterministic code applies the auditable policy and gives repeatable results.

• BA model reads each email and decides eligibility with temperature set to 0

  Incorrect

  Low temperature reduces variation but doesn't turn model judgment into the exact, auditable rules the lender needs.

• CModel router picks a model per application to decide eligibility

  Incorrect

  Routing picks a model for cost or quality but still leaves the decision to a model.

• DA model fine-tuned on past decisions decides eligibility directly

  Incorrect

  Fine-tuning learns patterns from history; it doesn't guarantee each threshold is applied as written.

Orchestrate each part with the right engine: language models for reading messy input, rules for decisions that must be exact and repeatable. Structured output is the contract between the two.

• AThe Developer deployment type for the agent's model

  Incorrect

  Deployment types don't change instructions.

• BA guardrail with stricter blocking levels

  Incorrect

  Guardrails filter harmful content; they don't improve instructions.

• CPrompt Optimizer on the agent's Instructions field

  Correct

  Prompt Optimizer uses an LLM to restructure and enhance instructions, guided by optional suggestions, then applies the result.

• DAn evaluation run that scores the current instructions

  Incorrect

  An evaluation measures how the agent performs; it doesn't rewrite the instructions.

Prompt Optimizer rewrites instructions with best practices and explains each change; run an evaluation with your own data afterwards to confirm the new instructions actually help.

• ARaise the maximum output tokens

  Incorrect

  Output limits don't shrink the prompt.

• BPresent the records as a compact table

  Correct

  Models understand tabular data well, and tables can be a space-efficient way to include data.

• CWrite each record as a longer paragraph

  Incorrect

  Longer prose uses more tokens.

• DRepeat each record twice for emphasis

  Incorrect

  Repetition doubles the token use.

Space efficiency matters even with large context windows: prefer tables, trim boilerplate and remember that token boundaries differ across formats.

• APut the instructions inside the pasted email text itself

  Incorrect

  Mixing them makes the confusion worse.

• BRemove all formatting from the prompt to keep it short

  Incorrect

  Less structure makes the boundary harder to see.

• CRaise the temperature of the summarizer model for each email

  Incorrect

  Randomness doesn't separate content from instructions.

• DSeparate instructions and pasted text with --- and headings

  Correct

  Clear syntax, such as separators and uppercase headings, communicates intent and makes outputs easier to parse.

Structure prompts with clear sections and separators; delimiters can also serve as stopping conditions for generation.

• AIt never affects latency, because it runs after the response

  Incorrect

  Prompts and completions both run through classification models.

• BAny developer with Foundry User can remove the filtering

  Incorrect

  Turning off output filtering is limited to customers approved for modified guardrails.

• CFiltering adds latency; low-risk cases may request changes

  Correct

  Content filtering increases safety but also latency; for lower-risk uses, requesting modified filtering policies can be explored.

• DIt applies only to image models, never to text models

  Incorrect

  The guardrail system applies to text and image content.

Latency tuning weighs model choice, generation size, streaming and filtering. Treat filter changes as a safety decision with approval, not a quick performance tweak.

• ALower the maximum output tokens for each of its answers

  Incorrect

  Shorter answers can still be fabricated.

• BAsk the model to be more confident in all of its answers

  Incorrect

  Confidence doesn't add facts.

• CRemove the paragraph and ask the model to recall it from memory

  Incorrect

  That relies even more on the model's own parameters.

• DLet it issue searches, then paste the results back in

  Correct

  Using an affordance such as search mitigates fabricated answers and adds up-to-date information.

Affordances (search, tools) give the model external information instead of relying on its parameters; the agent equivalent is tool calling.