Skip to content

GH-600 · Domain 4: Perform evaluation, error analysis, and tuning · 18% of the exam

Task 4.3: Tune agent behavior based on evaluation results

Revising instructions, workflows and constraints, and refining how an agent uses memory, tools and tool access after an evaluation.

Study it

  • Tuning instructions, workflows, constraints, memory and tool access

    Partly covered by: Evaluating LLMs

Sample questions

Try each one before opening the answer. Every option is explained, with the GitHub documentation page that proves it.

Question 1 · choose 1

Copilot cloud agent keeps pushing commits that fail the repository's CI checks, which the team then has to send back. The repository has no custom instructions. What tuning does GitHub recommend first?

  1. AAdd the build, test and lint commands to .github/copilot-instructions.md
  2. BTurn off Require approval for workflow runs so CI runs on every push
  3. CSwitch the cloud agent to a larger runner so the tests run faster
  4. DAsk reviewers to leave single comments so Copilot fixes one failure at a time, more carefully
Show the answer and why
  • AAdd the build, test and lint commands to .github/copilot-instructions.md

    Correct

    Copilot validates before pushing most reliably when instructions tell it how to build, test and validate.

  • BTurn off Require approval for workflow runs so CI runs on every push

    Incorrect

    That lets workflows run without review; it doesn't make Copilot validate before pushing.

  • CSwitch the cloud agent to a larger runner so the tests run faster

    Incorrect

    Runner size affects speed, not whether the agent knows how to validate its work.

  • DAsk reviewers to leave single comments so Copilot fixes one failure at a time, more carefully

    Incorrect

    Batching comments is recommended, and comments don't teach it to run the checks before pushing.

Copilot has its own ephemeral environment where it can run tests and linters before it pushes, and it is most likely to do so when given clear instructions. GitHub's troubleshooting guidance for changes that don't pass CI is a .github/copilot-instructions.md file describing how to build, test and validate the project, such as make build, make test and make ci.

Question 2 · choose 1

After an evaluation, a team sees Copilot cloud agent call a ticketing MCP server's close_ticket and reassign_ticket tools during tasks where it only needed to read ticket details. The repository configuration has "tools": ["*"]. What change does GitHub recommend?

  1. AKeep ["*"] and add an instruction asking the agent not to close tickets
  2. BSwitch the server's type from local to http so write tools are disabled
  3. CList only the read tools the tasks need in the server's tools array
  4. DEnable approval prompts for the MCP tools in the cloud agent's repository settings
Show the answer and why
  • AKeep ["*"] and add an instruction asking the agent not to close tickets

    Incorrect

    An instruction doesn't remove tools the agent can call without asking.

  • BSwitch the server's type from local to http so write tools are disabled

    Incorrect

    The type describes how the server runs; it doesn't filter tools.

  • CList only the read tools the tasks need in the server's tools array

    Correct

    GitHub strongly recommends allowlisting specific read-only tools, because the agent uses them autonomously.

  • DEnable approval prompts for the MCP tools in the cloud agent's repository settings

    Incorrect

    Cloud agent uses configured MCP tools without asking for approval; there is no prompt to turn on.

Once an MCP server is configured, Copilot cloud agent uses its tools autonomously without asking for approval. The tools key lists which of the server's tools are enabled; GitHub strongly recommends allowlisting specific read-only tools rather than using *, and enabling only the tools a task needs.

Question 3 · choose 1

An SDK app defines three custom agents with descriptions "Helps with code", "Works on tests" and "Does reviews." Evaluation shows the runtime often delegates Python coverage questions to the wrong agent. What should the team tune first?

  1. ARewrite the descriptions to state each agent's specific expertise
  2. BGive every agent tools: null so any of them can handle any task
  3. CSet infer: false on all three agents so the runtime chooses better
  4. DRename the agents' name fields to longer, more descriptive identifiers and keep the descriptions as they are
Show the answer and why
  • ARewrite the descriptions to state each agent's specific expertise

    Correct

    The runtime uses descriptions to match intent; vague ones lead to poor delegation.

  • BGive every agent tools: null so any of them can handle any task

    Incorrect

    Wider tools don't fix selection, and they weaken least privilege.

  • CSet infer: false on all three agents so the runtime chooses better

    Incorrect

    infer: false stops automatic selection entirely; agents are then used only when asked for.

  • DRename the agents' name fields to longer, more descriptive identifiers and keep the descriptions as they are

    Incorrect

    The runtime matches intent against names and descriptions, and GitHub's guidance is to make the description specific; here it stays vague.

When a request matches an agent's expertise, the SDK runtime delegates to that custom agent as a sub-agent, and it uses the description to match intent. GitHub's best practice is to keep descriptions specific, such as "Analyzes Python test coverage and identifies untested code paths," rather than vague ones like "Helps with code."

Question 4 · choose 1

What is the maximum length of the Markdown prompt below a custom agent profile's frontmatter?

  1. A4,000 characters
  2. B10,000 characters
  3. CThere is no limit
  4. D30,000 characters
Show the answer and why
  • A4,000 characters

    Incorrect

    The limit is 30,000 characters.

  • B10,000 characters

    Incorrect

    The limit is 30,000 characters.

  • CThere is no limit

    Incorrect

    The prompt is capped at 30,000 characters.

  • D30,000 characters

    Correct

    The prompt can be a maximum of 30,000 characters.

You define a custom agent's behavior, expertise and instructions in the Markdown body below the YAML frontmatter of its profile. The prompt can be a maximum of 30,000 characters, so long reference material may fit better in a skill.

Question 5 · choose 1

Which frontmatter property is required in a custom agent profile?

  1. Aname
  2. Btools
  3. Cdescription
  4. Dmodel
Show the answer and why
  • Aname

    Incorrect

    name is optional and defaults to the file name.

  • Btools

    Incorrect

    tools is optional; omitting it enables all tools.

  • Cdescription

    Correct

    description is required; it states the agent's purpose and capabilities.

  • Dmodel

    Incorrect

    model is optional; the default model is inherited.

In a custom agent profile's frontmatter only description is required; name defaults to the file name without .md or .agent.md, omitting tools enables all tools, and an unset model inherits the default model.

Question 6 · choose 1

A team writes .github/copilot-instructions.md to tune Copilot cloud agent. Which other features will also follow it?

  1. ACopilot Chat and Copilot code review
  2. BNone; the file is read only by cloud agent
  3. COnly inline suggestions in the IDE
  4. DOnly GitHub Agentic Workflows
Show the answer and why
  • ACopilot Chat and Copilot code review

    Correct

    Repository-wide instructions also apply to Copilot Chat and Copilot code review.

  • BNone; the file is read only by cloud agent

    Incorrect

    The instructions also apply to Chat and code review.

  • COnly inline suggestions in the IDE

    Incorrect

    The documented other uses are Chat and code review.

  • DOnly GitHub Agentic Workflows

    Incorrect

    The documented other uses are Chat and code review.

To add instructions for all tasks assigned to Copilot in a repository, create .github/copilot-instructions.md with information about the project and your coding standards. These instructions also apply to Copilot Chat and Copilot code review, so write them with every reader in mind.

Question 7 · choose 1

A team's MCP tools are rarely found by Copilot CLI's tool search. What does GitHub suggest?

  1. APrefix every tool name with tool_ so that tool search can recognize it as a tool
  2. BName tools for what they do and describe them in the words people search for
  3. CKeep descriptions empty to save tokens
  4. DSplit each tool into many smaller tools
Show the answer and why
  • APrefix every tool name with tool_ so that tool search can recognize it as a tool

    Incorrect

    Prefixes don't help matching; clear names and descriptions do.

  • BName tools for what they do and describe them in the words people search for

    Correct

    Tool search matches on name, description and parameter names and descriptions.

  • CKeep descriptions empty to save tokens

    Incorrect

    Descriptions are what tool search matches against.

  • DSplit each tool into many smaller tools

    Incorrect

    More tools make selection harder; clearer wording helps.

Tool search matches what the agent is trying to do against each tool's name, description, and parameter names and descriptions. Clear, specific wording, such as naming tools for what they do and describing them in the words people would search for, makes a tool show up for the right requests.

Practise domain 4 →Practise all domains →