Task 4.2: Analyze agent failures and identify root causes

• AThe recoverable flag, which separates model errors from tool errors

  Incorrect

  recoverable says whether recovery is possible, not where the error happened.

• BThe cwd field, since tool errors always happen outside the working directory

  Incorrect

  cwd is the working directory; it doesn't classify errors.

• CThe retryCount the hook returned last time it ran, which counts model errors only

  Incorrect

  retryCount is an output the hook can return for retries, not a classification of the error.

• DThe errorContext field, which is model_call, tool_execution, system or user_input

  Correct

  errorContext says where the error occurred, which is the basis for classifying it.

The onErrorOccurred hook receives a timestamp, the working directory, the error message, an errorContext of model_call, tool_execution, system or user_input, and whether the error is recoverable. Categorizing by errorContext lets you handle and track different kinds of failure separately, and GitHub's best practices also say to log errors even when you suppress them.

• AThe hook's timeoutSec is set below 30

  Incorrect

  A short timeout can make a hook time out, but it doesn't stop it from executing.

• BThe file has no "version": 1 field

  Correct

  Hook configuration must specify version: 1.

• CThe hook uses the cwd property

  Incorrect

  cwd is a supported optional property for the script's working directory.

• DTwo hooks are defined for the same event

  Incorrect

  Several hooks per event are allowed and run in order.

• EThe referenced script isn't executable or has no shebang line

  Correct

  Scripts called from a hook must be executable and have a proper shebang, such as #!/bin/bash.

If hooks are not executing, check that the JSON file is in the .github/hooks/ directory, that its syntax is valid (for example with jq), that version: 1 is specified, that the called script is executable (chmod +x) and that it has a proper shebang such as #!/bin/bash.

• AAutopilot can't run shell commands at all, so the tests can never run in this mode

  Incorrect

  Autopilot can run commands when permissions allow; the denials came from the chosen permission option.

• BThe model hit a reasoning error and stopped calling tools

  Incorrect

  The log shows tool runs refused, which points to permissions, not reasoning.

• CThat choice makes Copilot refuse any tool request that needs a prompt, so the test runs were refused

  Correct

  With manual approval, Copilot automatically denies tool requests that require approval, which can stop it finishing.

• DThe session reached its --max-autopilot-continues limit

  Incorrect

  A continuation cap ends the run; here the evidence is repeated refused tool requests.

On entering Autopilot without full permissions, Copilot CLI offers to enable all permissions, use manual approval for the session, or cancel. With manual approval, Copilot automatically denies any tool request that requires approval, which may prevent it from completing tasks; /allow-all can grant full permissions during the session. Repeated refused tool calls in the log point to this, not to reasoning.

• AAn eyes reaction from Copilot on the issue

  Correct

  Copilot leaves an eyes reaction, then opens a draft pull request shown in the timeline.

• BAn email from GitHub Support

  Incorrect

  The sign is on the issue itself.

• CA new label called copilot-working

  Incorrect

  No such label is added.

• DA check run on the default branch

  Incorrect

  The sign is a reaction on the issue.

After you assign an issue, Copilot leaves an eyes reaction, then opens a draft pull request linked to the issue. When a "Copilot started work" event appears in the pull request timeline, View session opens the live session logs.

• AA copilot-setup-steps.yml file written specifically for the new agent

  Incorrect

  Setup steps aren't needed for an agent to be listed.

• BAn enterprise owner's approval

  Incorrect

  Repository agents need no approval to be listed.

• CA tools property in the profile

  Incorrect

  tools is optional; omitting it gives all tools.

• DMerging the profile into the default branch, then refreshing

  Correct

  Custom agents appear in the dropdown after the profile is merged into the default branch.

To create a custom agent, add an .agent.md profile under .github/agents, commit it, and merge it into the default branch. Back on the agents tab, refresh if needed, and the agent appears in the dropdown.

• AThe #codebase variable

  Incorrect

  The requirement is the @github participant.

• BThe /delegate command

  Incorrect

  That is a Copilot CLI command; in IDE chat you mention @github.

• CThe @github chat participant

  Correct

  In VS Code, Visual Studio and JetBrains IDEs you must mention @github; on GitHub.com it can be omitted.

• DA link to the issue

  Incorrect

  The requirement is the @github participant.

If Copilot says it cannot create a pull request from chat, check that Copilot cloud agent is available. In VS Code, Visual Studio and JetBrains IDEs, you must mention the @github chat participant in your prompt; on GitHub.com you can omit it.

• AIt starts each server once to test it

  Incorrect

  Servers start in agent sessions; saving checks syntax.

• BIt validates the configuration's syntax

  Correct

  The configuration is validated to ensure proper syntax when saved.

• CIt sends the file for an owner to approve

  Incorrect

  There is no approval step on save.

• DIt commits the file to the default branch

  Incorrect

  The configuration lives in repository settings, not in a committed file.

Repository administrators enter MCP configuration as JSON in the repository's Copilot MCP settings and click Save MCP configuration, which validates the syntax.