Task 3.2: Design and implement a testing strategy for pipelines

• AA script step at the end of ring 1 that queries the alerts once and fails the job if any are active

  Incorrect

  One query doesn't re-check over two hours, and authors can edit or remove the step.

• BAn action group on the alert rules that emails the release managers, who cancel the run if needed

  Incorrect

  Waiting for someone to cancel the run doesn't hold ring 2 automatically.

• CA Query Azure Monitor alerts check on the ring 2 environment, re-evaluated for up to two hours

  Correct

  The check passes only when no alert rules are active, is re-evaluated on an interval until its timeout, and is owned by the resource administrator.

• DAn Exclusive lock check on the ring 2 environment

  Incorrect

  That check doesn't look at alerts or application health at all.

The Query Azure Monitor alerts check observes Azure Monitor after a deployment and succeeds only if no alert rules are activated at the time of evaluation. It's re-evaluated after each interval and fails if the stage doesn't start within the timeout. Checks are defined on resources, outside the YAML file.

• AKeep only the browser tests and run them on more parallel agents for each pull request

  Incorrect

  More agents shorten the wait but keep slow, broad tests that are hard to diagnose as the main feedback.

• BMove most checks into isolated unit tests on every pull request, and keep fewer integration and UI tests for later stages

  Correct

  Unit tests run components in isolation and give fast, precise feedback; integration and UI tests still cover interactions before release.

• CReplace the browser tests with load tests, which exercise every component of the application at once

  Incorrect

  Load tests check behavior under increasing load, not functional correctness of each change.

• DDrop automated tests from pull requests and rely on manual testing before each release

  Incorrect

  Manual testing before release gives the slowest feedback of all.

A testing strategy combines several layers of tests. Fast, automated tests that pinpoint failures belong early in the pipeline, while broader tests of interactions and the user interface run before release.

• AA PublishPipelineArtifact step that uploads the XML files as an artifact named TestResults

  Incorrect

  An artifact stores the files, but Azure Pipelines doesn't read them into test results.

• BA PublishCodeCoverageResults step that points at the JUnit files

  Incorrect

  That task publishes coverage reports in formats such as Cobertura, not test results.

• CA PublishTestResults step with testResultsFormat JUnit and the path to the XML files

  Correct

  The task reads JUnit, NUnit, xUnit, CTest or TRX files from any runner and publishes them for reporting and analytics.

• DA switch to the VSTest task, which publishes results automatically

  Incorrect

  VSTest runs .NET test assemblies; it isn't how a Node.js runner's results are published.

The Publish Test Results task publishes results produced by any runner in a supported format (JUnit, NUnit, VSTest TRX, xUnit or CTest), which gives the run a test report and feeds test analytics.

• AAdd a build step that fails the pipeline when total coverage is under 80 percent

  Incorrect

  Total coverage counts legacy code too, so every pull request would fail until the whole codebase reaches 80 percent.

• BAdd azurepipelines-coverage.yml at the repository root with a diff target of 80%

  Correct

  Diff coverage measures only changed lines, and the file sets its target, 70 percent by default.

• CAdd a pull request template asking reviewers to check coverage of new code by hand

  Incorrect

  A manual reminder doesn't measure or block anything.

• DAdd a branch policy on main that requires the code coverage status

  Correct

  A required status policy blocks completion when the posted coverage status fails.

• ESet a failure threshold of 80 percent on the PublishCodeCoverageResults step

  Incorrect

  Publishing reports coverage; the task doesn't enforce a diff target on pull requests.

Diff coverage is coverage of only the lines added or changed in a pull request. A file named azurepipelines-coverage.yml at the repository root sets the diff target (70 percent by default), and the coverage status posted to the pull request can be required by a branch policy. The pull request coverage feature is available for Azure Repos.

• Aavg(response_time_ms) < 500

  Incorrect

  This would fail the test when responses are fast, the opposite of the requirement.

• Bmax(response_time_ms) > 500

  Incorrect

  The requirement is about the average; a single slow request would fail this.

• Cavg(response_time_ms) > 500

  Correct

  A criterion is the condition that fails the test, so the average above 500 ms fails it.

• Dpercentage(error) < 2

  Incorrect

  This would fail the test when errors are low, the opposite of the requirement.

• Epercentage(error) > 2

  Correct

  The error metric is the percentage of failed requests, so more than 2 percent fails the test.

Fail criteria are conditional statements in the form Aggregate_function(client_metric) condition threshold, such as avg(response_time_ms) > 300 or percentage(error) > 50. When any criterion evaluates to true, the test fails, which fails the pipeline step that ran it.

• ADark launching

  Incorrect

  Dark launching hides features in production.

• BShift-left testing

  Correct

  Shifting left moves testing earlier in the lifecycle, where problems are cheaper to fix.

• CBake time

  Incorrect

  Bake time is how long a release runs before it expands.

• DCanary releasing to internal users

  Incorrect

  Canary releases expose a new version to a small group in production.

Shift-left testing moves testing earlier in the development process, for example into pull requests and local builds, so defects are found sooner and cost less to fix.

• AA unit test

  Incorrect

  Unit tests check components in isolation, not system capacity.

• BA stress test

  Correct

  Stress tests subject the system to excessive load to assess resiliency and find bottlenecks.

• CA smoke test

  Incorrect

  A smoke test is a quick check that a deployment basically works.

• DAn integration test

  Incorrect

  Integration tests check interactions between components, not limits under load.

Load testing increases load up to a predefined threshold; stress testing goes beyond it with excessive load to evaluate resiliency and capacity. Together they form performance testing.

• AInject the faults while a load test drives realistic traffic against the service

  Correct

  Chaos testing gives meaningful results once the application is under load, so it's typically run together with load testing.

• BInject the faults while the service is idle, so that no traffic affects the results

  Incorrect

  With no load there's little to observe; the service must be under load to show how it responds to injected faults.

• CRun a penetration test against the database and cache instead

  Incorrect

  Penetration testing looks for security vulnerabilities, not for how the service copes with failures.

• DRun unit tests in which mocked database and cache clients throw errors

  Incorrect

  Unit tests check components in isolation, so they don't show how the whole running system responds.

Chaos testing introduces simulated failures to assess the system's response and its resilience strategies. For the results to be meaningful, the application must first be placed under load, which is why chaos testing and load testing are typically performed together, in the same environment.

• AStatic suites

  Incorrect

  Static suites hold whatever cases you add by hand.

• BQuery-based suites built from a work item query

  Incorrect

  Query-based suites collect cases that match a query, not a story.

• CRequirement-based suites

  Correct

  Requirement-based suites are tied to a requirement and its linked test cases.

• DShared steps

  Incorrect

  Shared steps reuse steps across test cases; they aren't suites.

Test plans contain static, requirement-based or query-based test suites. Requirement-based suites link test cases to requirements such as user stories, which supports traceability from requirements to test results.