Task 4.2: Implement observability for generative AI applications and agents

• AA Foundry Owner connects Application Insights; on-call gets Log Analytics Reader and 180-day retention

  Correct

  Tracing starts when an owner connects Application Insights, and retention follows that resource.

• BThe engineer turns on tracing in the agent's settings, and Foundry keeps every trace for 180 days by default

  Incorrect

  Only an owner role enables tracing, and Foundry doesn't keep traces itself.

• CA Foundry Owner connects Application Insights, and the on-call team reads traces with its Foundry User role

  Incorrect

  Reading traces needs read permission on the connected Application Insights resource.

• DTracing is always on; the team only needs to give the on-call engineers the Foundry Owner role

  Incorrect

  Tracing is off by default, and Owner isn't the role for reading telemetry.

Tracing is off by default; no trace data is collected until a Foundry Account Owner or Foundry Owner enables it by connecting an Application Insights resource. Viewing traces needs read permission on the connected Application Insights or Log Analytics workspace, such as Log Analytics Reader, and Application Insights and Log Analytics configuration governs retention.

• AThe Monitoring Reader role for the operator on the Foundry resource

  Incorrect

  Monitoring Reader lets the operator see metrics; logs still need routing.

• BTracing turned on in each Foundry project, with Application Insights connected

  Incorrect

  Traces are separate from the resource's request logs.

• CA metric alert rule on the Model Requests metric for each deployment

  Incorrect

  Alerts don't create log data.

• DA diagnostic setting sending RequestResponse logs to a Log Analytics workspace

  Correct

  Logs are generated but kept only when a diagnostic setting routes them.

Azure Monitor collects metrics automatically, but resource logs must be routed to Azure Monitor logs by a diagnostic setting before you can save or query them. Categories include RequestResponse for inference requests and Audit for administrative operations. Viewing metrics needs at least Monitoring Reader; configuring diagnostic settings needs Monitoring Contributor or equivalent.

• AThe metric averages over 24 hours, which hides short spikes of 429s

  Incorrect

  The metric's definition, not its window, explains it.

• BThe metric counts only server errors (500 and above) against availability

  Correct

  Throttling 429s aren't server errors in this calculation.

• CThe metric excludes provisioned deployments from its availability figures

  Incorrect

  It covers model deployments generally.

• DThe metric refreshes only once per day, after the spike has passed

  Incorrect

  Metrics are near real time.

Model Availability Rate is calculated as (Total Calls - Server Errors) / Total Calls, where server errors are HTTP responses of 500 or higher, so 429 throttling responses don't reduce it.

• APer-project Foundry cost attribution for chargeback is generally available

  Incorrect

  Project-level cost attribution is in preview.

• BGrouping cost analysis by Meter shows which model series drive the costs

  Correct

  Meters break costs down by model input and output usage.

• CHTTP 200 status codes alone decide whether a request is billed

  Incorrect

  Billing depends on billable processing and meter behavior.

• DMeters for most models from partners and community appear under the resource group, not the Foundry resource

  Correct

  Scope cost analysis to the resource group to see them.

• ECost records appear the same minute the request is served

  Incorrect

  Cost and usage records can appear with delay.

In Cost Management, scope cost analysis to the resource group and set Group by to Meter to see costs per model series; Azure OpenAI and Microsoft models appear as meters under each Foundry resource, while some partner and community models appear under the resource group. HTTP status codes alone don't determine billing, records can arrive with delay, and project-level cost attribution is in preview.

• AModel Availability Rate

  Incorrect

  It's a percentage, not a count of calls.

• BModel Requests

  Correct

  It's the number of calls to the inference API.

• CTime To Response

  Incorrect

  It measures latency, not a count of calls.

• DProvisioned Utilization

  Incorrect

  It doesn't count calls to the inference API.

In the Models - Requests category, Model Requests is the number of calls made to the model inference API over a period of time, aggregated as a total count.

• AWorkbooks, Grafana and Power BI

  Correct

  These combine metrics, logs and other sources.

• BOnly the Foundry portal Home page

  Incorrect

  The article lists other visualization tools.

• CAzure Resource Graph only

  Incorrect

  Resource Graph queries resources, not metrics dashboards.

• DExcel exports of the activity log

  Incorrect

  That isn't the documented option.

Beyond metrics explorer, Workbooks provide customizable reports with text, metrics and log queries, Grafana builds operational dashboards across sources, and Power BI can import log data from Azure Monitor for interactive visualizations.

• AImmediately

  Incorrect

  Cost data lags the billing event.

• BAbout one week

  Incorrect

  The delay is much shorter.

• CAbout five hours

  Correct

  There's roughly a five-hour delay.

• DAt the end of the month only

  Incorrect

  Data appears continuously with a delay.

Cost data in Azure portal cost analysis shows actual post-consumption charges for model consumption, with approximately a five-hour delay from the billing event.

• AThe project's Home page in the Foundry portal

  Incorrect

  Home doesn't show per-deployment metrics.

• BThe chat history in the model's playground

  Incorrect

  The playground is for testing prompts.

• CThe Quota page for the resource's deployments

  Incorrect

  Quota shows allocations, not metrics.

• DThe deployment's Metrics tab in Models + endpoints

  Correct

  The Metrics tab gives an overview of common metrics.

In the Foundry portal, under My assets select Models + endpoints, select the deployment and open its Metrics tab for an overview of common metrics; a link leads to Azure Cost Management for cost details.