Generative AI Engineer Associate · Domain 6: Evaluation and Monitoring · 12% of the exam
Task 6.3: Use inference logging to assess deployed RAG application performance; Use inference tables and Agent Monitoring to track a live LLM endpoint; Use AI Gateway (Inference Tables, Usage Tables, and rate limiting) to track an LLM or agent deployed via Agent Framework
Inference tables in Unity Catalog, usage system tables, rate limits, production traces and scorers, and what each tells you about a live endpoint.
Study it
Inference tables, usage tables, rate limits and production traces
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 2
For a RAG agent that calls a Unity Gateway model service, an engineer must report weekly token use and latency by requester, and debug bad answers by reading the exact prompts and responses. Which TWO data sources should be used? (Choose TWO.)
AThe system.billing.usage table alone
BThe system.ai_gateway.usage system table
CThe AI Search index sync logs
DThe model version's signature
EThe model service's inference table
Show the answer and why
AThe system.billing.usage table alone
Incorrect
Billing records cost by product and SKU; they hold no prompts, responses or per-request latency.
BThe system.ai_gateway.usage system table
Correct
The usage table records requester, service, token usage and latency for Unity Gateway services.
CThe AI Search index sync logs
Incorrect
Index sync history concerns the retriever's data, not the model service's requests.
DThe model version's signature
Incorrect
A signature describes the model's input and output schema, not traffic.
EThe model service's inference table
Correct
Inference tables log request and response payloads to a Unity Catalog Delta table for debugging.
Unity Gateway logs usage to system.ai_gateway.usage and, when enabled, request and response payloads to inference tables in Unity Catalog.
A model service has a service limit of 1,000 QPM, a default user limit of 60 QPM, a custom limit of 300 QPM for the group batch-jobs, and a custom limit of 500 QPM for the service principal sp-etl, which belongs to batch-jobs. What limit applies to sp-etl?
A300 QPM, because group limits override individual limits
B500 QPM, still subject to the 1,000 QPM service-wide maximum
C60 QPM, because the default user limit applies to every caller of the service
D800 QPM, the sum of the group and the individual limits
Show the answer and why
A300 QPM, because group limits override individual limits
Incorrect
It is the other way round: individual limits take priority over group limits.
B500 QPM, still subject to the 1,000 QPM service-wide maximum
Correct
A user or service principal limit takes priority over group limits and overrides the default, while the service limit caps all traffic.
C60 QPM, because the default user limit applies to every caller of the service
Incorrect
Custom limits override the default user limit.
D800 QPM, the sum of the group and the individual limits
Incorrect
Limits are not added together; one applicable limit is enforced.
Unity Gateway enforces a service-wide maximum, a default per-user limit and custom limits for users, service principals and groups. Individual custom limits win over group limits, and when both QPM and TPM apply the more restrictive one is enforced.
Security wants to find the callers that still reach Unity Gateway model services with personal access tokens instead of OAuth. Which column of system.ai_gateway.usage answers this directly?
Arequester_type, which separates the two token kinds
Brequest_tags, where the client records its token type
Cuser_agent, which names the token in each request
Dauth_mode, which records PAT or OAUTH per request
Show the answer and why
Arequester_type, which separates the two token kinds
Incorrect
requester_type says whether the caller is a user, service principal or group, not which credential it used.
Brequest_tags, where the client records its token type
Incorrect
Request tags hold user-provided values sent with the request header.
Cuser_agent, which names the token in each request
Incorrect
user_agent identifies the client library, such as an SDK version.
Dauth_mode, which records PAT or OAUTH per request
Correct
auth_mode holds the Databricks credential type that authenticated each request: a personal access token (PAT) or an OAuth token (OAUTH).
Filtering the usage table on auth_mode = 'PAT' surfaces the traffic that still authenticates with personal access tokens, so the team can move it to OAuth.