Question 1 · choose 1
A summarization service gets frequent 429 responses from its Standard deployment, yet the usage metrics show it consumes well under half of its tokens-per-minute limit. Each request sets the maximum output to 16,000 tokens, although summaries are rarely longer than 500 tokens. What is the most effective change?
- ARequest a quota increase for the model in that region
- BSet the maximum output tokens close to the real summary length
- CTurn on streaming so that tokens are returned as they are generated
- DRaise the requests-per-minute limit independently of the TPM value
Show the answer and why
ARequest a quota increase for the model in that region
Incorrect
More quota would hide the problem at extra cost; the deployment isn't short of real tokens.
BSet the maximum output tokens close to the real summary length
Correct
The rate limiter counts an estimate that includes the max_tokens setting when each request arrives, so an oversized value uses up the TPM budget early.
CTurn on streaming so that tokens are returned as they are generated
Incorrect
Delivery style doesn't change how the rate limiter estimates each request.
DRaise the requests-per-minute limit independently of the TPM value
Incorrect
RPM is set in proportion to the TPM assignment; you can't raise it on its own, and requests aren't the bottleneck here.
TPM rate limits use an estimate made when a request is received, built from the prompt and the max_tokens setting. Billing uses the real count afterwards. Right-sizing max_tokens frees the budget.
Microsoft documentation