Task 4.5: Design efficient batch processing strategies

• AThrough the synchronous API in fast mode to get the whole audit done in one evening

  Incorrect

  Fast mode is about synchronous latency, not price, and is not available in batches; it works against the budget.

• BThrough the Message Batches API with stream set to true so progress can be watched as results arrive

  Incorrect

  Streaming is not supported inside batch requests; results come back as a file when processing ends.

• CThrough the Message Batches API, polling for the results and matching them by custom_id

  Correct

  Batches cost half the standard price and finish within 24 hours, which fits a weekend deadline with no one waiting.

• DThrough the synchronous Messages API with as much concurrency as the rate limits allow

  Incorrect

  This finishes sooner than needed but costs twice as much as a batch for latency nobody uses.

Latency-tolerant, non-blocking workloads such as weekend audits fit the Message Batches API: 50 percent lower cost and up to 24 hours of processing, with results correlated through custom_id.

• AThe whole batch again, so that every document is processed under the same conditions

  Incorrect

  Rerunning the successes doubles the cost for results you already have.

• BOnly the errored custom_ids, with the oversized documents split into chunks and the invalid requests fixed

  Correct

  Resubmitting just the failures, changed so that they can succeed, avoids paying again for 9,860 good results.

• CThe errored documents through the synchronous API without changes so that you can watch each one finish

  Incorrect

  A different endpoint does not make an oversized document fit or an invalid request valid.

• DOnly the errored custom_ids, unchanged, since errored requests are not billed and may succeed on a second try

  Incorrect

  Oversized documents and invalid requests fail the same way again unless the requests are changed.

Handle batch failures by custom_id: resubmit only the failed requests, modified to remove the cause, for example by chunking documents that exceeded the context limit.

• ABatches always cost more for small jobs

  Incorrect

  Batch usage is half the standard price.

• BBatches do not support code review prompts

  Incorrect

  Almost any Messages request can be batched.

• CBatches can take up to 24 hours to finish

  Correct

  A batch can take up to a day to return, and developers cannot merge until the check finishes.

• DBatches return results in random order, so they cannot be used

  Incorrect

  Ordering is not what rules batches out here; the wait for results is.

A blocking pre-merge check needs the synchronous API, because a batch can take up to 24 hours to finish.

• AThey are still downloadable from the Console, since the batch is still listed there

  Incorrect

  You may still view the batch, but its results are no longer available for download.

• BThey are gone: batch results are available for 29 days after the batch was created

  Correct

  Results can be downloaded for 29 days from creation; after that only the batch record remains.

• CThey can be regenerated by resubmitting the same batch ID

  Incorrect

  Submitting again creates a new batch with new processing and new costs, not the original outputs.

• DThey are available from the results URL saved when the batch ended, which stays valid

  Incorrect

  The results expire 29 days after creation, whatever URL you saved.

Batch results are available for 29 days after the batch is created. Keep the raw outputs yourself if you may need them later, for example for audits.

• ANo; the batch must be split into batches of ten

  Incorrect

  Batch size is not the cause.

• BNo; caching is guaranteed to hit for every request after the first

  Incorrect

  Concurrency makes hits best-effort.

• CYes; prompt caching is not supported in batches

  Incorrect

  It is supported, on a best-effort basis.

• DYes; cache hits in batches are best-effort

  Correct

  Caching works with batches, but hits are not guaranteed.

Prompt caching can be used with the Message Batches API, but because requests may be processed concurrently and in any order, cache hits are best-effort.

• ARemove stream and the fast-mode speed setting from each request, since a batch rejects them

  Correct

  Batch results come back as a file rather than a stream, and fast mode tunes synchronous latency; both parameters return a validation error.

• BDry run one request of the new shape through the Messages API first

  Correct

  A single dry run of the request shape catches validation errors before they hit the whole batch.

• CKeep stream so that results arrive as each request finishes

  Incorrect

  stream is not supported inside a batch; results are retrieved once processing ends.

• DSplit the batch so that every request in it uses the same model

  Incorrect

  Each request is processed independently, so different kinds of requests can be mixed in one batch.

• EAdd requests with max_tokens set to 0 to warm the cache for the rest of the batch

  Incorrect

  max_tokens 0 is not supported in a batch, because a cache entry written there would likely expire before the follow-up runs.

Almost any Messages API request can be batched, but a few parameters are rejected: stream, the fast-mode speed setting and max_tokens 0. Strip them, and dry run one request of the new shape with the Messages API before submitting the batch.

• AOthers are unaffected; errored ones are not billed

  Correct

  Each request is processed independently, and errored ones are free.

• BErrored requests are billed at the batch rate

  Incorrect

  Errored requests are not billed.

• CThe batch must be resubmitted entirely to recover the 30

  Incorrect

  Requests are processed independently, so the succeeded results stand.

• DThe whole batch failed and is billed in full

  Incorrect

  One request's failure does not affect others.

Batch requests are independent: a failure in one does not affect the others, and errored requests are not billed.