Task 4.1: Implement cost optimization and resource efficiency strategies.

• ARequest a higher tokens-per-minute quota and keep max_tokens at 32,000 for safety

  Incorrect

  A higher quota helps, but the oversized reservation keeps wasting most of it, so fixing max_tokens is the first step.

• BMove the requests to the Priority service tier so that they are admitted ahead of other traffic

  Incorrect

  Priority affects ordering and price, and on-demand quota is shared across Priority, Standard and Flex, so the reservation problem remains.

• CEnable prompt caching on the 3,000-token input so that cached input tokens no longer count toward the quota

  Incorrect

  Cache reads are not counted toward the quota, but each input here is a different document, and the 32,000-token output reservation would remain the main problem.

• DLower max_tokens to slightly above the longest expected summary, such as 1,500 tokens

  Correct

  At the start of each request, the input tokens plus max_tokens are deducted from the quota, and unused tokens are returned only at the end. A 32,000 reservation limits concurrency even though actual use is far lower.

Quota is reserved up front as input tokens plus max_tokens and corrected at the end to input plus cache writes plus output times the burndown rate. Setting max_tokens close to real output size is the cheapest way to raise concurrency within the same quota.

• ACheck cacheReadInputTokens and cacheWriteInputTokens in the response usage to confirm cache hits

  Correct

  The usage fields report how many tokens were read from and written to the cache, which shows whether requests hit the cache.

• BSubmit the questions through batch inference so that the cached rulebook is shared across records

  Incorrect

  Prompt caching is supported only for on-demand inference, not with the batch inference API, and batch does not suit interactive questions.

• CPut the user's question first and the rulebook after it so that the newest content is read first

  Incorrect

  A prefix that changes with every request causes cache misses, because caching reuses an identical prompt prefix.

• DSet the cache time to live to 24 hours so that the rulebook stays cached overnight

  Incorrect

  Cache TTLs are short, such as 5 minutes or 1 hour depending on the model, and reset with each hit. A steady request rate keeps the cache warm anyway.

• EPut the rulebook and instructions at the start of the prompt and add a cachePoint block right after them

  Correct

  Explicit caching caches the prefix up to the checkpoint. Static content must come first and stay identical, with the changing question after the checkpoint.

Prompt caching rewards stable prefixes. Order the prompt static-first, place the checkpoint after the static block, and confirm hits in the usage fields or the CacheReadInputTokenCount metric. Cache reads are billed at a lower rate and do not count toward the tokens-per-minute quota.

• AA semantic cache of question embeddings and answers in Amazon ElastiCache for Valkey, with a TTL

  Correct

  A semantic cache matches new questions to earlier ones by meaning and returns the stored answer above a similarity threshold, avoiding the model call. A TTL limits how long answers are reused.

• BPrompt caching of the system prompt so that each question costs fewer input tokens

  Incorrect

  Prompt caching lowers the cost of a repeated prefix, but every question still triggers a full model invocation and output generation.

• CAmazon CloudFront in front of the chat API so that identical requests are served from edge caches

  Incorrect

  CloudFront caches responses by cache key, such as the URL and selected headers or query strings, so paraphrased questions do not match.

• DAn exact-match cache in Amazon DynamoDB keyed by a hash of each question's text, with a one-week TTL

  Incorrect

  Hash keys only match identical strings, so rephrased questions, the majority here, would still invoke the model.

When users ask the same thing in different words, only a semantic cache avoids the model call. Start with a strict similarity threshold, lower it while watching accuracy, and set TTLs that match how often the answers change.

• ABuy Provisioned Throughput for the nightly window

  Incorrect

  Provisioned Throughput reserves capacity and is billed while it exists, not a discount for flexible work.

• BSend these requests with the Priority service tier

  Incorrect

  Priority costs more than standard pricing in exchange for speed.

• CRaise maxTokens so that each job finishes in fewer calls

  Incorrect

  A higher output limit does not lower the price per token.

• DSend these requests with the Flex service tier

  Correct

  Flex offers a pricing discount for workloads that can handle longer processing times, such as summarization and agentic workflows.

Match urgency to price. Work that can wait fits the Flex tier's discount.

• APurchase Provisioned Throughput for the large model and route all extraction traffic through it

  Incorrect

  Provisioned Throughput reserves dedicated capacity, but every request would still run on the large model, so the cost gap to the small model remains.

• BA Bedrock Model Distillation job with the large model as teacher and the small model as student

  Correct

  Distillation fine-tunes the smaller student on the teacher's responses for this use case. It can use prompt-response pairs from invocation logs, filtered by request metadata, so no labeled data or training infrastructure is needed.

• CSwitch to the small model and add twenty worked examples from the logs to every prompt

  Incorrect

  Twenty examples add input tokens to every one of 3 million calls, which eats into the saving, and they do not train the small model on the large model's behavior for this task.

• DFine-tune the small model on a few hundred invoices that analysts label by hand first

  Incorrect

  The team has no labeled data and no time to create it, and a few hundred examples is far less than the logged production traffic.

When a large model is good enough but too expensive, transfer its behavior for the specific task to a smaller model. Distillation automates that transfer from teacher responses, which production logs may already hold.

• AStart summarization from S3 event notifications so that only new or updated objects are processed

  Incorrect

  Because the export rewrites every object each night, a notification fires for all 500,000 manuals and nothing is skipped.

• BSubmit the nightly summaries as an Amazon Bedrock batch inference job instead of individual calls

  Incorrect

  Batch inference lowers the price per token, but it still summarizes every manual, including the 98% that did not change.

• CKeep a content checksum with each summary and call the model only when the checksum changes

  Correct

  A checksum of the content stays the same when an unchanged manual is rewritten. Amazon S3 stores one for objects uploaded through AWS clients, or the workflow can compute it, and a match means the model call is skipped and the stored summary is kept.

• DEnable prompt caching for the summarization instructions that precede each manual's text

  Incorrect

  Caching can only reuse the shared instructions. Each manual is still sent and summarized, and its text is different in every request.

The cheapest model call is the one you do not make. Fingerprinting the input detects real changes even when the upstream system rewrites everything, and it keeps existing outputs stable.

• AAdd "Rationale:" as a stop sequence in the inference configuration of each request

  Correct

  Generation stops as soon as the model produces the stop sequence, so the discarded paragraph is never generated or billed, whatever the length of the category line before it.

• BSet the maximum response length to 15 tokens on every request to cut off the rationale

  Incorrect

  A fixed cap would also cut off the longest category names, which the service must never do.

• CLower the temperature to 0 so that the model writes shorter responses

  Incorrect

  Temperature changes how likely tokens are chosen, not how long the response is, so the rationale is still generated.

• DEnable prompt caching for the fixed template that precedes each ticket

  Incorrect

  Prompt caching lowers the cost of repeated input tokens. The expensive part here is the output, which caching does not reduce.

Response size controls are a direct lever on token cost. A stop sequence ends generation at a known marker, which suits outputs whose useful part varies in length; a fixed token cap suits outputs of predictable size.

• ASend a short keep-alive request in each open session every four minutes to keep the cache warm

  Incorrect

  A cache hit does reset the TTL, but this adds a model call to every session every four minutes, which the team ruled out.

• BAdd more cache checkpoints inside the case file so that smaller parts of it are cached

  Incorrect

  Extra checkpoints change which prefixes can be cached, not how long a cache entry lives, so the entries still expire during the pauses.

• CSet the ttl field of the existing cache checkpoint to 1h for these sessions

  Correct

  For models that support it, a 1-hour TTL keeps the cached prefix across pauses longer than 5 minutes, which AWS describes as the case for prompts reused less often than every 5 minutes but within the hour.

• DMove the follow-up questions to Amazon Bedrock batch inference jobs

  Incorrect

  Prompt caching is not supported with the batch inference API, and attorneys need their answers interactively.

Match the cache TTL to the traffic pattern. The 5-minute TTL keeps refreshing under steady traffic, while sessions with longer pauses need the 1-hour TTL to avoid writing the same prefix to the cache again and again.