Task 2.2: Implement model deployment strategies.

• AThe Reserved tier, sized for the steady input and output tokens per minute and arranged with the AWS account team

  Correct

  The Reserved tier reserves prioritized capacity with separately sized input and output tokens per minute and automatically overflows to the Standard tier when traffic exceeds the reservation.

• BThe Flex tier for all requests, with retries that use exponential backoff to absorb throttling

  Incorrect

  Flex trades longer processing times for a discount and suits workloads that tolerate delay, not a service that cannot be throttled.

• CThe Standard tier with a request for a higher tokens-per-minute quota through Service Quotas

  Incorrect

  A higher quota raises the upper bound, but a quota is not a guarantee that every on-demand request is served immediately at peak demand.

• DThe Priority tier, set with the service_tier parameter on every request the service sends

  Incorrect

  Priority requests are served ahead of Standard and Flex without a reservation, but they still draw on the shared on-demand quota, so capacity at the steady level is not reserved.

Service tiers trade price for priority. Mission-critical, steady traffic that must not be throttled calls for reserved capacity; the Reserved tier sizes input and output separately and overflows to Standard so spikes are still served.

• ASwitch the endpoint to SageMaker AI Serverless Inference so that compute is allocated after the model is downloaded

  Incorrect

  Serverless Inference does not support GPUs, so it cannot host a model of this size.

• BRaise the model download timeout, the container startup health check timeout and the EBS volume size

  Correct

  For large models, SageMaker AI recommends a longer model download timeout, a longer container startup health check timeout and an EBS volume larger than the model when the instance has no local disk.

• CMove the model to an asynchronous inference endpoint because it allows payloads of up to 1 GB

  Incorrect

  Asynchronous inference handles large request payloads and long processing times, but the endpoint still has to download the model and pass its health check at startup.

• DIncrease the endpoint's InitialInstanceCount so that the download is spread across more instances

  Incorrect

  Every instance downloads and loads the full model, so more instances do not make the download or health check any faster.

LLM deployments fail in ways classic ML deployments rarely do: model artifacts of tens or hundreds of gigabytes must be downloaded and loaded before the container answers health checks. SageMaker AI exposes the download timeout, the startup health check timeout and the EBS volume size for exactly this.

• ADeploy each model as an inference component on one endpoint with its own compute requirements and copy count

  Correct

  Inference components share an endpoint's instances while each model declares its own CPU, accelerator and memory needs, scales its copies independently and can scale down to zero copies.

• BDeploy each model to a SageMaker AI serverless endpoint so that idle models cost nothing at night

  Incorrect

  Serverless endpoints scale to zero, but they do not support GPUs, which 7B-parameter models need for acceptable latency.

• CDeploy each model to its own real-time endpoint with a minimum of one GPU instance per endpoint

  Incorrect

  Separate endpoints scale independently, but six always-on GPU instances waste money on the quiet models and do not share hardware.

• DDeploy all models to one multi-model endpoint so that SageMaker AI loads them from Amazon S3 on demand

  Incorrect

  Multi-model endpoints load models on demand from S3, but you cannot reserve accelerator and memory per model or set copy counts for each model.

Inference components decouple models from the endpoint: several models share the instances, each with its own resource allocation and copy count, and copies can scale to zero to free room for busier models.

• ASend every ticket to both models in parallel and keep the answer with the higher self-reported confidence

  Incorrect

  Running both models on every ticket increases cost rather than reducing it.

• BEscalate a ticket to the large reasoning model only when the small model flags it as complex or its output fails validation

  Correct

  Cascading sends only the hard cases to the expensive model, so the complex answers keep their quality while most tickets never reach it.

• CPurchase Provisioned Throughput for the large model and keep sending every ticket to it

  Incorrect

  Fixed capacity for the large model keeps paying large-model rates for the 85% of tickets a small model could handle.

• DSend every ticket first to a smaller, cheaper model that returns the intent label and priority

  Correct

  Using a smaller model for the routine classification task cuts the per-token price for 85% of the traffic while tests show equal accuracy.

• EMove all requests to the Priority service tier so that the large model answers each ticket faster

  Incorrect

  The Priority tier costs more than Standard. It improves response time, not cost.

Right-size the model for the task. A small model for routine work plus a cascade that escalates only complex or invalid results to the large model is the standard way to cut cost without hurting the hard cases.

• AOne real-time endpoint for each customer's classifier

  Incorrect

  Dedicated endpoints fit models with high traffic or strict latency needs. Three thousand mostly idle endpoints would cost far more than shared hosting.

• BOne serverless endpoint for each customer's classifier

  Incorrect

  Serverless Inference suits intermittent traffic, but it does not support GPUs.

• CA multi-model endpoint on CPU instances for all classifiers

  Incorrect

  A CPU-backed multi-model endpoint fits models that are fast enough on CPU. These classifiers need GPUs to meet the target.

• DA multi-model endpoint on GPU instances for all classifiers

  Correct

  Multi-model endpoints host large numbers of models on a shared fleet and container, support GPU-backed models, and load a newly copied model from S3 on its first invocation, accepting cold-start latency for rarely used models.

Many similar, rarely used models belong on a multi-model endpoint. Check each option against the hard constraints first: GPU support rules out Serverless Inference and CPU fleets here.

• AQuantize the model weights to INT4-AWQ with an optimization job

  Incorrect

  Quantization reduces hardware requirements so the model can run on cheaper GPUs, but the quantized model might be less accurate than the source model.

• BApply speculative decoding with a draft model through an optimization job

  Correct

  Speculative decoding speeds up decoding without compromising the quality of the generated text: a smaller draft model proposes tokens and the original model verifies them.

• CMove the endpoint to an instance type with more GPU memory

  Incorrect

  A larger instance fits when the model or its batches do not fit in GPU memory. The endpoint already uses the largest instance the budget allows.

• DAdd instances and a target tracking auto scaling policy

  Incorrect

  Auto scaling adds instances as the workload rises, which serves more concurrent requests. It does not make a single response generate faster.

Pick the optimization by what may change: quantization trades accuracy for cheaper hardware, while speculative decoding cuts latency and leaves the output quality of the original model intact.

• AQuantize the model, rerun the tests, and host it on a smaller GPU instance

  Correct

  Quantization uses a less precise data type for weights and activations, which reduces hardware requirements so the model can run on less expensive GPUs. It might lose some accuracy, which the regression tests check.

• BDistill the model into a smaller student with Amazon Bedrock Model Distillation

  Incorrect

  Distillation fine-tunes a smaller student model on a teacher model's responses. That is the retraining the startup wants to avoid.

• CCompile the model for the current instance type with an optimization job

  Incorrect

  Compilation optimizes the model for the chosen hardware type without loss of accuracy and shortens deployment and scaling time. It does not move the model to cheaper hardware.

• DBuy a SageMaker AI Savings Plan that covers the current instance

  Incorrect

  Savings Plans lower prices in exchange for a one- or three-year usage commitment, which the startup cannot make.

When the constraint is hardware cost and some accuracy loss is acceptable, quantization is the lever that changes the instance a model needs.

• AA serverless endpoint at the largest memory size for the model

  Incorrect

  Serverless Inference scales to zero when there are no requests, but it does not support GPUs, and its largest memory size is 6 GB.

• BAn asynchronous endpoint whose instance count scales in to zero

  Incorrect

  Asynchronous Inference can scale to zero, but it queues requests and writes results to Amazon S3, and such an endpoint accepts only asynchronous invocations, so the tool could not stream responses.

• CAn inference component for the model on an endpoint that can scale to zero

  Correct

  An endpoint that hosts inference components can scale in to zero instances when MinInstanceCount is 0 and the component's minimum copy count is 0. A step scaling policy on a NoCapacityInvocationFailures alarm brings capacity back, which takes several minutes.

• DScheduled scaling actions that set the variant's instance count to zero after hours

  Incorrect

  Scheduled actions change capacity at set times for predictable load. A real-time endpoint can scale to zero instances only if it hosts inference components, so this variant cannot reach zero.

For an idle real-time GPU endpoint, scale-to-zero needs inference components plus a policy that adds capacity when the next request arrives.

• ASageMaker Asynchronous Inference

  Correct

  Asynchronous inference queues requests, supports large payloads and long processing times, and can scale to zero when idle.

• BA Lambda function that runs the model

  Incorrect

  Lambda functions stop after 15 minutes.

• CA real-time endpoint with a 30-minute client timeout

  Incorrect

  Real-time endpoints are for short synchronous requests, not 30-minute jobs with large payloads.

• DSageMaker Serverless Inference

  Incorrect

  Serverless endpoints have payload and duration limits far below these requests.

Large payloads and long processing call for asynchronous inference, which queues work and scales with demand.