Task 2.2: Train, fine-tune, and customize models for ML and AI solutions.

• ABayesian optimization

  Incorrect

  Bayesian optimization uses the results of completed jobs to choose the next combinations. Running many jobs at once means each choice is made with less information.

• BGrid search

  Incorrect

  Grid search supports only categorical hyperparameters, so it cannot search five continuous ranges.

• CRandom search

  Correct

  Random search picks each combination independently of earlier results, so the maximum number of concurrent jobs can run without changing the performance of the tuning.

• DA warm start job with IDENTICAL_DATA_AND_ALGORITHM

  Incorrect

  Warm start reuses earlier tuning jobs as a starting point. There is no parent job here, and warm start is not a way to run more jobs in parallel.

Bayesian optimization learns from finished jobs, which makes it efficient but sequential in spirit. Random search does not learn between jobs, so it parallelizes freely.

• AUse managed Spot training and restart interrupted jobs from scratch

  Incorrect

  Without checkpoints an interrupted job starts over, and AWS recommends checkpointing for any Spot job that does not finish quickly. A 30-hour job would risk repeating hours of work.

• BTurn on SageMaker AI managed warm pools for the job

  Incorrect

  Warm pools keep infrastructure after a job ends so the next similar job starts faster. They do not lower the price of the compute, and the retained instances are billable.

• CUse managed Spot training with checkpoints in Amazon S3

  Correct

  Managed Spot training can cut training cost by up to 90% compared with On-Demand. With checkpointing, SageMaker AI copies checkpoints to S3 and restores them after an interruption, so the job resumes instead of restarting.

• DMove the job to a larger On-Demand instance type

  Incorrect

  A larger instance may finish sooner but is billed at a higher On-Demand rate, so it does not give the large saving the team wants.

Spot capacity is the main lever for training cost; checkpoints are what make long jobs safe on it. Set EnableManagedSpotTraining and a MaxWaitTimeInSeconds larger than MaxRuntimeInSeconds.

• AStop training early, when validation loss stops improving

  Correct

  The rising validation loss shows the model is overfitting. Early stopping ends training before the model learns noise in the training data.

• BTrain for more epochs so the model can converge

  Incorrect

  More epochs help an underfit model. Here the model already fits the training data too closely, so more passes widen the gap.

• CIncrease regularization, such as a stronger weight penalty

  Correct

  Regularization reduces model flexibility by penalizing parameters that add little to predictions, which counters overfitting.

• DAdd more layers and engineered features to the model

  Incorrect

  A more complex model is more likely to learn the noise in the training data, which makes overfitting worse.

• ERaise the learning rate so training converges faster

  Incorrect

  A higher learning rate can speed convergence but risks instability. It does not address the gap between training and validation loss.

Falling training loss with rising validation loss is the classic sign of overfitting (high variance). Reduce flexibility or stop earlier; do the opposite only for underfitting.

• AIncrease the number of epochs on the claims data

  Incorrect

  More passes over the same narrow data push the model further toward the domain and away from what it knew before.

• BRaise the learning rate so the next run finishes sooner

  Incorrect

  A higher learning rate makes larger updates to the weights and can make training unstable. It does nothing to preserve general skills.

• CRemove the validation set so all claims data is used for training

  Incorrect

  Dropping the validation set adds a little data but removes the main way to see the problem during training.

• DTurn on data mixing so Nova training data is blended in

  Correct

  Data mixing combines the custom dataset with samples from Nova's own training data. AWS documents it as a way to prevent overfitting on custom data and catastrophic forgetting of the model's existing capabilities.

Catastrophic forgetting happens when training on new, narrow data overwrites what a model already knew. Mixing general data back into fine-tuning, and watching general benchmarks as well as domain metrics, are the usual defenses.

• AIncrease numberOfResults so more chunks come back

  Incorrect

  Returning more chunks adds more near-miss results to the context. It does not make the exact part number rank higher.

• BSet the knowledge base retrieval search type to hybrid

  Correct

  Hybrid search combines vector (semantic) search with search through the raw text, which helps with exact tokens such as part numbers. It is supported for OpenSearch Serverless stores that contain a filterable text field.

• CSwitch to 256-dimension Titan Text Embeddings V2 vectors

  Incorrect

  Smaller vectors reduce storage and cost, but they are still semantic embeddings and will still blur similar part numbers together.

• DTurn off chunking so each document is one chunk

  Incorrect

  With no chunking each whole document is one embedding, which makes matching on one specific part even less precise.

Semantic search is good at meaning and weak at exact identifiers. Hybrid search adds keyword matching on the raw text, which is the usual fix for codes, IDs and rare terms.

• AA larger instance for the training jobs

  Incorrect

  Instance size does not tell the tuning job how to read the metric from the logs.

• BThe Hyperband strategy instead of Bayesian

  Incorrect

  The strategy chooses hyperparameters; it still needs a metric definition to read the objective.

• CA warm start from a previous tuning job

  Incorrect

  Warm start reuses earlier results; it does not define how to parse the metric.

• DA metric definition with a name and a regex

  Correct

  For custom algorithms, you define metrics by specifying a name and a regular expression; tuning parses the training job's stdout and stderr with it to find the values.

Built-in algorithms emit known metrics; custom scripts need metric definitions so tuning can read the objective from the logs.

• ACheckpoints added to S3 mid-job are copied in

  Incorrect

  Checkpoints added to the S3 folder after the job has started are not copied to the training container.

• BFiles written to /opt/ml/checkpoints sync to S3

  Correct

  Checkpoint files are saved under a local directory, by default /opt/ml/checkpoints, and SageMaker AI copies them to S3 and keeps the directory in sync during training.

• CExisting S3 checkpoints are copied in at job start

  Correct

  Existing checkpoints in S3 are written to the container at the start of the job, which lets a job resume from a checkpoint.

• DDeleting a local checkpoint keeps the S3 copy

  Incorrect

  If a checkpoint is deleted in the container, it is also deleted in the S3 folder.

• ECheckpoints cannot use S3 Express One Zone

  Incorrect

  Checkpoints can use S3 Express One Zone for faster access.

Checkpointing syncs a local folder with S3 both ways at defined moments: restore at start, write during training, mirror deletions.

• AManaged warm pools with a keep-alive period

  Correct

  Managed warm pools retain provisioned infrastructure after a job completes so matching subsequent jobs reuse it, reducing start-up latency; you set KeepAlivePeriodInSeconds, and warm pools are billable.

• BUse managed Spot training for the jobs

  Incorrect

  Spot capacity lowers cost but can take longer to start and may be interrupted.

• CUse a larger instance for each job

  Incorrect

  A larger instance can run faster but still has to be provisioned for each job.

• DUse pipe input mode for the data

  Incorrect

  Input modes change how data is read, not how long instances take to be provisioned.

For rapid iteration with many similar jobs, warm pools trade some idle cost for much faster starts.

• ATwelve fine-tuned models, one per category

  Incorrect

  Fine-tuning twelve models adds cost and maintenance for differences a prompt variable can express.

• BA Prompt management prompt with variables

  Correct

  Prompt management lets you include variables in a prompt and supply their values when you test or invoke the model, so one prompt serves different use cases.

• CA guardrail with a denied topic per category

  Incorrect

  Guardrails filter content; they do not template prompts.

• DA knowledge base with one document per category

  Incorrect

  Retrieval adds context from documents; it does not parameterize the prompt itself.

Prompt templates with variables are a lightweight customization technique; reach for fine-tuning only when prompting cannot get the behavior.

• AContinued use of the large model with Priority tier

  Incorrect

  Priority makes responses faster for a premium, which increases cost.

• BA larger context window on the small model

  Incorrect

  More context does not transfer the larger model's capability.

• CDistillation from the large model to the small one

  Correct

  Distillation transfers knowledge from a larger, more capable teacher model to a smaller, faster and cost-efficient student by fine-tuning the student on responses the teacher generates.

• DProvisioned Throughput for the large model

  Incorrect

  Reserved capacity can add throughput but does not make the large model cheaper per request.

Combining models through distillation is a standard way to cut inference cost while keeping much of a larger model's quality.