Task 3.3: Describe the training and fine-tuning process for FMs.

• AFine-tuning builds a model from scratch, and pre-training then adjusts it with a small labeled dataset

  Incorrect

  The order is reversed. Pre-training on broad data produces the foundation model, and fine-tuning starts from that pre-trained model.

• BPre-training changes only the prompt, while fine-tuning is the step that adjusts the model's parameters

  Incorrect

  Pre-training is training: it is how the model's parameters are learned from massive datasets in the first place.

• CPre-training learns general skills from very large, broad data; fine-tuning then adapts the model to a task with a smaller labeled dataset

  Correct

  Foundation models are trained on a broad spectrum of generalized data. Supervised fine-tuning then adjusts the model's parameters with labeled examples to improve performance on specific tasks.

• DFine-tuning usually needs more data and more compute than pre-training, because the model has to relearn the whole language from the beginning

  Incorrect

  Adapting a pre-trained model needs a much smaller dataset and fewer resources than training a new model, which is the point of transfer learning.

Pre-training is the expensive, general step done once; fine-tuning is a cheaper, targeted step that reuses what the model already learned.

• AContinued pre-training on the ranked responses

  Incorrect

  Continued pre-training adapts a model to a domain using large amounts of unlabeled data. It does not use human preference rankings.

• BTransfer learning from a model trained on another task

  Incorrect

  Transfer learning fine-tunes a model pre-trained on one task for a new, related task. It does not describe learning from ranked preferences.

• CFew-shot prompting

  Incorrect

  Few-shot prompting places examples in the prompt and changes nothing in the model.

• DReinforcement learning from human feedback (RLHF)

  Correct

  RLHF trains a separate reward model from human preference ratings and then uses it as the reward function to optimize the language model.

Human rankings, then a reward model, then reinforcement learning against that reward: this is RLHF, widely used to make LLM output more helpful and harmless.

• ACurate accurate, high-quality examples and remove records with errors

  Correct

  Mistakes and outliers in the input data can significantly affect deep learning, and low-quality training data is a major cause of hallucinations.

• BAdd as much unfiltered web text as possible, whatever its quality

  Incorrect

  Volume does not replace quality: low-quality, limited or biased training data is a leading cause of hallucinations.

• CKeep only the most common question type so the dataset is uniform

  Incorrect

  An imbalanced dataset can bias training, so the model performs well on the majority class and fails on the rest.

• DMake the examples diverse and representative of real customer questions

  Correct

  A broad, diverse and unbiased dataset makes accurate responses more likely, and adding representative examples is a documented fix when training is not improving.

• ELeave customers' personal data in the records so the model learns more

  Incorrect

  Personal data is sensitive data to discover and protect, for example with Amazon Macie, not material to train a product assistant on.

For fine-tuning, a smaller curated, representative dataset beats a large noisy one, and sensitive data should be removed before training.

• AContinued pre-training on the unlabeled reports

  Correct

  Continued pre-training uses large amounts of unlabeled data to adapt a model from a general domain to a specific one, such as a field with its own technical jargon.

• BSupervised fine-tuning on the unlabeled reports

  Incorrect

  Supervised fine-tuning needs labeled input-output examples; fine-tuning directly for a new domain requires large amounts of labeled records.

• CModel distillation into a smaller student model

  Incorrect

  Distillation transfers a teacher model's skills to a smaller student model. It does not teach a model the company's domain from raw reports.

• DFew-shot prompts that quote a few report paragraphs

  Incorrect

  Few-shot examples in a prompt steer one response; they do not change the model or cover a very large collection of reports.

Unlabeled domain text points to continued pre-training; labeled input-output pairs point to fine-tuning.

• ADomain adaptation fine-tuning on a large body of domain-specific text

  Incorrect

  Domain adaptation fine-tunes a model on domain-specific data to learn its language; this team's data is prompt and response pairs.

• BPrompt caching of the example responses

  Incorrect

  Prompt caching reuses repeated prompt prefixes at inference time; it trains nothing.

• CRetrieval of the examples through a knowledge base

  Incorrect

  Retrieval adds information to prompts at query time; it does not change how the model follows instructions in general.

• DInstruction-based fine-tuning on the prompt and response examples

  Correct

  JumpStart documents instruction-based fine-tuning that uses prompt and response examples; fine-tuning further trains the model and changes its weights.

Prompt-response pairs teach instruction following (instruction tuning); raw domain text teaches the domain (domain adaptation).

• AIt replaces the student model in production once distillation ends

  Incorrect

  The point is the opposite: the smaller, faster student model is what you use afterwards.

• BIt generates responses to your prompts that are used to fine-tune the student

  Correct

  Bedrock generates responses from the teacher model for your prompts and uses them to fine-tune the student model, transferring the teacher's knowledge for your use case.

• CIt labels the company's data by hand for the fine-tuning job

  Incorrect

  The teacher is a model, not a human labeler; Bedrock uses data synthesis to generate its responses automatically.

• DIt checks the student model's outputs for harmful content at run time in production

  Incorrect

  Filtering harmful content at run time is what Amazon Bedrock Guardrails does.

A larger, more capable teacher produces training data; a smaller student learns from it and then serves requests faster and more cheaply.

• ASupervised fine-tuning on labeled prompt and answer pairs

  Incorrect

  Supervised fine-tuning needs a labeled dataset of inputs paired with the desired outputs, which the company does not have.

• BContinued pre-training on public source code

  Incorrect

  Continued pre-training adapts a model to a domain with large amounts of unlabeled data; it does not optimize for passing the company's tests.

• CReinforcement fine-tuning with a reward function

  Correct

  Reinforcement fine-tuning uses rewards instead of labeled pairs and suits tasks with measurable success criteria, such as code generation checked by rule-based graders.

• DFew-shot prompting with three sample programs

  Incorrect

  Few-shot examples steer a single response at inference time; the company wants to improve the model itself.

Labeled answers point to supervised fine-tuning; a reliable way to score answers, without one right answer, points to reinforcement fine-tuning.

• ALabels are discarded after training, so their accuracy has no effect on the model

  Incorrect

  Labels are the ground truth the model learns from and is assessed against; their accuracy carries into the model.

• BAccurate labels allow the company to skip evaluating the model

  Incorrect

  Labeled ground truth is also what a model is assessed against; good labels make evaluation meaningful, not unnecessary.

• CAccurate labels make the model's responses fully deterministic

  Incorrect

  Generative models still sample their outputs; label quality affects accuracy, not determinism.

• DThe trained model's accuracy depends on the accuracy of its ground truth

  Correct

  A properly labeled dataset is the ground truth used to train and assess a model, and the trained model's accuracy depends on how accurate that ground truth is.

Garbage in, garbage out: mislabeled examples teach the wrong answers, so label quality is one of the cheapest levers on model quality.

• AIt leaves the model unchanged and only edits the prompt

  Incorrect

  Editing the prompt is prompt engineering; fine-tuning changes the model itself.

• BIt further trains the model and changes its weights

  Correct

  Fine-tuning is a customization method that involves further training and changes the model's weights.

• CIt can help a model work with industry jargon and specialized terms

  Correct

  Fine-tuning is useful when a model must work with domain-specific language such as industry jargon and technical terms.

• DIt must start from a randomly initialized model with no prior training

  Incorrect

  Fine-tuning starts from a pre-trained foundation model; that is what makes it an affordable way to reuse the model's broad capabilities.

• EIt is the only way to give a model access to data that changes daily

  Incorrect

  RAG gives a model current information at query time without retraining, which suits daily changes better.

Fine-tuning reuses a pre-trained model's general abilities and adapts its weights to your tasks and vocabulary; fast-changing facts are a RAG job.