Skip to content
BytePatterns

AIF-C01 · Domain 1: Fundamentals of AI and ML · 20% of the exam

Task 1.3: Describe the AI/ML development lifecycle.

The stages from business goal to a monitored model in production, the AWS tools for each stage, the habits of MLOps, and the model and business metrics used to judge the result.

Study it

  • The ML lifecycle and MLOps with SageMaker AI

    Lesson coming

  • Model metrics and business metrics: accuracy, precision, recall, F1, ROI

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the AWS documentation page that proves it.

Question 1 · choose 1

A data science team trains a new version of a model every week. It wants one place to catalog the versions, record each version's training metrics and approval status, and allow only approved versions to be deployed to production. Which Amazon SageMaker AI capability provides this?

  1. ASageMaker Feature Store
  2. BSageMaker Model Registry
  3. CSageMaker Data Wrangler
  4. DSageMaker JumpStart
Show the answer and why
  • ASageMaker Feature Store

    Incorrect

    Feature Store creates, stores and shares features, the inputs to models. It does not track model versions or approvals.

  • BSageMaker Model Registry

    Correct

    Model Registry catalogs models for production, manages model versions and their metadata such as training metrics, manages approval status and deploys models to production.

  • CSageMaker Data Wrangler

    Incorrect

    Data Wrangler imports, prepares, transforms and analyzes data before training. It is a data preparation tool, not a model catalog.

  • DSageMaker JumpStart

    Incorrect

    JumpStart provides pretrained, open-source models and solution templates to get started. It does not govern the team's own versions.

Versioning, approval and promotion of trained models belong to the deployment stage of the lifecycle, which SageMaker Model Registry handles.

Question 2 · choose 1

A hospital is evaluating a binary classification model that flags scans for a rare condition. Missing a patient who has the condition is far more costly than sending a healthy patient for an extra check. Which metric should the team focus on most?

  1. APrecision
  2. BAccuracy
  3. CRoot mean squared error (RMSE)
  4. DRecall
Show the answer and why
  • APrecision

    Incorrect

    Precision measures how many positive predictions were right. It matters most when a false positive is costly, which is the less expensive error here.

  • BAccuracy

    Incorrect

    Accuracy counts all correct predictions together. For a rare condition the data is imbalanced, and AWS points to balanced accuracy as the better measure in that case; neither targets missed positives directly.

  • CRoot mean squared error (RMSE)

    Incorrect

    RMSE measures the error of numeric predictions in regression. This is a classification problem.

  • DRecall

    Correct

    Recall is the share of actual positives the model finds, TP / (TP + FN). It is the metric to raise when false negatives are costly, such as in cancer testing.

Choose the metric by the cost of each error: costly false negatives call for recall, costly false positives for precision, and F1 balances the two.

Question 3 · choose 2

A company's data scientists train models in personal notebooks and copy the results to production by hand. Which practices would move the team toward MLOps? (Choose TWO.)

  1. AAutomate the stages from data preparation through training, validation and deployment so each run is repeatable
  2. BDeploy each model once and avoid retraining it, so that its behavior in production never changes
  3. CVersion the training code, data and models so that results can be reproduced and rolled back
  4. DKeep each scientist's experiments private to that person's notebook so the results stay independent
  5. EPromote a model to production as soon as an experiment finishes, without recording any test results
Show the answer and why
  • AAutomate the stages from data preparation through training, validation and deployment so each run is repeatable

    Correct

    MLOps automates the stages of the ML pipeline, from data ingestion and preprocessing through training and validation to deployment, for repeatability, consistency and scale.

  • BDeploy each model once and avoid retraining it, so that its behavior in production never changes

    Incorrect

    MLOps includes continuous monitoring and continuous training: models are retrained and redeployed as data and results change.

  • CVersion the training code, data and models so that results can be reproduced and rolled back

    Correct

    Version control of ML assets makes training reproducible and auditable and lets the team roll back to a previous version.

  • DKeep each scientist's experiments private to that person's notebook so the results stay independent

    Incorrect

    MLOps tracks experiments and manages training pipelines centrally; private, untracked experiments are the problem it replaces.

  • EPromote a model to production as soon as an experiment finishes, without recording any test results

    Incorrect

    MLOps adds automated testing and validation before release, so problems are found early and releases are auditable.

MLOps brings software delivery habits to ML: automated, repeatable pipelines, versioned assets, testing, and continuous monitoring with retraining.

Question 4 · choose 1

A startup wants to add text summarization to its product with a foundation model. It has no machine learning operations staff, does not want to manage servers or endpoints, and wants to pay only for what it uses. Which approach fits best?

  1. AHost an open-source model on Amazon EC2 instances that the startup patches and scales itself
  2. BDeploy a model from SageMaker JumpStart to a real-time endpoint backed by an instance type the startup chooses
  3. CCall a foundation model through Amazon Bedrock, a serverless managed API
  4. DPre-train a new foundation model on the startup's own documents before using it
Show the answer and why
  • AHost an open-source model on Amazon EC2 instances that the startup patches and scales itself

    Incorrect

    On Amazon EC2 the customer manages the guest operating system, patches and installed software, which is exactly the work the startup wants to avoid.

  • BDeploy a model from SageMaker JumpStart to a real-time endpoint backed by an instance type the startup chooses

    Incorrect

    This is a self-hosted endpoint: real-time inference uses a persistent endpoint backed by the instance type you choose, so there is an endpoint to size and run.

  • CCall a foundation model through Amazon Bedrock, a serverless managed API

    Correct

    Amazon Bedrock is serverless, so there is no infrastructure to manage, and one API gives access to foundation models from several providers.

  • DPre-train a new foundation model on the startup's own documents before using it

    Incorrect

    Foundation models are already trained on broad data and can perform general tasks such as summarization; pre-training a new one is the most expensive path and needs ML expertise.

A managed API service is the lowest-effort way to use a foundation model in production; self-hosting trades that convenience for more control.

Question 5 · choose 1

A data science team computes the same customer features separately in its training code and in its production application, and the two versions keep drifting apart. It wants one place to create, store and share features for both training and inference. Which Amazon SageMaker AI capability fits?

  1. ASageMaker Model Registry
  2. BSageMaker Feature Store
  3. CSageMaker JumpStart
  4. DAmazon SageMaker Canvas
Show the answer and why
  • ASageMaker Model Registry

    Incorrect

    Model Registry catalogs model versions and their approvals. It does not store feature values.

  • BSageMaker Feature Store

    Correct

    Feature Store creates, stores and shares features for training and inference and ingests them in a consistent way, which reduces training-serving skew.

  • CSageMaker JumpStart

    Incorrect

    JumpStart offers pretrained models and solution templates; it does not manage the team's features.

  • DAmazon SageMaker Canvas

    Incorrect

    Canvas lets users generate predictions without code. It is not a shared store of features for existing pipelines.

A feature store is the single source of truth for model inputs, so training and inference see the same values.

Question 6 · choose 1

A team runs data preparation, training, evaluation and model registration by hand, one notebook cell at a time. It wants these steps defined once as a workflow that runs the same way every time, on infrastructure it does not manage. Which Amazon SageMaker AI capability fits?

  1. ASageMaker Feature Store
  2. BSageMaker Serverless Inference
  3. CSageMaker Pipelines
  4. DSageMaker Model Cards
Show the answer and why
  • ASageMaker Feature Store

    Incorrect

    Feature Store stores and shares features. It does not orchestrate the steps of the workflow.

  • BSageMaker Serverless Inference

    Incorrect

    Serverless Inference hosts a trained model for predictions; it does not run training workflows.

  • CSageMaker Pipelines

    Correct

    Pipelines is a purpose-built workflow orchestration service that automates ML development on serverless infrastructure that SageMaker AI provisions and scales.

  • DSageMaker Model Cards

    Incorrect

    Model Cards document a model's intended use, risk and evaluation for governance; they do not run steps.

Repeatable, automated workflows are the core of MLOps, and in SageMaker AI they are built with Pipelines.

Question 7 · choose 1

A binary classifier for loan defaults is judged on two goals at once: it should catch most real defaults, and most of the applicants it flags should really default. The team wants a single score that balances both. Which metric fits?

  1. AF1 score
  2. BRecall
  3. CPrecision
  4. DRoot mean squared error (RMSE)
Show the answer and why
  • AF1 score

    Correct

    F1 is the harmonic mean of precision and recall, so it rises only when both are good.

  • BRecall

    Incorrect

    Recall covers only the first goal, finding the actual positives; on its own it can be maximized by flagging everyone.

  • CPrecision

    Incorrect

    Precision covers only the second goal, the quality of positive predictions.

  • DRoot mean squared error (RMSE)

    Incorrect

    RMSE measures the error of numeric predictions in regression, not the quality of class labels.

Recall asks "did we find them?", precision asks "were we right when we said so?", and F1 combines the two in one number.

Question 8 · choose 1

A demand forecasting model was accurate at launch, but its error has grown month after month as customer behavior changed. Which MLOps practice addresses this?

  1. AFreeze the model so its behavior stays the same over time
  2. BReplace the model with a larger one trained on the same historical data
  3. CEvaluate the model only once a year to reduce operating cost
  4. DMonitor model quality in production and retrain on recent data
Show the answer and why
  • AFreeze the model so its behavior stays the same over time

    Incorrect

    A frozen model keeps making the same mistakes as the world moves on; MLOps plans for monitoring and retraining instead.

  • BReplace the model with a larger one trained on the same historical data

    Incorrect

    A bigger model trained on the same old data does not learn the new behavior the forecasts are missing.

  • CEvaluate the model only once a year to reduce operating cost

    Incorrect

    Less frequent evaluation would hide the problem longer; MLOps calls for continuous monitoring.

  • DMonitor model quality in production and retrain on recent data

    Correct

    MLOps includes continuous monitoring with metrics and continuous training that automatically retrains models for redeployment, and monitoring events can trigger retraining.

Models decay as the world changes. Monitoring catches it, and automated retraining on fresh data fixes it.

Question 9 · choose 2

A team reports on its new recommendation model. Which of the following are business metrics rather than model performance metrics? (Choose TWO.)

  1. APrecision of the model's predictions
  2. BCost per user of running the model
  3. CF1 score on the test set
  4. DReturn on investment of the project
  5. EAccuracy on the validation set
Show the answer and why
  • APrecision of the model's predictions

    Incorrect

    Precision is a model performance metric: the share of positive predictions that are correct.

  • BCost per user of running the model

    Correct

    The exam guide lists cost per user among the business metrics used to evaluate ML models.

  • CF1 score on the test set

    Incorrect

    F1 is a model performance metric that combines precision and recall.

  • DReturn on investment of the project

    Correct

    Return on investment (ROI) is one of the business metrics the exam guide lists.

  • EAccuracy on the validation set

    Incorrect

    Accuracy is a model performance metric: the ratio of correct predictions to all predictions.

Model metrics say how well the model predicts; business metrics say whether the project pays off. A good model can still be a bad investment.

Practise domain 1 →Practise all domains →