Skip to content

Machine Learning Associate · Domain 3: Model Development · 31% of the exam

Task 3.1: Use ML foundations to select the appropriate algorithm for a given model scenario; Identify methods to mitigate data imbalance in training data; Compare estimators and transformers; Develop a training pipeline

Matching a problem to regression, classification, trees or clustering, class weights and resampling for imbalance, fit versus transform in Spark ML, and Pipeline stages from raw columns to a fitted model.

Study it

  • Choosing an algorithm for a model scenario

    Partly covered by: What Is Machine Learning

  • Imbalanced training data: class weights and resampling

    Lesson coming

  • Spark ML estimators, transformers and training pipelines

    Lesson coming

Sample questions

Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.

Question 1 · choose 1

Using pyspark.ml on Databricks, a team must predict which of five subscription plans each customer will choose, and it needs a probability for every plan. Which estimator configuration fits?

  1. ALogisticRegression with family="multinomial"
  2. BLogisticRegression with family="binomial"
  3. CLinearRegression trained on the plan index as the label
  4. DKMeans with k=5 on the customer features
Show the answer and why
  • ALogisticRegression with family="multinomial"

    Correct

    Multinomial logistic regression predicts a multiclass outcome and gives a probability for each class.

  • BLogisticRegression with family="binomial"

    Incorrect

    Binomial logistic regression predicts a binary outcome, not one of five classes.

  • CLinearRegression trained on the plan index as the label

    Incorrect

    Linear regression predicts a continuous value, not a class with class probabilities.

  • DKMeans with k=5 on the customer features

    Incorrect

    k-means groups data points into a chosen number of clusters without using the labels, so it does not learn the plan customers choose.

In spark.ml, logistic regression predicts a binary outcome with the binomial family or a multiclass outcome with the multinomial family, and its model outputs class probabilities. Linear regression predicts a continuous response, and k-means is a clustering algorithm that does not learn from labels.

Question 2 · choose 1

On Databricks Runtime 17.3 LTS ML, an AutoML classification run on a fraud dataset with 2% positives notes that it handled the imbalance. A reviewer finds that the test split still has about 2% positives and asks whether the team should rebalance it before trusting the reported test metrics. What is the right answer?

  1. AYes; rerun with a manual split column whose test rows are chosen to have equal numbers of each class
  2. BYes; set sample_weight_col so that AutoML also reweights the validation and test splits
  3. CNo; AutoML does not handle class imbalance, so the splits simply show the raw data
  4. DNo; AutoML downsamples the majority class and adds class weights in the training split only, so test metrics reflect the true distribution
Show the answer and why
  • AYes; rerun with a manual split column whose test rows are chosen to have equal numbers of each class

    Incorrect

    AutoML deliberately keeps the test and validation splits at the true class distribution, so a balanced test split would misstate performance.

  • BYes; set sample_weight_col so that AutoML also reweights the validation and test splits

    Incorrect

    Sample weights adjust the importance of each class during model training; they do not rebalance the evaluation splits.

  • CNo; AutoML does not handle class imbalance, so the splits simply show the raw data

    Incorrect

    Since Databricks Runtime 11.3 LTS ML, AutoML reduces the imbalance of the training split when it detects one.

  • DNo; AutoML downsamples the majority class and adds class weights in the training split only, so test metrics reflect the true distribution

    Correct

    AutoML balances only the training dataset, so the model is always evaluated on data with the true class distribution.

When AutoML detects an imbalanced classification dataset, it downsamples the major class or classes and adds class weights, scaled so that the model stays calibrated, but only in the training dataset. The validation and test datasets keep the true input class distribution, which is what the reported metrics should reflect.

Question 3 · choose 2

A data scientist sorts the stages of a pyspark.ml workflow into those she must fit on training data and those that only transform a DataFrame. Which two of these are Estimators? (Choose TWO.)

  1. AVectorAssembler
  2. BStringIndexer
  3. CHashingTF
  4. DLogisticRegression
  5. ETokenizer
Show the answer and why
  • AVectorAssembler

    Incorrect

    VectorAssembler is a feature transformer that merges columns into a vector; it has nothing to learn.

  • BStringIndexer

    Correct

    StringIndexer is fitted on data to learn the label-to-index mapping, and fit() returns a StringIndexerModel.

  • CHashingTF

    Incorrect

    HashingTF is a Transformer that maps words to feature vectors through its transform() method.

  • DLogisticRegression

    Correct

    A learning algorithm such as LogisticRegression is an Estimator; fit() trains a LogisticRegressionModel.

  • ETokenizer

    Incorrect

    Tokenizer is a Transformer that splits text into words through its transform() method.

In Spark ML, a Transformer implements transform() to turn one DataFrame into another, while an Estimator implements fit(), which learns from a DataFrame and produces a Model, itself a Transformer. StringIndexer and learning algorithms such as LogisticRegression are Estimators; Tokenizer, HashingTF and VectorAssembler are Transformers.

Question 4 · choose 1

A training pipeline is defined as follows; city is a string column and age and income are numeric. indexer = StringIndexer(inputCol="city", outputCol="city_idx"); assembler = VectorAssembler(inputCols=["city_idx", "age", "income"], outputCol="features"); pipeline = Pipeline(stages=[assembler, indexer, lr]) What happens when pipeline.fit(train_df) runs?

  1. AIt succeeds, because a Pipeline orders its stages by their input and output column names
  2. BIt fails, because the stages run in the listed order and city_idx does not exist yet when VectorAssembler runs
  3. CIt succeeds, because VectorAssembler treats the missing city_idx column as invalid data and skips it under its default setting
  4. DIt succeeds, but VectorAssembler puts the raw city strings into the feature vector
Show the answer and why
  • AIt succeeds, because a Pipeline orders its stages by their input and output column names

    Incorrect

    The data flow is read from column names, but the stages must be listed in topological order; they run in the order given.

  • BIt fails, because the stages run in the listed order and city_idx does not exist yet when VectorAssembler runs

    Correct

    Stages run in order, and the Pipeline checks the DataFrame schema at run time, so the missing city_idx column stops the fit.

  • CIt succeeds, because VectorAssembler treats the missing city_idx column as invalid data and skips it under its default setting

    Incorrect

    VectorAssembler's handleInvalid deals with NULL and NaN values in its input columns, not with a column that is missing.

  • DIt succeeds, but VectorAssembler puts the raw city strings into the feature vector

    Incorrect

    VectorAssembler accepts only numeric, boolean and vector input columns, not strings.

A Pipeline is an ordered array of stages, and the input DataFrame passes through them in order; even a non-linear pipeline must list its stages in topological order. Pipelines check types at run time against the DataFrame schema. Here StringIndexer must come before VectorAssembler so that city_idx exists when the assembler runs.

Question 5 · choose 1

After pipeline.fit(train_df) returns a PipelineModel, a data scientist must score test_df with exactly the same feature processing as training. What should she do?

  1. ACall transform(test_df) on the fitted PipelineModel
  2. BCall pipeline.fit(test_df) and then transform test_df with the result
  3. CRe-create each stage and fit it on test_df
  4. DTransform test_df stage by stage with newly created, unfitted stage objects
Show the answer and why
  • ACall transform(test_df) on the fitted PipelineModel

    Correct

    The PipelineModel passes test data through the fitted stages in order, so training and test data get identical feature processing.

  • BCall pipeline.fit(test_df) and then transform test_df with the result

    Incorrect

    Fitting on the test data learns new feature mappings and a new model instead of reusing the trained ones.

  • CRe-create each stage and fit it on test_df

    Incorrect

    Refitting the stages on test data breaks the identical processing that the PipelineModel guarantees.

  • DTransform test_df stage by stage with newly created, unfitted stage objects

    Incorrect

    New stage objects have not learned anything from the training data, so the processing would not match training.

At test time, PipelineModel.transform() passes the data through the fitted stages in order, each stage updating the dataset for the next one, which ensures that training and test data go through identical feature processing.

Databricks documentation

Question 6 · choose 1

In a notebook, docs_df has the columns id and text. A data scientist runs words_df = Tokenizer(inputCol="text", outputCol="words").transform(docs_df). What is words_df?

  1. Adocs_df itself, now holding an added words column
  2. BA fitted TokenizerModel that can be applied to other DataFrames
  3. CA DataFrame with only the words column
  4. DA new DataFrame with id, text and the appended words column
Show the answer and why
  • Adocs_df itself, now holding an added words column

    Incorrect

    transform() outputs a new DataFrame; it does not change the input DataFrame in place.

  • BA fitted TokenizerModel that can be applied to other DataFrames

    Incorrect

    Tokenizer is a Transformer, and transform() returns a DataFrame, not a model.

  • CA DataFrame with only the words column

    Incorrect

    The output column is appended to the existing columns; id and text are kept.

  • DA new DataFrame with id, text and the appended words column

    Correct

    A Transformer's transform() converts one DataFrame into another, generally by appending columns, here words.

A Transformer implements transform(), which converts one DataFrame into another, generally by appending one or more columns: a feature transformer reads a column such as text, maps it into a new column and outputs a new DataFrame with the mapped column appended. The Tokenizer API example shows a result row that holds both text and words.

Question 7 · choose 1

A single deep decision tree fits its training data closely. According to Spark's MLlib guide, which tree-based algorithm combines many decision trees to reduce the risk of overfitting?

  1. ARandom forests of decision trees
  2. BA single decision tree with more bins
  3. CK-means clustering
  4. DLinear regression
Show the answer and why
  • ARandom forests of decision trees

    Correct

    Random forests are ensembles of decision trees that combine many trees to reduce the risk of overfitting.

  • BA single decision tree with more bins

    Incorrect

    More bins change how continuous features are discretized; it is still one tree.

  • CK-means clustering

    Incorrect

    K-means is a clustering algorithm, not an ensemble of trees.

  • DLinear regression

    Incorrect

    Linear regression is not a tree-based ensemble.

Random forests are ensembles of decision trees, combined to reduce the risk of overfitting. Spark's implementation supports binary and multiclass classification and regression with continuous and categorical features.

Question 8 · choose 1

A team models how long customers stay subscribed. For customers who are still active, the end date is not yet known (the data is right-censored). Which spark.ml algorithm is designed for censored data?

  1. ALogistic regression
  2. BSurvival regression (AFT model)
  3. CK-means clustering
  4. DLinear regression on the observed durations
Show the answer and why
  • ALogistic regression

    Incorrect

    Logistic regression predicts a categorical outcome; it does not handle censored durations.

  • BSurvival regression (AFT model)

    Correct

    Spark's survival regression implements the accelerated failure time model, a parametric survival model for censored data.

  • CK-means clustering

    Incorrect

    K-means groups data points; it does not model durations.

  • DLinear regression on the observed durations

    Incorrect

    Ordinary linear regression treats censored durations as complete observations.

In spark.ml, survival regression implements the accelerated failure time (AFT) model, a parametric survival regression model for censored data, such as subscription lengths of customers who have not churned yet.

Question 9 · choose 1

A data scientist adds a column weight to the training data, larger for rows of the rare class, and wants pyspark.ml's LogisticRegression to use it during training. Which parameter should she set?

  1. AweightCol="weight"
  2. Bthreshold=0.1
  3. CregParam=0.5
  4. Dfamily="multinomial"
Show the answer and why
  • AweightCol="weight"

    Correct

    weightCol names the instance weight column; without it all weights are treated as 1.0.

  • Bthreshold=0.1

    Incorrect

    threshold sets the binary prediction cutoff; it does not weight training rows.

  • CregParam=0.5

    Incorrect

    regParam sets the regularization strength, not per-row weights.

  • Dfamily="multinomial"

    Incorrect

    family chooses the label distribution model, not instance weights.

LogisticRegression's weightCol parameter names the column of instance weights; if it is not set or empty, all instance weights are treated as 1.0. Giving rare-class rows larger weights is one way to reduce the effect of class imbalance in training.

Question 10 · choose 1

A helper function returns the same HashingTF instance, and a data scientist puts that one instance into a Pipeline's stages list twice. What does the ML Pipelines guide say?

  1. AThis is fine; Spark runs the instance twice
  2. BSpark removes the duplicate stage automatically
  3. CIt is allowed only if the two uses have different outputCol values
  4. DTwo separate HashingTF instances, since stages must be unique
Show the answer and why
  • AThis is fine; Spark runs the instance twice

    Incorrect

    Pipeline stages must have unique IDs, so one instance cannot appear twice.

  • BSpark removes the duplicate stage automatically

    Incorrect

    The guide says stages should be unique instances; it does not describe removing duplicates.

  • CIt is allowed only if the two uses have different outputCol values

    Incorrect

    One instance has one set of parameters and one ID, so it cannot be two stages.

  • DTwo separate HashingTF instances, since stages must be unique

    Correct

    Each stage needs a unique ID; two different instances of HashingTF get different IDs and can both be used.

A Pipeline's stages should be unique instances: the same instance should not be inserted twice, because stages must have unique IDs. Different instances of the same class, such as myHashingTF1 and myHashingTF2, have different IDs and can be put into the same Pipeline.

Databricks documentation

Practice domain 3 →Practice all domains →