Machine Learning Associate · Domain 3: Model Development · 31% of the exam
Task 3.1: Use ML foundations to select the appropriate algorithm for a given model scenario; Identify methods to mitigate data imbalance in training data; Compare estimators and transformers; Develop a training pipeline
Matching a problem to regression, classification, trees or clustering, class weights and resampling for imbalance, fit versus transform in Spark ML, and Pipeline stages from raw columns to a fitted model.
Imbalanced training data: class weights and resampling
Lesson coming
Spark ML estimators, transformers and training pipelines
Lesson coming
Sample questions
Try each one before opening the answer. Every option is explained, with the Databricks documentation page that proves it.
Question 1 · choose 1
Using pyspark.ml on Databricks, a team must predict which of five subscription plans each customer will choose, and it needs a probability for every plan. Which estimator configuration fits?
ALogisticRegression with family="multinomial"
BLogisticRegression with family="binomial"
CLinearRegression trained on the plan index as the label
DKMeans with k=5 on the customer features
Show the answer and why
ALogisticRegression with family="multinomial"
Correct
Multinomial logistic regression predicts a multiclass outcome and gives a probability for each class.
BLogisticRegression with family="binomial"
Incorrect
Binomial logistic regression predicts a binary outcome, not one of five classes.
CLinearRegression trained on the plan index as the label
Incorrect
Linear regression predicts a continuous value, not a class with class probabilities.
DKMeans with k=5 on the customer features
Incorrect
k-means groups data points into a chosen number of clusters without using the labels, so it does not learn the plan customers choose.
In spark.ml, logistic regression predicts a binary outcome with the binomial family or a multiclass outcome with the multinomial family, and its model outputs class probabilities. Linear regression predicts a continuous response, and k-means is a clustering algorithm that does not learn from labels.
On Databricks Runtime 17.3 LTS ML, an AutoML classification run on a fraud dataset with 2% positives notes that it handled the imbalance. A reviewer finds that the test split still has about 2% positives and asks whether the team should rebalance it before trusting the reported test metrics. What is the right answer?
AYes; rerun with a manual split column whose test rows are chosen to have equal numbers of each class
BYes; set sample_weight_col so that AutoML also reweights the validation and test splits
CNo; AutoML does not handle class imbalance, so the splits simply show the raw data
DNo; AutoML downsamples the majority class and adds class weights in the training split only, so test metrics reflect the true distribution
Show the answer and why
AYes; rerun with a manual split column whose test rows are chosen to have equal numbers of each class
Incorrect
AutoML deliberately keeps the test and validation splits at the true class distribution, so a balanced test split would misstate performance.
BYes; set sample_weight_col so that AutoML also reweights the validation and test splits
Incorrect
Sample weights adjust the importance of each class during model training; they do not rebalance the evaluation splits.
CNo; AutoML does not handle class imbalance, so the splits simply show the raw data
Incorrect
Since Databricks Runtime 11.3 LTS ML, AutoML reduces the imbalance of the training split when it detects one.
DNo; AutoML downsamples the majority class and adds class weights in the training split only, so test metrics reflect the true distribution
Correct
AutoML balances only the training dataset, so the model is always evaluated on data with the true class distribution.
When AutoML detects an imbalanced classification dataset, it downsamples the major class or classes and adds class weights, scaled so that the model stays calibrated, but only in the training dataset. The validation and test datasets keep the true input class distribution, which is what the reported metrics should reflect.
A data scientist sorts the stages of a pyspark.ml workflow into those she must fit on training data and those that only transform a DataFrame. Which two of these are Estimators? (Choose TWO.)
AVectorAssembler
BStringIndexer
CHashingTF
DLogisticRegression
ETokenizer
Show the answer and why
AVectorAssembler
Incorrect
VectorAssembler is a feature transformer that merges columns into a vector; it has nothing to learn.
BStringIndexer
Correct
StringIndexer is fitted on data to learn the label-to-index mapping, and fit() returns a StringIndexerModel.
CHashingTF
Incorrect
HashingTF is a Transformer that maps words to feature vectors through its transform() method.
DLogisticRegression
Correct
A learning algorithm such as LogisticRegression is an Estimator; fit() trains a LogisticRegressionModel.
ETokenizer
Incorrect
Tokenizer is a Transformer that splits text into words through its transform() method.
In Spark ML, a Transformer implements transform() to turn one DataFrame into another, while an Estimator implements fit(), which learns from a DataFrame and produces a Model, itself a Transformer. StringIndexer and learning algorithms such as LogisticRegression are Estimators; Tokenizer, HashingTF and VectorAssembler are Transformers.
A training pipeline is defined as follows; city is a string column and age and income are numeric.
indexer = StringIndexer(inputCol="city", outputCol="city_idx"); assembler = VectorAssembler(inputCols=["city_idx", "age", "income"], outputCol="features"); pipeline = Pipeline(stages=[assembler, indexer, lr])
What happens when pipeline.fit(train_df) runs?
AIt succeeds, because a Pipeline orders its stages by their input and output column names
BIt fails, because the stages run in the listed order and city_idx does not exist yet when VectorAssembler runs
CIt succeeds, because VectorAssembler treats the missing city_idx column as invalid data and skips it under its default setting
DIt succeeds, but VectorAssembler puts the raw city strings into the feature vector
Show the answer and why
AIt succeeds, because a Pipeline orders its stages by their input and output column names
Incorrect
The data flow is read from column names, but the stages must be listed in topological order; they run in the order given.
BIt fails, because the stages run in the listed order and city_idx does not exist yet when VectorAssembler runs
Correct
Stages run in order, and the Pipeline checks the DataFrame schema at run time, so the missing city_idx column stops the fit.
CIt succeeds, because VectorAssembler treats the missing city_idx column as invalid data and skips it under its default setting
Incorrect
VectorAssembler's handleInvalid deals with NULL and NaN values in its input columns, not with a column that is missing.
DIt succeeds, but VectorAssembler puts the raw city strings into the feature vector
Incorrect
VectorAssembler accepts only numeric, boolean and vector input columns, not strings.
A Pipeline is an ordered array of stages, and the input DataFrame passes through them in order; even a non-linear pipeline must list its stages in topological order. Pipelines check types at run time against the DataFrame schema. Here StringIndexer must come before VectorAssembler so that city_idx exists when the assembler runs.
After pipeline.fit(train_df) returns a PipelineModel, a data scientist must score test_df with exactly the same feature processing as training. What should she do?
ACall transform(test_df) on the fitted PipelineModel
BCall pipeline.fit(test_df) and then transform test_df with the result
CRe-create each stage and fit it on test_df
DTransform test_df stage by stage with newly created, unfitted stage objects
Show the answer and why
ACall transform(test_df) on the fitted PipelineModel
Correct
The PipelineModel passes test data through the fitted stages in order, so training and test data get identical feature processing.
BCall pipeline.fit(test_df) and then transform test_df with the result
Incorrect
Fitting on the test data learns new feature mappings and a new model instead of reusing the trained ones.
CRe-create each stage and fit it on test_df
Incorrect
Refitting the stages on test data breaks the identical processing that the PipelineModel guarantees.
DTransform test_df stage by stage with newly created, unfitted stage objects
Incorrect
New stage objects have not learned anything from the training data, so the processing would not match training.
At test time, PipelineModel.transform() passes the data through the fitted stages in order, each stage updating the dataset for the next one, which ensures that training and test data go through identical feature processing.
In a notebook, docs_df has the columns id and text. A data scientist runs words_df = Tokenizer(inputCol="text", outputCol="words").transform(docs_df). What is words_df?
Adocs_df itself, now holding an added words column
BA fitted TokenizerModel that can be applied to other DataFrames
CA DataFrame with only the words column
DA new DataFrame with id, text and the appended words column
Show the answer and why
Adocs_df itself, now holding an added words column
Incorrect
transform() outputs a new DataFrame; it does not change the input DataFrame in place.
BA fitted TokenizerModel that can be applied to other DataFrames
Incorrect
Tokenizer is a Transformer, and transform() returns a DataFrame, not a model.
CA DataFrame with only the words column
Incorrect
The output column is appended to the existing columns; id and text are kept.
DA new DataFrame with id, text and the appended words column
Correct
A Transformer's transform() converts one DataFrame into another, generally by appending columns, here words.
A Transformer implements transform(), which converts one DataFrame into another, generally by appending one or more columns: a feature transformer reads a column such as text, maps it into a new column and outputs a new DataFrame with the mapped column appended. The Tokenizer API example shows a result row that holds both text and words.
A single deep decision tree fits its training data closely. According to Spark's MLlib guide, which tree-based algorithm combines many decision trees to reduce the risk of overfitting?
ARandom forests of decision trees
BA single decision tree with more bins
CK-means clustering
DLinear regression
Show the answer and why
ARandom forests of decision trees
Correct
Random forests are ensembles of decision trees that combine many trees to reduce the risk of overfitting.
BA single decision tree with more bins
Incorrect
More bins change how continuous features are discretized; it is still one tree.
CK-means clustering
Incorrect
K-means is a clustering algorithm, not an ensemble of trees.
DLinear regression
Incorrect
Linear regression is not a tree-based ensemble.
Random forests are ensembles of decision trees, combined to reduce the risk of overfitting. Spark's implementation supports binary and multiclass classification and regression with continuous and categorical features.
A team models how long customers stay subscribed. For customers who are still active, the end date is not yet known (the data is right-censored). Which spark.ml algorithm is designed for censored data?
ALogistic regression
BSurvival regression (AFT model)
CK-means clustering
DLinear regression on the observed durations
Show the answer and why
ALogistic regression
Incorrect
Logistic regression predicts a categorical outcome; it does not handle censored durations.
BSurvival regression (AFT model)
Correct
Spark's survival regression implements the accelerated failure time model, a parametric survival model for censored data.
CK-means clustering
Incorrect
K-means groups data points; it does not model durations.
DLinear regression on the observed durations
Incorrect
Ordinary linear regression treats censored durations as complete observations.
In spark.ml, survival regression implements the accelerated failure time (AFT) model, a parametric survival regression model for censored data, such as subscription lengths of customers who have not churned yet.
A data scientist adds a column weight to the training data, larger for rows of the rare class, and wants pyspark.ml's LogisticRegression to use it during training. Which parameter should she set?
AweightCol="weight"
Bthreshold=0.1
CregParam=0.5
Dfamily="multinomial"
Show the answer and why
AweightCol="weight"
Correct
weightCol names the instance weight column; without it all weights are treated as 1.0.
Bthreshold=0.1
Incorrect
threshold sets the binary prediction cutoff; it does not weight training rows.
CregParam=0.5
Incorrect
regParam sets the regularization strength, not per-row weights.
Dfamily="multinomial"
Incorrect
family chooses the label distribution model, not instance weights.
LogisticRegression's weightCol parameter names the column of instance weights; if it is not set or empty, all instance weights are treated as 1.0. Giving rare-class rows larger weights is one way to reduce the effect of class imbalance in training.
A helper function returns the same HashingTF instance, and a data scientist puts that one instance into a Pipeline's stages list twice. What does the ML Pipelines guide say?
AThis is fine; Spark runs the instance twice
BSpark removes the duplicate stage automatically
CIt is allowed only if the two uses have different outputCol values
DTwo separate HashingTF instances, since stages must be unique
Show the answer and why
AThis is fine; Spark runs the instance twice
Incorrect
Pipeline stages must have unique IDs, so one instance cannot appear twice.
BSpark removes the duplicate stage automatically
Incorrect
The guide says stages should be unique instances; it does not describe removing duplicates.
CIt is allowed only if the two uses have different outputCol values
Incorrect
One instance has one set of parameters and one ID, so it cannot be two stages.
DTwo separate HashingTF instances, since stages must be unique
Correct
Each stage needs a unique ID; two different instances of HashingTF get different IDs and can both be used.
A Pipeline's stages should be unique instances: the same instance should not be inserted twice, because stages must have unique IDs. Different instances of the same class, such as myHashingTF1 and myHashingTF2, have different IDs and can be put into the same Pipeline.