Machine Learning Interview Questions and Answers

Supervised learning uses labelled examples, learning a mapping from inputs to known outputs. Tasks include classification (predict a category, such as spam or not) and regression (predict a number, such as price). Algorithms include linear and logistic regression, decision trees, gradient boosting and neural networks.

Unsupervised learning works with unlabelled data, finding structure. Tasks include clustering (grouping similar customers with k-means), dimensionality reduction (PCA, t-SNE) and anomaly detection. There is no ground-truth label to score against.

A third setting, self-supervised learning, generates labels from the data itself and underpins modern language and vision models. Reinforcement learning is another paradigm where an agent learns from reward signals.

Choose supervised when you have reliable labels and a clear target; choose unsupervised for exploration and segmentation when labels are unavailable or expensive.

Overfitting is when a model learns noise and idiosyncrasies of the training set instead of the underlying pattern. It performs well on training data but poorly on new data. Signs include a large gap between training and validation metrics and unstable performance across folds.

Prevention:

• More data and data augmentation.
• Simpler models or fewer features.
• Regularization: L1 or L2 for linear models, weight decay for neural networks.
• Early stopping based on validation loss.
• Cross-validation to detect instability.
• Dropout and batch normalization for deep networks.
• Pruning and depth or leaf limits for trees.
• Ensembling and bagging to reduce variance.

The counterpart is underfitting, where the model is too simple and misses signal. Balance the two using the bias-variance tradeoff and always keep a held-out test set for the final check.

Choose metrics that match the problem.

• MAE: mean absolute error, in the target's units, robust to outliers.
• MSE and RMSE: penalize large errors more; RMSE is in target units and sensitive to outliers.
• R-squared: fraction of variance explained, but it always rises with more features, so use adjusted R-squared.
• MAPE: percentage error, useful across scales but undefined when the actual is zero.

from sklearn.metrics import mean_absolute_error, root_mean_squared_error, r2_score

Always compare against a baseline, such as predicting the mean or the previous value. Inspect residual plots for patterns, heteroscedasticity and non-linearity, and check errors across segments, not just overall. Use cross-validation for stable estimates and a held-out test set for the final report.

Bias is error from overly simple assumptions; a high-bias model underfits and misses patterns in both training and test data. Variance is error from sensitivity to the training sample; a high-variance model overfits, fitting noise so training error is low but test error is high.

Expected error decomposes into bias squared, variance and irreducible noise. As model complexity increases, bias falls and variance rises, so the sweet spot minimizes total error.

Diagnose with cross-validation: high bias means both training and validation error are high, so add features or complexity or reduce regularization. High variance means a large gap between low training error and high validation error, so get more data, simplify the model, add regularization, or use bagging. Regularization, early stopping, pruning and ensembling are the main levers.

Class imbalance means one class is far rarer, so accuracy becomes misleading and models favor the majority.

Strategies:

• Resampling: oversample the minority (SMOTE synthesizes examples) or undersample the majority. Do this only on training folds, never before splitting.
• Class weights: set class_weight="balanced" or scale the loss so minority errors cost more.
• Threshold tuning: move the decision threshold to optimize the metric you care about, such as recall or F1.
• Metric choice: use precision-recall AUC, F1 or cost-based measures instead of accuracy or ROC-AUC.
• Anomaly framing: treat the rare class with one-class or isolation methods.
• Collect more minority data if feasible.

model = LogisticRegression(class_weight="balanced")

Always evaluate on a stratified holdout or with stratified cross-validation so each fold keeps the class ratio.

Precision is the fraction of predicted positives that are truly positive: TP / (TP + FP). Recall, or sensitivity, is the fraction of actual positives that were found: TP / (TP + FN).

Prefer precision when false positives are costly: a spam filter that flags real email, a fraud alert that blocks a legitimate purchase, or a recommendation that annoys users.

Prefer recall when false negatives are costly: cancer screening, intrusion detection, or catching defective products before shipping.

F1 is the harmonic mean of the two, useful when you need a single balanced number. There is always a tradeoff: lowering the decision threshold generally raises recall and lowers precision.

from sklearn.metrics import precision_recall_curve
p, r, thresholds = precision_recall_curve(y_true, y_scores)

Choose the operating point from the business cost of each error type, not from a default threshold.

Cross-validation repeatedly splits data into training and validation folds to estimate how well a model generalizes. In k-fold, the data is split into k parts; the model trains on k-1 and validates on the remaining one, rotating so every point is validated once. The average score is the estimate.

from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5, scoring="roc_auc")

Variants: stratified k-fold preserves class ratios for imbalanced data; grouped k-fold keeps all rows of one entity together to avoid leakage; time series split respects temporal order.

Use it for model selection, hyperparameter tuning and comparing algorithms. Pitfalls: preprocessing such as scaling or SMOTE must be fit inside each fold via a pipeline, otherwise information leaks and the score is optimistic. Keep a separate untouched test set for the final estimate.

Regularization adds a penalty on model complexity to the loss, discouraging large weights and reducing overfitting.

L2 (ridge) adds the sum of squared weights. It shrinks weights smoothly toward zero but rarely makes them exactly zero, and it handles correlated features well.

L1 (lasso) adds the sum of absolute weights. It drives some coefficients exactly to zero, performing feature selection automatically, which helps with sparse or high-dimensional data.

Elastic net combines both, with alpha controlling overall strength and l1_ratio the mix.

Ridge(alpha=1.0)
Lasso(alpha=0.01)
ElasticNet(alpha=0.1, l1_ratio=0.5)

The strength parameter must be tuned by cross-validation. In neural networks L2 appears as weight decay, and dropout is another regularization technique. Regularization always trades a little training fit for better generalization.

Gradient descent minimizes a loss function by repeatedly moving parameters in the direction that decreases the loss. The gradient is the vector of partial derivatives, and you step opposite to it.

for epoch in range(epochs):
    preds = X @ w + b
    error = preds - y
    grad_w = (2/n) * X.T @ error
    grad_b = (2/n) * error.sum()
    w -= lr * grad_w
    b -= lr * grad_b

The learning rate is critical: too large and the loss diverges or oscillates, too small and training is slow. Variants: batch gradient descent uses all data per step (stable, slow), stochastic uses one example (noisy, fast), and mini-batch uses a small batch and is the standard. Momentum and Adam adapt the step per parameter. Always scale features, monitor the loss curve, and consider schedules or early stopping.

Both are tree ensembles, differing in how trees are combined.

Random forest builds many deep, independent trees on bootstrap samples with random feature subsets, then averages or votes. Trees train in parallel and reduce variance. It is robust, hard to overfit, and needs little tuning.

Gradient boosting builds trees sequentially, each fitting the residual errors of the current ensemble. It reduces bias and often achieves higher accuracy, but is sensitive to hyperparameters and can overfit without regularization. Modern implementations add shrinkage, subsampling and column sampling.

RandomForestRegressor(n_estimators=500, max_features="sqrt")
GradientBoostingRegressor(n_estimators=500, learning_rate=0.05, max_depth=3)

Use a forest as a strong baseline with minimal tuning; use boosting when you need the best tabular performance. HistGradientBoosting, XGBoost and LightGBM are fast defaults.

MLOps applies DevOps practices to machine learning: versioning data, code and models; automating training and deployment; and monitoring models in production.

Key components:

• Experiment tracking and reproducibility, for example MLflow or Weights and Biases.
• Feature stores for consistent training and serving features.
• CI/CD for retraining, testing and rollout with canary or shadow deployment.
• A model registry with lineage and approval.
• Monitoring for performance, latency and cost.

Monitoring matters because models decay. Data drift is when input distributions shift away from training data; concept drift is when the relationship between inputs and target changes. Both degrade accuracy even when the code is unchanged.

Alert on drift, prediction distributions and business KPIs, and define a retraining trigger. Labels may arrive late, so use proxies until ground truth is available.

Data leakage is when information unavailable at prediction time seeps into training, producing optimistic validation scores that collapse in production.

Common sources:

• Target leakage: a feature that is a proxy for the label, such as account_closed_date when predicting churn.
• Train-test contamination: fitting scalers, imputers, target encoders or SMOTE on the full dataset before splitting.
• Temporal leakage: using future data to predict the past, as with random splitting of time series.
• Group leakage: the same entity appears in both train and test.

Prevention:

• Split first, then fit all preprocessing on training folds only, ideally inside a pipeline.
• Use time-based splits for temporal problems.
• Audit features for whether they are known at prediction time.
• Use grouped splits when rows share an entity.

Pipeline([("scaler", StandardScaler()), ("clf", LogisticRegression())])

Leakage often shows as suspiciously high accuracy, so treat such results with suspicion.

ROC-AUC is the probability that a randomly chosen positive example is ranked above a randomly chosen negative one. It summarizes performance across all thresholds, and 0.5 is random.

from sklearn.metrics import roc_auc_score
roc_auc_score(y_true, y_scores)

Misleading cases:

• Heavy class imbalance: ROC-AUC can look strong while precision is terrible, because the large negative class dominates the false-positive rate. Precision-recall AUC is more informative when the positive class is rare.
• Threshold independence: AUC says nothing about the operating point you will deploy.
• Calibration: a model can rank well but produce badly calibrated probabilities, which matters when decisions depend on the probability value.
• Ranking versus cost: if errors have asymmetric costs, AUC ignores them.

Report AUC alongside PR-AUC, calibration curves and the confusion matrix at the chosen threshold.

Start with the objective and data: implicit signals such as views, clicks and purchases, plus explicit ratings and item or user features.

A common two-stage design:

• Candidate generation: retrieve a few hundred relevant items quickly. Use collaborative filtering such as matrix factorization or ALS, content-based similarity, or approximate nearest neighbour search over embeddings. This scales to millions of items.
• Ranking: score candidates with a richer model, gradient boosting or a neural network, using user, item and context features to optimize the target event.

user_vec = als_model.user_factors[user_id]
scores = item_factors @ user_vec
candidates = np.argpartition(scores, -200)[-200:]

Add business rules for diversity, freshness and filtering. Evaluate offline with recall@k and NDCG, then run online A/B tests on engagement and revenue. Address the cold start problem with popularity or content features, and monitor feedback loops and drift.