Machine Learning Interview Questions and Answers

Bias is error from overly simple assumptions; a high-bias model underfits and misses patterns in both training and test data. Variance is error from sensitivity to the training sample; a high-variance model overfits, fitting noise so training error is low but test error is high.

Expected error decomposes into bias squared, variance and irreducible noise. As model complexity increases, bias falls and variance rises, so the sweet spot minimizes total error.

Diagnose with cross-validation: high bias means both training and validation error are high, so add features or complexity or reduce regularization. High variance means a large gap between low training error and high validation error, so get more data, simplify the model, add regularization, or use bagging. Regularization, early stopping, pruning and ensembling are the main levers.

Class imbalance means one class is far rarer, so accuracy becomes misleading and models favor the majority.

Strategies:

• Resampling: oversample the minority (SMOTE synthesizes examples) or undersample the majority. Do this only on training folds, never before splitting.
• Class weights: set class_weight="balanced" or scale the loss so minority errors cost more.
• Threshold tuning: move the decision threshold to optimize the metric you care about, such as recall or F1.
• Metric choice: use precision-recall AUC, F1 or cost-based measures instead of accuracy or ROC-AUC.
• Anomaly framing: treat the rare class with one-class or isolation methods.
• Collect more minority data if feasible.

model = LogisticRegression(class_weight="balanced")

Always evaluate on a stratified holdout or with stratified cross-validation so each fold keeps the class ratio.

Precision is the fraction of predicted positives that are truly positive: TP / (TP + FP). Recall, or sensitivity, is the fraction of actual positives that were found: TP / (TP + FN).

Prefer precision when false positives are costly: a spam filter that flags real email, a fraud alert that blocks a legitimate purchase, or a recommendation that annoys users.

Prefer recall when false negatives are costly: cancer screening, intrusion detection, or catching defective products before shipping.

F1 is the harmonic mean of the two, useful when you need a single balanced number. There is always a tradeoff: lowering the decision threshold generally raises recall and lowers precision.

from sklearn.metrics import precision_recall_curve
p, r, thresholds = precision_recall_curve(y_true, y_scores)

Choose the operating point from the business cost of each error type, not from a default threshold.

Cross-validation repeatedly splits data into training and validation folds to estimate how well a model generalizes. In k-fold, the data is split into k parts; the model trains on k-1 and validates on the remaining one, rotating so every point is validated once. The average score is the estimate.

from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5, scoring="roc_auc")

Variants: stratified k-fold preserves class ratios for imbalanced data; grouped k-fold keeps all rows of one entity together to avoid leakage; time series split respects temporal order.

Use it for model selection, hyperparameter tuning and comparing algorithms. Pitfalls: preprocessing such as scaling or SMOTE must be fit inside each fold via a pipeline, otherwise information leaks and the score is optimistic. Keep a separate untouched test set for the final estimate.

Regularization adds a penalty on model complexity to the loss, discouraging large weights and reducing overfitting.

L2 (ridge) adds the sum of squared weights. It shrinks weights smoothly toward zero but rarely makes them exactly zero, and it handles correlated features well.

L1 (lasso) adds the sum of absolute weights. It drives some coefficients exactly to zero, performing feature selection automatically, which helps with sparse or high-dimensional data.

Elastic net combines both, with alpha controlling overall strength and l1_ratio the mix.

Ridge(alpha=1.0)
Lasso(alpha=0.01)
ElasticNet(alpha=0.1, l1_ratio=0.5)

The strength parameter must be tuned by cross-validation. In neural networks L2 appears as weight decay, and dropout is another regularization technique. Regularization always trades a little training fit for better generalization.

Gradient descent minimizes a loss function by repeatedly moving parameters in the direction that decreases the loss. The gradient is the vector of partial derivatives, and you step opposite to it.

for epoch in range(epochs):
    preds = X @ w + b
    error = preds - y
    grad_w = (2/n) * X.T @ error
    grad_b = (2/n) * error.sum()
    w -= lr * grad_w
    b -= lr * grad_b

The learning rate is critical: too large and the loss diverges or oscillates, too small and training is slow. Variants: batch gradient descent uses all data per step (stable, slow), stochastic uses one example (noisy, fast), and mini-batch uses a small batch and is the standard. Momentum and Adam adapt the step per parameter. Always scale features, monitor the loss curve, and consider schedules or early stopping.

Both are tree ensembles, differing in how trees are combined.

Random forest builds many deep, independent trees on bootstrap samples with random feature subsets, then averages or votes. Trees train in parallel and reduce variance. It is robust, hard to overfit, and needs little tuning.

Gradient boosting builds trees sequentially, each fitting the residual errors of the current ensemble. It reduces bias and often achieves higher accuracy, but is sensitive to hyperparameters and can overfit without regularization. Modern implementations add shrinkage, subsampling and column sampling.

RandomForestRegressor(n_estimators=500, max_features="sqrt")
GradientBoostingRegressor(n_estimators=500, learning_rate=0.05, max_depth=3)

Use a forest as a strong baseline with minimal tuning; use boosting when you need the best tabular performance. HistGradientBoosting, XGBoost and LightGBM are fast defaults.

MLOps applies DevOps practices to machine learning: versioning data, code and models; automating training and deployment; and monitoring models in production.

Key components:

• Experiment tracking and reproducibility, for example MLflow or Weights and Biases.
• Feature stores for consistent training and serving features.
• CI/CD for retraining, testing and rollout with canary or shadow deployment.
• A model registry with lineage and approval.
• Monitoring for performance, latency and cost.

Monitoring matters because models decay. Data drift is when input distributions shift away from training data; concept drift is when the relationship between inputs and target changes. Both degrade accuracy even when the code is unchanged.

Alert on drift, prediction distributions and business KPIs, and define a retraining trigger. Labels may arrive late, so use proxies until ground truth is available.