Skip to content
TryHired.
Interview prep · Technical · Python 3.12.10 · NumPy 2.5.2

Machine learning interview questions: answers with real output

Core machine learning ideas, from overfitting to attention, answered in plain words with small NumPy examples you can run.

Updated Nov 12, 2026 · Examples run on Python 3.12.10 · NumPy 2.5.2 · questions: TryHired
Quick answer

Machine learning interviews test whether you understand how models learn and fail: the bias-variance trade-off, overfitting and regularization, evaluation metrics beyond accuracy, cross-validation, gradient descent, common algorithms (logistic regression, trees, k-means, PCA) and practical problems such as data leakage, class imbalance and drift. Below are 20 questions; every example was run with Python and NumPy and shows its real output.

Basic

Basic machine learning interview questions

Advice

Basic1. What is the difference between supervised, unsupervised and reinforcement learning?

Supervised learning learns from labeled examples to predict a target (classification or regression). Unsupervised learning finds structure in unlabeled data (clustering, dimensionality reduction). Reinforcement learning learns actions by trial and error to maximize a reward.

Basic2. What is the bias-variance trade-off?

Bias is error from a model too simple to capture the pattern (underfitting); variance is error from a model so flexible it fits noise (overfitting). As flexibility grows, training error keeps falling while test error falls and then rises. You choose the complexity where test error is lowest.

Python
import numpy as np
rng = np.random.default_rng(0)
x = np.sort(rng.uniform(-3, 3, 40)); y = np.sin(x) + rng.normal(0, 0.3, 40)
xt = np.sort(rng.uniform(-3, 3, 200)); yt = np.sin(xt) + rng.normal(0, 0.3, 200)
for deg in (1, 3, 9, 15):
    c = np.polyfit(x, y, deg)
    tr = np.mean((np.polyval(c, x) - y) ** 2); te = np.mean((np.polyval(c, xt) - yt) ** 2)
    print(f'degree {deg:>2}: train MSE {tr:.3f}  test MSE {te:.3f}')
Output (Python 3.12.10 · NumPy 2.5.2)
degree  1: train MSE 0.250  test MSE 0.301
degree  3: train MSE 0.108  test MSE 0.109
degree  9: train MSE 0.077  test MSE 0.133
degree 15: train MSE 0.062  test MSE 0.147

Basic3. How do you reduce overfitting?

Get more data, simplify the model, add regularization, use early stopping, apply dropout in neural networks, reduce or engineer features, and use cross-validation to choose settings. Always measure on data the model has not seen.

Basic4. Why split data into training, validation and test sets?

The model learns on the training set, you tune choices on the validation set, and the test set gives one honest final estimate. Reusing the test set for tuning leaks information and makes results look better than they are. k-fold cross-validation reuses data efficiently when it is scarce.

Python
import numpy as np
idx = np.arange(10)
folds = np.array_split(idx, 5)
for k, val in enumerate(folds):
    train = np.setdiff1d(idx, val)
    print(f'fold {k}: validate {val.tolist()} train {train.tolist()}')
Output (Python 3.12.10 · NumPy 2.5.2)
fold 0: validate [0, 1] train [2, 3, 4, 5, 6, 7, 8, 9]
fold 1: validate [2, 3] train [0, 1, 4, 5, 6, 7, 8, 9]
fold 2: validate [4, 5] train [0, 1, 2, 3, 6, 7, 8, 9]
fold 3: validate [6, 7] train [0, 1, 2, 3, 4, 5, 8, 9]
fold 4: validate [8, 9] train [0, 1, 2, 3, 4, 5, 6, 7]

Basic5. Why is accuracy misleading on imbalanced data?

If 95% of cases are negative, a model that always predicts negative is 95% accurate and useless. Look at the confusion matrix, precision (how many predicted positives are right), recall (how many real positives are found) and F1, their harmonic mean.

Python
import numpy as np
y_true = np.array([1, 0, 1, 1, 0, 0, 1, 0, 0, 0])
y_pred = np.array([1, 0, 0, 1, 0, 1, 1, 0, 0, 0])
tp = int(((y_pred == 1) & (y_true == 1)).sum()); fp = int(((y_pred == 1) & (y_true == 0)).sum())
fn = int(((y_pred == 0) & (y_true == 1)).sum()); tn = int(((y_pred == 0) & (y_true == 0)).sum())
precision = tp / (tp + fp); recall = tp / (tp + fn)
print('TP FP FN TN:', tp, fp, fn, tn)
print(f'accuracy {np.mean(y_true == y_pred):.2f}  precision {precision:.2f}  recall {recall:.2f}  F1 {2 * precision * recall / (precision + recall):.2f}')
Output (Python 3.12.10 · NumPy 2.5.2)
TP FP FN TN: 3 1 1 5
accuracy 0.80  precision 0.75  recall 0.75  F1 0.75

Basic6. What is the difference between classification and regression?

Classification predicts a category, such as spam or not spam; regression predicts a number, such as a price. Some models do both, and many classifiers output probabilities that you turn into a decision with a threshold.

Intermediate

Intermediate machine learning interview questions

Advice

Intermediate7. What does ROC AUC measure?

ROC AUC is the probability that the model scores a random positive example above a random negative one. 0.5 is random ranking and 1.0 is perfect. It ignores the decision threshold, so pair it with precision and recall at the threshold you will use.

Python
import numpy as np
scores = np.array([0.9, 0.8, 0.7, 0.6, 0.55, 0.4, 0.3, 0.2])
labels = np.array([1, 1, 0, 1, 0, 1, 0, 0])
pos, neg = scores[labels == 1], scores[labels == 0]
auc = np.mean([(p > n) + 0.5 * (p == n) for p in pos for n in neg])
print('AUC =', auc)
Output (Python 3.12.10 · NumPy 2.5.2)
AUC = 0.8125

Intermediate8. How does logistic regression learn?

It passes a weighted sum of features through the sigmoid to get a probability, and gradient descent adjusts the weights to minimize log loss (cross-entropy). The coefficients are changes in log-odds per unit of each feature.

Python
import numpy as np
rng = np.random.default_rng(0)
X = rng.normal(size=(200, 2)); true_w = np.array([2.0, -1.0])
y = (rng.uniform(size=200) < 1 / (1 + np.exp(-(X @ true_w + 0.5)))).astype(float)
w = np.zeros(2); b = 0.0
for _ in range(2000):
    p = 1 / (1 + np.exp(-(X @ w + b)))
    w -= 0.1 * X.T @ (p - y) / len(y); b -= 0.1 * np.mean(p - y)
acc = np.mean((1 / (1 + np.exp(-(X @ w + b))) > 0.5) == y)
print('weights', np.round(w, 2), 'bias', round(b, 2), 'train accuracy', round(acc, 3))
Output (Python 3.12.10 · NumPy 2.5.2)
weights [ 2.47 -1.24] bias 0.48 train accuracy 0.835

Intermediate9. What is the difference between L1 and L2 regularization?

Both add a penalty on coefficient size to the loss. L2 (ridge) shrinks all coefficients smoothly toward zero. L1 (lasso) can set some exactly to zero, which performs feature selection. A stronger penalty means a simpler, more biased, lower-variance model.

Python
import numpy as np
rng = np.random.default_rng(0)
X = rng.normal(size=(30, 5)); y = X @ np.array([3.0, 0, 0, 1.5, 0]) + rng.normal(0, 1, 30)
for lam in (0.0, 10.0, 100.0):
    w = np.linalg.solve(X.T @ X + lam * np.eye(5), X.T @ y)
    print(f'ridge lambda={lam:>5}: {np.round(w, 2)}')
Output (Python 3.12.10 · NumPy 2.5.2)
ridge lambda=  0.0: [ 3.09  0.15 -0.27  1.09  0.05]
ridge lambda= 10.0: [ 1.63  0.25 -0.27  0.57 -0.09]
ridge lambda=100.0: [ 0.33  0.1  -0.09  0.11 -0.08]

Intermediate10. How does the learning rate affect gradient descent?

Gradient descent steps against the gradient to reduce the loss. Too small a learning rate converges slowly; a large one overshoots the minimum and bounces from side to side (0.9 below), and too large a one diverges (1.1). Schedules and adaptive optimizers such as Adam adjust the step size during training.

Python
def descend(lr, steps):
    x = 5.0                       # minimize f(x) = x^2, gradient 2x; the minimum is x = 0
    for _ in range(steps):
        x -= lr * 2 * x
    return x
for lr in (0.01, 0.1, 0.9, 1.1):
    print(f'lr={lr}: after 3 steps {descend(lr, 3):.4g}, after 20 steps {descend(lr, 20):.4g}')
Output (Python 3.12.10 · NumPy 2.5.2)
lr=0.01: after 3 steps 4.706, after 20 steps 3.338
lr=0.1: after 3 steps 2.56, after 20 steps 0.05765
lr=0.9: after 3 steps -2.56, after 20 steps 0.05765
lr=1.1: after 3 steps -8.64, after 20 steps 191.7

Intermediate11. How does a decision tree choose a split?

At each node it tries candidate splits and keeps the one that most reduces impurity, measured by Gini impurity or entropy for classification, or variance for regression. Deep trees overfit, so depth limits, minimum leaf sizes or pruning are used.

Python
def gini(labels):
    n = len(labels)
    return 1 - sum((labels.count(c) / n) ** 2 for c in set(labels))
parent = ['yes'] * 5 + ['no'] * 5
left, right = ['yes'] * 4 + ['no'], ['yes'] + ['no'] * 4
weighted = (len(left) * gini(left) + len(right) * gini(right)) / len(parent)
print('parent gini', gini(parent), ' after split', round(weighted, 3))
Output (Python 3.12.10 · NumPy 2.5.2)
parent gini 0.5  after split 0.32

Intermediate12. What is the difference between random forests and gradient boosting?

A random forest trains many deep trees independently on bootstrap samples and random feature subsets, then combines their votes, which reduces variance. Gradient boosting trains shallow trees one after another, each fitting the errors of the ensemble so far, which reduces bias and is often more accurate but needs careful tuning to avoid overfitting.

Intermediate13. How does k-means clustering work?

Pick k starting centers, assign each point to its nearest center, move each center to the mean of its points, and repeat until assignments stop changing. Results depend on the starting centers and on scaling, and k must be chosen, for example with the elbow method or silhouette score.

Python
import numpy as np
rng = np.random.default_rng(0)
pts = np.vstack([rng.normal(c, 0.5, size=(30, 2)) for c in ((0, 0), (5, 5), (0, 5))])
centers = pts[rng.choice(len(pts), 3, replace=False)]
for _ in range(10):
    lab = np.argmin(((pts[:, None] - centers) ** 2).sum(-1), axis=1)
    centers = np.array([pts[lab == k].mean(0) for k in range(3)])
print(np.round(centers[np.argsort(centers[:, 0] * 10 + centers[:, 1])], 2))
Output (Python 3.12.10 · NumPy 2.5.2)
[[-0.06  0.14]
 [-0.07  5.  ]
 [ 5.06  5.03]]

Intermediate14. What does PCA do?

Principal component analysis finds new axes, ordered by how much variance they capture, as combinations of the original features. Keeping the first few components reduces dimensions while keeping the bulk of the information. Standardize features first so units do not dominate.

Python
import numpy as np
rng = np.random.default_rng(0)
t = rng.normal(size=200)
X = np.column_stack([t, 2 * t + rng.normal(0, 0.3, 200), rng.normal(size=200)])
Xc = (X - X.mean(0)) / X.std(0)
_, s, _ = np.linalg.svd(Xc, full_matrices=False)
print('variance explained per component', np.round(s ** 2 / (s ** 2).sum(), 3))
Output (Python 3.12.10 · NumPy 2.5.2)
variance explained per component [0.665 0.331 0.004]
Advanced

Advanced machine learning interview questions

Advice

Advanced15. What is data leakage?

Leakage is information in training that would not be available at prediction time, such as a feature recorded after the outcome, or scaling and imputation fitted on the whole dataset before splitting. It produces excellent validation scores that collapse in production. Split first, fit preprocessing on training data only, and split by time for time-dependent problems.

Advanced16. How do you handle class imbalance?

Use metrics that reflect the minority class (precision, recall, PR AUC), adjust the decision threshold, weight classes in the loss, resample (undersample the majority or oversample the minority, inside cross-validation only), and get more minority examples if you can.

Advanced17. What does softmax do, and why use cross-entropy loss?

Softmax turns a vector of scores into probabilities that sum to 1. Cross-entropy then penalizes the model by the negative log of the probability it gave the correct class, which gives strong gradients when the model is confidently wrong.

Python
import numpy as np
logits = np.array([2.0, 1.0, 0.1])
p = np.exp(logits - logits.max()); p /= p.sum()
print('probabilities', np.round(p, 3), 'sum', round(p.sum(), 3))
print('cross-entropy if class 0 is correct', round(-np.log(p[0]), 3))
print('cross-entropy if class 2 is correct', round(-np.log(p[2]), 3))
Output (Python 3.12.10 · NumPy 2.5.2)
probabilities [0.659 0.242 0.099] sum 1.0
cross-entropy if class 0 is correct 0.417
cross-entropy if class 2 is correct 2.317

Advanced18. What causes vanishing gradients, and how are they addressed?

In deep networks, gradients are products of many small derivatives, so they can shrink toward zero in early layers and stop them learning (saturating activations such as sigmoid make it worse). ReLU-family activations, careful initialization, normalization layers and residual connections address it.

Python
import numpy as np
sig_grad_max = 0.25   # the sigmoid's derivative is at most 0.25
for layers in (5, 10, 20):
    print(f'{layers} sigmoid layers: gradient scale at most {sig_grad_max ** layers:.2e}')
Output (Python 3.12.10 · NumPy 2.5.2)
5 sigmoid layers: gradient scale at most 9.77e-04
10 sigmoid layers: gradient scale at most 9.54e-07
20 sigmoid layers: gradient scale at most 9.09e-13

Advanced19. What is attention in a transformer?

Each token builds a query, key and value vector. Attention scores are query-key dot products, scaled by the square root of the key dimension and passed through softmax; each token's output is the score-weighted sum of value vectors. Self-attention lets every token use context from every other token in one step.

Python
import numpy as np
rng = np.random.default_rng(0)
Q, K, V = (rng.normal(size=(3, 4)) for _ in range(3))   # 3 tokens, dimension 4
scores = Q @ K.T / np.sqrt(4)
w = np.exp(scores - scores.max(1, keepdims=True)); w /= w.sum(1, keepdims=True)
print('attention weights (rows sum to 1):'); print(np.round(w, 3))
print('output shape', (w @ V).shape)
Output (Python 3.12.10 · NumPy 2.5.2)
attention weights (rows sum to 1):
[[0.226 0.475 0.298]
 [0.144 0.598 0.259]
 [0.668 0.235 0.097]]
output shape (3, 4)

Advanced20. How do you monitor a model in production?

Track input data drift (feature distributions versus training), prediction drift, and the real outcome metrics once labels arrive, alongside latency and errors. Set alerts, keep the training data and model version for every prediction, and plan retraining and rollback.

Practice

How to practice with this page

Advice
  • Implement one algorithm from scratch (logistic regression or k-means) and explain each line.
  • For every metric, know when it misleads and what you would use instead.
  • Prepare a model you shipped: data, baseline, metric, validation, deployment and monitoring.
  • Practice the system design side: how the model is served, updated and monitored.
Next

Related interview prep

Disclosure: TryHired makes HIRED, a paid job-search program, so we have a conflict of interest when we mention it. This question bank is free, with no TryHired signup.

FAQ

Questions people ask

What machine learning questions are asked in interviews?
Bias-variance, overfitting and regularization, evaluation metrics, cross-validation, gradient descent, logistic regression, trees and ensembles, clustering, PCA, data leakage, imbalance and, for many roles, neural networks and transformers.
Do machine learning interviews include coding?
Often: implementing a simple algorithm in Python, data manipulation, or an ML system design discussion. See the Python interview questions and system design interview questions.
How is a machine learning engineer interview different from a data scientist interview?
ML engineer interviews put more weight on production: serving models, pipelines, scaling, monitoring and software engineering. Data scientist interviews put more weight on statistics, experiments and analysis.
Do I need deep learning for a machine learning interview?
For roles that work with images, text or large models, yes: neural network basics, backpropagation, common architectures and transformers. For tabular-data roles, classical methods matter more.
How do I prepare for a machine learning interview?
Re-derive the core methods, implement two or three from scratch, and prepare one end-to-end project story with your metric, validation and what happened in production.
Method

How this page is built

TryHired wrote these questions and answers. Every Python block on this page was executed with Python 3.12.10 · NumPy 2.5.2 when the page was built, and the output shown is the real output; the build stops if any example fails. The question list is general practice: it does not predict what a particular employer will ask.

What this page can't tell you
  • Interviewers ask different questions for different roles and levels.
  • Answers are short explanations, not complete references.
  • Examples are deliberately small; production work uses libraries such as scikit-learn or PyTorch.
Sources
  1. NumPy documentation, linear algebra: https://numpy.org/doc/stable/reference/routines.linalg.html
  2. Google Machine Learning Crash Course: https://developers.google.com/machine-learning/crash-course
  3. Hastie, Tibshirani and Friedman, The Elements of Statistical Learning (free PDF): https://hastie.su.domains/ElemStatLearn/
  4. Vaswani et al., Attention Is All You Need (2017): https://arxiv.org/abs/1706.03762