Data Science & Statistics Interview Questions and Answers

Imagine we assume a change has no real effect, which is the null hypothesis. The p-value answers: if that were true, how surprising would the data we observed be? A p-value of 0.03 means that under the no-effect assumption we would see a result this extreme about 3 percent of the time by chance alone.

When the p-value is below a threshold, usually 0.05, we call the result statistically significant and reject the null hypothesis. It does not prove the effect is real, large or important; it only says the data are unlikely under the null. A tiny p-value with a trivial effect size can still be meaningless for the business.

Significance also depends on sample size and power. Always report the effect size and a confidence interval alongside the p-value, and remember that many tests increase the chance of a false positive.

The Central Limit Theorem says that the sampling distribution of the sample mean approaches a normal distribution as the sample size grows, regardless of the shape of the population distribution, provided observations are independent and the variance is finite.

means = [np.mean(np.random.exponential(2, 50)) for _ in range(10000)]

Implications: it justifies normal-based confidence intervals and t-tests for means even when the underlying data are skewed, as long as n is reasonably large. The common rule of thumb is n greater than about 30, but heavily skewed or heavy-tailed data need more.

It also explains why the standard error shrinks with the square root of n: quadrupling the sample halves the margin of error. It does not apply directly to small samples, dependent data, or statistics other than means without care.

A Type I error is a false positive: rejecting the null hypothesis when it is actually true. Its probability is the significance level alpha, usually 0.05. In a fraud system it means flagging a legitimate transaction.

A Type II error is a false negative: failing to reject the null when a real effect exists. Its probability is beta, and statistical power is 1 minus beta, the chance of detecting a true effect. In a medical test it means missing a real disease.

There is a tradeoff: lowering alpha reduces false positives but increases false negatives for a fixed sample size. Increasing sample size or effect size raises power.

from statsmodels.stats.power import TTestIndPower
n = TTestIndPower().solve_power(effect_size=0.2, power=0.8, alpha=0.05)

Choose alpha and power based on which error is more costly, and compute the sample size before the experiment.

A confidence interval gives a range of plausible values for a population parameter, along with the uncertainty in the estimate. A 95 percent interval means that if we repeated the study many times, about 95 percent of the intervals we constructed would contain the true parameter.

from scipy import stats
mean, se = data.mean(), stats.sem(data)
ci = stats.t.interval(0.95, len(data)-1, loc=mean, scale=se)

It is not the probability that the true value lies in this specific interval under a frequentist view, though that is the common shortcut. Wider intervals mean more uncertainty, coming from smaller samples or noisier data.

Overlapping intervals between two groups suggest no clear difference, but the correct test compares the difference directly rather than eyeballing overlap. Report the point estimate, the interval and the sample size together so stakeholders can judge practical significance.

Start with a clear hypothesis and a primary metric tied to the business, plus guardrail metrics for regressions.

Key steps:

1. Define the unit of randomization, usually the user, and keep it stable across the test.
2. Compute sample size from the minimum detectable effect, power of 0.8, alpha of 0.05 and baseline variance. Underpowered tests waste traffic.
3. Randomize with a hash so assignment is consistent and balanced.
4. Run for at least one full business cycle, avoiding peeking that inflates false positives.
5. Analyze with the pre-declared metric and test, using intention-to-treat.

treatment = hash(user_id) % 100 < 50

Watch for novelty effects, sample ratio mismatch, and interactions between concurrent tests. Segment analyses are exploratory unless pre-registered. Combine statistics with judgment about practical significance before shipping.

Power is the probability of detecting a real effect of a given size, that is, correctly rejecting a false null hypothesis. The conventional target is 0.8, meaning an 80 percent chance of finding the effect if it exists.

Power depends on four linked quantities: significance level alpha, effect size, sample size and variance. Fix any three and the fourth follows. Larger effects, lower noise, higher alpha or more data all increase power.

from statsmodels.stats.power import TTestIndPower
n = TTestIndPower().solve_power(effect_size=0.3, alpha=0.05, power=0.8)

Consequences: an underpowered study may miss a real effect, and when it does find significance it tends to overestimate the magnitude. Always run a power calculation before collecting data, and acknowledge that small pilot studies rarely have enough power. Increasing duration or traffic is the usual remedy.

Bayes theorem updates a belief given evidence: P(A|B) = P(B|A) * P(A) / P(B). The prior P(A) is your initial belief, the likelihood P(B|A) is how probable the evidence is under A, and the posterior P(A|B) is the updated belief.

Example: a disease affects 1 percent of people. A test is 99 percent sensitive and has a 5 percent false positive rate. If you test positive, what is the chance you have it?

p_disease = 0.01
sensitivity = 0.99
false_pos = 0.05
p_pos = sensitivity*p_disease + false_pos*(1-p_disease)
p_posterior = sensitivity*p_disease / p_pos
# 0.99*0.01 / (0.99*0.01 + 0.05*0.99) = 0.1667

Even with a positive test the probability is only about 17 percent, because the disease is rare. This base rate effect is why screening uses confirmatory tests and why base rates matter in fraud and spam detection.

Both test hypotheses about means, and the choice hinges on whether the population standard deviation is known and the sample size.

Use a z-test when the population standard deviation is known and the data are normal, or when the sample is large enough that the sample standard deviation is a good estimate thanks to the Central Limit Theorem, typically n greater than 30 or 50.

Use a t-test when the population standard deviation is unknown and the sample is small. The t-distribution has heavier tails that account for the extra uncertainty of estimating the variance from data; as n grows it converges to the normal.

from scipy import stats
stats.ttest_ind(group_a, group_b, equal_var=False)

Other choices matter too: paired tests for matched observations, Welch's t-test when variances differ, and non-parametric tests such as Mann-Whitney for heavily skewed or ordinal data.