Run a mock round

First understand why data is missing, because the mechanism determines the right approach.

• MCAR: missing completely at random, safe to drop rows with a small loss of power.
• MAR: missing at random given observed variables, so imputation using other features is reasonable.
• MNAR: missing not at random, where the missingness itself carries information, such as high earners refusing to report income. Simple imputation then biases results.

Options: drop rows or columns, mean or median imputation, model-based imputation such as MICE or k-NN, or flag missingness with an indicator variable and let the model use it. For time series, forward-fill carefully and never leak future values backward.

df["income"] = df["income"].fillna(df["income"].median())
df["income_missing"] = df["income"].isna().astype(int)

Always report how much data was missing and test sensitivity across methods.

Start from decisions, not charts. Ask what action each viewer takes and which questions they need answered.

Principles:

• Lead with a small number of headline KPIs, then allow drill-down.
• Show trends and comparisons, not just current values, and include targets.
• Make the time grain and definitions visible.
• Put the most important metric top-left where the eye lands.
• Use the right chart: lines for trends, bars for comparisons, tables for precision.
• Avoid pie charts for many categories and never truncate a bar axis.
• Highlight anomalies and add period-over-period change.

Agree on metric definitions and a single source of truth so teams do not argue about numbers. Design for the audience: executives want outcomes, operators want granular, actionable detail. Validate with a small group, iterate, and retire unused tiles. A dashboard is a product, not a one-off export.

Attribution assigns credit for a conversion to the marketing touchpoints along the customer journey. The model choice changes how budget is allocated.

Common models:

• Last click: simple, overcredits closing channels such as branded search.
• First click: credits awareness, ignores nurturing.
• Linear: equal credit across touchpoints.
• Time decay: more credit to recent touches.
• Position based: weights first and last.
• Data driven: Shapley values or Markov chains estimate each channel's marginal contribution.

SELECT conversion_id, MAX_BY(channel, touch_time) AS channel
FROM touchpoints GROUP BY conversion_id;

Challenges include cross-device and offline journeys, view-through impressions, and correlation between channels. None of these models proves causality; incremental lift requires experiments or geo holdouts. Use attribution for directional budgeting and experiments for causal truth.

Cohort analysis groups users by a shared start event, usually their first activity period, and tracks behaviour over subsequent periods. Retention is the share of a cohort still active in period n.

SELECT
  date_trunc('week', first_seen) AS cohort,
  week_number,
  COUNT(DISTINCT user_id) AS active
FROM user_activity
GROUP BY 1, 2;

A retention triangle or heatmap shows cohorts as rows and periods as columns, making it easy to compare whether newer cohorts retain better. Common metrics are day-1, day-7 and day-30 retention, and the flattening point of the curve indicates a stable core of users.

Pitfalls: define active precisely, avoid mixing calendar time with cohort age, and watch for survivorship bias. Comparing retention curves before and after a product change is one of the most reliable ways to judge whether the change improved engagement.

Vanity metrics look impressive but do not guide decisions: total registered users, page views, cumulative downloads. They only go up, lack context and cannot be tied to a specific action.

Actionable metrics are tied to a behaviour you can influence and have a clear link to value: activation rate, weekly active users, conversion rate, retention, average order value. They can move up or down and prompt a response.

For example, ten million page views is vanity, while the percentage of new users who complete onboarding within 24 hours is actionable because a product change can shift it.

The test: if the metric doubled or halved, would you change what you are doing? If not, it is probably vanity. Pair metrics with segments and a time frame for context so teams focus on outcomes rather than activity.

Start with the decision the audience needs to make, then structure the narrative around it: situation, complication, question and answer. Lead with the recommendation and supporting evidence, and keep details in an appendix.

Guidelines:

• Know the audience. Executives want the headline, impact and ask; analysts want methodology.
• Use one clear message per chart, with a descriptive title that states the takeaway.
• Quantify impact in business units such as revenue, churn or hours saved.
• Address the strongest counterargument and state limitations honestly.
• Provide a confidence level and range, not false precision.
• End with concrete next steps and owners.

Avoid jargon, avoid dumping every query, and never overwhelm with a wall of numbers. Rehearse the first 30 seconds, when attention is highest. Good storytelling makes the insight memorable and the decision easy.

Averages hide the shape of a distribution. Percentiles describe typical and extreme behaviour, which matters for user experience and capacity planning.

For example, average page load time may look fine while the 95th percentile is terrible for a meaningful share of users. Latency objectives are usually expressed as p95 or p99. Revenue per user is heavy-tailed, so the median is a better typical value than the mean, and the top 1 percent of customers often drive a large share of revenue.

SELECT
  percentile_cont(0.5) WITHIN GROUP (ORDER BY load_ms) AS p50,
  percentile_cont(0.95) WITHIN GROUP (ORDER BY load_ms) AS p95
FROM page_loads;

Use histograms and box plots to inspect skew and outliers, and always state which percentile you mean. Segment percentiles by device, region or plan to find users having the worst experience.

A metric tree decomposes a top-level business goal into the drivers that influence it. For example, revenue equals active customers times purchase frequency times average order value. Each node can be broken down further, linking strategy to measurable inputs.

Benefits: it shows where a team can have leverage, prevents optimizing a proxy that harms the top goal, and connects daily work to outcomes.

Metric governance is the discipline of defining, documenting and owning metrics. Without it, different teams compute active user or churn differently, meetings devolve into debates about numbers, and dashboards contradict each other. Practices include a metrics layer with version-controlled SQL, a data dictionary, a single owner per metric, and change review.

metric: weekly_active_users
definition: distinct users with a qualifying event in a 7-day window
owner: growth-analytics

Governance turns metrics into an asset rather than a source of confusion.

Outliers can be data errors, rare genuine events, or informative signals. Decide whether to remove, cap or keep them based on context, never automatically.

Detection methods:

• Visual: box plots and scatter plots.
• Z-score: flag points more than three standard deviations from the mean, assuming normality.
• IQR rule: flag points below Q1 - 1.5*IQR or above Q3 + 1.5*IQR, robust to skew.
• Model-based: isolation forest or DBSCAN for multivariate cases.

q1, q3 = df.amount.quantile([0.25, 0.75])
iqr = q3 - q1
mask = (df.amount < q1 - 1.5*iqr) | (df.amount > q3 + 1.5*iqr)

Handle by correcting errors at the source, winsorizing extreme values for modelling, or using robust statistics such as the median. Investigate before deleting: a spike in orders might be a real event or a pricing bug, and both matter.