Statistics
Statistics is the logic layer underneath every model you'll ever train.
Not a math requirement to get through - it's how you read data honestly, reason about uncertainty in code, and understand what a model is actually doing when it "learns." This page is your complete reference: concepts, formulas, problems, and a quiz.
Download statistics notes (PDF)Why statistics, really
Why you need it
Every dataset lies a little through noise, bias, and randomness. Statistics is the only formal toolkit for telling signal from coincidence — without it you're debugging model behavior by vibes instead of evidence.
How it shapes coding & logic
Conditionals mirror conditional probability. Loops that aggregate data are computing means and variances. Edge-case thinking ("what if this is an outlier?") is applied dispersion. Statistical thinking makes your logic more defensive and your code more correct on data it hasn't seen yet.
How it drives model training
Loss functions are likelihoods. Regularization is a prior. Train/test splits are sampling design. Evaluation metrics are hypothesis-testing concepts in disguise. Every "why did my model do that" question resolves to a statistics answer.
Deep-dive notes
Everything you need to know
17 core topics, ordered from foundations to applied machine learning. Tap any topic to expand its notes and key formula.
The vocabulary you reach for before anything else: mean, median, mode, and range describe where your data sits and how it's shaped.
- Mean is sensitive to outliers; median is not — know when each is the honest summary.
- Mode matters most for categorical or multi-modal data (e.g. bimodal user behavior).
- Range is fragile (only two points decide it); pair it with IQR for a robust spread.
- In code: this is what you compute before writing a single line of model logic — sanity-check your dataset first.
Quick reference
Cheat sheet
Every formula on this page, in one scannable grid — bookmark this section for quick lookups while coding.
Mean
x̄ = Σx / n
Central tendency, sensitive to outliers
Variance
σ² = Σ(x − x̄)² / n
Average squared spread
Std. Deviation
σ = √σ²
Spread in original units
Z-score
z = (x − μ) / σ
How many σ from the mean
Confidence Interval
x̄ ± z·(σ/√n)
Range for the true parameter
Correlation (Pearson r)
r = cov(X,Y) / (σx·σy)
Strength of linear relationship
Bayes' Theorem
P(H|E) = P(E|H)P(H)/P(E)
Update belief with evidence
Binomial PMF
P(X=k) = C(n,k)pᵏ(1−p)ⁿ⁻ᵏ
k successes in n trials
Normal PDF
f(x) = (1/σ√2π)e^(−(x−μ)²/2σ²)
The bell curve
Chi-Square
χ² = Σ(O − E)²/E
Categorical fit / independence
F-statistic (ANOVA)
F = variance between / variance within
Compare 3+ group means
R² (coefficient of determination)
R² = 1 − SSres/SStot
Variance explained by model
Practice
Problems to work through
Try each problem before revealing the answer — struggling with it first is where the learning actually happens.
Problem 1
A dataset has values 4, 8, 6, 5, 3. Find the mean and the variance (population).
Hint: Mean first, then average of squared deviations from that mean.
Problem 2
A coin is flipped 10 times. What's the probability of getting exactly 6 heads? (p = 0.5)
Hint: Use the binomial PMF with n=10, k=6, p=0.5.
Problem 3
A sample of 36 users has a mean session time of 12 minutes with a standard deviation of 3 minutes. Construct a 95% confidence interval for the true mean.
Hint: z for 95% ≈ 1.96. CI = x̄ ± z(σ/√n).
Problem 4
Two variables have a Pearson correlation of r = 0.9. Does this mean one causes the other?
Hint: Think about confounding variables and the classic warning in statistics.
Problem 5
You run an A/B test and get p = 0.03 with α = 0.05. What do you conclude, and what does the p-value *not* mean?
Hint: p-value = probability of the data given H₀ is true, not the probability H₀ is true.
Problem 6
A model has low training error but much higher validation error. Which half of the bias-variance tradeoff is the likely culprit, and why?
Hint: Low bias, high variance = overfitting to the training sample.
Check yourself
Quiz
7 questions. Select an answer for each, then submit to see your score and explanations.
1. Which measure of central tendency is least affected by outliers?
2. What does a p-value of 0.02 actually tell you?
3. The Central Limit Theorem says that as sample size grows, the distribution of the sample mean approaches:
4. In the bias-variance tradeoff, an overfit model typically has:
5. Which distribution best models 'number of server errors in one hour' when errors are rare and independent?
6. A correlation coefficient of r = -0.85 means:
7. Standardizing a feature (z-score scaling) before training a model primarily addresses:
Answer every question to unlock submit.
Keep the full notes offline
Every topic, formula, and worked example from this page, packaged as a PDF you can study from anywhere.
Download statistics notes (PDF)