Probability & Statistics

A first-principles guide to probability: from sample spaces and axioms to conditional probability, independence, and Bayes' theorem.

Discrete and continuous random variables, PMF, PDF, CDF, expectation, variance, covariance, and joint distributions explained.

A deep dive into the key distributions used in ML: Bernoulli, Binomial, Poisson, Gaussian, Exponential, Beta, and Multivariate Normal.

A unifying framework for probability distributions: sufficient statistics, conjugate priors, and why most ML distributions share a common structure.

Why averages become Gaussian: the Law of Large Numbers, types of convergence, and the Central Limit Theorem explained.

How to find the best parameters for a model by maximizing the probability of observed data.

Bayesian parameter estimation: combining prior beliefs with data for more robust models.

Expectation-Maximization: how to fit models with latent variables, from Gaussian Mixture Models to missing data problems.

p-values, significance levels, Type I/II errors, t-tests, and confidence intervals — the foundations of statistical inference.

Distribution-free methods: kernel density estimation, rank tests, bootstrap, and nonparametric Bayesian models for when assumptions fail.

Full Bayesian reasoning: posterior distributions, conjugate priors, predictive distributions, and how Bayesian methods differ from frequentist approaches.

Bayesian networks, Markov random fields, and how graphs encode conditional independence for tractable reasoning over complex distributions.

Monte Carlo, rejection sampling, importance sampling, MCMC, Metropolis-Hastings, and Gibbs sampling for computational inference.