The Fundatmentals of Machine Learning Week 2 - 1
Overview
1. Probability density functions
2. Properties & parameters of probability distributions
3. Multivariate probability distributions
4. The normal distribution
5. Conditional probabilities and Bayes Theorem
INTRO
Why is probability densitiy destimation useful in ML?:
- A lost easier to classify data if you have the underlying distributions.
- Build up a probability distribution from previous instances.
- See where a new test case lies on the distribution and infer how likely it is to belong to the same class.
- Understand how distribution from two or more classes overlaps, to inform choice of machine learning algorithm.
- See how distribution fits with assumptions of chosen model/algorithm.
1. Probability Density Functions
1) Probability Theory
2) Probability Density Functions
Function known as a probability density function (pdf):
*The higher the probability density at x and around x, the more likely that the value of a datapoint will be close to x.
- Area under the curve gives the probability so if pdf is p(x),
- Area under the whole curve must sum to 1.
2. Properties / Parameters of Distributions
- Distributions and pdfs are often described/specified by parameters, commonly mean and variance.
- The mean is the usual average, also known as the expected value E(x) of <X>:
- Variance governs the spread of the data and is the square of the standard deviation. It is given by the expected square of the deviation from the mean:
- Other notations
- Example:
Fair 6-sided dice: P(X=1) = P(X=2) = ... = P(X = 6) = 1/6