Machine Learning from First Principles
Derive, implement from scratch and evaluate the classical ML algorithms, from linear regression to neural networks and clustering.
34 lessons · about 42 hours
What you'll be able to do
- Derive the classical ML algorithms from the math in Courses 1–3
- Implement them from scratch in NumPy
- Evaluate models correctly and explain every equation in a scikit-learn model's documentation
Before you start
Courses 1, 2 and 3, or equivalent knowledge.
Syllabus
Module 1
Linear regression
Simple and multiple regression, least-squares and MLE views, gradient-descent solution, ridge regression, the bias–variance decomposition.
4 lessons coming soon
Module 2
Model evaluation and regularisation
Train, validation and test splits; k-fold and leave-one-out cross-validation; overfitting; L1 vs L2 regularisation; evaluation metrics.
3 lessons coming soon
Module 3
Logistic regression
The sigmoid, cross-entropy from MLE, gradient derivation, convexity, softmax and multinomial logistic regression.
4 lessons coming soon
Module 4
Generative classifiers
Naive Bayes, Gaussian discriminant analysis, LDA revisited, generative vs discriminative models.
3 lessons coming soon
Module 5
Nearest neighbours and distance
k-NN, distance metrics, the curse of dimensionality.
3 lessons coming soon
Module 6
Support vector machines
Margins, hard and soft margin, the primal problem, the Lagrangian dual and KKT conditions, kernels and the kernel trick, hinge loss.
5 lessons coming soon
Module 7
Decision trees
Entropy and Gini impurity, information gain, growing and pruning trees.
3 lessons coming soon
Module 8
Neural networks
Perceptron, multilayer perceptrons, universal approximation (intuition), backprop in matrix form, initialisation, the softmax–cross-entropy gradient.
5 lessons coming soon
Module 9
Unsupervised learning
k-means and k-medoids, hierarchical clustering (bottom-up and top-down), PCA and GMM/EM revisited.
4 lessons coming soon