Machine Learning | Linear Regression, Optimization, Regularization, Generalization and Extensions

Speakers: Loh Ken Yaw (Vincent) Ong Kok Rhui - Data Science student @ Monash University Malaysia References: Monash University Malaysia FIT5201 Machine Learning unit Week 3 Pattern Recognition and Machine Learning by Christopher Bishop An Introduction to Statistical Learning in Python by Gareth James et. al Timestamps: 00:00 Introduction to linear models for regression 02:03 Training error and Mean Squared Error (MSE) 03:36 Expected squared error and LOTUS principle 07:00 Central Limit Theorem and noise distribution 10:29 Basis functions and non-linear transformations 14:45 Optimizing parameters via partial derivatives 18:44 Gradient descent and stochastic gradient descent 23:53 Overfitting and training error vs generalization error 26:23 Regularization techniques to punish extreme weights 29:54 Ridge (L2) vs. Lasso (L1) regression 35:22 Simple and multiple linear regression overview 40:31 Estimating coefficients with the least squares approach 43:55 Assessing model accuracy and standard error of the mean 56:31 Hypothesis testing, null hypotheses, and p-values 1:02:09 Assessing model fit with RSE and R-squared statistics 1:09:57 Multiple linear regression and variable correlation 1:16:29 Using the F-statistic to test predictor relationships 1:23:48 Forward, backward, and mixed variable selection methods 1:29:50 Extending linear models and using interaction terms 1:40:20 Common issues and handling non-linearity in models