Do not FEAR p-values and regressions (a basic intro) - Week 6 - IAT 461

Statistical Significance, P-Values, Effect Size & Regression (Stats vs ML) This lecture explains statistical significance intuitively for data science research, focusing on populations vs. random samples, sampling error, and how larger samples reduce variability. It introduces hypothesis testing via the null hypothesis, clarifies what a p-value is (and is not), and demonstrates simulation-based approaches such as label-shuffling/randomization tests and bootstrapping to estimate p-values and 95% confidence intervals. The session contrasts statistical significance with practical importance using effect size (including Cohen’s d), discusses how sample size affects p-values, and covers Type I/II errors and power analysis. It then shifts to linear regression: residuals, ordinary least squares, interpreting slope and intercept, R², and p-values for models and coefficients, including multiple regression and “holding others constant.” Finally, it contrasts the statistics lens with a machine-learning lens emphasizing train/test splits, loss (RMSE), and overfitting. This course is offered by Dr. Alireza Karduni from school of Interactive Arts and Technology from Simon Fraser University. For more information visit https://datadialogue.vercel.app/ https://www.sfu.ca/siat.html 00:00 Lecture Kickoff 01:12 Why Sampling Matters 02:32 Population vs Sample 04:06 Sampling Distributions 06:11 Comparing Two Groups 06:50 Null Hypothesis Basics 10:53 What P Values Mean 13:49 Central Limit Theorem 18:06 Permutation Test Intuition 21:40 Effect Size vs Significance 24:07 Bootstrap and Confidence 27:03 Confidence Intervals Basics 27:58 Classic Tests vs Simulation 29:04 P Values and Effect Sizes 31:49 Power and What to Report 33:58 Linear Regression Intuition 36:16 Best Fit Line OLS 38:46 R Squared and Model P Value 41:48 Multiple Regression Coefficients 46:02 Machine Learning Evaluation 50:26 Wrap Up and Midterm Notes #datascience #machinelearning #statistics