So You Think You Know How to Take Derivatives? | Steven Johnson | ASE60

So you think you know how to take derivatives? Derivatives are seen as the "easy" part of learning calculus: a few simple rules, and every function's derivatives are at your fingertips! But these basic techniques can turn bewildering if you are faced with much more complicated functions like a matrix determinant (what is a derivative "with respect to a matrix" anyway?), the solution of a differential equation, or a huge engineering calculation like a fluid simulation or a neural-network model. And needing such derivatives is increasingly common thanks to the growing prevalence of machine learning, large-scale optimization, and many other problems demanding sensitivity analysis of complex calculations. Although many techniques for generalizing and applying derivatives are known, that knowledge is currently scattered across a diverse literature, and requires students to put aside their memorized rules and re-learn what a derivative really is: linearization. In 2022 and 2023, Alan and I put together a one-month, 16-hour "Matrix Calculus" course at MIT that refocuses differential calculus on the linear algebra at its heart, and we hope to remind you that derivatives are not a subject that is "done" after your second semester of calculus. Contents 00:00 Welcome and introduction 02:15 Derivatives: the “easy” part of calculus? 06:41 Why matrix calculus matters 11:24 Automatic differentiation: from calculus to compilers 14:44 Derivatives as linearization 19:20 Gradients as linear operators 24:41 The chain rule and forward vs reverse mode 27:49 “Adjoint method” for engineering optimization 29:05 Final comments and questions S/O to https://github.com/agchesebro for the video timestamps! Want to help add timestamps to our YouTube videos to help with discoverability? Find out more here: https://github.com/JuliaCommunity/You... Interested in improving the auto generated captions? Get involved here: https://github.com/JuliaCommunity/You...