Lecture 17: 3D Vision

Lecture 17 discusses ways for incorporating 3D structure into neural networks. We talk about different representations for 3D data including depth maps, voxel grids, implicit functions, point clouds, and triangle meshes. We show how graph convolution is a neural network primitive that respects the structure of triangle meshes. We discuss metrics for comparing 3D shapes including intersection over union (IoU), Chamfer distance, and F1 scores. We briefly talk about camera systems, and specifically the difference between canonical and view-centric coordinate systems. We discuss datasets for 3D shape prediction including ShapeNet and Pix3D, and discuss the Mesh R-CNN architecture for performing joint object detection and 3D shape prediction. Slides: http://myumi.ch/jxDrl _________________________________________________________________________________________________ Computer Vision has become ubiquitous in our society, with applications in search, image understanding, apps, mapping, medicine, drones, and self-driving cars. Core to many of these applications are visual recognition tasks such as image classification and object detection. Recent developments in neural network approaches have greatly advanced the performance of these state-of-the-art visual recognition systems. This course is a deep dive into details of neural-network based deep learning methods for computer vision. During this course, students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in computer vision. We will cover learning algorithms, neural network architectures, and practical engineering tricks for training and fine-tuning networks for visual recognition tasks. Course Website: http://myumi.ch/Bo9Ng Instructor: Justin Johnson http://myumi.ch/QA8Pg