EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera - CVPR 2020 (Oral)

Abstract: The high frame rate is a critical requirement for capturing fast human motions. In this setting, existing markerless image-based methods are constrained by the lighting requirement, the high data bandwidth and the consequent high computation overhead. In this paper, we propose EventCap — the first approach for 3D capturing of high-speed human motions using a single event camera. Our method combines model-based optimization and CNN-based human pose detection to capture high frequency motion details and to reduce the drifting in the tracking. As a result, we can capture fast motions at millisecond resolution with significantly higher data efficiency than using high frame rate videos. Experiments on our new event-based fast human motion dataset demonstrate the effectiveness and accuracy of our method, as well as its robustness to challenging lighting conditions. L. Xu, W. Xu, V. Golyanik, M. Habermann, L. Fang and C. Theobalt, EventCap: Monocular 3D Capture of High-Speed Human Motions using an Event Camera, Proc. of IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2020 (Oral)