Brain-Aligned Tactile Representations for Dexterous Robot Learning [Trinity Chung CMU MSR Thesis]

Trinity Chung CMU MS Robotics Thesis Defense Talk. Thursday, June 25, 2026. Event Details: https://www.ri.cmu.edu/event/brain-al... Title: Brain-Aligned Tactile Representations for Dexterous Robot Learning Abstract: Touch is the essential sensory modality through which animals and robots physically negotiate the world. While much of robotic touch focuses on the capabilities of currently available tactile hardware, this thesis asks a more general question: what forms of tactile processing and representation could allow robots to approach the dexterity of animals? This thesis argues that simulated force and torque provide a privileged representation of touch: one that is biologically grounded, computationally scalable, and effective for dexterous behavior. For biological touch, the rodent vibrissal pathway provides a tractable model system for studying tactile processing. We train task-optimized temporal networks on realistic whisker force/torque sequences to identify shapes and compare their internal representations against recordings from rats' barrel cortex. We find that convolutional recurrent models (ConvRNNs) align very closely with neural data, and contrastive self-supervision with tactile-specific augmentations matches supervised alignment as a label-free proxy. In robotic touch, we test whether the same class of representation is useful for dexterous behavior. We develop a GPU-parallel tactile sensor simulator that exposes a family of tactile abstractions under one interface, from binary contact and per-taxel force/torque to marker displacement and temperature, fast enough to serve as the front-end for dexterous reinforcement learning. For in-hand manipulation tasks, we find that sensor placement matters more than sensor type or resolution, that whole-hand coverage closes most of the gap to a privileged teacher. Across 3 dexterous tasks, per-taxel force/torque emerges as the most useful and robust observation. Together, these results argue that force and torque are not merely convenient signals to simulate, but a biologically grounded and behaviorally effective representation of touch. With recurrence and broad spatial coverage, simulated force and torque provide a common representational substrate for understanding tactile computation in brains and building tactile intelligence in robots. Committee: Prof. Aran Nayebi (Chair, CMU) Prof. Katerina Fragkiadaki (CMU) Akash Sharma (Amazon Frontier AI & Robotics) Timestamps: 00:00 Introduction 03:11 Task-Optimized ConvRNNs and Rodent Tactile Processing 13:03 Tactile Genesis: Tactile Sensors for Dexterous Tasks 21:57 Takeaways 23:35 Questions Papers: 1. T. Chung*, Y. Shen*, N.C.L. Kong, A. Nayebi. "Task-optimized convolutional recurrent networks align with tactile processing in the rodent brain". Advances in Neural Information Processing Systems (NeurIPS) Oral, Volume 38 (2025): 68925-68951. https://arxiv.org/abs/2505.18361 2. T. Chung, K. Yamazaki, D. Patel, A. Duburcq, Y. Qiao, K. Fragkiadaki, A. Nayebi. "Tactile Genesis: exploring tactile sensors at scale for learning dexterous tasks". 2026. https://arxiv.org/abs/2606.22332 Trinity's website: https://trinityjchung.com/