Yoshua Bengio - Towards Neural Nets for Conscious Processing and Causal Reasoning

A major gap between current state-of-the-art deep learning and human generalization abilities regards out-of-distribution scenarios, where our best AI systems suffer a significant drop in accuracy, compared with us. Interestingly, when humans are confronted with new or surprising situations, they tend to switch from system-1 types of behaviors relying on quick habitual responses to system-2 types of cognitive processes, which are slower, require conscious attention and generate verbalizable thoughts. This form of computation seems to rely on a modular decomposition of knowledge into pieces that can be recombined in novel ways on-the-fly using attention mechanisms to sequentially generate these pieces forming the elements of our thoughts, suggesting that this provides a form of more powerful systematic generalization than the system-1 habitual responses. In this presentation, we will describe a research plan and inductive biases for introducing this type of system-2 knowledge representation, inference and learning in neural networks, as well as early results on the neural machinery we propose for this, called GFlowNets. Recognized worldwide as one of the leading experts in artificial intelligence, Yoshua Bengio is most known for his pioneering work in deep learning, earning him the 2018 A.M. Turing Award, “the Nobel Prize of Computing,” with Geoffrey Hinton and Yann LeCun. In 2019, he was awarded the prestigious Killam Prize and in 2021, became the second most cited computer scientist in the world. He is a Fellow of both the Royal Society of London and Canada and Officer of the Order of Canada.