【深層学習】GPT-2 - 大規模言語モデルの可能性を見せ、社会もざわつかせたモデルの仕組み【ディープラーニングの世界vol.33】#113 #VRアカデミア #DeepLearning

This article introduces GPT-2, a language model that was previously shelved after its ability to generate supernatural sentences caused a stir. Staying away from the social sensationalism, we'll explain what kind of model it is, what task it solves, how it solves it, and what makes it so great! ▼Related Video Watch the Transformer video here!    • 【深層学習】Transformer - Multi-Head Attentionを理...   For busy people →    • 【深層学習】忙しい人のための Transformer と Multi-Head At...   GPT →    • 【深層学習】GPT - 伝説の始まり。事前学習とファインチューニングによるパラダイム...   The World of Deep Learning    • Deep Learning の世界   Natural Language Processing Series    • 自然言語処理シリーズ   ▼References Radford, Alec, et al. "Language models are unsupervised multitask learners." OpenAI blog 1.8 (2019): 9. http://www.persagen.com/files/misc/ra... Original paper! People were excited about this amazing AI, training a huge language model with huge data sets, but behind the scenes there was a lot of thoughtful, meticulous design and effort. As people who create AI, I think it would be good to read this book to understand what lies behind the spectacular results. He, Kaiming, et al. "Identity mappings in deep residual networks." European conference on computer vision. Springer, Cham, 2016. https://link.springer.com/chapter/10.... Residual connection is not just a solution to vanishing gradients, it's also an element in identity mapping learning. This paper is written with that philosophy in mind. I like it. Here's a video on ResNet explaining its philosophy:    • 【深層学習】 CNN 紹介 "ResNet" 言わずとしれた CNN の標準技術が登...   Reddy, Siva, Danqi Chen, and Christopher D. Manning. "Coqa: A conversational question answering challenge." Transactions of the Association for Computational Linguistics 7 (2019): 249-266. https://direct.mit.edu/tacl/article/d... This is a paper on the CoQA dataset, one of the tasks. It's a collection of TOEIC-style questions, and it's amazing that a deep learning model can solve them. Even just flipping through it should give you a good idea! [2019 Edition] Summary of Representative Natural Language Processing Models and Algorithms Timeline - Qiita https://qiita.com/LeftLetter/items/14... I use this as a reference for various videos. ▼In Closing Thank you for watching! If you enjoyed the video, please like and subscribe. If you have any questions or comments about the video, please leave a comment or message me on Twitter! For business or collaboration requests, please contact me via my official website or Twitter DM. AIcia Solid Project - Official Website - https://sites.google.com/view/aicia-o... Video Creation: AIcia Solid (Twitter:   / aicia_solid  ) Video Editing: AIris Solid (Younger Sister) (Twitter:   / airis_solid  ) ======= Logo: TEICA (  / t_e_i_c_a  ) Model: http://3d.nicovideo.jp/works/td44519 Model by: W01fa (  / w01fa  )