How did the Attention Mechanism start an AI frenzy? | LM3

The attention mechanism is well known for its use in Transformers. But where does it come from? It's origins lie in fixing a strange problems of RNNs. Support me on Patreon!   / vcubingx   Language Modeling Playlist:    • Language Modeling   3blue1brown series on Transformers:    • Transformers, the tech behind LLMs | Deep ...   The source code for the animations can be found here: https://github.com/vivek3141/dl-visua... These animation in this video was made using 3blue1brown's library, manim: https://github.com/3b1b/manim Sources (includes the entire series): https://docs.google.com/document/d/1e... Chapters 0:00 Introduction 0:22 Machine Translation 2:01 Attention Mechanism 8:04 Outro Music (In Order): Helynt - Route 10 Helynt - Bo-Omb Battlefield Helynt - Underwater Philanthrope, mommy - embrace https://chll.to/7e941f72 Helynt - Twinleaf Town Follow me! Website: https://vcubingx.com Twitter:   / vcubingx   Github: https://github.com/vivek3141 Instagram:   / vcubingx   Patreon:   / vcubingx