Топ вопросов NLP собеседований | трансформер, механизм внимания, берт | Часть 1

The secret to successful employment is meticulous interview preparation! Knowledge of the attention mechanism and transformers is the foundation for any interview at any level in NLP! Contents: Transformer architecture Attention mechanism Positional embeddings Tokenization Transformer architectures (BERT, GPT, etc.) Timecodes: 00:00 What awaits you? 00:16 Welcome 00:51 What is a transformer? 01:12 How does a transformer differ from other architectures? Compare it with RNN and CNN 03:17 What does a transformer architecture look like? What parts does it consist of? 05:41 How does the attention mechanism work? 08:04 What types of attention are there? 11:33 Where can I find useful materials for a more detailed study of this information? 11:52 Why are positional embeddings needed? 12:31 How does positional encoding work? 13:02 What types of positional encoding are there? 14:40 What types of tokenizers are there? 15:15 Why Byte-Level? Why not split by charms? 16:08 What are the main hyperparameters of tokenizers? 17:49 How can you expand the vocabulary? 18:35 The most effective way to get a job in NLP 19:11 What transformer architectures are there? 20:38 What is BERT? 23:01 What other problems can encoder models solve? 24:31 What are bi-encoder and cross-encoder? When should you use each type? 26:25 How to vectorize text using BERT? 29:01 What other types of Bert do you know? Notes with analyzed questions: https://t.me/rockaux Real interview reviews and lots of useful content: https://boosty.to/lokis_alexandr Turnkey offer mentoring, roadmap, and consultation: https://sites.google.com/view/lokisml... Personal Telegram: https://t.me/abletobetable