How I-JEPA Works

We dive into the I-JEPA Paper by Meta AI, McGill, Mila, and NYU, a non-generative approach for self-supervised learning from images. -- Get Oxen 🐂 https://oxen.ai/ Oxen.ai makes versioning your datasets as easy as versioning your code! Even is millions of unstructured images, we quickly handle any type of data so you can build cutting-edge AI. -- ImageNet 🔢 https://www.oxen.ai/ox/ImageNet-1k Paper 📜 https://arxiv.org/abs/2301.08243 I-JEPA 📜 https://www.oxen.ai/blog/arxiv-dives-... Masked Auto Encoder 📜 https://arxiv.org/abs/2111.06377 Latent Space Diagram 📜   / vision-transformer-vit-under-the-magnifyin...   Join Arxiv Dives 🤿 https://oxen.ai/community Discord 🗿   / discord   -- Chapters 0:00 Intro to I-JEPA 2:05 Semantic Image Representations 3:05 Latent Representation 4:30 Invariance Based Pre-Training 5:45 Generative Pre-Training 6:53 What is I-JEPA 9:07 I-JEPA vs. Previous Approaches 11:55 ViT Method 13:00 Sampling Context and Targets 15:35 Prediction and Loss 17:30 Latent Space 19:49 Attention Head 22:50 Evaluation on Image Classification 30:20 Conclusion and Conversation