Can LLM Drive Robot to Another City?

⌨️ Code I created: https://github.com/Grigorij-Dudnik/Ro... Can an LLM agent drive a robot from one city to another, through mud, snow, ice, fields and forests, completely by itself? In this video I'm showing the full story of building an autonomous robot navigation system on top of the Earth Rover Mini+ - a four-wheel offroad robot with GPS-RTK, cameras and a SIM card. I extended my RoboCrew framework to support it, giving the embodied LLM agent tools to move and turn, and feeding it augmented camera views plus a rotating map showing robot orientation and bearing to the target. The LLM agent running on the Gemini Flash, which in my experience works far better than other models for AI robot control tasks like this. Getting the basics working was just the beginning though - the real challenge turned out to be making the robot actually navigate long distance without constantly flipping over, getting stuck in mud, or spinning in circles because the compass was lying. A huge chunk of this project ended up being raw robot compass calibration - the Earth Rover's built-in orientation data falls apart the moment the robot hits uneven terrain, because magnetic bias from the onboard motors corrupts the readings. I went low-level into the magnetometer, gyroscope and accelerometer data and wrote my own sensor fusion solution, including magnetic declination correction, that gives reliable heading even when the robot is tilted. After fixing navigation, I kept running into new problems across multiple days of offroad robot testing - mud that swallowed the wheels, corn roots, steep embankments, rate limits from the Gemini API mid-run, and eventually complete darkness. The final run is 1.5 kilometers from one city to another, through genuinely rough terrain. Will our robot be up for that challange? 0:00 - Intro 0:55 - Earth Rover Intro 1:12 - Defaut LLM agent tests 2:42 - Earth Rover SDK 4:20 - Augmenting camera images 4:49 - First try outdors (Run 1) 5:35 - Improving prompt and camera aug 6:12 - Second try (Run 2) 7:55 - Map augmentation improvements 8:34 - Autonomous ride through park (Run 3) 10:43 - Making robot think faster 11:24 - A few hundred meters try (Run 4) 12:48 - Solving wrong compas orientation 14:54 - Figting magnetic biases 16:25 - Declination correction 16:55 - Going completely offroad (Run 5) 20:44 - Ride summary and problems solving 21:51 - Open source models 22:52 - Let's sum up 23:55 - The Grand Finale (Final Run) 26:08 - First problems 28:15 - Ground becames impossible to pass 28:48 - It moves! 33:24 - Better architecture for LLM navigation 34:30 - Will it cross a road? 36:43 - Into the night 39:32 - Ride summary 39:58 - Outro