Running Gemma 4 Local AI

How to run Gemma 4 open local AI models in LM Studio on a computer with integrated graphics (like most laptops and mini PCs). Although everything here will also work just fine on a system with a dedicated graphics card. You can download the version of LM Studio shown in the video here: https://lmstudio.ai/download I note this because, in the days since the video was made, a new agentic application called "LM Studio Bionic" has been released, and it is this program that is linked on the LM Studio homepage. If you enjoy this video, you may also like my episode on Agentic AI:    • Explaining Agentic AI: The Good, the Bad &...   CONFIGURATION NOTES There are lots of settings that you can experiment with when running LLMs like Gemma 4 locally, especially if you have a powerful dedicated GPU with a lot of VRAM. But on my system with integrated graphics, I experimented with the following changes: CONTEXT LENGTH: Go to “My Models”, click on the gear icon to access the settings for a particular model, and then click on the “Load” tab. Here you can set the “Context Length” for a model, which on my system for Gemma 4 12B defaulted to 8192 tokens. However, I tried reducing this to 4096 to save resources and increase speed. But I put this back to 8196 for the test summarizing the 3900 word book chapter. CPU THREAD POOL SIZE: in the settings for a model, under the “Inference” tab you will find a control for the “CPU Thread Pool Size”, which by default on my system was set to 4. I increased this to 6, which is the number of physical CPU cores in my Ryzen 5600G processor, and setting the CPU Thread Pool Size to your number of physical CPU cores is often advised. Having made these two changes, I obtained a slight increase in tokens a second. But it was very slight, and less than the variance when running different tasks. So because of this I chose not to include a section in the video explaining this. The video was already long, and I thought it better to get to the practical use of Gemma 4 five minutes earlier! :) I also conducted many experiments with my BIOS settings in attempt to run all or part of a Gemma 4 model on the 5600G’s integrated GPU (iGPU). But Gemma 4 12B QAT cannot be partially “offloaded”, and there are limits on the amount of RAM that an iGPU can access (well, at least on older/traditional hardware ike a 5600G APU). Using the GPU Offload setting, I did manage to offload part of the Gemma 4 E4B Q_K_M model to the iGPU, but with no practical advantage (due I imagine to the model remaining in the same RAM, and bottle necks being created with the model split between the CPU and GPU). I did find it beneficial to off-load the model’s Key-Value (KV) cache to the GPU, and did this with a fixed 2GB RAM allocated to the iGPU. My conclusion this far is to trust LM Studio to get the settings right, and only to adjust the Context Length when required to process large text files. --- You can access most ExplainingComputers videos by category from this page: https://explainingcomputers.com/video... More videos on film and other making, plus retro tech, can be found on my Christopher Barnatt channel:    / @christopherbarnatt   Chapters: 00:00 Titles & Intro 00:56 Choosing Your Model 04:25 LM Studio 08:51 Speed & Application 13:55 Images & Text Files 17:20 Coding 18:57 Gemma 4 E4B 21:31 Wrap #Gemma4 #12B #E4B #LMStudio #LocalAI