Ollama not using GPU? Start by checking where your model is loaded, then investigate drivers, device selection, and available video memory. A slow reply alone does not tell you which component is responsible.
This guide covers the native Windows application, with separate NVIDIA and AMD checks. It is for local models; a cloud model does not run inference on your PC’s graphics card. The workflow is based on official documentation reviewed on October 24, 2026, not a hands-on benchmark.
1. Ollama not using GPU? Confirm the loading status
Open PowerShell and run:
ollama listChoose a local model you already have. In the next command, replace YOUR_MODEL with its exact name and tag; do not type the placeholder literally.
ollama run YOUR_MODEL "Write one sentence about rain."
ollama psRun the second command promptly, while the model is still loaded. The PROCESSOR column reports placement: 100% GPU means fully on the GPU, 100% CPU means system memory, and a CPU/GPU split means partial GPU loading. See the official Ollama FAQ.
| Result | Next action |
|---|---|
| 100% GPU | GPU loading works. Investigate the workload or application delay. |
| 100% CPU | Continue with the driver and device checks below. |
| CPU/GPU split | Start with the memory comparison in step 6. |
| No model listed | Run a local model again before drawing a conclusion. |
| Cannot connect to server | Restore the server connection before diagnosing acceleration. |
When investigating Ollama not using GPU, focus on whether your model is loaded on the expected device. Save this first result so you can compare it after each change.
2. Restart Ollama and check the correct server
Before investigating Ollama not using GPU, establish a repeatable test. Save your work, finish active generations, quit Ollama from its tray menu, and reopen it. If an update is ready, use the application’s restart-to-update option.
Repeat the same model and prompt from step 1. Changing the model and driver together makes it harder to tell which change mattered.
Ollama’s Windows application runs a background server. Its usual local API address is http://localhost:11434, according to the Windows documentation. A separate chat interface may point to another computer, WSL, or a container.
Compare the interface’s configured server address with your intended local installation. If the interface uses a different server, checking your Windows installation will not diagnose that server’s hardware. For broader connection troubleshooting, see our AI API integration guide.
3. Check NVIDIA support and the driver
For an NVIDIA system with Ollama not using GPU, run these read-only commands:
nvidia-smi
nvidia-smi -LThe first displays driver and device information; the second lists GPU identifiers. NVIDIA explains the output in its nvidia-smi reference. Record the card name and driver version before changing anything.
Check both Ollama’s hardware compatibility page and Windows requirements. At review time, the hardware page requires compute capability 5.0 or higher and driver 550 or newer, with driver 570 or newer for compute capability 5.0–6.2. The Windows page specifies NVIDIA driver 551.61 or newer. Meet the stricter applicable requirement for your card and operating system.
If a driver update is needed, obtain a compatible driver from NVIDIA or your PC manufacturer’s official support channel. Complete the installation and any requested restart, then repeat your baseline test.
A command-not-found message is not proof that the card is unsupported. Check the installed driver and tool availability. Conversely, a successful device listing confirms visibility to NVIDIA’s utility, not successful model loading inside Ollama.
4. Check the AMD backend and compatibility
On AMD hardware, Ollama not using GPU needs a different compatibility check. NVIDIA commands and CUDA settings do not diagnose a Radeon installation.
The current Windows documentation describes ROCm acceleration through a ROCm v7 / HIP7-capable driver stack and Vulkan acceleration through a Vulkan-capable Radeon driver. It recommends Vulkan as a fallback for some Radeon RX 6000 systems that do not expose the required ROCm support.
Find your exact card in Ollama’s current hardware guidance. Do not assume that a Linux compatibility entry also applies to Windows. On a laptop, record both integrated and discrete graphics devices rather than reporting only the processor’s brand.
Update the appropriate supported driver, restart Ollama, and repeat the same local model test. If it still fails, collect the backend and device information from the logs in step 7 before trying an override copied from an unrelated setup.
5. Review settings that hide or override the GPU
If Ollama not using GPU began after a tutorial or configuration change, inspect the settings it introduced. An invalid CUDA_VISIBLE_DEVICES value can hide NVIDIA GPUs; Ollama documents -1 as a way to force CPU use.
Search Windows for Edit environment variables for your account. Record existing values before editing. Check user variables and, if you manage the computer, system variables. Focus on settings you intentionally added, including CUDA_VISIBLE_DEVICES and OLLAMA_LLM_LIBRARY.
Remove an obsolete user override only after confirming why it was set. On a shared or managed workstation, ask the administrator about system-wide settings. Quit and relaunch Ollama after the change so the server inherits the updated environment.
Changing a variable in one PowerShell window does not reconfigure an already running tray application. That distinction can explain why a seemingly correct fix had no effect. Restore the recorded value if the change does not help or affects another workload.
6. Ollama not using GPU with larger models: compare memory demand
A CPU/GPU split is different from Ollama not using GPU at all. Treat it as a capacity clue before assuming device discovery is broken.
Use a smaller local model already installed on the same machine. Keep the server and driver unchanged. If that model loads fully on the GPU while your original one does not, the result points toward the larger workload’s requirements.
Context length also matters: increasing it raises memory requirements. For a short diagnostic prompt, you can try a 4096-token context in an interactive session:
ollama run YOUR_MODELAt the model’s interactive prompt, enter:
/set parameter num_ctx 4096
Write one sentence about rain.Check ollama ps from a second terminal. This is a diagnostic setting, not a universal recommendation for your real workload. A task that needs a longer document may need its original context restored.
See Ollama’s context-length guidance for the memory tradeoff. Close your own unnecessary GPU-heavy applications before repeating the comparison, and record whether the loading split changes.
If your request instead exceeds a model’s input allowance, see our AI token-limit troubleshooting guide. That is a different symptom from hardware detection.
7. Read the logs and verify the fix
If Ollama not using GPU persists, inspect the current Windows server log:
Get-Content "$env:LOCALAPPDATA\Ollama\server.log" -Tail 100Look around the time of your test for device discovery, backend selection, or memory errors. Keep several surrounding lines; an isolated word such as “CUDA” does not establish that your model loaded successfully.
For additional diagnostics, Ollama’s troubleshooting documentation says to quit the tray application first, then launch it from PowerShell with debug logging:
$env:OLLAMA_DEBUG="1"
& "ollama app.exe"If the executable cannot be found, check the actual installation location instead of downloading a replacement from an unofficial source. After collecting the diagnostic output, quit that debug instance and reopen Ollama normally from Start.
When reporting Ollama not using GPU, include Windows version, Ollama version, GPU name, driver version, exact model tag, context setting, and before-and-after placement. Remove usernames, private paths, prompts, and other sensitive details from logs before sharing them publicly.
Call the GPU-loading issue resolved when your intended local model reports the expected placement and completes the same prompt. If it already reports full GPU placement but remains slow, compare a shorter prompt and a direct local request before changing more drivers.
Ollama not using GPU: frequently asked questions
Does partial CPU usage mean acceleration is broken?
No. A CPU/GPU split indicates that at least part of the model is on the GPU. Compare a smaller model and shorter context before treating the result as a detection failure.
Does Ollama not using GPU require a complete reinstall?
First record your baseline, compatibility details, overrides, and log errors. Reinstalling before collecting evidence can leave the underlying cause unidentified and makes before-and-after comparisons less useful.
Why does the problem affect only my chat interface?
If Ollama not using GPU affects only one interface, compare its exact server, model tag, and context with your terminal test. A successful local test narrows the investigation to differences between those two requests.
What should I try next?
For Ollama not using GPU, use the first failing check as your next action: server connection, hardware support, driver visibility, configuration, or workload size. For more related topics, browse AI Tools & Platforms.
Editorial note: This article was prepared with AI assistance from the linked primary documentation. The hero image is an AI-generated illustration; the diagrams are explanatory graphics. No hardware benchmark or guaranteed speed improvement is claimed.
