Local models

SurfSense can download a model and run it on your computer, with its own built-in runtime. You don't need to install anything else. Once a model is downloaded, chat works with the network off, and your questions and documents never leave your computer. A chat that uses plugins is the exception: their servers get what the agent sends them.

Download a chat modelCopy Anchor Link

What you see on each modelCopy Anchor Link

• Your computer. At the top, SurfSense shows what it found: Graphics card, Apple Silicon GPU or Runs on your processor, and how much memory it can use. Every model below is priced against this computer. You don't run a scan yourself. If it reads Graphics card not detected by the runtime. Reinstall to fix, your computer has a graphics card that SurfSense can't use. See GPU support.
• Builds. A model comes in several builds: the same model compressed by different amounts. A smaller build needs less memory and loses a little quality. Each row leads with the build that suits your computer best, and the others are one click away.
• Fit. A build with no badge runs at full speed. Reduced speed means part of it runs on the processor, so it works but is slower. Won't fit means your computer doesn't have the memory, and the download is refused.
• Vision. The model can read images you attach in chat.

SurfSense also lowers how much a model can read at once, its context window, when memory is tight, so the model stays on the graphics card instead of slowing down.

Search Hugging FaceCopy Anchor Link

Not every model is in the tested list. Under All models, search Hugging Face for any model in the GGUF format. In onboarding, click Not listed? Search Hugging Face.

• Results are sorted by downloads. They are not reviewed by SurfSense.
• Open a result to see its builds. Sizes are exact. Fit is estimated and checked before download. An estimate is marked with ~.
• SurfSense checks the file before it downloads anything. A file that isn't a chat model, or doesn't fit, is refused with the reason.
• A repository marked Needs an account can't be downloaded. SurfSense does not sign in to Hugging Face.

Searching sends what you type to huggingface.co as you type. If you turned that off, search says so, and tested and downloaded models keep working.

DownloadsCopy Anchor Link

• One at a time. A second download waits for the first: Waiting for the download ahead of it.
• Keep working. A download carries on when you close Settings, and you get a message when it ends.
• Cancel stops a download. Starting it again picks up where it stopped.
• Checked. Each file is checked against the exact file SurfSense expects. If it doesn't match, the download fails with The downloaded file did not match the expected one. Retry the download.
• Disk space. SurfSense checks there is room before it starts and tells you how much the download needs.

Models are saved in the models folder of your data folder. If you already have a GGUF file, you can copy it there and it appears under This computer. For a model that reads images, name its vision file mmproj-<model>.gguf, after the model's file name. SurfSense tells you the name to use if it finds one that doesn't match.

Delete a modelCopy Anchor Link

In Settings › Text gen, find the model under This computer and click Delete. Delete model removes its files from your computer. If it is the model in use, you choose another afterwards. Files another model still needs stay.

GPU supportCopy Anchor Link

• Vulkan comes with your graphics driver. On Windows there is nothing to install. On Linux, install your distribution's Vulkan loader (libvulkan1 on Debian and Ubuntu, vulkan-loader on Fedora) and an up-to-date driver.
• CUDA is not used. NVIDIA cards run through Vulkan.
• Integrated graphics count as the processor. Graphics built into a PC processor share its memory, so SurfSense sizes models against your system memory.
• One card. When a computer has several graphics cards, SurfSense uses the first one the runtime reports and never adds their memory together.
• On a Mac, the GPU and the processor share one pool of memory, and SurfSense leaves part of it for macOS and your other apps.

If your computer has a graphics card but SurfSense reports Graphics card not detected by the runtime. Reinstall to fix, update the graphics driver, then reinstall SurfSense. On Linux, check the Vulkan loader is installed.

MemoryCopy Anchor Link

A local model needs room for itself and for what it is reading. If other apps fill your memory, replies get slower. Settings › Resources shows how much memory, graphics memory and processor time SurfSense and your other apps are using.

SurfSense keeps one chat model loaded at a time, and serves several replies from it at once. Making an image or voicing a podcast on this computer briefly unloads the chat model to free memory. A chat sent meanwhile waits, then the chat model loads again.

RelatedCopy Anchor Link

• Models: local, server or plan, and which models run Agentic.
• Image, voice and search models: local image, podcast and embedding models.
• Privacy: what leaves your computer, and how to switch it off.