Skip to content
Wayland Docs

Use a Local Model

You can run Wayland against a model on your own machine. A local server that speaks the OpenAI API, such as Ollama or vLLM, connects through the OpenAI-compatible provider. Nothing leaves your machine for inference.

As of Wayland Core v0.12.15, a local self-hosted endpoint is keyless: a local Ollama or llama.cpp endpoint needs no provider key at all. The engine recognizes a local base URL and stops asking for an OpenAI key, so you can run local models fully private and free with zero keys.

In the desktop

Section titled “In the desktop”
  1. Open Settings > Models and choose the OpenAI-compatible provider.
  2. Set the base URL to your local server. For Ollama, that is a URL like http://localhost:11434/v1.
  3. Set the model id to a model your server has, for example qwen2.5:32b.
  4. Save, then pick that model in a conversation to use it. A local endpoint needs no API key, so you can leave the key field empty.

In the engine

Section titled “In the engine”

Wayland Core reaches a local model the same way, through a profile that uses the OpenAI-compatible path:

[profiles.ollama]
provider = "openai"
model = "qwen2.5:32b"
base_url = "http://localhost:11434/v1"

No api_key line is needed for a local endpoint. The engine treats a local self-hosted base URL as keyless and connects without one.

Then run with that profile:

Terminal window
wayland-core --profile ollama "Summarize this directory"

The engine also has a dedicated ollama: route through a bundled plugin, where the model string carries the model name, for example --model ollama:qwen3-coder. That route reads OLLAMA_BASE_URL if you need to point it at a different endpoint.

Tool-incapable models degrade gracefully

Section titled “Tool-incapable models degrade gracefully”

Not every local model can do native tool calls. The engine’s capability-first tools gate handles this: a tool-incapable model (some Ollama and llama.cpp models, for instance) degrades gracefully instead of erroring, and the engine remembers the choice per model so it does not re-probe a model it already knows. You can run a small local model for plain chat and a tool-capable one for agent work without hand-tuning anything. See the Capability Status Matrix for the current state.

Verify it

Section titled “Verify it”

Send a short test message. If you get a reply, the local model is wired up. If the engine cannot reach the server, confirm it is running and that the base URL and port are correct.

Next

Section titled “Next”