Skip to content
Wayland Docs

Use a Local Model

You can run Wayland against a model on your own machine. A local server that speaks the OpenAI API, such as Ollama or vLLM, connects through the OpenAI-compatible provider. Nothing leaves your machine for inference.

In the desktop

Section titled “In the desktop”
  1. Open Settings -> Models and choose the OpenAI-compatible provider.
  2. Set the base URL to your local server. For Ollama, that is a URL like http://localhost:11434/v1.
  3. Set the model id to a model your server has, for example qwen2.5:32b.
  4. Save, then pick that model in a conversation to use it.

In the engine

Section titled “In the engine”

Wayland Core reaches a local model the same way, through a profile that uses the OpenAI-compatible path:

[profiles.ollama]
provider = "openai"
model = "qwen2.5:32b"
api_key = "ollama"
base_url = "http://localhost:11434/v1"

Then run with that profile:

Terminal window
wayland-core --profile ollama "Summarize this directory"

The engine also has a dedicated ollama: route through a bundled plugin, where the model string carries the model name, for example --model ollama:qwen3-coder. That route reads OLLAMA_BASE_URL if you need to point it at a different endpoint.

Verify it

Section titled “Verify it”

Send a short test message. If you get a reply, the local model is wired up. If the engine cannot reach the server, confirm it is running and that the base URL and port are correct.

Next

Section titled “Next”