Connecting another server
Point Aldwin at any OpenAI-compatible chat server, such as vLLM, llama.cpp or a company gateway.
On this page
Aldwin can talk to any server that speaks the OpenAI chat format. That covers servers you run yourself, like vLLM or llama.cpp, and gateways your company runs in front of its models. This guide shows how to add one with Other server in /model.
What counts as another server#
A server works with Aldwin if it offers two OpenAI-style routes under one base address:
GET /models, which lists the models it servesPOST /chat/completions, which answers chat messages
Most tools that say "OpenAI-compatible API" do this. Some common ones:
| Server | Typical address | Notes |
|---|---|---|
| vLLM | http://localhost:8000/v1 | Start it with vllm serve <model>. For tool use, add --enable-auto-tool-choice and the --tool-call-parser for your model. |
llama.cpp llama-server | http://localhost:8080/v1 | Add --jinja so the model's tool format is used. Pick another port if SearXNG already uses 8080. |
| LiteLLM proxy | http://localhost:4000/v1 | One address in front of many providers. Use the proxy key it gives you. |
| A company gateway | https://... | Ask whoever runs it for the base address and a key. |
Ollama and LM Studio have their own guides: Ollama, LM Studio.
What you need#
- The server running, and its base address (usually ending in
/v1) - A key, if the server asks for one
- Aldwin installed (
uv tool install aldwin)
Aldwin's tools work only with models that can call tools. Check your server's docs for how to turn tool calling on.
Connect Aldwin#
In the web app#
- Open Settings from the bottom of the sidebar and choose Models.
- Select Add a server.
- Give it a short name, such as
office, and its address, such ashttp://localhost:8000/v1. Add an API key only if the server needs one. - Select Connect. Aldwin asks the server for its model list.
- Under Model in use, pick one of the server's models.
In the terminal#
- Start Aldwin and type
/model. - Pick Other server and press
Enter. - At
Server address (e.g. http://localhost:8000/v1):, type the base address and pressEnter. - At
API key (Enter to skip):, paste the key (the input is hidden), or pressEnterif the server has none. - At
Provider name (e.g. office-gpu):, type a short name. Use lowercase letters and numbers, joined with_if you need two words, such asofficeoroffice_gpu. Avoid dashes in the name if the server needs a key. - Aldwin asks the server for its model list. When that works you see
✓ Added provider office. - Your server now appears under Available providers in
/model. Pick it, then pick a model.
Addresses on your own computer (localhost or 127.0.0.1) can use http://. Any other address must use https://, so prompts and keys are encrypted on the way.
What Aldwin saves#
The server goes under providers: in ~/.aldwin/config.yaml:
providers:
office:
shape: openai_chat
base_url: https://llm.example.com/v1
title: office
local: false
secret: office_api_keyThe key goes into ~/.aldwin/secrets.yaml as office_api_key. You can also set it as an environment variable named after the provider, such as OFFICE_API_KEY. An environment variable wins over a saved key.
You can write this entry by hand instead of using /model. Aldwin reads it the next time it starts.
Try it#
aldwinType /model, pick your server and a model, then ask:
List the files in this folder.
If the model can call tools, you see a list_dir step and then the answer. The status line at the bottom shows the model and your provider name.
Privacy notes#
- Prompts and files Aldwin reads go to the server you added, and from there wherever that server sends them.
- A server on your own computer keeps everything on your machine.
- Details: Privacy.
Troubleshooting#
| What Aldwin says | Fix |
|---|---|
Invalid server address '...'. Address must begin with http:// or https://. | Type the full address, including http:// or https://. |
Non-local server '...' must use https://. | Use the server's https:// address. Plain http:// is allowed only for your own computer. |
Could not connect to server at ... | The server isn't running, the address is wrong, or the key was refused. Check that curl <address>/models works, then try again. Aldwin waits up to 5 seconds. |
Your office key was rejected. ... | The server refused the key. Check it with whoever runs the server. |
office isn't running. Start it and try again. | The server on your computer stopped. Start it again. |
Model ... isn't available to your key. ... | The model name isn't served. Pick one from the list in /model. |
| The model answers but never uses tools | Turn on tool calling in the server (see the table above), or pick a model that supports tools. |