Skip to content
Guide

Connecting another server

Point Aldwin at any OpenAI-compatible chat server, such as vLLM, llama.cpp or a company gateway.

On this page

Aldwin can talk to any server that speaks the OpenAI chat format. That covers servers you run yourself, like vLLM or llama.cpp, and gateways your company runs in front of its models. This guide shows how to add one with Other server in /model.

What counts as another server#

A server works with Aldwin if it offers two OpenAI-style routes under one base address:

  • GET /models, which lists the models it serves
  • POST /chat/completions, which answers chat messages

Most tools that say "OpenAI-compatible API" do this. Some common ones:

ServerTypical addressNotes
vLLMhttp://localhost:8000/v1Start it with vllm serve <model>. For tool use, add --enable-auto-tool-choice and the --tool-call-parser for your model.
llama.cpp llama-serverhttp://localhost:8080/v1Add --jinja so the model's tool format is used. Pick another port if SearXNG already uses 8080.
LiteLLM proxyhttp://localhost:4000/v1One address in front of many providers. Use the proxy key it gives you.
A company gatewayhttps://...Ask whoever runs it for the base address and a key.

Ollama and LM Studio have their own guides: Ollama, LM Studio.

What you need#

  • The server running, and its base address (usually ending in /v1)
  • A key, if the server asks for one
  • Aldwin installed (uv tool install aldwin)

Aldwin's tools work only with models that can call tools. Check your server's docs for how to turn tool calling on.

Connect Aldwin#

In the web app#

  1. Open Settings from the bottom of the sidebar and choose Models.
  2. Select Add a server.
  3. Give it a short name, such as office, and its address, such as http://localhost:8000/v1. Add an API key only if the server needs one.
  4. Select Connect. Aldwin asks the server for its model list.
  5. Under Model in use, pick one of the server's models.

In the terminal#

  1. Start Aldwin and type /model.
  2. Pick Other server and press Enter.
  3. At Server address (e.g. http://localhost:8000/v1):, type the base address and press Enter.
  4. At API key (Enter to skip):, paste the key (the input is hidden), or press Enter if the server has none.
  5. At Provider name (e.g. office-gpu):, type a short name. Use lowercase letters and numbers, joined with _ if you need two words, such as office or office_gpu. Avoid dashes in the name if the server needs a key.
  6. Aldwin asks the server for its model list. When that works you see ✓ Added provider office.
  7. Your server now appears under Available providers in /model. Pick it, then pick a model.

Addresses on your own computer (localhost or 127.0.0.1) can use http://. Any other address must use https://, so prompts and keys are encrypted on the way.

What Aldwin saves#

The server goes under providers: in ~/.aldwin/config.yaml:

providers:
  office:
    shape: openai_chat
    base_url: https://llm.example.com/v1
    title: office
    local: false
    secret: office_api_key

The key goes into ~/.aldwin/secrets.yaml as office_api_key. You can also set it as an environment variable named after the provider, such as OFFICE_API_KEY. An environment variable wins over a saved key.

You can write this entry by hand instead of using /model. Aldwin reads it the next time it starts.

Try it#

aldwin

Type /model, pick your server and a model, then ask:

List the files in this folder.

If the model can call tools, you see a list_dir step and then the answer. The status line at the bottom shows the model and your provider name.

Privacy notes#

  • Prompts and files Aldwin reads go to the server you added, and from there wherever that server sends them.
  • A server on your own computer keeps everything on your machine.
  • Details: Privacy.

Troubleshooting#

What Aldwin saysFix
Invalid server address '...'. Address must begin with http:// or https://.Type the full address, including http:// or https://.
Non-local server '...' must use https://.Use the server's https:// address. Plain http:// is allowed only for your own computer.
Could not connect to server at ...The server isn't running, the address is wrong, or the key was refused. Check that curl <address>/models works, then try again. Aldwin waits up to 5 seconds.
Your office key was rejected. ...The server refused the key. Check it with whoever runs the server.
office isn't running. Start it and try again.The server on your computer stopped. Start it again.
Model ... isn't available to your key. ...The model name isn't served. Pick one from the list in /model.
The model answers but never uses toolsTurn on tool calling in the server (see the table above), or pick a model that supports tools.
Was this page helpful?