Skip to content

Workspace

Self-hosted runners

Run Codebot's reviews and agent on machines you control, with models you host or your own keys.

Self-hosted runners are in early access. Ask Codebot to turn them on for your organization; Settings → Runners appears once they're on.

A runner is a small program you run on your own machine with Docker. It picks up your organization's reviews and agent requests and runs each one in its own container on that machine: the repository is cloned there, and the model is called from there. Use it to keep runs on infrastructure you control, for example to use models you host yourself (vLLM, SGLang, any OpenAI-compatible server) or a private endpoint such as Azure OpenAI.

What runs where

On your runnerThrough Codebot
Cloning the repository, running the agent✓
Calls to your model, with keys set on the runner✓
The pull request's diff and context, prepared for the run✓
Reviews, comments and pull requests, posted to your git host✓
The run's transcript: what the agent read, ran and wrote✓
The run's steps: tools used, files touched, timings, model usage, errors✓
Run status, review verdicts and summaries, agent answers✓

From runner 0.2.0, the runner reads your git host and posts reviews itself, and sends Codebot the run's steps without their content: your code, diffs and agent transcripts don't pass through Codebot. The run page shows the steps and says the full transcript stayed on your runner. For GitHub, Codebot gives the runner a short-lived token for each run's repository; for Azure DevOps, set CODEBOT_ADO_PAT on the runner (without it, Azure DevOps runs read the pull request through Codebot, as before).

Webhooks from GitHub, Azure DevOps, Linear and Discord still arrive at Codebot, which decides what to run and sends it to your runners.

Keep only run metadata

By default Codebot keeps what it shows you about a run on your runners: its status and steps, review verdicts and summaries, and the agent's answers. If your security team wants less, Codebot can turn on metadata only for your organization (ask us; it's part of the highest tier). An admin then chooses it in Settings → Runners → Run data:

  • Codebot keeps run status, timings, counts and links only.
  • Requests, answers, review summaries and error details still pass through Codebot on their way to your git host or chat app, and are not kept. Your reviews, comments and replies are posted as usual; a failed run's page says only that it failed.
  • Follow-up mentions read the conversation from the pull request, issue or thread instead of from Codebot.

Add a runner

  1. In Settings → Runners, choose Add runner and give it a name, such as the machine's name.
  2. Copy the command Codebot shows. The runner's token is shown only once; if you lose it, remove the runner and add a new one.
  3. Run the command on a Linux machine with Docker that can reach your git host, your models and Codebot:
docker run -d --pull always --name codebot-runner --restart unless-stopped --stop-timeout 300 \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -e CODEBOT_API_URL=https://app.trycodebot.dev \
  -e CODEBOT_RUNNER_TOKEN=cbr1_… \
  -e CODEBOT_RUNNER_HOSTNAME="$(hostname)" \
  ghcr.io/simonepriuli/codebot-runner:latest

The runner shows as Online within a few seconds. It needs the Docker socket to start a container per run; run containers don't get the socket or the runner's token, only a token for their own run.

Send repositories to your runners

Settings → Runners has a default for the whole organization and a choice per repository:

  • Codebot cloud: runs execute in Codebot's sandboxes, as without runners.
  • Your runners: runs wait for one of your runners to pick them up.

A run sent to your runners waits while they are busy. If none of your runners is online for 10 minutes, the run fails and the pull request's check says so. Codebot never moves such a run to its cloud on its own.

Runs on your runners don't count as compute on your plan. Usage shows them separately.

Use your own models

Keys and endpoints you set on the runner replace the ones stored in Settings → Models, for runs on that runner only. Anything you don't set falls back to the stored keys, so to keep keys out of Codebot entirely, set them on the runner and store none.

VariableWhat it sets
CODEBOT_KEY_ANTHROPIC, CODEBOT_KEY_OPENAI, CODEBOT_KEY_GOOGLE, CODEBOT_KEY_OPENROUTER, CODEBOT_KEY_GROQ, CODEBOT_KEY_MISTRAL, CODEBOT_KEY_DEEPSEEKThat provider's API key
CODEBOT_KEY_EXAWeb search for the agent
CODEBOT_OPENAI_COMPATIBLEAn OpenAI-compatible endpoint (vLLM, SGLang, LiteLLM…), as JSON: {"name":"Corp GPU","baseUrl":"https://llm.corp.internal/v1","apiKey":""}

Models only your network can reach

Put a model endpoint that only your network can reach, such as GPUs behind your VPN, on the runner. Codebot never calls it: the runner asks the endpoint which models it serves (GET <baseUrl>/models) when it starts and every 5 minutes, and tells Codebot their names and context windows. It never sends the endpoint's address or key.

Those models then appear in the model pickers on Code review and Agent, in a group named Your runners followed by the endpoint's name. Pick one as you would any other model. Settings → Runners shows what each runner's endpoint serves, or why the runner could not list its models.

  • The run containers call the endpoint too, so its baseUrl must be reachable from inside a run container, not only from the runner. localhost is the container itself, so use the LAN address of the machine that serves the model, for example http://192.168.1.20:1234/v1 for LM Studio on port 1234. Docker's bridge network does not resolve host.docker.internal on Linux unless the host is configured for it. Allow the port in the host's firewall. If the runner lists the models but runs using them fail with a connection error, this is the cause.
  • Running the runner's run containers on the host network (CODEBOT_RUNNER_NETWORK=host) may also let them reach localhost. This is not verified yet, so check that a run reaches the endpoint before you rely on it.
  • If your endpoint has no /models route, list the models in the JSON: "models":[{"id":"qwen3-coder","contextWindow":262144}]. Context windows you set there win over the ones the endpoint reports; contextWindow sets the default.
  • Runs that use such a model only go to runners whose endpoint serves it. If none of your online runners serves it for 10 minutes, the run fails and says so.
  • Repositories that run on Codebot cloud cannot use such a model: their runs fail with a message to route them to your runners. The model picker warns you when this applies.
  • Put the endpoint's host in NO_PROXY when the runner uses a proxy, and your internal CA in NODE_EXTRA_CA_CERTS.
  • If the runner's host resolves internal names with a local resolver (systemd-resolved on 127.0.0.53), run containers on Docker's default network can't use it: set CODEBOT_RUNNER_DNS to your DNS servers.

Configure the runner

From runner 0.5.0, the agent answers each Discord channel from a host: one long-lived container per channel (docker ps shows codebot-host-<id>) that answers every thread of the channel, several at once. Each channel gets a Docker volume, codebot-host-<id>, mounted at /data, which keeps each thread's conversation, clone and worktree between messages and across restarts. A host stops after 10 minutes without a message, and the volume stays: the next message starts it again where the thread left off. If a host stops in the middle of an answer, the runner starts it again and the answer carries on. Volumes grow with every thread; past CODEBOT_RUNNER_HOST_DISK_GB, the runner removes the volumes of the channels quiet the longest, and those channels' next threads start from the conversation Codebot keeps. When the runner stops, it first lets its hosts finish the answers in progress, for up to 5 minutes: give its container that long to stop (docker stop -t 300, or stop_grace_period in Compose).

After the agent answers in a Linear issue or a pull request, its run container stays up for 6 minutes. If the next message in that conversation, on the same repository, arrives meanwhile, the same container answers it: the repository is already cloned, so the answer starts sooner. A waiting container holds one of the runner's slots, and gives it up as soon as a run on another conversation needs it.

VariableDefaultWhat it does
CODEBOT_RUNNER_CONCURRENCY2Runs it executes at once
CODEBOT_RUNNER_HOSTNAMEthe runner container's idThe machine's name, shown for the runner in Codebot; the command Codebot gives sets it to $(hostname)
CODEBOT_RUNNER_MEMORYunlimitedMemory per run container, e.g. 4g
CODEBOT_RUNNER_CPUSunlimitedCPUs per run container, e.g. 2
CODEBOT_RUNNER_NETWORKDocker's defaultDocker network for run containers
CODEBOT_RUNNER_DNSDocker'sDNS servers for run containers, comma-separated, e.g. 10.0.0.2,10.0.0.3
CODEBOT_RUNNER_RUN_IMAGEthe runner's imageImage for run containers, e.g. one built FROM the runner with your toolchains
HTTPS_PROXY, HTTP_PROXY, NO_PROXYnonePassed to run containers
NODE_EXTRA_CA_CERTSnonePath of a CA bundle inside the runner's container (mount it with -v); the runner copies it into each run container
CODEBOT_ADO_PATnoneAzure DevOps personal access token the runner reads pull requests and posts reviews with; it never leaves your network
CODEBOT_RUNNER_GATEWAYonoff sends run containers' calls straight to Codebot, which then reads your git host as before 0.2.0
CODEBOT_RUNNER_GATEWAY_PORT8787Port run containers reach the runner on, inside Docker's network only
CODEBOT_RUNNER_GATEWAY_URLfound from DockerAddress run containers reach the runner at, when the runner does not run in a container itself
CODEBOT_RUNNER_HOSTSonoff runs Discord answers in a container each, as before 0.5.0; hosts also need the gateway
CODEBOT_RUNNER_HOST_DISK_GB20Disk the hosts' volumes may take before the runner removes those of the channels quiet the longest
CODEBOT_HOST_IDLE_SECONDS600How long a host waits for a message before it stops
CODEBOT_HOST_DRAIN_SECONDS300How long a stopping host waits for its answers in progress

Run containers run as an unprivileged user and are removed an hour after they finish, so docker logs codebot-run-<run id> works in between.

Sizing your runners

Settings → Usage → Runners shows how your runners keep up:

  • Queue wait is how long runs waited between being created and a runner taking them. A few seconds is normal. When the p90 runs into minutes, runs are queuing for a free slot.
  • Peak is the most runs a runner had at once, against its CODEBOT_RUNNER_CONCURRENCY. A runner whose peak reaches its concurrency had runs waiting for it at least once.
  • Didn't start counts runs that failed before any runner took them. These usually come from no runner being online, or no runner serving the run's model.

When the wait is long and runners reach their peak, raise CODEBOT_RUNNER_CONCURRENCY if the machine has memory and CPU to spare. Otherwise add a runner on another machine. Runs go to whichever runner is free.

Update or remove a runner

To update, remove the runner's container and run the same docker run command again: --pull always fetches the newest image. Without it, Docker reuses the image already on the machine, however old. Runs in progress keep going; the restarted runner picks them up.

docker rm -f codebot-runner
docker run -d --pull always --name codebot-runner … # the same command as before
``` The runner page marks runners that are too old for Codebot, and such a runner stops with a message saying which version to install.

To remove a runner, choose **Remove runner**. Its token stops working right away and the runs it was working on fail. Stop its container on your machine too.

<Callout type="warning">
  Anyone with a runner's token can take your organization's runs, including the run tokens that post reviews. Keep it like a password, and remove the runner if it leaks.
</Callout>