Any GPU. Any cloud. One inference service you run yourself.

EKS, GKE, AKS, k3s, RKE2 — any Kubernetes ≥ 1.28 with a default StorageClass. About two minutes with warm images.

1. 1

   Label your GPU nodes

   The chart's device plugins and VRAM probes schedule by this label. opodcp infra up does it for you on bare machines.

   kubectl label node <gpu-node> opod.io/gpu-vendor=nvidia

2. 2

   Check the cluster (optional — the chart runs the same preflight as a pre-install Job)

   docker run --rm --network host -v ~/.kube:/kube:ro \
     ghcr.io/opod-io/opodcp:0.1.0 preflight --kubeconfig /kube/config

3. 3

   Install the control plane

   One Deployment with the console inside, a preflight Job, RBAC and per-vendor probes. No databases.

   export ADMIN_TOKEN=$(openssl rand -hex 16)
   helm install opod oci://ghcr.io/opod-io/charts/opod --version 0.1.0 \
     -n opod --create-namespace --set admin.token=$ADMIN_TOKEN
   
   kubectl -n opod port-forward svc/opod-opodcp 8080:8080
   # console: http://127.0.0.1:8080 — sign in with $ADMIN_TOKEN

4. 4

   Create an endpoint

   In the console, Endpoints → New, or from the API. Leave the node out and the placer picks a GPU with room.

   export OPOD=http://127.0.0.1:8080
   curl -s -X POST $OPOD/api/v1/endpoints \
     -H "Authorization: Bearer $ADMIN_TOKEN" -H 'Content-Type: application/json' \
     -d '{"name":"chat","model":"qwen2.5-0.5b-gguf",
          "plan":{"model":{"id":"qwen2.5-0.5b-gguf","engine":"llamacpp"},
                  "workers":[{"replicas":1,"engine":"llamacpp"}]}}'

5. 5

   Wait for a real token, then chat

   pending → converging → serving, and serving only after the leader produced a token. In the cluster the endpoint is http://opod-ep-chat.opod.svc:8080/v1.

   curl -s -H "Authorization: Bearer $ADMIN_TOKEN" $OPOD/api/v1/endpoints/<id>/status | jq .phase,.proof
   
   curl -s -X POST $OPOD/api/v1/endpoints/<id>/gateway/v1/chat/completions \
     -H "Authorization: Bearer $ADMIN_TOKEN" -H 'Content-Type: application/json' \
     -d '{"model":"qwen2.5-0.5b-gguf","messages":[{"role":"user","content":"hello"}]}'

The same product on a laptop with a llama.cpp CPU worker — Apple silicon works. This is also what our CI runs.

1. 1

   Make a cluster and install

   kind create cluster --name opod
   export ADMIN_TOKEN=$(openssl rand -hex 16)
   helm install opod oci://ghcr.io/opod-io/charts/opod --version 0.1.0 \
     -n opod --create-namespace --set admin.token=$ADMIN_TOKEN --wait
   kubectl -n opod port-forward svc/opod-opodcp 8080:8080 &

2. 2

   Create a CPU endpoint

   With no GPU, name the node — the placer never picks a CPU-only node by itself.

   export OPOD=http://127.0.0.1:8080
   curl -s -X POST $OPOD/api/v1/endpoints \
     -H "Authorization: Bearer $ADMIN_TOKEN" -H 'Content-Type: application/json' \
     -d '{"name":"chat","model":"qwen2.5-0.5b-gguf",
          "plan":{"model":{"id":"qwen2.5-0.5b-gguf","engine":"llamacpp"},
                  "workers":[{"node":"opod-control-plane","replicas":1,"engine":"llamacpp"}]}}'

3. 3

   Chat

   curl -s -H "Authorization: Bearer $ADMIN_TOKEN" $OPOD/api/v1/endpoints/<id>/status | jq .phase,.proof
   curl -s -X POST $OPOD/api/v1/endpoints/<id>/gateway/v1/chat/completions \
     -H "Authorization: Bearer $ADMIN_TOKEN" -H 'Content-Type: application/json' \
     -d '{"model":"qwen2.5-0.5b-gguf","messages":[{"role":"user","content":"hello"}]}'

Measured cold on an Apple-silicon laptop, nothing cached (2026-09-12): helm install --wait 52 s; serving · first token in 144 ms 253 s after the create — mostly the first image pulls and the model download.

Linux machines with GPUs and SSH, no Kubernetes. opodcp infra builds k3s over SSH, labels each node by GPU vendor and writes the kubeconfig.

1. 1

   Get the opodcp CLI

   Signed binaries come with the first tagged release; until then the CLI ships inside the image. Run on a Linux box that can reach the others over SSH.

   id=$(docker create ghcr.io/opod-io/opodcp:0.1.0)
   docker cp "$id":/opodcp ./opodcp && docker rm "$id"

2. 2

   Describe the machines

   # machines.yaml
   cell: lab
   ssh: { user: ubuntu, keyFile: ~/.ssh/id_ed25519, sudo: true }
   server: { name: cp-1, host: 192.0.2.10 }
   agents:
     - { name: gpu-1, host: 192.0.2.11 }   # GPU vendor detected on the host
     - { name: gpu-2, host: 192.0.2.12 }

3. 3

   Plan, build, check, install

   Then continue from step 4 of the Kubernetes path. --local when the server is the machine you're on.

   ./opodcp infra plan -f machines.yaml    # host facts + gate, changes nothing
   ./opodcp infra up   -f machines.yaml    # k3s server + agents over SSH
   ./opodcp preflight --kubeconfig ~/.opodcp/lab/kubeconfig
   
   export KUBECONFIG=~/.opodcp/lab/kubeconfig
   helm install opod oci://ghcr.io/opod-io/charts/opod --version 0.1.0 \
     -n opod --create-namespace --set admin.token=$(openssl rand -hex 16)

Measured on a design-partner cell: infra up built a 9-machine cluster in 80 s; preflight to first token on an RTX A6000 took 127 s.

Only the open-source runtime, on a Mac or Linux box — no Kubernetes, no control plane. One OpenAI URL in front of the engine you already run.

1. 1

   Build it (Go 1.25+)

   Prebuilt binaries arrive with the first tagged release; building takes seconds.

   git clone https://github.com/opod-io/opod-core
   cd opod-core && go build -o opod ./cmd/opod

2. 2

   Run an engine, start a leader

   Ollama is simplest; vLLM, llama.cpp and MLX-LM work too — ./opod doctor says which fits.

   # macOS: brew install --cask ollama && open -a Ollama
   # Linux: curl -fsSL https://ollama.com/install.sh | sh
   OPOD_DEFAULT_MODEL=llama-3.2-1b ./opod up    # prints the API URL and an admin key, once

3. 3

   Point your tools at it, add machines

   curl http://127.0.0.1:8080/v1/chat/completions \
     -H "Authorization: Bearer sk-orc-…" -H 'Content-Type: application/json' \
     -d '{"model":"llama-3.2-1b","messages":[{"role":"user","content":"hello"}]}'
   
   ./opod connect --list                                  # Cursor, Aider, Continue, Codex CLI, Open WebUI…
   ./opod token create --node                             # on the leader
   ./opod join "http://<leader>:8080?token=<node-token>"  # on each other machine