Deploying Models

Yokai turns a deployment choice into a normalized Docker specification and sends it through the daemon to the selected device agent.

Workload types

vLLM

Use vLLM for high-throughput OpenAI-compatible text, vision, audio, embedding, or reranking services. The deploy flow supports Hugging Face search, image-tag discovery, GPU selection, tensor parallel flags, runtime options, BKC matching, plugins, and a pre-deploy memory estimate.

llama.cpp

Use llama.cpp for GGUF models, consumer GPUs, and smaller resource envelopes. Yokai queries the selected Hugging Face repository for GGUF variants and records one or more selected files in the service spec.

ComfyUI

Use ComfyUI for image workflows. Yokai deploys its configured ComfyUI image with GPU access and exposes the web port through the device connection.

Guided deployment

Open Deploy with 3.

1. Workload

Choose the runtime. Defaults and later fields change with the workload.

2. Device

Select an online target. The TUI displays available hardware and uses the device profile, GPU count, and VRAM to rank compatible BKC entries.

3. Model

Search Hugging Face or enter an exact repository ID. vLLM search applies the text-generation filter; llama.cpp can continue into a GGUF-variant picker.

4. Configuration

Use:

• B to match and apply a Best-Known Config;
• M to call the vLLM memory estimator;
• F to apply or edit flags;
• Tab to move through editable fields.

Review image, service name, port, GPU IDs, extra arguments, environment variables, volumes, plugins, IPC mode, shared memory, and ulimits.

5. Review and deploy

The review screen submits to POST /deploy. The daemon resolves plugins and assets, then asks the agent to pull the image and create the service. Deployment progress and errors return to the TUI.

Service actions

From Dashboard, open a service with Enter. The detail view supports:

• S stop;
• R restart;
• T test the exposed endpoint;
• X remove;
• Shift+L open streaming logs.

CLI examples

vLLM

yokai services deploy \
  --device gpu-box-01 \
  --type vllm \
  --model Qwen/Qwen3-30B-A3B \
  --port 8000:8000 \
  --gpu-ids all

llama.cpp with a custom image

yokai services deploy \
  --device homelab-3090 \
  --type llamacpp \
  --model bartowski/Meta-Llama-3.1-8B-Instruct-GGUF \
  --image ghcr.io/ggml-org/llama.cpp:server-cuda \
  --port 8080:8080

Local or private image

yokai services deploy \
  --device gpu-box-01 \
  --type vllm \
  --image registry.example.com/vllm:tested \
  --skip-pull

The CLI outputs JSON. Use yokai services list, stop, restart, remove, and logs --follow for lifecycle automation.