Why Yokai
Yokai turns a collection of GPU machines into one operable AI fleet. It is for people who want the control and economics of self-hosted inference without rebuilding an internal platform from shell scripts, Docker commands, SSH sessions, and disconnected dashboards.
The problem
Section titled “The problem”A single GPU is easy to experiment with. A fleet is not. As soon as models span a workstation, a homelab server, a remote host, and a shared accelerator, operators inherit several separate jobs:
- inventory machines and credentials;
- choose a runtime and a configuration that fits the hardware;
- deploy and upgrade containers consistently;
- keep tunnels, ports, tokens, and endpoints straight;
- watch health, VRAM, temperature, power, throughput, and logs;
- update every AI client when a model endpoint changes.
Most teams solve each job with a different tool. Yokai makes them one workflow.
The goal
Section titled “The goal”Yokai’s goal is to make private AI infrastructure feel like a coherent product instead of a collection of machines. The operator should be able to answer three questions from one terminal:
- What hardware and services are available?
- What can I safely deploy here?
- Is the fleet healthy and useful right now?
The project does not replace Docker, Tailscale, Prometheus, Grafana, vLLM, llama.cpp, or ComfyUI. It coordinates them behind a purpose-built operator experience.
Who it is for
Section titled “Who it is for”Yokai is designed for:
- homelab operators with one or more GPU machines;
- developers running private coding models;
- inference engineers evaluating models across mixed hardware;
- small teams that need a repeatable GPU deployment layer without a Kubernetes control plane;
- anyone who prefers a fast keyboard-driven terminal interface.
What makes it different
Section titled “What makes it different”One release, two binaries
Section titled “One release, two binaries”Each release archive contains the Go control binary and the compiled OpenTUI sidecar. There is no Python environment or Node runtime to install on an operator or target machine.
Hardware-aware recipes
Section titled “Hardware-aware recipes”The built-in Best-Known Config catalog currently contains 85 validated recipes across 80 unique models. Recipes capture model, image, runtime flags, quantization, minimum GPU resources, supported device profiles, and optional plugins.
Agent-based control, local authority
Section titled “Agent-based control, local authority”The local daemon owns configuration and opens SSH tunnels to small authenticated agents on GPU devices. Device tokens and SSH material remain under the operator’s control.
Operations and developer experience in one loop
Section titled “Operations and developer experience in one loop”Yokai does more than launch a model. It surfaces metrics and logs, manages service lifecycle actions, provisions monitoring, and registers OpenAI-compatible endpoints with VS Code Copilot, OpenCode, OpenClaw, Claude Code, and Codex.
Product principles
Section titled “Product principles”- Terminal first. Fast navigation, low overhead, and no browser required for fleet control.
- Useful defaults, visible escape hatches. Start with a validated BKC, then adjust images, model IDs, ports, GPU selection, environment variables, volumes, plugins, or runtime flags.
- Secure by default. Agents require bearer tokens and are normally reached through SSH tunnels.
- Backend first. The daemon owns discovery, settings, bootstrap, and integrations so the TUI stays a thin client over REST and SSE.
- Inspectable automation. CLI commands return JSON, configuration is local, and each deployed service remains a normal Docker container.