Build it
Nothing is released yet, so Ymir builds from source. You need Rust 1.99 and a llama-server from
llama.cpp. On an NVIDIA PC, take the release's
ubuntu-cuda-12.8-x64 build and unpack its matching cudart- archive into the same directory
(no CUDA toolkit needed). llama.cpp offloads every layer that fits, so node.toml needs only
llamacpp = "system:/path/to/llama-server".
git clone https://github.com/WeirdBrains/ymir && cd ymir
cargo build --release -p ymird -p ymirctl
Set up and run
ymirctl hw # what this machine has and the model budget
ymirctl setup --llama-server /path/to/llama-server --model /path/to/model.gguf
# or: ymirctl setup --download # catalog models that fit, asks before downloading
ymird & # model, inference and app roles in one process
ymirctl chat "Hello" # streams through the loopback bridge
ymirctl status # roles, models and their state
ymirctl load <model> / ymirctl unload <model>
ymirctl bridge token # API key for Jan: base URL http://127.0.0.1:8484/v1
ymirctl service install # or setup --service: a systemd user unit / launchd agent
The bridge serves /v1/models, /v1/chat/completions, /v1/completions and
/v1/embeddings, on 127.0.0.1 only. While a big model loads, a stand-in can answer, flagged with
the x-weirdnet-standin header. Send x-wait: true, or use ymirctl chat --wait,
to wait for the real model.
Some settings
One file per node, ~/.config/ymir/node.toml (move it with YMIR_HOME; model files move with
WEIRDNET_MODELS_DIR). Secrets are env: or file: references, never inline.
[[model.local]]
id = "qwen3-8b"
swap_group = "main"
idle_ttl = "10m" # unload when idle
[[model.local]]
id = "smollm2-360m"
role = "standin" # answers while the main model cold-starts
[[model.local]]
id = "qwen2.5-7b"
backend = "vllm"
wrap = "http://127.0.0.1:8000/v1" # a stock server Ymir didn't start (loopback only)
wrap_api_key = "env:VLLM_API_KEY"
A chat GGUF used for embeddings needs args = ["--pooling", "mean"] on llama-server.
More machines
Machines join over Weirdnet by being linked to your own identity: ymirctl pool link on a node prints a
code, and you open it on the other device (Morph, or ymirctl pool add). Both sides show the same six
digits to compare. Roles over Weirdnet work today, but they have so far only been tested on one computer. Several model
nodes in one pool (M4) is in progress; see the roadmap.