Self-hosted · Open source · Over Weirdnet

Your own cloud of LLM models

Ymir turns a computer with a decent GPU, or just a CPU, into part of a private model cloud with one command. Start with one machine and add more. The pool acts as one endpoint, and the models are spread across your own devices over Weirdnet: no open HTTPS port, no reverse proxy, no API keys.

Three roles, placed freely

Run all three on one box, or spread them over any number of devices. They speak the same protocols whether a peer is local or remote.

Model nodes

Run the models: llama.cpp everywhere, vLLM where it is supported, or a stock server Ymir wraps. They advertise what they have and load or unload on command.

Inference nodes

Coordinate the pool: one model catalog, routing, a stand-in model while a big one loads, and failover. Load balancing, cloud-wide swapping and synced context are still planned.

App nodes

Ask for models and send prompts: Bragi, ymirctl, or the loopback-only OpenAI-compatible bridge for Jan and scripts on 127.0.0.1.

What works today

Status from the design doc, 2026-11-19. Nothing is released yet.

  • One box runs all three roles in one ymird, with llama.cpp and vLLM (managed, or wrapping a server that is already running).
  • An OpenAI-compatible endpoint on loopback: chat, completions and embeddings, so Jan and scripts work.
  • A small stand-in model answers while a big one loads, clearly marked; models unload when idle and restart with backoff after a crash.
  • Roles talk across devices over Weirdnet, and only devices in your own roster are let in. So far this has been tested on one computer, not across real machines.
  • Milestones done: M0, M1, M2, M3. In progress: M4 Several model nodes.

Linux packages (tar.gz, zip, .deb and a TAUR recipe) build locally; nothing is published. See the full roadmap.

Part of the WeirdBrains family

Ymir holds only the Ymir-specific app. The generic parts are published Weirdnet crates.

Bragi

The chat client for desktop and phone. It finds your Ymir nodes and streams answers from your own machines.

Weirdnet

The peer-to-peer network under every WeirdBrains app: identities, devices, relays and the inference protocol.

Open source

BSD-3-Clause. Source and issues on GitHub.