the container that wasn't there

A friend of ours has a 12 GB card and a healthy suspicion of software. When we asked him to try hosting, he did not ask what he would earn. He asked: "so what Docker images are you going to run on my PC?"

It is exactly the right question, and the honest answer surprised him: none. Not "a small one", not "a sandboxed one". There is no image, no daemon, no root. Getting to that answer took us through a few months of design, and it is worth writing down — because the reasoning applies to anyone deciding what a stranger's computer should be asked to run.

the obvious design, and why we didn't ship it

Every serving stack in 2026 ships as a container. vLLM publishes CUDA and ROCm images; so does every inference framework worth the name. The obvious architecture writes itself: the control plane tells a host which image to pull, the host runs it, everyone goes home.

We built the plan, costed it out for a home machine, and stopped at the first line of the estimate. A vLLM image is 10 to 30 GB. On top of that the host needs a container runtime, a GPU container toolkit, driver plumbing that matches, and — on most distributions — a privileged install step. That is a substantial ask before a single token has been served, and it is a lot of surface area running on a machine that also holds someone's photographs.

Meanwhile the alternative was sitting right there. llama.cpp is a ~5 MB binary. One build of it — the Vulkan one — serves AMD, NVIDIA and Intel GPUs from the same file. It starts in seconds, runs as an unprivileged user, and binds only to localhost.

For the machines we actually want in the pool — gaming PCs, workstations, the RX 6900 XT under a desk — the 5 MB binary is not the compromise. It is the better engineering.

what we ended up installing

Two files, and you can read the source of both.

mahout is our agent: one Rust binary that probes the hardware, enrolls the machine with a one-time code, downloads and verifies model weights, supervises the engine, polls for work and signs a receipt for every job. Roughly 5 MB, open source, no root.

The engine is a pinned llama.cpp release, downloaded once and checked against a SHA-256 that is compiled into the mahout binary itself. If the file that arrives is not the file we pinned, it is discarded and nothing runs.

That is the entire footprint. A host machine ends up with an agent, an engine and some model weights in a cache directory — all of it named, checksummed and removable with one command.

the part where containers do earn their keep

None of this means containers are wrong. It means they are wrong here, for now.

When a single card serves many concurrent requests, vLLM's continuous batching and paged attention pull decisively ahead. That is a workstation and datacenter story — the tier where a machine is bought to serve, not borrowed between gaming sessions. We will want it, and the architecture already has the seam: in our catalogue, the engine belongs to the build, not to the host. Adding vLLM is adding an entry, not rewriting the agent.

When that lands, three rules hold, and they are the same rules that already govern model weights:

• Pinned by digest, never by tag. vllm/vllm-openai@sha256:… is a specific set of bytes. :latest is a promise that someone else can quietly change.
• Allow-listed inside the client. The list of images a host may run is compiled into the mahout release. The control plane can ask for one of them; it cannot introduce a new one. If we were ever compromised, we still could not tell your machine to run something arbitrary — we would have to ship a new signed binary, in public, for everyone to see.
• No egress, read-only weights. The container talks to mahout on localhost and to nothing else.

"and where do the images live?"

Our friend's follow-up, naturally. Here we surprised ourselves.

The instinct of every infrastructure engineer is to mirror: run your own registry, pull everything through it, control the supply chain. We nearly did. Then we asked what mirroring an upstream image actually buys, given that we already reference it by digest — and the answer was: nothing. A digest is verified by the container runtime on the way in. If the bytes match the digest, they are the bytes we tested, whether they came from Docker Hub or from a server in our own rack. Re-hosting would add tens of gigabytes of storage, a backup story and a monitoring story, in exchange for zero additional guarantee.

So we don't mirror. We pin.

There is exactly one case where we will run our own registry, and it is not about control of the supply chain — it is that the image will be ours. Verifying that a host really ran the model it claims requires an engine build that emits a cryptographic commitment to its own activations as it decodes. That is a patched engine, which means an image we build ourselves, which means somewhere to publish it: an EU registry, signed, pinned by digest, exactly like everything else. Building that infrastructure the day we need it, rather than the day we imagine needing it, has been the quiet theme of the whole project.

what this looks like from the outside

Our friend installed it in about four minutes:

curl -fsSL https://elephantpool.ai/install-mahout.sh | sh
mahout models
mahout up --code HIS-CODE
mahout bench --model gpt-oss-20b

mahout models told him what his 12 GB card could serve before he committed to a single download. The benchmark measured what it really does rather than what a table predicted — and that measurement, not our estimate, is what his dashboard now shows as earnings per serving hour.

He never installed Docker. He never typed sudo. And when he wants it gone, mahout uninstall prints a deletion plan with sizes before it touches anything.

That, in the end, is the whole argument. Software that runs unattended on someone else's hardware should be small enough to read, pinned tightly enough to verify, and easy enough to remove that leaving is never a fight.

Read the full architecture · What runs on your machine · See the models · Install mahout