When a job goes wrong
A host disappears mid-request, a model will not load, the queue is empty. What happens to your request, what you are charged, and what the host is paid.
The machines serving your request are somebody's desktop. They get switched off, they lose Wi-Fi, their owner starts a game. A network built on that has to treat failure as ordinary rather than exceptional — and be plain about what you are charged when it happens.
A host takes a job and vanishes
Every claimed job has a deadline for producing its first output. Miss it and the job is taken back and returned to the queue for another machine — once. You see a slower response; you do not see an error, and you are not charged twice.
The retry is deliberately only before the first chunk. Once any output has reached you the job is never re-queued: replaying a stream you have already partly seen would hand you a response that contradicts itself halfway through. Past that point a failure is a failure and is reported as one.
Nothing can serve it
If the request carries a placement requirement no live
host satisfies, you get a 503 immediately, naming what was required. If the queue
simply has no free machine, the request waits, and a wall clock — two minutes by default — ends it
with a timeout rather than letting it hang.
What you pay
Tokens that were produced. A job that fails before producing anything costs nothing. A job that fails partway bills the tokens you actually received, which is also what the host is paid 80% of. No failure surcharge, no minimum. How the price itself is set is in what you pay and why.
What the host sees
A model that will not load on a machine is recorded as a failed capability for that build, with the error, on the machine's page — not silently dropped. Usually the honest answer is that the card does not have the VRAM the catalogue estimated, which is why a benchmark on your own hardware is the only figure the dashboard treats as real.
A host that repeatedly claims work and fails to deliver earns nothing from those jobs, because payment follows produced tokens.
What is not there yet
- No alerting. If your machine stops taking work at three in the morning, you find out by looking at the dashboard. Email notification for a host that goes quiet is scheduled and not built.
- No cross-host continuation. A stream cannot move to another machine mid-generation and pick up where it left off. Retry-before-first-chunk covers the common case — a machine that dies while loading — but not one that dies halfway through a long answer.
Both are named here because a reliability page that only describes what works is not a reliability page.
Related
Model catalogue · Download mahout · How the whole system works · Blog