Troubleshooting

Spark does not start

Symptom	Cause and fix
Exit code 173 (0xBAD) and "SPARK_ADMIN_PASSWORD is required"	Set SPARK_ADMIN_PASSWORD, at least 8 characters.
Address already in use	Choose another SPARK_PORT.
"spark-runner was not found beside the server"	The device probe warns when spark-runner is not next to spark. Keep the archive's folder together, or set SPARK_RUNNER_PATH.

A request fails

Status and message	Fix
401 "A valid API key is required"	Send a key, Authorization: Bearer sk-spark-… or x-api-key. A disabled or wrong key is refused even when keys are not required.
404 "The model … does not exist on this server"	Use an id or alias from GET /v1/models. Or set SPARK_DEFAULT_MODEL and SPARK_UNKNOWN_MODEL_FALLBACK.
400 "Only n = 1 is supported"	Remove n, or set it to 1.
400 "max_tokens: Field required"	The Anthropic route needs max_tokens.
400 "… is a chat model; embeddings need an embedding model"	Call /v1/embeddings with a bert, nomic-bert or gemma-embedding2 model.
400 for a prompt longer than the context	Raise the model's context in its settings, or send less.
503 for a model set to cuda	The machine has no usable CUDA device. Set the device to auto or cpu.
A model is refused because it does not fit	The message shows the arithmetic. Unload other models, lower the context, pick a smaller quantization, or add memory.

The first request is slow

The first request to a model loads it. A model unused for SPARK_IDLE_UNLOAD_MINUTES is unloaded and loads again on the next request. Turn Unloaded when idle off in the model's settings to keep it loaded.

Requests queue

A runner answers one request at a time. Concurrent requests to the same model wait their turn. The timeline of a request shows the time spent waiting. Load the same model on a node as well, or use a smaller model.

Generation is slow

On the CPU, decode speed depends on the model's size and quantization, and on the CPU. As a reference, on 4 CPU cores: about 48 tokens per second for SmolLM2 135M and about 16 for Qwen3 0.6B. Check the speed on the Requests page.

A runner keeps restarting

Open Runners and read the runner's log. A runner that exits with code 132 (SIGILL) on its first request runs a native library built for another CPU. Releases ship a portable build. A build from source needs to be rebuilt on the machine that runs it.

Images do not work

• The model needs a vision projector. The model page shows whether one is beside it.
• With SPARK_IMAGE_URLS=public an image URL that points at a private address is refused. Send the image as base64.
• A request over SPARK_MAX_BODY_SIZE (64 MB) is refused.

Streams arrive all at once

A reverse proxy buffers the response. Turn buffering off for /v1/.

Downloads fail

• A gated repository needs SPARK_HF_TOKEN.
• Check the Downloads page for the error, then use Resume. A download resumes from its .partial file.
• The server must reach huggingface.co. Behind a proxy, give the server the proxy settings through its environment.

The console shows no CUDA device

Spark probes the devices at start. Install the NVIDIA driver and cuBLAS 12 (libcublas.so.12), restart, and use Measure again on the Runners page.