Models

Spark serves GGUF files. The model library is the models folder, SPARK_MODELS_FOLDER (default $SPARK_STORAGE/models). This page covers how models get there, how they are identified and what you can set per model.

The library

• Any .gguf file copied into the models folder appears in the library on the next scan. Subfolders are allowed.
• Spark reads the file's header without loading it. It shows the architecture, quantization, parameter count, trained context, chat template, size and an estimate of the memory it needs.
• A split GGUF (-00001-of-00003.gguf and so on) counts as one model. Vision projector files (mmproj) are not listed as models.
• GET /v1/models lists only the models Spark can run.

Model ids and aliases

The id is the file name without .gguf and without a split suffix, in lower case, with unsafe characters replaced by dashes. SmolLM2-135M-Instruct-Q8_0.gguf becomes smollm2-135m-instruct-q8_0.

A request's model is resolved in this order, ignoring case: the id, the alias, the file name, the id of the name given, then the model's display name. When nothing matches, the request fails with 404, unless SPARK_UNKNOWN_MODEL_FALLBACK=true, which serves unknown names with the default model. A request that names no model gets SPARK_DEFAULT_MODEL, or the only model when there is one.

Set an alias on the model page to serve clients that hard-code a name, for example gpt-4o-mini. GET /v1/models lists the id and the alias.

Add models with Discover

Discover searches Hugging Face for GGUF repositories. The filters follow the Hub's own model list.

Filter	What it does
Search and sort	Most downloaded, trending, most liked or recently updated.
Task, size, license, language, author	The Hub's filters.
Runs on Spark	Keeps only repositories whose architecture Spark runs. Discover marks a repository it cannot run, and says why.

Open a repository to see its files by quantization and size, and pick one. A repository's vision projector is shown and downloaded with the model. A split model downloads all its parts. Download to picks this server or, on a central, a node.

From a URL downloads any URL that points at a .gguf file. A Hugging Face /blob/ link is rewritten to its download link. Set SPARK_HF_TOKEN for gated repositories and to avoid rate limits.

Downloads

• Downloads are resumable. They write .partial files and use range requests, and the file's GGUF magic number is checked at the end.
• Download jobs are kept in the database, so a download that the server was stopped in the middle of carries on from its partial files when it starts again.
• A dropped connection is retried 5 times with back-off, each from where the file ends.
• A failed or cancelled download has Resume on the Downloads page. Starting the same file again resumes the earlier job.
• Downloads are not checked against the Hub's checksums yet.

Per-model settings

The model page sets these. They apply the next time the model loads, and a change reloads a loaded model.

Setting	Effect
Alias	An extra name clients can request.
Context (tokens)	The KV cache size. Empty uses SPARK_CONTEXT_LENGTH (4096). A prompt longer than the context is refused with 400.
Device	auto, cpu, cuda or cuda:N. Empty uses SPARK_DEVICE.
Vision projector	The mmproj file images go through. The default is the one beside the model. none makes the model text only.
Temperature, Top P, Top K, Min P, Max tokens	Defaults for requests that do not set them.
System prompt	Prepended as a system message when a request carries none.
Unloaded when idle	Turn off to keep the model loaded. It is then never unloaded for being idle.

The page also has Load, Unload, Clear KV snapshots and Delete. Deleting removes the file (every part of a split model), its record and its KV snapshots, and kills its runner without saving.

Supported architectures

Spark has two engines behind one interface.

Engine	Architectures
Spark's transformer	llama, mistral, qwen2, qwen3, granite. Quantizations Q4_0 to Q8_0, K-quants, IQ-quants, F16 and BF16.
TensorSharp families	Gemma 3 and 4, Qwen 3.5, 3.6, 3.8 and Next, GPT-OSS, Nemotron-H, Mistral 3, DiffusionGemma, DeepSeek 4, GLM, Hunyuan and others, with their vision projectors, thinking and tool call formats.
Embedding models	bert, nomic-bert, gemma-embedding2. They answer /v1/embeddings.

An embedding model refuses chat requests, and a chat model refuses /v1/embeddings, each with a 400.

Architectures neither engine runs yet include phi3, olmo2, qwen3moe and mixtral. Image and video generation families load but have no endpoint.

Images

A model with a multimodal projector reads images. Spark pairs a model with the mmproj file beside it, or with the one named in its settings. Both APIs accept images as base64, data URIs or http(s) URLs. SPARK_IMAGE_URLS decides how URLs are fetched: public (the default) fetches only from public addresses, any from anywhere, off refuses URLs.