OpenAI compatible API

Spark answers the OpenAI routes on the same port as the console. Any OpenAI SDK works with base_url = http://<host>:8080/v1 and a Spark key as the API key. The parameters each route reads are in the compatibility matrix.

Chat

curl http://localhost:8080/v1/chat/completions \
-H "Authorization: Bearer $SPARK_KEY" -H "Content-Type: application/json" \
-d '{
"model": "smollm2-135m-instruct-q8_0",
"messages": [{"role": "user", "content": "Say hello in five words."}],
"max_tokens": 24
}'

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8080/v1", api_key="sk-spark-…")

reply = client.chat.completions.create(
model="smollm2-135m-instruct-q8_0",
messages=[{"role": "user", "content": "Say hello in five words."}],
)
print(reply.choices[0].message.content)

# streaming
stream = client.chat.completions.create(
model="smollm2-135m-instruct-q8_0",
messages=[{"role": "user", "content": "Count to three."}],
stream=True,
)
for chunk in stream:
if chunk.choices:
print(chunk.choices[0].delta.content or "", end="")

import OpenAI from "openai";

const client = new OpenAI({ baseURL: "http://localhost:8080/v1", apiKey: "sk-spark-…" });

const stream = await client.chat.completions.create({
model: "smollm2-135m-instruct-q8_0",
messages: [{ role: "user", content: "Hello!" }],
stream: true,
});
for await (const chunk of stream) process.stdout.write(chunk.choices[0]?.delta?.content ?? "");

using System.ClientModel;
using OpenAI;
using OpenAI.Chat;

var options = new OpenAIClientOptions { Endpoint = new Uri("http://localhost:8080/v1") };
var client  = new ChatClient("smollm2-135m-instruct-q8_0", new ApiKeyCredential("sk-spark-…"), options);

ChatCompletion reply = await client.CompleteChatAsync(new UserChatMessage("Hello!"));
Console.WriteLine(reply.Content[0].Text);

await foreach (StreamingChatCompletionUpdate update in client.CompleteChatStreamingAsync(new UserChatMessage("Count to three.")))
{
foreach (ChatMessageContentPart part in update.ContentUpdate) Console.Write(part.Text);
}

The response

The response follows OpenAI's shape. usage carries prompt_tokens, completion_tokens, total_tokens and prompt_tokens_details.cached_tokens, the number of prompt tokens served from the KV cache. The x-request-id response header is the id of the request in Spark's request log.

Streaming

stream: true returns server-sent events, one chat.completion.chunk per piece, ending with data: [DONE]. Add stream_options: {"include_usage": true} to get the usage in a final chunk.

System messages

The developer role is treated as system. If a request carries no system message, the model's configured system prompt is used.

Tools

Send tools as usual. Spark renders them into the model's chat template and parses the model's tool calls into tool_calls. Send results back as tool messages with tool_call_id.

• tool_choice: "none" turns tools off. Other values of tool_choice are not read, so you cannot force a tool.
• Whether a model produces tool calls depends on the model and its template. A small model may answer in text instead.

Reasoning

Models that think return their reasoning in message.reasoning_content (and in streaming deltas), the field that vLLM, llama.cpp and DeepSeek use. Thinking is on when the request sets reasoning_effort to anything but none or minimal, or think: true, or chat_template_kwargs: {"enable_thinking": true}, or reasoning: {"effort": ...}. Messages in a request may carry reasoning_content, or reasoning, for earlier turns.

JSON mode and JSON schema

response_format takes {"type": "json_object"} or {"type": "json_schema", "json_schema": {"schema": {...}}}. Spark enforces the output with a grammar. Without a schema in json_schema, it falls back to JSON mode.

Images

{
  "model": "smolvlm-256m-instruct-q8_0",
  "messages": [{
    "role": "user",
    "content": [
      {"type": "text", "text": "What does the sign say?"},
      {"type": "image_url", "image_url": {"url": "data:image/png;base64,…"}}
    ]
  }]
}

The model needs a vision projector. See Models. input_audio and file parts are refused with 400.

Completions

POST /v1/completions takes prompt as a string, or an array with one string. Token arrays and batches are not supported. max_tokens defaults to 16.

Embeddings

curl http://localhost:8080/v1/embeddings \
  -H "Authorization: Bearer $SPARK_KEY" -H "Content-Type: application/json" \
  -d '{"model": "all-minilm-l6-v2-q8_0", "input": ["first text", "second text"]}'

input is a string or a non-empty array of strings. encoding_format is float (the default) or base64. dimensions shortens a Matryoshka model's vectors, which are truncated and normalized again. Asking for more dimensions than the model has is refused with 400. Add an embedding model such as all-MiniLM-L6-v2 under Discover.

Sampling parameters

temperature, top_p, top_k, min_p, presence_penalty, frequency_penalty, repetition_penalty (or repeat_penalty), seed and stop are read. max_completion_tokens and max_tokens set the length. Defaults come from the model's settings. n must be 1.

Errors

Errors use OpenAI's shape:

{"error": {"message": "The model 'nope' does not exist on this server (see GET /v1/models).", "type": "not_found_error", "param": null, "code": null}}

When an error happens after a stream has started, it arrives as a final event in the stream.