Decision Providers

A decision provider is a hosted model that answers StructuredAI's typed questions (choice, score and yes/no) with its own probabilities and confidence. It is an alternative to the local checkpoints for workspaces that have not downloaded one, for inputs longer than a checkpoint's context, and for languages the English checkpoint does not read.

Three providers are supported:

All three bill per input token only. Output tokens are not charged.

Set up a provider

1. Go to Manage → AI → Decision Providers (#/manage/ai/decision-providers).
2. Pick the provider in the list on the left.
3. Click Set up, or turn on one of its models, which opens the same form.
4. Paste the API key. Get an API key opens the vendor's console.
5. Optionally, enter an Endpoint: a URL that serves the same API, such as a gateway in front of the vendor. Leave it empty for the vendor's own.
6. Click Test. The form asks three questions about the sample text (a choice, a score and a yes/no) using the settings as they are on the form, saved or not. It shows the answers, the model that answered, the time taken and the input tokens. A wrong key shows the vendor's own error message.
7. Click Set up to save.

The API key is stored in the graph, encrypted on disk when MSK_GRAPH_MASTER_KEY is set, and never sent back to the browser. To change other settings without retyping it, leave the key field empty when you save.

The test is a real request. It is billed, and it is recorded in LLM usage like any other decision.

Models

The right-hand pane lists every model of the selected provider:

• The toggle sets whether code may ask for the model by name.
• Make default sets the model a call answers with when it names none. The default model is always enabled; make another one the default to turn it off.

Each row shows the model's context size and input price.

Use it from code

Name the provider with its generated constant. The provider's settings card shows the exact line to copy:

using Mosaik.AI;

var options = StructuredAIOptions.Decision(DecisionProviders.MercuryDecide, userUID: CurrentUser);

var answers = await StructuredAI.AskAsync(ticket.Body, q => q
    .Choice("department", "Which team should handle this?", o => o
        .Option("billing",   "payments, invoices and refunds")
        .Option("technical", "bugs or integration problems")
        .Option("other"))
    .YesNo("urgent",      "The message conveys urgency")
    .Score("frustration", "How frustrated is the customer?", "calm", "annoyed", "very angry"),
    options, CancellationToken);

To use a model other than the default, pass it by name. It must be enabled on the provider:

var pinned = StructuredAIOptions.Decision(DecisionProviders.TypeSafeJev, "jev-1.14.0");

userUID attributes the request's usage to a user. Without it the usage is recorded without one.

The generated name is fixed per provider and does not change when you rename the provider in the settings form. Code naming DecisionProviders.TypeSafeJev keeps compiling if the provider is removed and set up again. If it is removed and not set up again, that code no longer compiles.

What a request looks like

• One request per input. Every question about an input goes in the same request. AskManyAsync runs up to MaxConcurrentRequests (4 by default) inputs at a time.
• Re-ranking sends one yes/no request per candidate, with the query and the candidate as its input.
• There is no prompt. StructuredAIOptions.Instructions and UseStructuredOutput only apply to LLM providers. Put guidance in the question texts and the option descriptions.
• Retries. A request the vendor answers with 429, 502, 503 or 529 is retried twice, after one and then two seconds. Each attempt times out after 60 seconds.
• ActionProbability equals Confidence, because the hosted models have no separate action head.

Errors

A call throws DecisionProviderException when:

• the provider is not set up, or has no API key;
• the model asked for is not enabled on it (the message lists the enabled ones);
• the vendor's API refused the request. StatusCode and ResponseBody hold its answer, and the message carries the vendor's error text.

An answer that cannot be read, or a question the vendor refused to answer, throws StructuredAIPredictionException.

Check StructuredAI.IsAvailable(options) first in code that must keep working when no provider is set up.

Usage and cost

Every request is recorded on Manage → Operate → LLM Usage (#/manage/operate/llm-usage), under the model the vendor reports and attributed to the vendor (typesafe, openai, inception). An alias is recorded as what it resolved to: jev-latest is recorded as jev-1.13.0. Failed requests are recorded with their error. Cost is calculated from the reference prices when cost calculation is enabled.

Differences between the providers

Mercury Decide speaks the same request format as TypeSafe's System One API, so the two behave the same from code.