Curiosity - Structured Decisions (StructuredAI)

Structured Decisions (StructuredAI)

StructuredAI answers typed questions about a piece of text: pick one of several options, place the text on an ordered scale, or say whether a statement holds. Every answer comes with a probability for each option and a confidence, so code can act on a clear answer and route an unclear one to a person. It also re-ranks a list of candidates against a query.

It is a decision API, not a generator. It never writes text; it only chooses between options you name. For free text out (a summary, a reply), use ChatAI.

Every code scope has it as StructuredAI, next to ChatAI and AgentAI: endpoints, AI tools, scheduled tasks and connectors. The types live in Mosaik.AI, so add using Mosaik.AI; at the top of the script.

Asking questions

using Mosaik.AI;

var answers = await StructuredAI.AskAsync(ticket.Body, q => q
    .Choice("intent", "What does the customer want?", o => o
        .Option("refund",         "money returned or a duplicate charge reversed")
        .Option("technical_help", "a bug, outage or integration problem")
        .Option("other",          "none of the other options fits"))
    .YesNo("is_urgent",   "Does the message communicate time pressure or a deadline?")
    .Score("frustration", "How frustrated does the customer sound?",
        "calm and neutral", "concerned but civil", "clearly annoyed", "very angry"),
    cancellationToken: CancellationToken);

string intent = answers.ChoiceOf("intent");          // "refund"
bool   urgent = answers.IsYes("is_urgent");          // P(yes) >= 0.5
double p      = answers.ProbabilityOf("is_urgent");  // 0..1
int    level  = answers["frustration"].Level;        // 0..3, nearest whole level
double score  = answers.ScoreOf("frustration");      // expected level, fractional

Kind	Builder	Answer
Choice	`.Choice(id, question, options)`	`Choice` is the winning option key. Option order is the order the model sees; the description tells it what the option means. Always include an escape option such as `other`.
Score	`.Score(id, question, levels...)`	`Score` is the expected level, from 0 (the first level) to `levels - 1`, fractional. `Level` and `LevelText` give the nearest whole level.
Yes/No	`.YesNo(id, statement, yesMeans, noMeans)`	`Probability` is the probability that the statement holds; `IsYes` is `Probability >= 0.5`.

Every StructuredAnswer also carries Confidence (0 to 1) and Probabilities (one entry per option, level index, or yes/no).

The input is a string, or an object that is serialised to JSON. With an object, name its fields in backticks in the question text:

var answers = await StructuredAI.AskAsync(new { subject = mail.Subject, body = mail.Body }, q => q
    .YesNo("billing", "Is `body` about an invoice or a payment?"));

AskManyAsync(inputs, questions) answers the same questions for many inputs in one call. Use it instead of a loop of AskAsync. StructuredQuestions.Create(q => ...) builds a set once for reuse, and StructuredQuestions.Triage(), Email(), Guard() and Moderation() are ready-made sets.

Re-ranking

RerankAsync scores every candidate by the probability, from 0 to 1, that it is relevant to a query. Sort by the score to re-rank:

Dictionary<UID128, float> scores = await StructuredAI.RerankAsync(query, candidates, cancellationToken: CancellationToken);

var top = scores.OrderByDescending(kv => kv.Value).Select(kv => kv.Key).Take(10).ToList();

candidates maps each node to a short text: a title and a snippet, rather than a whole document. Pass question: to ask something other than relevance, naming the two fields query and document.

What answers: `StructuredAIOptions`

Every call takes an optional StructuredAIOptions that picks the engine:

StructuredAIOptions local    = StructuredAIModel.Multilingual;                                       // a local checkpoint
StructuredAIOptions decision = StructuredAIOptions.Decision(DecisionProviders.TypeSafeJev, userUID: CurrentUser);
StructuredAIOptions llm      = StructuredAIOptions.LLM(LLMProviders.OpenAI, CurrentUser);

Engine	What it is	Data leaves the workspace	Cost
Local checkpoint (default)	A Laya model that runs in-process on the CPU. English, Multilingual and Typed Decisions checkpoints, downloaded by an administrator from Manage → AI → Models.	No	None
Decision provider	A hosted decision model: TypeSafe Jev, OpenAI Decisions or Inception Mercury Decide. See Decision Providers.	Yes, to the vendor	Per input token
LLM provider	Any configured LLM provider, asked to predict a probability for every option. Each input is one recorded agent run.	Yes, to the provider	Input and output tokens

Omitted, a call uses the local English checkpoint.

DecisionProviders and LLMProviders are generated helper classes, like Agents and Secrets. Each has one constant per provider set up in the workspace, so code that names a provider that does not exist fails to compile instead of failing at run time. LLMProviders.<Name> converts to the provider's UID128 wherever an API takes one.

How to choose:

Start with the local checkpoint. It needs no key and keeps the text in the workspace.
Use a decision provider when no checkpoint is downloaded, when the inputs are longer than a checkpoint's context (512 or 1024 tokens), or for a language the English checkpoint does not read.
Use an LLM provider when a question needs world knowledge or a long, nuanced rubric. Its probabilities are the model's estimate, not calibrated ones.

Checking before calling

StructuredAI.IsAvailable(options) returns whether the engine can answer right now:

for a local checkpoint, that it is downloaded;
for a decision provider, that it is set up, has an API key, and has the requested model enabled;
for an LLM provider, that its task exists.

Code that must degrade gracefully checks it first.

Errors

Exception	When
`StructuredAIModelNotInstalledException`	A local checkpoint was asked for before an administrator downloaded it. The message names the model and the page to download it from.
`DecisionProviderException`	The decision provider is not set up, the model asked for is not enabled on it, or the vendor's API refused the request. `StatusCode` and `ResponseBody` carry the vendor's answer when there is one.
`StructuredAIPredictionException`	A provider's answer could not be read, or a question was left unanswered or refused. `RawOutput` holds what it answered.

Concurrency

Every member is safe to call from parallel requests and tasks. A local checkpoint is loaded once, on first use, and released after 30 minutes idle; its forward passes run one at a time because a single pass already uses every core. Pass parallelOptions: to cap the threads a local call uses. A provider call runs up to MaxConcurrentRequests (4 by default) requests at a time.