LLM providers

Malspark's agents call language models through one of four provider backends. Which one a deployment uses is a setting, not code: core.llm.provider picks the backend, and each backend has its own credentials, endpoint and model names. Every setting on these pages is edited under Settings → Configuration → Models, or walked through by the Connect a language model setup guide.

At a glanceCopy Anchor Link

Analyst and judge modelsCopy Anchor Link

Each backend has two model names: expert_model, which the analysts use, and judge_model, which the judge uses. The analysts run at temperature 0.1 and the judge at 0.0 unless an agent's own entry says otherwise. A job can switch the backend for itself with llm_provider in its config; see Running an analysis.

Per-agent models and fallbacksCopy Anchor Link

llm.agents holds one optional entry per agent key — provider, model, temperature and, for a provider that declares one (openai, ollama), a base URL — so the analysts and the judge need not share one model, or one server. It is ordinarily edited from the Agents page, one agent at a time.

{
  "static": {
    "provider": "openai",
    "model": "qwen3.6-35b-a3b",
    "base_url": "http://127.0.0.1:8080/v1",
    "fallbacks": [
      {"provider": "ollama", "model": "gemma4:12b"}
    ]
  }
}

An entry may name the models the agent falls back to, in order. The next model is asked only when the one before failed as a provider — a refused or dropped connection, a timeout, an HTTP 5xx, 408 or 429, a model the server does not have, a refused credential — and never because of what a model said. Which model answered is recorded on every turn. The full rule is in Per-agent model overrides.

A provider answers for itselfCopy Anchor Link

Every fact the platform needs about a provider lives on that provider's adapter (maljan.llm.provider_base.LLMProvider): which metadata requests describe a model's window and which setting declares one, the global effort setting and the levels it takes, whether structured output is worth asking for, which SDK packages raise its errors and whether its own client retries them, whether a model left reasoning spends its output cap on it, whether its requests write the prompt cache for an hour (the spend meter prices that write only for that provider's models), whether its endpoint serves concurrent requests, whether an agent may give it an endpoint of its own, the keyword a call's output cap goes under, the reasoning it asks to have sent back, and the one short turn the settings probe asks it for and how that answer is read. No shared module names a provider. llm.provider, an agent entry's provider and a job's llm_provider each take any registered provider's name: llm.provider is refused when saved if nothing registered it, and a stored name registered only in another process still loads.

A provider registered at runtime with register_provider subclasses LLMProvider and declares a settings_model. Its block lives under llm.<name> (its credential, address, effort and expert_model / judge_model), is read with that model, and is listed in the catalogue with the annotations the adapter declares. A selected provider with no block of its own names no role model, and saving it is refused, so it never runs on another provider's models.

A model is probed before a job may name itCopy Anchor Link

With llm.require_probe on (the default), a job is refused while any model its team names has not answered a connection test. The probe asks for one short answer at the endpoint and on the model the run will use, through each provider's own completion API. See A model is probed before a job may name it.

Behaviour common to every providerCopy Anchor Link

• Request timeout. Every provider's client is built with an 1800 s request timeout until the model's pace is measured; after that each request is sized for its own output cap. See A call waits as long as its answer takes at the model's pace.
• Every tool call is answered. No request sends a tool call without its reply, in each provider's own message shape, because every provider's API refuses a history with a call left unanswered.
• The context window is learned, not guessed. The window a model is served with decides how much of a tool answer it may read and how large its output cap is. It is taken from a setting, a metadata endpoint or a vendored table, and a run that could learn none says so. See How much of a tool answer a model sees.
• A spend ceiling, when you want one. core.llm.max_spend_usd_per_job bounds what one job may spend on its models, priced from core.llm.model_prices. It is empty by default. See Loops have no default limit.