Advanced
Configuration
The bootstrap environment contract, Settings → Configuration, the setup guides, connection probes, JSON export and import, secret storage.
Malspark is configured in two places and no others. A small bootstrap contract comes from the process environment and is validated once at startup; every other application setting lives in the settings store in Postgres and is edited from the web console. This document describes both, the export and import format, and how secrets are stored.
The bootstrap contract
apps/api/app/config.py builds APISettings from the process environment
only — no .env file is discovered or read. Construction never refuses;
apps/api/app/bootstrap.py performs every refusal at startup and raises one
bootstrap: ... line naming every problem at once, so a misconfigured
deployment does not have to be restarted once per missing variable.
The full surface is documented in
bootstrap.env.example.
The table below is the
same contract in short form.
| Variable | Required | Default | Notes |
|---|---|---|---|
DATABASE_URL | yes | postgresql+asyncpg://maljan:[email protected]:5433/maljan | Async driver. |
REDIS_URL | yes | redis://127.0.0.1:6379/0 | Job queue, events, rate-limit counters. |
MINIO_ENDPOINT | yes | 127.0.0.1:9000 | Host and port, no scheme. |
MINIO_ACCESS_KEY | yes | minioadmin | |
MINIO_SECRET_KEY | yes | — | Refused outside debug while unset or left at minioadmin. |
SETTINGS_ENCRYPTION_KEY | yes | — | Fernet key; an invalid or missing key aborts startup. |
JWT_SECRET_KEY | outside debug | — | Refused when unset, under 32 characters, or a known placeholder. |
DB_POOL_SIZE | no | 5 | |
DB_MAX_OVERFLOW | no | 10 | |
DB_POOL_RECYCLE_SECONDS | no | 1800 | |
RUN_MIGRATIONS_ON_STARTUP | no | false | Leave off in production; migrate as a deploy step. |
MINIO_BUCKET | no | maljan-samples | |
MINIO_SECURE | no | false | |
AUDIT_ANCHOR | no | — | database, object_store or none: where the audit chain's head is kept outside the database (see Operations). Unset means database when AUDIT_ANCHOR_DATABASE_URL is set, else none; any other value is refused at startup. Read from the environment only. |
AUDIT_ANCHOR_DATABASE_URL | with AUDIT_ANCHOR=database | — | A role that may only INSERT and SELECT audit_chain_anchor; startup warns when it is the application's own role. A platform secret. |
JWT_ALGORITHM | no | HS256 | |
JWT_ISSUER | no | maljan-api | |
JWT_AUDIENCE | no | maljan-clients | |
JWT_KEY_ID | no | v1 | kid stamped on new tokens. |
JWT_PREVIOUS_SECRET_KEY | no | — | Accepted alongside the current secret during a rotation window; set JWT_PREVIOUS_SECRET_NOT_AFTER beside it to give that window an end. |
JWT_PREVIOUS_KEY_ID | no | v0 | |
JWT_PREVIOUS_SECRET_NOT_AFTER | no | — | ISO-8601 moment (UTC when it carries no offset) after which a token signed with the previous secret is refused. Unset means the window has no end, which the startup check warns about at every start; a value that does not read as a moment is refused at startup. |
APP_NAME | no | Maljan | |
APP_VERSION | no | 0.1.0 | |
DEBUG | no | false | Also enables /docs, /redoc and /openapi.json. |
SQL_ECHO | no | false | Independent of DEBUG. |
CORS_ORIGINS | no | ["http://localhost:3000","http://127.0.0.1:3000"] | JSON list. |
CORS_ALLOW_METHODS | no | GET, POST, PUT, PATCH, DELETE, OPTIONS | JSON list. |
CORS_ALLOW_HEADERS | no | Authorization, Content-Type, X-Correlation-Id, X-API-Key | JSON list. |
COOKIE_SECURE | no | inverse of DEBUG | Secure flag on the refresh cookie. |
AUTH_DISABLED | no | false | Local development only; refused when DEBUG is false. |
AUTH_DISABLED_USER_ID / _EMAIL / _FULL_NAME | no | seeded dev admin | Only read when the bypass is on. |
SAMPLES_DIR | no | data/samples | Host directory the worker copies each job's sample under, one subdirectory per server that needs a copy. A containerised server mounts it at its declared sample_mount. |
UPLOAD_TEMP_DIR | no | data/uploads/.tmp | Scratch directory for uploads and worker tempfiles. |
No variable in this table names a place inside another container. Where a
containerised tool server sees SAMPLES_DIR is that server's own declared
sample_mount (see What a server declares about its tools): the
worker copies each sample under SAMPLES_DIR on its own host and hands each
server the same file under its own mount, or the host path itself when the
server declares none. GHIDRA_CONTAINER_SAMPLES_PATH is retired; the migration
that retired it read a value other than /data/samples once into
core.static.ghidra.sample_mount, and a worker that still has it set says at
start which setting holds it now. The worker states, for each static provider
the settings select, where its copy goes and the path its server is handed, in
one line at start (Static provider 'ghidra' mirrors a sample to ... and is handed it as /data/samples/.work/<sha256>.).
Seven of these variables open the platform's own stores: DATABASE_URL,
REDIS_URL, MINIO_ACCESS_KEY, MINIO_SECRET_KEY, SETTINGS_ENCRYPTION_KEY,
JWT_SECRET_KEY and JWT_PREVIOUS_SECRET_KEY. Each is marked once, on
PlatformSecrets in src/maljan/core/platform_secrets.py, which APISettings
inherits and gives each its default, with what it opens and the names the
shipped Compose file writes the same value under (POSTGRES_PASSWORD, REDIS_PASSWORD,
MINIO_ROOT_USER, MINIO_ROOT_PASSWORD). Those names are what no tool server's
environment may carry (Which directories a sidecar may read
says how that is held).
One warning does not block startup: COOKIE_SECURE false outside debug, which
means the refresh cookie crosses the wire unencrypted unless a trusted proxy
terminates TLS.
Outside Compose, keep these in the gitignored bootstrap.env at the repository
root. make dev-up, make dev-down, make dev-logs and make migrate source
it; for a bare process, set -a; . ./bootstrap.env; set +a first.
Settings → Configuration
Everything else is a catalog entry. The catalog is derived from the core
Settings model (src/maljan/core/config.py) plus the editable and read-only
API fields (apps/api/app/services/settings_catalog_api.py), and each entry
carries its own title, description, type, bounds, choices and the group it
belongs to (src/maljan/core/settings_annotations.py).
The backend exposes eighteen groups, in this order:
| Group | Covers |
|---|---|
| LLM & model | Which backend the analysts and the judge call, and the per-call limits. |
| Providers | Credentials, endpoints and model names per LLM vendor. |
| Frontier arms | Evaluation-only comparison endpoints and their cost accounting. |
| Static analysis provider | The static analyst's provider and its connection details. |
| Sandbox provider | Where samples are detonated, or which uploaded report stands in. |
| Tool servers (MCP) | The servers agents may call and the tools each may expose. |
| Memory / LTM | The backend, its own settings under memory.<backend>, and how many neighbours are recalled. The enrichment worker, the purge and the API's health probe build the store memory.backend selects from these same keys; there is no second, API-side copy of them. |
| Analysis layers | Deterministic pre-analysis layers, reference data and thresholds. |
| Negotiation | Rounds and the consensus condition. |
| Chunking | How large inputs are split before they reach a model. |
| Reporting | Report contents and the metadata stamped on it. |
| Agents | The analysts, the active profile and the ReAct limits. |
| Live events | The conversation feed the console draws a running analysis from, and how long the record of one is kept. |
| Tracing | LangSmith tracing of model calls. |
| Enrichment / threat intelligence | Lookups for the indicators a report names, and whether they run on the enrichment worker or beside the analyses. |
| Provider plugins | The providers installed packages add, and the allow-list that decides which of them load; none loads by default. |
| API | Request limits and login protection; applied immediately. |
| Deployment (read-only) | Bootstrap values, shown for reference. |
The console maps them onto five sections — Models, Analysis tools, Agents and
pipeline, Layers and reporting, Platform — and adds one synthesised group,
Profiles, carved out of Agents
(apps/web/src/app/(app)/settings/configuration/sections.ts). A backend group
the console does not list explicitly falls through to Platform, so a new group
appears without a frontend change.
Editing, review and apply
Edits are staged in the browser, not written per keystroke. The toolbar shows
what is staged, the review step lists each change as a before/after pair, and
applying sends one PATCH /api/v1/settings with the whole set. Each entry
declares when it takes effect: live, next_job, or restart for read-only
deployment values.
A value is saved whole, so a page loaded before another writer changed a key
would drop that change unseen: an upload's accepted report format adds to
core.sandbox.report_formats and core.sandbox.upload.allowed_formats, a
credential bootstrap enables its server in core.mcp.servers, and another
admin may save any key. The PATCH therefore carries versions: per changed
key, the updated_at the page was served it with (null for no stored row).
A key whose stored row is not that one any more (saved, accepted or reset
since) is refused with 409 naming it, and nothing is saved; the console drops
that key's staged edit, reloads it, and says so, so the change can be made
over what is stored now. A key sent without a version is not checked, so a
script that means to overwrite still can; an import always writes what its
file holds.
An entry that has an override can be reset: DELETE /api/v1/settings/{key}
removes one override, DELETE /api/v1/settings removes a whole group's. The
value then falls back to the catalog default and the console shows it as such.
Per-agent model overrides
llm.agents holds one optional override per agent key — provider, model,
temperature, base URL and reasoning effort — so the analysts and the judge need
not share a single model. It is a JSON leaf of its own and is ordinarily edited from the
Agents page, one agent at a time.
Two keys of it belong to model-calling roles that are no agent definition:
mediator, the debate stage's mediator, and summarizer, the function
summariser. The Agents page lists both under Roles with a model entry only,
with the same model fields and a Test model button that asks each model of
the entry for one short answer and files it for the probe gate. With no entry
either runs on the global expert model at the global effort, exactly as before
the keys existed. The mediator does not fall back to llm.agents.judge: a judge
entry moves the verdict calls and never the mediation. No agent definition may
take either key; one stored under it before is renamed to <key>_custom with
every reference, its own llm.agents entry included.
The function summariser used to name its model with
preprocessing.summarizer_provider and preprocessing.summarizer_model. Neither
ever reached a model — the summariser ran on the global expert model whatever
they held, and the two values travelled as top-level fields of every summariser
request. The Anthropic Messages API refuses a field it does not know, as does an
OpenAI-compatible server that refuses unknown parameters, so on those endpoints
every summary request failed and the summariser passed its raw input to the
analysts. Its requests now carry neither field, so there it summarises on the
expert model and the analysts receive summaries where they used to receive raw
text. Both settings are gone and their stored rows deleted; no entry
is made from them, since that would move a summariser onto a model it has
never called. llm.agents.summarizer is what sets it now, and its model is
still built with no output cap of its own, each call held to what its window
leaves.
The base URL is per agent, and applies to the openai and ollama providers
only: Anthropic and Gemini are vendor APIs with no endpoint to override, and an
override set against them is rejected on save. Two agents can therefore sit on
two different local OpenAI-compatible servers (llama.cpp / ik_llama.cpp) or two
different Ollama hosts, while the global llm.openai.base_url and
llm.ollama.base_url stay the fallback for everything that sets none. The
credential is not per agent: an openai entry with its own endpoint still
authenticates with llm.openai.api_key. A per-agent endpoint gets the same
treatment a global one does — the llama.cpp sampler keys and the structured
output the local servers handle badly are decided from the endpoint the agent
will actually call.
An entry may also name the models the agent falls back to, in order, under
fallbacks — each one a provider, a model and, for openai and ollama, a
base URL of its own, written exactly like the entry's first model. A fallback
with no temperature takes the entry's. The Agents page edits the list under
the model override (add, remove, move up and down). The next model is asked
only when the one before failed as a provider: a refused or dropped
connection, a timeout, an HTTP 5xx, 408 or 429, a model the server does not
have, a refused credential, or a refusal the provider reports as an error.
Never on what a model said: an answer the validation loop rejects is sent back
to the model that wrote it, and a parse or validation error a model's answer
raises reaches the loop rather than the next model. A timeout is a provider
failure because every model on a list but the last has its own turn deadline —
core.llm.fallback_turn_share (0.5) of what is left, at that turn, of the
budget the current loop runs under (never less than one second), so inside an
ask it is a share of the ask's clock and a stall late in a loop is still
replaced before the loop cancels it; a share of the loop because the loop is
what would otherwise cancel a stalled model first. The reporter's list starts
over before the narrative round, against its 600 s, and again before the
composer sections, against core.reporting.composer_per_section_timeout —
and every provider's client has a 1800 s request timeout until the model's
pace is measured, after which each request is sized for its own output cap
(see A call waits as long as its answer takes). A 429 or 503 whose
Retry-After (seconds or an HTTP date) asks for at most thirty seconds is waited out on the same model
once before the list moves on. Once the list has moved, the model that answered
stays for the rest of that loop (a stalled first model costs one deadline, not
one per turn), and the next loop starts at the first model again.
A model named twice in one list is refused on save. An entry without
fallbacks is the single-model form every entry had before, unchanged.
Which model answered is recorded on every turn — on the ledger entry of each
call the turn asked for and per agent in run_summary.models — and the switch
is recorded once, with the reason in words: in the run summary and as a
model_fallback event the conversation draws whether or not deltas stream.
Which dialect an OpenAI-compatible endpoint speaks
llm.openai.compat says whether the endpoint behind base_url is llama.cpp or
a hosted OpenAI-compatible API, because the two disagree about what a request
body may contain. Three llama.cpp-only fields exist for good reasons — the
repetition penalty that stops a small local model looping on ATT&CK id recall,
the n_predict echo of the output cap that llama.cpp reads where it ignores
max_completion_tokens, and chat_template_kwargs.enable_thinking — and a
hosted API answers all three with 400 Unsupported parameter.
llama.cpp's DRY sampler is a fourth, opt-in: llm.openai.dry_multiplier,
dry_base, dry_allowed_length and dry_penalty_last_n are forwarded in the
same extra_body when set, each on its own, and none is set by default. It
penalises a token that extends a sequence already repeated in the context.
| Value | What is sent |
|---|---|
auto (default) | llama_cpp when the base URL host is loopback, link-local, .local or a private address; standard otherwise |
llama_cpp | the three extras, whatever the host — for a local server reached through a public name |
standard | OpenAI-standard fields only — for a hosted API, or a local vLLM that validates its body |
deepseek | DeepSeek's API: the output cap as max_tokens as well, and disable_thinking as thinking.type: disabled; none of the llama.cpp extras |
An endpoint that rejects one of the extras anyway is retried once without them,
recorded for the rest of the process, and named in a warning that says to set
this value explicitly. base_url unset means api.openai.com, which never
receives them in any mode.
auto guesses which server software answers; it is not the rule a run's
disclosure block places a host by, and the two differ on purpose. A single
label (a compose service's name), a .internal name and the shared range
100.64.0.0/10 get standard here, because a proxy in front of a hosted API
sits there as often as llama.cpp does and refuses the extras, while the
disclosure block takes those hosts as inside the deployment
(What left this machine).
deepseek is a value of its own because no one body serves both APIs. OpenAI's
clients send the output cap as max_completion_tokens, which DeepSeek's chat
completions ignore (measured: a cap of 5 came back as 88 tokens with thinking
off and 138 with it on, both ending stop); DeepSeek reads max_tokens, with
reasoning counted against it, which OpenAI's own API refuses beside
max_completion_tokens for its reasoning models. Under auto a DeepSeek base
URL is a hosted API like any other, so no cap reaches it: the value is set, not
guessed from the host. DeepSeek also accepts chat_template_kwargs and ignores
it, so deepseek is the only value under which disable_thinking reaches it.
A DeepSeek thinking model returns its reasoning as reasoning_content beside
content, and on a request that carries tools that reasoning has to be sent
back on its assistant turn in every later request: DeepSeek's thinking-mode
guide (https://api-docs.deepseek.com/guides/thinking_mode, on tool calls) says
the API answers 400 otherwise. The OpenAI client reads it from no answer and
writes it into no request, so under deepseek the provider keeps each turn's
reasoning_content exactly as returned and sends it back on that turn, on
every request that carries tools. A request without tools is sent without it:
the guide says it is not needed there and is ignored if sent, so it would only
be input read for nothing. The answer's content is untouched, and a turn sent
again is sent byte for byte, so the request's front stays what DeepSeek has
cached. The reasoning sent back counts toward the conversation's size in the
window budget, and its tokens are in the run's counts (reasoning_tokens); its
text stays with the turn in the loop's conversation and is not published. The
cap goes out as max_tokens on each request, so a cap bound for one call
reaches DeepSeek as the model's own does.
On every provider the pipeline builds its models from (openai under every
compat value and through either OpenAI API, anthropic, gemini and
ollama), no request sends a tool call without its reply. A turn can hold a
call no tool ran (its arguments were cut inside a string, so the call stays in
invalid_tool_calls, which the loop's tool node does not run and the OpenAI
client still writes into the request), and every provider's API refuses a
history with a call left unanswered: DeepSeek answers 400, the Responses API
wants a function_call_output for every function_call, Anthropic wants each
tool_use block answered by a tool_result block at the front of the next
user turn, and Gemini wants as many functionResponse parts as the model turn
has functionCall parts. Each request is completed as it is sent, in the
provider's own message shape (maljan.llm.tool_replies): a call with no reply
anywhere in the conversation gets one saying no reply was recorded (and, for a
call whose arguments did not parse, that it was not run), and a warning says
how many.
| Provider | Where the reply goes |
|---|---|
openai, chat completions (every compat) | a tool message after the turn; the turn's tool messages are put in the order of its calls, after DeepSeek's reasoning passback |
openai, Responses API (a model the client sends there: a codex or pro model, or reasoning, include, text, truncation or context_management set) | a function_call_output item right after the run of call and output items the call stands in, in call order; a call counts as answered wherever its output stands |
ollama | the same tool message as chat completions, in the shape the Ollama client writes its own |
anthropic | a tool_result block at the front of the next user turn, after the results it already has and before its text; a turn followed by another assistant turn gets a user turn holding only the replies |
gemini | a tool reply in the conversation before the client serializes it, placed so the turn's functionResponse parts come out in call order: Gemini's parts carry no id and pair by name and order. A call counts as answered when any tool reply in the conversation carries its id, the client's own rule, so a request the client already builds right is sent unchanged |
A reply that was recorded is never said missing. Chat completions, Ollama and Anthropic pair a call only with the replies right after its turn. A reply recorded further on, where it answers no call of the turn it follows, is moved to its call when only platform messages (a nudge, a user turn) stand between them, and a warning names the calls moved. When a model turn stands between them, the reply is not moved: that would put the model's later words after a result it had not seen when it wrote them. Nothing is written for that call either; it is sent as the client built it, the provider may refuse the request, and a warning names the call. Anthropic's client joins every reply and user message between two assistant turns into one user turn, so there a reply standing elsewhere is always after a model turn and is never moved.
The completion never changes who speaks last in a request: a history that ends on the model's turn of calls would end on their replies once completed, so it is sent as the client built it, and a warning names the calls left without a reply.
The Anthropic, Gemini and Ollama clients do not write a call whose arguments
did not parse into the request at all, so there it needs no reply; a
tool_use block the turn itself holds is answered like any other, and said
not run when its input did not parse. The call stays in the turn as the model
wrote it, a history that is already well formed is sent as it was, and the
loop's own conversation is not changed. The one model built outside the
providers, the evaluation harness's frontier comparison arm
(core/frontier.py), is a plain OpenAI client and is not completed.
llm.openai.compat, like every llm.openai setting, is global: it applies to
every model built on the openai provider, per-agent entries and fallbacks at
their own endpoints included. Under deepseek, an openai entry pointing at a
local llama.cpp server gets DeepSeek's fields and none of the llama.cpp extras,
so a run that mixes the two keeps its local entries on another provider
(ollama) or runs them under llama_cpp in a separate configuration.
Reasoning effort
llm.openai.reasoning_effort is sent as the request's top-level
reasoning_effort field, on every request of every dialect, exactly as written:
DeepSeek takes low, high and max (its thinking-mode guide,
https://api-docs.deepseek.com/guides/thinking_mode, maps xhigh to high and
ultra to max), OpenAI's reasoning models minimal to high. Empty, the
shipped value, sends nothing and leaves the endpoint's own default (high on
DeepSeek). A value the endpoint does not know is its own 400, and the llm
connection test asks with the value, so it is found there rather than on a
job's first call. The setting is global, sent to every openai-provider model
including per-agent entries and fallbacks that name no effort of their own.
Per agent. An entry of llm.agents — and each of its fallbacks — may name
its own effort, sent where the global value is sent: Anthropic's
output_config.effort (one of low, medium, high, xhigh, max; a level
the Models API says the model does not take refuses the job, naming the field
it was read from — llm.agents.<key>.effort, or
llm.agents.<key>.fallbacks[<i>].effort for a fallback) or the
OpenAI-compatible reasoning_effort, as written. Left empty it inherits llm.anthropic.effort or
llm.openai.reasoning_effort — a fallback inherits the global value of its own
provider, not the entry's effort — so with no per-agent effort every request
is the one sent before. An ollama or gemini entry that names one is refused
on save: neither provider sends a reasoning effort. Nothing sets a level by
default.
Every model-calling role reads an entry: each analyst its own key (its
revision round runs on the same model), judge the verdict calls, reporter
the narrative and the composer sections, mediator the debate stage's
mediation and summarizer the function summariser. An absent entry falls back
to a global role model: judge for the reporter (the build line then says
role=judge), expert for every other key, the mediator included. What
follows an entry follows these two as well — the output cap derived from the
entry's model and logged as Output cap for mediator, the model each ledger
row and run_summary.models records the call under, the Building dedicated LLM for agent 'mediator' line, the run's window, and the probe gate, which
checks a role whenever the run calls it (a debate stage, the summariser
switched on): on its entry, or with none on the global model it falls back to.
The Agents page offers the field once an entry names a model, with the levels
the backend serves for that provider and model (GET /api/v1/settings/effort-options): where the Models API described the model,
every level it does not refuse, a level it said nothing about marked "not
described for this model" (the run sends it and the API answers); else the
setting's own list; and a text box for an OpenAI-compatible endpoint. Each model's build line in the worker log names
the effort in force and where it came from, e.g. Building anthropic/claude-haiku-5-5 (role=judge, temp=0.0, effort=max from llm.anthropic.effort) or
effort=high from llm.agents.reporter.effort.
Every probe asks each model at the effort its run sends — the agent probe and the provider's Test alike — and a model whose entry names its own effort is filed under that model and that effort. A model with no effort of its own is filed and looked up as before, so its earlier results stay valid; changing an entry's effort is a change the probe gate asks about, on save and at submit, until the model has been probed at the new level.
Setup guides
Settings → Setup offers seven guided flows
(apps/web/src/app/(app)/settings/setup/guides.ts): llm, static,
sandbox, tool-server, agent, memory, enrichment. A guide walks
provider choice, credentials, a connection test and a review step, and applies
the result as one write, so a half-configured provider is never left behind.
Connection probes
The "Test" buttons are each provider's own connection test
(apps/api/app/services/settings_probes.py holds no table of them). The API
answers four probes of its own — llm, mcp, agent and redis — and every
registered provider's id: a static or sandbox provider is built from the
candidate settings and its own probe() is asked, a memory backend's
readiness(), a reputation source's probe(). The llm probe lists models,
builds its one short turn and reads the answer through the selected LLM
provider's adapter (maljan.llm.provider_base). A provider registered at
runtime has a "Test" button the day it is registered, and the test and the job
read one object built one way. They are reached at
POST /api/v1/settings/test/{probe}, with /test/mcp and /test/agent taking
a body naming the server or agent; an id no provider registered answers 404. A
probe that fails answers 200 with the failure as data — a connection test that
fails is an answer, not an error.
What the console shows about a provider — its title, its one-line description,
its credential keys, whether it is hosted or run locally, its test and the label
of its button, whether an agent may give it an endpoint of its own, its effort
setting — is the provider's own declaration, served by GET /settings/schema
on its selector's entry (choice_titles, choice_descriptions,
choice_facts) and in probe_labels. The built-in and reserved tool-server
keys come on the server map's entry. The console keeps no table of providers.
A probe runs against the staged values on top of the stored ones, so an input
the caller did not stage comes from the store — including the credential. That
means a staged endpoint is sent the stored key for that provider, which is a
secret the console never shows in the clear. Every probe therefore writes one
audit row (settings.probe) naming the probe, the endpoints it was pointed at
(as labels: scheme and host), the keys that were staged for it and whether it
succeeded. The values themselves are never in the row. The routes are
admin-only, as they have always been; what was missing was the record.
Enrichment sources
Post-verdict enrichment asks the sources api.enrichment_sources declares,
laid over the two the platform ships as declarations (virustotal and
abuseipdb). The secret setting a shipped source reads its credential from is
generated from its declaration's credential: its title is the source's label,
its description says which indicator kinds leave the host for which host, and
its Test button is the source's own declared probe; no table in the API or
the console names a source (the shipped two keep api.virustotal_api_key and
api.abuseipdb_api_key). An entry under a
shipped key changes only the fields it names ({"abuseipdb": {"enabled": false}} turns one off); any other key adds a source. For every domain and IP
worth asking about, the configured sources that answer that kind are asked in
their order, and the first record is kept in network.domains[].reputation
or network.ips[].reputation, where the report, the export and the console
read it whichever source wrote it.
| Field | Meaning |
|---|---|
label, enabled, order | the name shown, whether it is asked, and its place (lower first) |
kinds | the indicator kinds it answers: domain, ip (the kinds the report holds a reputation slot for). A source naming hash or url is refused on save: the sample's hash reputation is declared as a reputation lookup tool (a server tool of kind reputation that takes the hash), which the triage pack and the judge ask |
server, tool, arguments | a tool of a server in mcp.servers whose roles holds reputation; the indicator goes in the argument the tool's declaration in force names for its kind (its block's tool_facts.<tool>.arguments, what the server declares of itself, or what is read of its listing), or the one arguments names (only for kinds the source answers, and only beside tool; a REST source takes none). With no tool, the server is asked through its indicator lookup for each kind: the first tool its tools list ticks whose declaration in force is a lookup about an indicator (indicator_report, indicator_reputation) and names an argument for the kind, never one that sends the sample |
requests | otherwise one https request per kind: method (GET or POST), url, params, headers and json_body, where {value} stands for the indicator and {credential} for the credential. Every placeholder is filled in one pass, so text the indicator brings (a {credential} in a domain name) is sent as written and never filled; the indicator is URL-encoded as one path segment in the URL, URL-encoded by the client in a query parameter, and an opaque value in a header or the body. A URL that carries a query of its own keeps its pairs, in order and repeated keys as written, and the declared params follow them. The host is the URL's own; the credential never goes in the URL's path |
credential, api_key | the settings key the credential is read from, or the source's own key, stored encrypted beside the map and shown masked |
probe | the request its Test button sends, under the source's key |
timeout_seconds, rate_limit_seconds | per request, and the pause after each one |
answer_root, record | where the answer's object sits, and each field of the record: name, path, read (count, value, whole, texts with limit). A REST source needs a record. A server source without one reads its tool's answer as a reputation answer, through the paths the tool's declaration gives, else by the shape every reputation answer has: malicious, suspicious, total, engines, labels (five at most), first_seen, last_seen, link and the service the answer names, each only where the answer gave it; its malicious_when may name the four counts |
record_source, malicious_when, age_from | the record's source (default: the key), the thresholds that mark an indicator suspicious, and the field holding the answer's date in epoch seconds (older than 90 days is stale and marks nothing) |
GET /api/v1/system/status reports how many sources are configured
(enrichment_configured) to anyone, and lists every source in force as
enrichment_sources: [{key, label, configured}] to a signed-in caller only; a
server source is configured only while its server is enabled. The enrichment
guide offers one step per source.
api.enrichment_reputation_servers (off by default) makes every enabled tool
server whose roles hold reputation, and that no source names, a source of
its own, under mcp:<server key>: asked about domains and IPs through its
indicator lookups as a server source with no tool is, its answer read as a
reputation answer, recorded under the server's key, and ordered by its
reputation_service.rank. A lookup that fails is followed by the server's
next one; a lookup whose listing requires an argument besides the indicator,
or that its listing is read as sending the sample, is never asked, whatever a
declaration says of it. After each question it pauses
reputation_service.rate_limit_seconds; where the block says none, it pauses
as long as the slowest shipped source does (16 s today, read from the shipped
declarations), so a server that relays a service a shipped source asks
directly never asks it faster. reputation_service.malicious_when (a list of
{field, at_least} over malicious, suspicious, total and engines)
gives its records a verdict; where the block says none, its records carry no
malicious_verdict and mark no indicator suspicious. Each sends the report's indicators to that server
and the service behind it, so it is the operator's consent: off, a server is
asked only through a source that names it. The run's disclosure names every
such server before the first question and every question asked.
Where the enrichment runs
api.enrichment_dedicated_worker decides whether a report's reputation lookups
are queued for the enrichment worker or beside the analyses, and it ships off:
a deployment that runs one process keeps working. On the shared queue the
enrichment yields — it re-enqueues itself a minute later whenever an analysis
is waiting, up to a total of 30 minutes, after which it runs anyway — so an
analysis submitted after an enrichment does not wait for the whole of it, and
the enrichment is never starved. Turn it on where the second process actually runs
— the compose stack starts one and sets the default beside it. With it on and
nothing reading that queue, the analysis worker logs one warning at startup and
GET /api/v1/system/status reports enrichment_worker as down; the
enrichments stay queued and run when a worker appears.
Flipping the setting takes effect on the next enrichment: an arq job id is one per report and queue, so a report queued under the old setting can be queued again for the other worker straight away, and two triggers for one report on one queue still coalesce into one job. What does not move is an enrichment already sitting in the queue it was put in — it runs when that queue's worker runs, which for the analysis queue is between analyses.
One consequence of that, if a flip happens while an enrichment is still queued: the report can be enriched twice, once from each queue. Both runs read the report and write the same fields, and both completion events take their own sequence numbers, so the feed's count still matches its last number; what it costs is a second set of provider lookups against a rate-limited key and two completion events for one report. Flipping the setting when nothing is queued avoids it.
That worker runs ENRICHMENT_MAX_JOBS (default 2) at a time. More than one
because each job waits on somebody else's HTTP; not many more because they
share each source's one key, and reputation services rate-limit per key rather
than per job. api.enrichment_max_lookups still caps each
report; this multiplies how many reports are in flight against the same limit.
A model is probed before a job may name it
What the probe does. Both the llm and the agent probe end by asking for
one short answer — one turn, eight tokens, at the endpoint and on the model the
run will use, through each provider's own completion API (/chat/completions
for an OpenAI-compatible server, /api/generate for Ollama, /v1/messages for
Anthropic, :generateContent for Gemini). Listing a provider's catalogue comes
first and is not enough on its own: a server can offer a name it will not load,
a key can be refused for one model and not another, and a misspelling can land
on a name the catalogue happens to hold. A call that came back with nothing in
it — an empty choices, a candidate that was filtered away — is a failure too.
The OpenAI-compatible body carries the same chat_template_kwargs.enable_thinking
switch a run would send, under the same llm.openai.compat rule and at every
endpoint asked, and a reply whose only text is reasoning_content counts as an
answer: a model that reasoned is a model that loaded on a key that was accepted.
Ollama is asked the same way and read the same way. Its thinking arrives in its
own thinking field, which counts as an answer for the same reason — without
that, every reasoning model served by Ollama failed the probe, because eight
tokens are spent thinking and response comes back empty, and with
llm.require_probe on the deployment could then create no job at all.
llm.ollama.disable_thinking sends think: false, so the budget is spent on
the answer instead; it is off by default, because Ollama refuses the field for
a model that has no thinking mode. A model whose thinking field is empty as
well — some do not fill it at eight tokens — still comes back as answered
nothing, so that failure and a timeout both carry a sentence naming the
setting and what it does. A measured 12B reasoning model failed the probe in
55 s at the default and passed in 243 ms with the setting on; with
llm.require_probe on, every job is refused in between. See the low-memory
option in getting-started.md.
Ollama's body also carries options.num_ctx and keep_alive from
llm.ollama.num_ctx and llm.ollama.keep_alive, the two fields a run sends
with every call that decide which instance Ollama keeps loaded. Ollama loads a
model at the context size the request names and reloads it when a later
request names another, so a probe asked at the server's default (4,096) left
the model at that size and the job's first call paid a full reload out of its
analyst's time budget.
The completion gets ninety seconds of its own, because a local server reloads a
model it had unloaded and a large one is not a ten-second load.
What is written down. One row per (endpoint, model) pair the probe
actually completed a call with, carrying the provider, whether the model
answered and the sentence it came back with. The llm probe completes one call
per pair it will file: the selected provider's expert model at its own
endpoint, and every per-agent override at its own endpoint, which is how a
second llama.cpp or a second Ollama host is proved. Anthropic and Gemini have
one endpoint apiece, named rather than addressed, so a per-agent entry there
differs only in its model — and each is still asked, because a key may be
refused for one model and not another. The same pair named twice is one call
and one row. The judge model is listed and never called, so it is not filed.
The agent probe files the pairs its agent would use: its first model, and
every model it falls back to, one after another — the agent passes only when
every model on its list answered. The llm probe asks the fallbacks served by
the selected provider along with the per-agent entries.
Where a call goes is worked out in one place (maljan.core.model_assignments)
for the probe and for the gate alike, and folded there the way a URL folds —
lower-case scheme and host, the scheme's default port dropped, no trailing
slash — so http://box:8080/v1/, HTTP://BOX:8080/v1 and http://box:80/v1
file and resolve under one spelling instead of four.
A call that ran out of time leaves no row at all — neither a pass nor a failure. Nothing was learned about that pair, and writing a cold model down as a missing one would lock the operator out of their own jobs; the probe says so and asks to be run again once the model is warm.
What it costs. The llm probe asks its pairs one after another — a single
local server told to load several models at once is the failure this project
has already diagnosed — with ninety seconds for each call and five minutes for
the whole probe, counted from the moment the probe starts, so the catalogue
listing in front of the calls comes out of the same five minutes rather than
being added to them. A pair there was no room left to ask is named in the answer as
not tried and files no row, exactly as a timeout does; pressing Test again
asks it. In a failing pair's sentence an endpoint is printed as its scheme and
host, so a base URL that carries credentials does not reach the screen or the
stored row.
Where the gate stands. Submitting a job reads that record for every model
the run can reach — fallbacks included, each named in the refusal as the model
the agent falls back to — each read on the model its builder calls: the agents of
the team's analysis stages and every agent those can ask through ask_<key>, and so
on; the mediator of a debate and the function summariser while it is on, each on its
entry or else the global expert model; and the judge, and the reporter while
reporting.enabled is on, on their entries or else the global judge model — and refuses with 422 when one of them
has no passing row, naming the agent, the model, the endpoint and the probe's
last message. The endpoint appears there as its label — scheme and host — and
never as the value a call is made with: that refusal is read by whoever
submitted the job, not only by an admin, and a base URL configured with
userinfo would otherwise show them the endpoint's credentials. Saving a
per-agent model (core.llm.agents.*) is refused with
the same sentence, because an operator who saves a model nothing can reach has
made the mistake the gate is about and the settings page is where it can be
fixed. The console shows the sentence as written in both places.
The pair is also the invalidation. A changed endpoint or a changed model is a different question, finds no row, and is refused until it is probed: nothing has to expire a result, because a result is never read for a pair it was not taken against. The endpoint half of the pair is folded the way a URL folds — lower-case scheme and host, the scheme's default port dropped, no trailing slash — so one server typed four ways is one key rather than four. A row written before that folding existed is filed under the spelling of the day it was taken; re-run the probe if the gate refuses a model you have tested.
core.llm.require_probe is on, and turning it off is the only way past the
gate — neither a job nor a save can ask to skip it. It is there for an
air-gapped batch run, where the endpoint is known good and nobody is at a
console to press a button.
A team that needs Ghidra waits for it
A static provider that degrades (r2, a generic MCP server, capa/YARA) costs a run some evidence when it is missing, and the run says so. Ghidra does not degrade: a static run with no decompiler is a confident report grounded in nothing, so the agent that needs it fails the run when it starts — minutes and a paid model call after the sample was accepted.
So POST /api/v1/jobs asks first, in the same place as the model gate and
with the same 422 (apps/api/app/services/provider_readiness.py). Every agent
the chosen team can run — its stages' agents and every agent they can ask —
that opens a static provider (the agents only the mediator may ask are not
among them: an ask of one whose provider is not ready is a failed call on the
ledger, not a refused job) is resolved to the provider it would open, as the
run resolves it: the team's forced provider, the definition's own
static_provider, the job's static_provider, or core.static.provider.
Each distinct provider that does not degrade is asked whether it is ready:
Ghidra over http answers when GET <url>/mcp/schema with the configured token
returns below 400, which loads and analyses nothing. Ghidra over stdio is
started by the job itself, so what is checked is that core.static.ghidra.command
is set and names an executable the API host finds (by its last path segment in
the refusal). The check runs where the API runs, on its PATH and filesystem, so for a worker on another host or in another container it says only what the API can see. The shipped transport is stdio with no command, so an operator
who switches Ghidra on without setting transport to http is told that here
rather than when the agent starts. The refusal
names each agent, the provider and its address as scheme and host:
A static provider this team needs is not ready, and a run without it fails when
that agent starts. Start it, switch it on or correct its address, or choose a
team that does not need it. agent 'all_tools_reverser_ghidra' needs static provider
'ghidra' at http://ghidra-mcp:8089, which is not ready: ConnectError: ...Ghidra switched off (core.static.ghidra.enabled false) attaches nothing and
fails nothing, and it is the shipped default, so it is refused only for an
agent that was given it by name — by its definition, by the team or by the
job's static_provider. The loud failure inside the run stays; this is a check
before it.
Format routing and the sandbox
No sample is refused for its format. Routing detects the file type from magic
bytes (pe, elf, mach-o, apk, dex, ipa, jar, ole2, ooxml,
pdf, rtf, chm, img, vhd, lnk, the script types, the archive types)
and maps it to a platform. Each of the four newest is read by the bytes its
specification fixes and nothing more: RTF's {\rtf header; compiled HTML
help's ITSF header with version 2 or 3; a virtual hard disk's conectix
footer cookie, at its start (a dynamic or differencing disk's copy) or, for a
raw disk image whose last 512 bytes are that footer, there (a fixed disk;
only those 512 bytes are read); and a raw disk image (img) by its boot
sector: the boot signature with a GPT header, an MBR with a partition
(status 0x00 or 0x80, a type, a first sector past the MBR and a length) that
lies inside the file, or a volume boot record (a jump, EB xx 90 or
E9 xx xx, and a BIOS parameter block of 512, 1024, 2048 or 4096 bytes per
sector and a power-of-two sectors per cluster, or exFAT's own fields). An
executable, a script by its shebang and a script by its extension are named
as such before a boot sector is looked for. No shipped reader opens them: they
route to whichever tool declares them in its routed_formats or pack step
declares them in its formats, and a run with no such reader says the
container was not parsed. Their platform stays unknown; rtf is a
document and the other three are archive in the category an
identification reports.
The platform vocabulary is windows, linux, macos, android, ios,
multi and unknown. It is a plain string rather than a closed set: an
unlisted value degrades the rule filtering that reads it and nothing else.
multi is a sample that does not bind to one OS — a JAR, a macro document, a
PDF — and unknown is the honest answer when the bytes did not say.
Each sandbox is asked for the options its format needs:
- CAPEv2.
sandbox.cape2.package_by_formatmaps a file type to a CAPE analysis package, e.g.{"apk": "apk", "elf": "generic", "pdf": "pdf", "ooxml": "doc"}.*is the fallback key; a format with no entry is submitted without a package, so CAPE picks one. The guest platform is sent when CAPE has a name for it (windows,linux,android) and left unset otherwise.sandbox.cape2.submit_optionsis sent verbatim as further form fields (machine,tags,options,timeout, anything elsetasks/create/fileaccepts). - Hatching Triage.
sandbox.triage.profile_by_formatmaps a file type to a VM profile, with*as its fallback andsandbox.triage.profilebehind that, so an operator who never touches the map keeps the profile they had.sandbox.triage.analysis_secondsis how long the VM runs the sample, sent as the submission'sdefaults.timeout; empty, the default, sends nothing and Triage's own default applies.sandbox.triage.timeout_seconds, how long the platform waits for the report, must be longer: the wait covers the run and Triage's processing of it, and settings validation refuses one that is not; no margin for the processing is guessed, so leave room for it. A value the account does not allow is refused by Triage, and the submission error quotes Triage's own words and names the setting. The run summary states the run-time limit Triage set for the task (its behavioural tasks'timeoutin the overview): a limit, not a measured duration, and never the value that was asked for. - The REST DSL.
sandbox.rest.submit.submit_fieldsis passed through verbatim as extra multipart fields, beside the existingextra_fields.sandbox.rest.mapping.channelsmaps an operator-chosen channel name to a JSONPath for anything the report schema has no field for; namespace the name by platform, e.g.{"android.permissions": "$.apk.permissions[*]"}. Those rows land inSandboxReport.channelsand are capped and counted like every other channel.
ATT&CK domains
The technique universe spans all three ATT&CK domains. data/attck_valid_ids.json
carries one sorted id list per domain (enterprise, mobile, ics), and
data/attck_techniques.json carries, per technique id, its domain, its name,
its tactic slugs and its MITRE platforms, plus the tactic catalogue (slug to
TA-id and display name) per domain. The two files come from the same bundles
and the same script, so they never disagree about which domain an id belongs
to. Mobile and ICS techniques carry their own matrices' tactics: each bundle
files its kill-chain phases under its own name, and the extractor reads all
three. Between them they answer every dictionary question the pipeline asks about
a technique — validity, name, tactics, domain, platforms — with no network and
no bundle load: tools.knowledge.attck_lookup, attck_scope, attck_validate
and the capability matrix all read them and build nothing.
data/attck_retired_ids.json, written by the same script from the catalogue it
overwrites, names the ids a previous release had, the release that retired them
and, where the bundle states one, the id that revoked them — so an older
report's T1562.001 is reported as retired rather than as an invented id, and
the data builders can retarget it mechanically.
src/maljan/memory/attck_loader.py downloads and caches each domain's STIX
bundle under ~/.cache/maljan/attck/ (or MALJAN_ATTCK_CACHE) for the ranked
index alone — tools.knowledge.resolve_technique and the alignment gate — and
consults it for platforms only when a real id is missing from the vendored
table. Enterprise is required; Mobile and ICS are additive, and a box that can
reach neither keeps working with a narrower catalog. Regenerate all three files
with uv run python scripts/knowledge/prepare_attck_malware_fixtures.py.
A failed index build is remembered for validation.index_retry_seconds
(default 900) and then attempted again, so one unreachable moment does not cost
a worker its index for the life of the process; 0 never re-attempts. The value
travels to the knowledge tool server — the process where the build happens — as
MALJAN_INDEX_RETRY_SECONDS, which is on that server's env_allow and cannot
be taken off it.
Keeping the cached bundles current
validation.attck_refresh_days (default 7; Analysis layers → ATT&CK data
in the console) is how many days the cached bundles may age before they are
compared with MITRE's release index (index.json in attack-stix-data) and
replaced when a newer release is out. 0 never checks on its own; the longest
interval accepted (2,885,414 days) is the longest whose next-check date is
still a calendar date for a cache written before 2100. The analysis worker runs
the automatic check hourly and acts only once a bundle is older than the
interval; a domain found current has its age counted from that check. After a
failed automatic check it waits one hour, then twice as long after each failure
in a row, never longer than the interval, and logs when it will try again; the
console's next-check line shows that time. The loader's own refresh at the first index build after the interval
reads the same setting, which reaches it and the knowledge tool server as
MALJAN_ATTCK_REFRESH_DAYS (on that server's env_allow, written by the
container from the store). MALJAN_ATTCK_MAX_AGE_DAYS is retired: the
migration that retired it read a whole number other than 7 once into the
setting, where the store held none (0 or a negative number, which the old
loader read as "never check", as 0, and so is a number past the setting's
bound), and a worker that still has it set warns at start and names the
setting. A stored value the settings model refuses, of this or any key, is
never replaced by a default where something runs: a job fails (it no longer
runs on the shipped defaults, for this or any refusal of its settings, nor when
the stored settings cannot be read; a database or connection error is named by
its class with the advice to run the job again once the database answers, any
other failure of the read by its class alone, and a refusal by the keys or
fields it names, else by its class, never by the value refused), an API read
answers 409 and the ATT&CK cron skips its hour, each naming the key, the reason
and the remedy (fix or reset it in the console). The settings page alone reads
past it so it can open: a banner above every settings page lists the refused
keys, and each row says its stored value is refused.
Under the setting, the console shows per domain the cached release, its date,
when the file was written, its age and the next automatic check, and the last
check and update with who ran them. Check now reads the release index and
downloads nothing: it says the cache is up to date with its versions, says
plainly when the index could not be read (never "up to date"), or, when a
newer release is out, opens a dialog naming the cached and the latest release
with its date and offering Update now or Wait for the scheduled update,
which names the date of the next automatic check. Update now downloads each
outdated domain's release from the platform's constant address, reads it back
whole (it parses, it is the release the index named, it carries techniques)
and only then renames it over the cached file, keeping the previous one as
<domain>-attack.json.bak; one domain that fails leaves the whole cache as it
was and says which and why. A swap that fails part way puts the domains it had
already replaced back from their backups, and names any it could not. An
update the worker gives up on (its job timeout, a shutdown) is told to stop
and has a deadline of its own, so it ends before the swap; the console then
says it stopped and that the table shows what the cache holds. Both need an
administrator and are audited.
Each download follows a redirect only to HTTPS on the same host and is bounded in size by four times the same domain's cached bundle, when that bundle is known to have been read whole; otherwise (a domain missing or truncated, the release index) by the response's declared length, and with none it is unbounded, which is logged as a warning. A download that passes its bound is stopped and logged. The download as a whole has a deadline, not only each read.
The cache is on the analysis worker's volume and that process is its only
writer: the API asks it through a queue of its own (arq:queue:attck), read by
a small worker inside the analysis worker's process beside its one analysis
slot, so a check is answered in seconds while an analysis runs. Every write of
a bundle, the update's and the loader's own refresh, takes the cache
directory's lock, and an update that finds it held says who holds it (another
update and who started it, or the loader's own refresh) rather than waiting.
An update record whose lock is free belongs to a process that stopped part
way, and the console shows it as an update that did not finish, with when it
started and who started it. An analysis reading a bundle during an update reads
the old file or the new one, never half of either; analyses already running
keep the catalogue they loaded, and the next one reads the new files. A whole
bundle is parsed in a short-lived child process, one at a time, so the half
gigabyte a parse costs leaves the analysis worker with the child. The child
gets a time limit from the file's size (30 s plus 2 s per megabyte) and is
killed past it, only PATH, PYTHONPATH and HOME of the environment, and
an answer file of its own in the cache directory. A parse that does not finish leaves that domain's
release "not known right now", which is neither unreadable nor outdated: a
check then fails with that reason and an update downloads nothing. The
temporary files a killed process leaves in the cache are removed when the
worker starts: those written under the update or state lock with both locks
held, and the facts and answer files only once they are older than the longest
parse limit, so two workers sharing the volume never remove each other's.
The top-level-domain list
data/tlds-alpha-by-domain.txt is IANA's root-zone list as published, with its
# Version line, and the string sweep reads hosts against it
(tools.strings.read_tld_list, loaded at import). Refresh it with
uv run python scripts/knowledge/refresh_iana_tlds.py, which downloads the
registry's file, checks it with the same reader and writes it unchanged; no
hand edits.
The API catalogue
data/api_behaviour_map_v1.json and data/api_attck_map_v1.json are written
by scripts/knowledge/build_api_capability_db.py (make prepare-api-db) from
the curated tables in its source; no hand edits. Both carry one block per
platform — windows for a PE's imports, linux for an ELF's dynamic symbols —
and a caller asks one at a time, because the two vocabularies share names
(connect, send, system). tools.knowledge.api_capability takes
platform and the triage pack passes the routed format's, so an ELF's symbols
are never given Win32 categories. A routed format with no block — a Mach-O, an
APK — is not asked at all: the catalogue answering about the wrong system is
worse than it saying nothing, and the import table is in the format entry
either way.
The Linux block is narrower on purpose: there is no registry, and persistence, keylogging, screen capture and credential access have no unambiguous libc vocabulary to author from.
Both blocks are measured rather than judged, and both read alike. A group
whose bare presence would label ordinary software is an informational
association carrying corroborated_by — the names that would give it weight —
instead of a tier the catalogue calls suspicious. One group per platform keeps
a label and each carries flags_with, so the label waits for the combination
that makes the group mean something: process_injection on Linux waits for a
name that reaches into another process, keylogging on Windows for the input
hook, raw-input device or whole-keyboard read that is the capture rather than
the key-state poll a game does every frame. A technique rule is kept only where
its combination is the act the technique describes.
One bar, and one trigger for a second look. The bar asks whether a rule carries information: one whose size-matched lift over a known-bad corpus is at or below 1.5 fires no more often on malware than on ordinary software, and it goes. The trigger is a rule above 4% of ordinary Windows software whose lift is under 3 — not a threshold that deletes it, because the measured rate is carried precisely so a common association can ship honestly and let the reader weigh it, but a sign that the rule should be argued on its own terms. Three were, and each went for a different reason: one was the sole row on no malware profile at all and so told a reader nothing another row did not; one was weak on its own numbers; one failed the test that decides whether a name belongs in a rule, applied to the whole rule, because the act its names describe is not the act the technique describes. A rule is deleted when one of those arguments carries, and not for its rate.
Every association carries the rate it was measured at, under measured:
seen_on_benign_percent is the share of a named benign corpus the association
fired on, seen_on_benign_files the count behind that share, and — on Windows —
held_out_malware_profiles how many profiles the combination was not chosen on
that it fires on, stated as 0 where it fires on none, because an absent count
and a count of zero read the same and mean opposite things. The rate reaches the
model in the api_capability answer, the report's import-technique table and
the console's cell, and a reader weighs it.
Read those numbers in the one direction they were measured in: they say how
often a rule fires on software that is not a sample, and none of them is a
probability that a given sample is benign. There is no technique-level ground
truth for either malware corpus, so what a malware number says is that a
combination separates binaries already known to be bad from binaries already
known to be good, never that a sample performs the technique; roughly three
malware samples in ten import nothing an import rule can see at all, because
they are packed, .NET, or resolve everything at runtime. An association that has
not been measured carries no measured block, and the surfaces say so rather
than printing a zero.
A technique id in either block is retargeted, or dropped and listed, against
the vendored catalogue's revoked_by when a release retires it. Every rule
carries name — the catalogue's name for the id — a rule label saying which
of two rules on one technique matched and what combination it keys on, and
ordinary_use: one sentence naming the software that is not a sample and
imports the same names, because a mechanism with ordinary users that does not
say so reads as an accusation. Almost every rule needs two or more names; one
keys on a single name that is the act itself, which the row states by setting
min_apis to one.
Rerun the measurement after an ATT&CK refresh, after adding a group or a rule, or against software that is not what the block was written against:
uv run python scripts/knowledge/measure_api_behaviour_block.py \
--fail-over 1 /usr/bin /usr/sbin /usr/lib/systemd
uv run python scripts/knowledge/measure_api_behaviour_block.py --platform windows \
--fail-over 1 --write-inventory corpus.jsonl.gz /srv/windows-corpus
uv run python scripts/knowledge/measure_api_behaviour_block.py --platform windows \
corpus.jsonl.gzA Linux run reads the dynamic symbol imports of the ELF files under those
directories; a Windows run reads PE import tables with pefile, the way
extractors/pe_extractor.py does, recording an ordinal-only import as
Ordinal_<n> so a binary that resolves everything by ordinal stays in the
denominator. Either way the corpus is deduplicated by content, so a suite that
ships the same runtime DLL in twenty packages counts once. It prints per group
and per rule how many binaries each appears on and labels, and names the ones
carrying a label or a technique row so a reader can judge whether that
population is the one the technique describes. --fail-over exits non-zero
when anything is above that share. --write-inventory saves what was read as
gzipped JSON lines, so the same corpus can be measured again after the files
are gone, and a Windows run takes such a file in place of a directory. It needs
pyelftools or pefile, reaches no network, and no test runs it over real
binaries: a test that read a host's software would answer differently on every
machine.
Rule corpora
The Sigma and YARA corpora belong to the analysis tool server, which is what
runs the scans. Point it at your own with two environment entries on that
server (Settings → Tool servers → analysis → env):
| Name | What it names | Default |
|---|---|---|
MALJAN_SIGMA_RULES_DIR | Directory of Sigma rule YAML, loaded recursively. | data/sigma_rules |
MALJAN_YARA_RULES_DIR | The YARA rule file the scan compiles. | data/yara_ttp_rules.yaml |
An unset value means the corpus the project ships; a path that does not exist
means an empty corpus and no matches, not a failure. analysis.sigma_rules_dir
was the previous name for the first of these — a stored override moves into the
server's env automatically on upgrade.
A rule in the YARA corpus fires when any of its patterns is in the sample's
bytes, and its confidence travels with the hit into every agent's pack. A
pattern must therefore be a fact about a sample rather than a word that
describes one: the persistence rules name key paths and not the API that writes
a value, and the packing rules name section names and packer banners and not
the words AES, packed or compress, which any program that speaks a
protocol carries.
A rule may also carry an all_of group beside its patterns: the patterns
fire one at a time, the group fires only whole. That is for a technique that
is a pair rather than a string — MiniDumpWriteDump is in a crash reporter
and lsass.exe is in every process lister, and only the two together are
credential dumping.
A rule that cannot be made that specific is written as a note: it omits
technique_id, and a rule with no technique may not carry a confidence
either. It fires, it says in its description what is in the file, and it
asserts nothing — which is what web_client_apis and file_enumeration_apis
are for. Importing an HTTP client is a fact; calling it a command-and-control
channel is a claim no substring can support.
How much of a tool answer a model sees
core.preprocessing.max_tool_output_chars is 0 by default, and 0 does not
mean "no cap" — it means the cap is worked out at the moment of each call from
the context window the served model was found to have. A positive value is an
explicit operator cap and behaves as this setting always did: that many
characters, on every answer, whatever the window.
The arithmetic lives in one function (maljan.llm.context_window) and reads:
the served window, less the tokens held back for the model's own reply (the
larger of core.llm.expert_max_tokens and core.llm.judge_max_tokens where an
operator set them, never more than a quarter of the window, and a quarter of it
where both are 0), less what the conversation already holds,
converted at 3 characters per token, times the eighth of what is left
that one answer may take. Three characters per token is measured rather than
assumed — a recorded conversation of about 114,000 characters was reported by
the server at 38,868 tokens — and is deliberately denser than the four the
token estimate uses for prose, because what this bounds is JSON and decompiled
C.
A cap never exceeds the room that is really left. Below 2,000 characters the share stops falling and the floor applies, but only while the room affords it; where it does not, the cap is what is left. Because each answer is measured against what is free at that moment, and because what it takes is charged as soon as it is handed out — a model turn may call several tools at once — the answers of one conversation add up to less than the room it started with, on every window the vendored table ships.
The share decides how large the first answer is and how quickly they shrink: at 32,768 tokens the first is 9,216 characters and about twelve clear the floor; at 131,072 the first is 46,080; at a million, 371,928. At the floor the answer meets the structural shortener exactly as any other does and carries the same notice naming the arguments that would narrow it.
An answer over the cap only because of its whitespace is not shortened. A tool
that indents its JSON spends a quarter or more of its characters on layout, and
a derived cap makes that gap the common case: a recorded elf_info was 8,628
characters indented and 5,577 compact against a cap of 8,486. Such an answer is
handed over whole, written without the whitespace — parseable, every value the
tool's, and with no notice, because nothing was left out — and the run summary
counts it as tool_output_compacted. Only a compact form that still does not
fit meets the shortener, and what the shortener then cuts is the compact form.
When the room runs out, the tool phase ends. Below about a thousand
characters an answer cannot survive its own notice, so nothing of it is handed
over. The model is told once, in one sentence, that the conversation has no
room left for a tool answer; the whole answer stays on the evidence ledger
under the call's id, and the run summary counts it as tool_output_no_room.
From there that agent's tool calls are not run — a server's time is not
spent on an answer with nowhere to go — and a call made anyway returns one
short line, filed on the ledger as the guard's answer with no tool run. The run-state block carries the same fact on every model turn,
replaced rather than appended, so it costs the same whether the loop reads it
once or forty times. The loop then ends on that same step, the way a repeating
loop does, whatever the model asks next: no_room on its budget record with
the reason, on the stage_ended_at_cap event and under the agent's caps in
run_summary.budget, and the forced synthesis turns what was gathered into the
answer. The graph's own step-limit sentence ("need more steps") is never shown
or handed on as an agent's words, and the platform writes none of its own in
their place: where a loop a cap ended leaves no answer and the salvage writes
none, the agent's answer is empty and its status no_claims, and why is on the
budget record and the stage_ended_at_cap event. The node does not run such an
analyst a second time over the same material, which would meet the same full
window.
Both notices come out of the tool budget — the window less the room kept back for the model's reply — and are withheld when they would not fit. So does the marker a character cut leaves behind, which is kept back out of the cap rather than appended after it, the way the shortener already reserves room for its own notice. That is what leaves the reply reserve whole: the forced synthesis above is this design's answer to a full conversation, and spending its room on saying that the room ran out would take it from the one thing left to do. Measured by driving the guardrail itself over every window the vendored table ships, at twenty, forty and sixty rounds, with a chunk preloaded and at fan-outs of thirty-two and a hundred and twenty-eight: the tool budget is never exceeded, and the whole reserve survives — 8,192 tokens on a 131,072-token window, 2,048 on 8,192, 1,024 on 4,096.
What is outside that guarantee is the model's own output: its tool requests and its prose are not the platform's to cap, and on a very small window they reach the window before the platform's text does.
What a conversation is measured at is what its next request will weigh, not the
messages alone. The definitions of the loop's tools go with every request, and
they are counted: a static analyst holding the default toolset — 35 tools
from the analysis, knowledge and VirusTotal servers — carries about 20,500
characters of them, 28% of a 32,768-token window's 73,728-character tool
budget, and a count that left them out said there was room until the server
refused. With the framing, about 46,000 characters (63%) are left for answers
and the model's own turns. On a small window, or with a large toolset, untick
the tools an agent does not need in each server's tick list in the Tools step:
that list is the only thing that narrows what a server sends an agent, and
every unticked tool is its definition's characters back on every request. Where the
server reported how many tokens the last request really took, that figure,
converted at the same three characters per token, plus what the conversation
gained since, is a floor under the measure, so content that tokenises worse
than three characters a token — pages of strings noise do — or the template
the server wraps each message in cannot hide room that is gone.
And where a server says the window is full anyway — llama.cpp's "context shift
is disabled" or "the request exceeds the available context size", an
OpenAI-compatible "maximum context length", Anthropic's "prompt is too long" —
after the analyst's loop has gathered at least one tool answer, that agent's
tool phase ends with no_room ("the model server reported its context window
full" on the record and the stage event), and the forced synthesis writes the
answer from what was gathered, rather than the agent failing and its work being
lost. Only an error the provider's SDK raised for the server's answer counts.
A server that names the reply cap is read by its numbers. vLLM words a full
conversation as "'max_tokens' … is too large: 8192. This model's maximum
context length is 32768 tokens and your request has 24808 input tokens": the
cap fits the window on its own and the prompt grew until the two together did
not, so that is a full window. A cap at least as large as the window, or one
named with no numbers to read, is a configuration fault no conversation could
avoid. The same full-window sentence on the first request, before anything was
gathered, means the framing alone does not fit; and an error that is not a
server's answer is the platform's own. Each of those faults still fails the
agent, because there is nothing to salvage and the failure is the true
statement. After the server has said the window is full, the final-answer
nudge is not sent — it would re-send the conversation the server just refused
— while after the platform's own budget ended the phase it still is, because
that conversation is inside the tool budget with the reply reserve whole.
The judge's tool loop is accounted the same way, under the judge's own name:
its conversation and its tool definitions are measured before every model
turn, with the server's reported count as a floor; its answers are capped from
its own room; its loop is streamed, ends on the step it runs out of room, and
ends with no_room on the same strict full-window answer once it has gathered
something. Its reasoning is then asked for once, with no tools, from what it
gathered, and mediation reads the verdict from that; a judge loop that fails
before gathering anything fails as before. Both salvages re-send the
conversation trimmed to the loop's own room less a margin: the window the
budget counts on less the reply reserve (never less than a quarter of the
window, whatever the generation cap), less an eighth of the window for what the
character measure gets wrong (the tool answers after the server's last count
were never counted), at the budget's characters a token, less the tool
definitions and the salvage's own question sent beside the turns. After the
server refused the conversation as over the window, the margin is a quarter of
the window. That window is the smaller of the declared
and the probed one, so a context_size left larger than the served window
cannot size a salvage close to the request the server just refused; how the
room was sized and how many messages were dropped are on the loop's budget
record. Each gets only what is left of its loop's time. An
analyst's salvage is also held to what that time can read and answer at the
model's measured rates, and is not sent when not even the task fits (see the
time cap in architecture).
The window itself is learned free of charge and without asking the operator anything. In order:
| Source | Where it comes from |
|---|---|
declared | core.llm.ollama.num_ctx, which the provider sends with every call, or core.llm.openai.context_size where an operator has set it. It does not short-circuit the probe: where a window was also probed, the smaller of the two wins, so a model that holds less than num_ctx asks for — and a context_size left behind by a server restarted smaller — cannot overflow the real window |
probed | llama.cpp GET /props (default_generation_settings.n_ctx, then n_ctx_per_seq); an OpenAI-compatible GET /v1/models (max_model_len for vLLM, context_length for OpenRouter); Ollama POST /api/show (model_info.<arch>.context_length, with a Modelfile num_ctx winning); Text Generation Inference GET /info (max_total_tokens) |
table | data/model_context_windows_v1.json, keyed by model-id family, for the vendor APIs that publish a window without serving it, and for a vendor API that serves it when that probe failed (DeepSeek's deepseek-flash, deepseek-v4-flash and deepseek-v4-pro at 1,048,576, documented as 1M on DeepSeek's Models & Pricing page; each row's page and the day it was read are under window_sources) |
fallback | nothing answered, and nothing is derived from it: one tool answer is capped at the documented 6,000 characters — exactly what this platform did before the window was learned at all — and every surface says the window is unknown |
No generation call is ever made — the probe reads metadata endpoints only, a
guard test drives both entry points through a transport that records every
request, and a probe that fails never fails a run and never blocks a settings
save. One question is asked per (provider, endpoint, model) rather than per
agent, both outcomes are remembered for fifteen minutes, and the whole plan
runs under one four-second wall clock.
A window an endpoint reports is untrusted input and is believed only up to ten million tokens. Past that the figure is refused rather than clamped, with the reason in words, because a proxy reporting its window in bytes produces a cap larger than any answer there will ever be — and a cap that large makes every answer fit, which switches the shortener, the summariser and the character cut off for the whole run.
A run whose agents sit on different models takes the smallest of their
windows, because one cap is handed to every tool server the job opens. The
models an agent falls back to count as models it sits on: a fallback with a
smaller window than the first model governs the cap, because the turn it
answers reads the same conversation. The agents counted are the ones whose
tool answers the cap sizes, each on the model it really calls: every analyst
and every agent they can ask through ask_<key> (a lead's specialists sit in
no stage); the mediator whenever the team has a debate stage and the function summariser
whenever it is on, each on its own llm.agents entry or else the global expert
model; and the judge on its own entry, or else the global judge model
(core.llm.<provider>.judge_model), never the expert one. A global judge model
with a smaller window than the analysts' therefore sizes the whole job. These
are the models the probe gate checks, but for the reporter: its narrative and
composer calls take no tool answer under this cap, and the composer sizes each
section's evidence on its own models' windows. The triage pack the narrative and
the composer read is cut at the smaller of the job's bound and the one the
reporter's own models' window derives.
Where to see what applied: the Settings page prints the detected window beside
the field, with the source word itself, and run_summary.truncation records
the window, its source, the characters-per-token figure and the smallest and
largest cap the run used. A run that landed on fallback is the one to act on
— set core.llm.openai.context_size to the window the server was started with,
and the cap is derived from then on.
A call waits as long as its answer takes at the model's pace
Two calls have an output budget of their own: the judge's verdict
(core.llm.judge_max_tokens, derived from the window by default, under the judge's
own time limit where an operator set one — none by default) and each composer section
(core.reporting.composer_section_max_tokens, under
core.reporting.composer_per_section_timeout, 120 s). A timeout chosen for a
fast model cuts a slow one off: at 3.8 tokens a second only about 2,280 of the
judge's tokens fit in 600 s.
The analysts' and the judge's output caps are derived too.
core.llm.expert_max_tokens and core.llm.judge_max_tokens ship at 0,
which derives each agent's cap in three cases (context_window.derived_reply): where the model's maximum
output is declared — by the probe's model list (max_output_tokens,
max_completion_tokens, OpenRouter's top_provider.max_completion_tokens) or
by a vendored max_output row, each carrying the vendor page it is documented
on (gpt-4o and gpt-4o-mini 16,384, gpt-4.1 32,768, DeepSeek's deepseek-flash,
its legacy name deepseek-v4-flash and deepseek-v4-pro 393,216, written 384K
on DeepSeek's Models & Pricing page)
— the smaller of that and a
quarter of the window; for a runtime we run — one that answered the window
probe as a runtime: llama.cpp /props, Ollama /api/show or TGI /info — a
quarter of the window, since no API limits its output (a loopback address alone
is not one: a gateway on localhost:4000 forwarding to a hosted API has that
API's limit); and for a hosted API that declares
no maximum, the documented fallback of 8,192 (never above a quarter of the
window), which a quarter of a hosted model's window is routinely past.
On a local 32,768-token window that is 8,192; on a local 131,072, 32,768; on
the shipped gpt-4o, 16,384.
A window nothing reported derives nothing: the documented fallback of 8,192
applies and the sentence says the window is unknown. The reply reserve follows
the same rule, bounded by an operator's generation cap where one is set; so
does the report stage, below. A value above 0 is
the operator's and is used as set; a stored setting keeps its value. Each
agent's cap is derived over the models its own model is built from: its
llm.agents entry and that entry's fallbacks, the smallest of them, or else
the global model of its role. The judge's cap follows llm.agents.judge when
one is set, and the global judge model only without one. Each
derivation is logged and recorded in run_summary.generation.output_caps
({agent: {tokens, derivation}}), and the judge's is printed beside the
verdict wait. 0 no longer means unbounded.
The report stage derives its budget as the judge does. Each composer section and the narrative round is one reply written into the window its prompt already fills, so for each model of the reporter's list, in order:
- the operator's value:
composer_section_max_tokensabove 0 for a section (plus the reporter's own budget for reasoning, below), otherwisellm.judge_max_tokensabove 0 — the reporter runs on the judge role and has always been built with the judge's cap;llm.expert_max_tokensis the analysts' and no longer reaches the report stage; - else the derivation above (
context_window.derived_reply), the one every role's cap follows: the smaller of a quarter of the window and the model's declared maximum output, a quarter of the window for a runtime we run, the documented 8,192 for a hosted API that declares nothing.
It is never more than the model's maximum — its declared maximum output, or
the window it serves when it declares none — and a value held at it says so;
the reasoning room is inside that bound too. Sized at the declared maximum
alone, a model whose window is at or below its declared maximum output (for
example a context_size set to 200,000 under DeepSeek's 393,216) was given the
whole window as its answer, and
the narrative was recorded as degraded and sections as not written. With
nothing set, a section on DeepSeek's deepseek-flash with a 1,048,576-token
window may write 262,144 tokens and the remaining 786,432 hold its evidence;
on Claude Haiku 5.5's 1,000,000-token window, 128,000, its declared maximum,
leaving 872,000; on a local 65,536-token window, 16,384.
A section's evidence is sized against the window only when the window was
learned. The 8,192-token fallback a failed probe leaves is a number printed
beside the word fallback, not a fact: the section's claims and tool answers
are then shown whole, and its budget is the operator's value, else the
documented 8,192, never above the model's declared maximum. Where the window is
known, the evidence room is the window less the budget, never below zero,
and each call is held to what the window leaves after that call's own prompt
(at three characters a token) when its budget would not fit beside it — a
hosted API refuses a request whose prompt and max_tokens pass the window.
The call's cap is lowered for that call alone, under the field each server
reads — max_completion_tokens for OpenAI, max_tokens for DeepSeek and
Anthropic, max_tokens and n_predict in llama.cpp's request extras
(whether or not the model was built with a cap: the models the container
builds for a run carry a cap derived from the window and already sent these
keys, and a model built without one — the function summarizer's, or the
provider called directly — is now held to the call's cap too),
max_output_tokens for Gemini — and the worker log says so. An Ollama model
keeps the cap it was built with, since its client takes no per-call cap; a
model the llama.cpp self-heal rebuilt without the extras is sent the cap only
as max_completion_tokens, which ik_llama.cpp does not read. A prompt larger
than the whole window is recorded as a degradation: Ollama fits such a prompt
to num_ctx by cutting it from the front, which can drop the system prompt
and the round's rules, and says nothing. A budget that fills the
window — a gateway that declares its window as its maximum output, a local
llm.judge_max_tokens at or past the window — so leaves the section no room
for claims and tool answers. A section whose facts do not fit what the window
leaves records the degradation "the section's prompt without its claims and
tool answers … exceeds the … its model's context window leaves after the
reply", its claims and tool answers are shown as the no-room sentence, and
the section is still asked.
Every list a section's model writes — flow steps, configuration items, commands, flags, C2 channels, citations — is kept whole; no count cuts it.
The narrative round (the executive summary, key findings and recommendations)
takes the same budget, is held per call the same way, and waits the way a
composer section does: 600 s until the reporter's pace is measured, then the
time its budget takes at that pace for each call it may make (its answer, the
one retry, and the structured attempt where the endpoint supports one).
Appendix B prints it as narrative:round. The worker log prints one line per section ("ReportComposer: section
'…' output budget: …"), and the run summary prints the derivation beside the
section's wait ("Output budget of composer:section: 262144 tokens — the
smallest of a quarter (262144) of the model's 1048576-token context window
(probed), the model's declared maximum output of 393216"). A fixed budget dropped a live report's
section when the model's answer outgrew it. The run summary keeps one
composer:section row, the wait last sized: written one after another, that
is the last section's, sized from the pace measured over the sections before
it; written at once, every section is sized before any has answered, so the
row is the last section's wait at the pace measured before the sections began
(or a later re-sizing of a wait still running). The row describes the wait
that section got; it is read off wall-clock rates and was never the same
between two runs, and nothing in the report body reads it.
So each of those calls waits max(configured, derived). Where the model's reading rate is
measured and the call gives its prompt size (a composer section, the verdict),
derived = (prompt_tokens / reading rate + max_tokens / generation rate) × 1.5;
otherwise derived = max_tokens / rate × 1.5 with the rate that includes the
prompt read. Appendix B prints which one each timeout took. The rates are
the model's own for this job, read off every answer it has already given
without a token of its own: Ollama's eval_count over eval_duration,
llama.cpp's timings.predicted_n over predicted_ms (the openai provider
carries the timings object the OpenAI-compatible client would drop into each
answer), and on an endpoint that reports neither the answer's output token
count over the call's wall clock (which includes reading the prompt, so that
rate is lower than the server's and the wait longer). The reading rate is
Ollama's prompt_eval_count over prompt_eval_duration or llama.cpp's
timings.prompt_n over prompt_ms. The margin, 1.5, covers the spread between
turns, and the prompt read where it is not timed on its own.
The HTTP request carrying a call is sized the same way. Every provider's
client is built with a 1,800 s request timeout
(PROVIDER_REQUEST_TIMEOUT_SECONDS; Gemini's was a fixed 90 s and is now the
same). httpx reads it as the longest silence it waits through, not as a
deadline for the whole answer, so it ended an answer only on a server that
sends nothing until it has finished — a non-streaming llama.cpp server, whose
answers it held to about 1,800 s of generation (a llama.cpp server's
answer is now read as a stream; see Loops have no default limit below). A
streamed answer, and DeepSeek's, which sends keep-alive lines while it
generates, were not ended by it. Once the model's pace is measured, an
OpenAI-compatible (chat
completions or Responses API), Anthropic or Gemini request whose output cap
takes longer at that pace than its client allows carries its own timeout, the
SDK's per-request option: that time, by the same arithmetic and margin, its
prompt counted at three characters a token. Any other request keeps its
client's timeout. So no derived wait is held under 1,800 s any more: at 3.8
tokens a second the judge's 8,192 tokens need 8,192 / 3.8 × 1.5 ≈ 3,234 s and
get it, where 600 s allowed about 2,280 of them, and a 393,216-token section
at 40 tokens a second gets 393,216 / 40 × 1.5 ≈ 14,746 s. The Ollama client
streams every answer, so its 1,800 s bounds the silence between two pieces of
an answer. The function summariser's wait follows its request's.
These waits have no upper bound of their own. A server that stays connected
but stops generating, or a call left running on a single-slot llama.cpp
server after its loop gave up on it, is held for the whole derived time — at
3.8 tokens a second 32,768 tokens take about 12,900 s — where 1,800 s used to
release it. The last resort is the job timeout, core.job_timeout, which is
empty — no limit — unless you set one (see "The job timeout" below). A composer section is its answer and the one retry its validation
allows, so its wait holds two calls of its output cap: at 3.8 tokens a second
a budget of 8,192 needs 2 × 8,192 / 3.8 × 1.5 ≈ 6,467 s; an operator's budget
of 900 with the reporter's disable_thinking on needs 2 × 900 / 3.8 × 1.5 ≈
710 s.
A fast model's derived time falls under its
configured one, which then stands. Until a model has answered once, and for a
call with no output budget, the configured value stands. Rates are kept per
model and per server, so one tag served by a local and a remote Ollama is two
paces. The verdict call starts its model list on its sized wait. The report
stage starts the reporter's list once; each section then measures the list's
turn deadline against its own wait, with the job's llm.fallback_turn_share,
without putting the list back on its first model — a model that failed as a
provider in one section is not waited out again in the next, and a switch
holds for the rest of the report stage.
The section budget is also the section's real cap, and a model's reasoning
counts against it: Ollama's num_predict and llama.cpp's n_predict include
the thinking channel. The budget at 0 already is the model's whole reply
room. With an operator's own composer_section_max_tokens, where the
reporter's provider has been told to keep reasoning out
(llm.ollama.disable_thinking or llm.openai.disable_thinking), the
composer's model is capped at that value alone; where it has not, the cap is
that value plus the reporter's own output cap (judge_max_tokens, else
derived) for the reasoning, and the sum is held at the model's maximum. Each
model of the reporter's list is capped by its own provider's switch, and the
wait is sized from the largest cap: the platform
cannot tell a reasoning tag from its name, and sending think: false to a
model that does not reason is an error on Ollama. A section's evidence room is
what every model of the list leaves, each by its own window less its own cap;
where one model's cap does not fit beside a call's prompt, that call is held
to the least any model of the list may write, never past a model's own cap,
and the log names the limit that held it (a model's own output cap, or what its
window leaves). A model whose whole window the prompt already passes cannot
answer it, is recorded as such, and does not hold the others' call.
A section the cap cut is
recorded as cut at that cap, not as a schema failure. On Ollama every output
cap — this one, judge_max_tokens, expert_max_tokens — now reaches the
server as num_predict, which ChatOllama otherwise drops, so a thinking
model's reasoning counts against the judge's and the analysts' caps too;
disable_thinking, or a larger cap, is the remedy. The verdict call records
whether it reached judge_max_tokens, Ollama's done_reason: "length"
included. run_summary.generation records
each model's rate, tokens, seconds, calls and source, its prompt reading rate
where the server reports one (prompt_tokens_per_second, prompt_tokens,
prompt_seconds, prompt_sources), and for each sized call
the configured value, the budget, the rate, the derived and the applied
seconds; the report's Run Summary prints the same numbers.
Loops have no default limit
No agent loop has a step or time limit unless you set one.
core.react_agent_max_steps and core.react_agent_timeout are empty by
default, the two deprecated core.react_agent_*_overrides maps ship empty, no
built-in agent definition carries max_steps or timeout_seconds, and an
ask's core.agents.delegation_steps / delegation_timeout_seconds are empty
too. A number you set — on an agent's card, in a map, or deployment-wide — is
kept to exactly as before. The judge's tool loop reads its budget the same way
every agent's loop does (loop_limits("judge")): the judge entries of the
maps, then the deployment's values.
A loop with no limit ends when its model answers, or at one of the stops that
are not a count: the repeat guard (a stall limit: eight refused calls, each
identical in tool and arguments to one already answered; a served retry and a
call that differs in any argument or page never count), the conversation's room (a tool answer that no longer fits the window),
and the job's spend ceiling below. The job timeout below, when you set one, is
the last resort. A loop with no time limit has no clock of its own. Every model request still has
a whole-call deadline Malspark enforces itself: the request timeout sized in the
section above (its output cap at the model's measured pace, prompt read
included), or the client's own timeout — PROVIDER_REQUEST_TIMEOUT_SECONDS,
1,800 s — where nothing is measured. The client's timeout stays as a second
guard, on silence: httpx reads it as the longest gap it waits through, so a
server that trickles keep-alive bytes or answers slowly but steadily was held
by nothing else. A model list gives no turn deadline in such a loop; a
stalled model is ended at its whole-call deadline, which the list reads as a
provider failure and moves on from. The run-state block says budget remaining: no step limit, no time limit rather than a number, and the run
summary's budget rows carry max_steps / timeout_s as null.
Where nothing is measured, the 1,800 s bounds only the silence before the first generated piece of an answer. A generated piece is a streamed chunk that carries text, reasoning or a tool call; the opening chunk that names only the role, and a closing chunk with only a finish reason or usage, are not. Once two pieces arrive, the call has a pace of its own, from its first piece to its last; the time before the first piece is the prompt read and is never counted as generating. Its deadline becomes the time to its first piece and its output cap at that pace, times 1.5, wherever that is longer than the deadline it started with. A call with no output cap is sized the same way from the room its model's window leaves after the prompt, where the window is declared in the settings or reported by the server. A call that keeps producing is therefore never cut by the unmeasured value, and one that stops is cut when the deadline its pace set passes. Where a producing call has nothing to size from (one piece so far, or no output cap and no known window), it is held only to the silence after its last piece: the provider's request timeout. A piece is one chunk, and chunks and generated units differ by a few in either direction: a unit a server holds back sends no chunk, and a server may send two chunks for one unit; the margin covers the difference.
The pace of a call that is cut, fails or is ended from outside is recorded for its model as a generation rate, so a retry is sized from it. A call with fewer than two pieces records nothing, and the next call keeps the rule it had. On a streamed answer (and Ollama's, which its client streams), the connection's own read timeout after pieces have arrived is the same deadline, the silence after the last piece, and ends the call as one (a failed turn the loop salvages from, not a dropped connection); before the first piece it is raised as it always was. The deadline message says which rule applied — the model's measured pace, the pace measured in this call, silence before the first generated piece, or silence after the last — and its numbers.
Pieces are seen only where the answer is streamed. A llama.cpp server's answer
is read as a stream for this reason, with its usage on the closing chunk and
langchain-openai's own 120 s gap limit off. The chunks are joined, as they
arrive and holding none of them, into the
answer the server would have sent whole: the same text, finish reason,
timings and usage (the last the stream sent, wherever it came, since
ik_llama.cpp repeats a running total on every chunk), and tool calls read as
strictly as a whole answer's, so a call cut mid-argument stays an invalid
call. A chunk whose content is not a string is joined by langchain's own rule,
with the text before it and after it kept in its place. The join works on its
own copy of each chunk, so the chunks the run's callbacks were handed (a
LangSmith trace keeps them) stay as the server sent them. A server error sent inside the stream is raised as the status error it
is on a whole answer, and a connection that drops while the answer streams as
the APIConnectionError it is on a whole answer. Ollama's client streams every
answer. DeepSeek (compat: deepseek) is read as a stream too, its reasoning
kept and sent back, so its calls take the streamed deadline: the pace of the
call's own pieces, silence before the first piece and after the last. Another
hosted API's answer is read whole, as before, and its first piece is its whole
answer, so there the provider's timeout still bounds the call until the model's
pace is measured.
The other fixed limits were decided one by one: core.negotiation.max_iterations
stays an explicit setting (5), the runaway stop on a negotiation that never
converges; core.react_agent_tool_call_budget only ever logs a warning;
capa and FLOSS on the analysis server have no wall clock of their own and run
for as long as their caller asks (the triage pack passes
core.static.capa.timeout_seconds and what is left of
core.triage.budget_seconds), and their manifest declares none and marks them
long_running. A model's call of either is waited for with no client deadline
unless you set core.mcp.breaker.call_timeout_seconds; every other call waits
the call_timeout_seconds its server's block declares, or, where it declares
none (every built-in), core.mcp.breaker.default_call_timeout_seconds; a timeout or a
cancellation of such a call is not counted by the breaker; a call the client
gives up on, or a job that ends, is cancelled at the server, which kills the
tool's child process with its process group; and a second call of the same
run (the same tool, sample and arguments) joins the one already going instead
of starting another child;
ANSWER_SHARE, the share of the window one tool answer is sized from, stays a
documented derivation constant; the Ghidra sink pre-pass — run for every agent
whose own static provider is Ghidra, the static analyst, a clone of it and a
generic agent given Ghidra's tools alike, as the provider's own open and call
graph — waits one call's budget per request (Ghidra's block's
call_timeout_seconds where declared, else
core.mcp.breaker.default_call_timeout_seconds, or
core.mcp.breaker.call_timeout_seconds where set), or as long as Ghidra takes
when that budget is zero.
The spend ceiling. core.llm.max_spend_usd_per_job is the most one job may
spend on its models, in US dollars. It has no default: while it is empty a job
has no spend ceiling, nothing is priced against one, and no call is held or
refused for spend. Set it on a paid provider. It is a hard
bound against the platform's own measure of a prompt — its characters over
three, the measure the window accounting uses — so nothing is sent that could
take the job past it by that measure. A prompt that tokenises denser than that
(long runs of hex or base64) costs more input than was reserved for it, and the
job can pass the ceiling by that difference; the charged cost is what is
settled either way.
What a call costs is what it was charged. Where the provider reports the
call's cost in its answer (an OpenRouter-style cost), that figure is used and
no price is read. Otherwise the call's provider-reported usage — the input
tokens read from the prompt cache, the other input tokens and the output
tokens, reasoning included — is priced at the rates in force when its request
was sent (every model client stamps its answers with that time), from
core.llm.model_prices first, keyed by the model name the provider serves:
{"deepseek-v4-pro": {"input_usd_per_mtok": 0.66,
"cached_input_usd_per_mtok": 0.022,
"output_usd_per_mtok": 1.98,
"source": "our contract",
"windows": [{"utc_from": "01:00", "utc_to": "04:00",
"days": ["mon", "tue", "wed", "thu", "fri"],
"input_usd_per_mtok": 1.32,
"cached_input_usd_per_mtok": 0.044,
"output_usd_per_mtok": 3.96,
"source": "our contract, peak hours"}]}}then a prices row of the vendored model table
(data/model_context_windows_v1.json), which carries DeepSeek's documented
prices for deepseek-flash and deepseek-v4-pro with the page they are
documented on — data, not a limit. A row's own figures are its price outside
every window; a window is a span of the day in UTC (its start in it, its end
not; a window whose end is before its start runs past midnight), on the
weekdays it names (none is every day), with its own figures and source. The
vendored rows carry DeepSeek's peak hours, 01:00–04:00 and 06:00–10:00 UTC
Monday to Friday, at twice the off-peak rate of every other hour. DeepSeek
also takes Chinese public holidays out of its peak hours, which no window
names, so a call in a peak window on such a day is counted at the peak rate,
above what it cost; its page does not say whether a request is timed at its
start or its end, and the platform times it at its start. A price key keeps a
model's tag (qwen3:8b and qwen3:32b are two models); the base name answers
only where no row names the tag. run_summary.spend.prices_from names, per
model, every rate its calls were priced at, or provider-reported. A call
whose provider reported no usage is counted, and the spend is then said as "at
least X; N calls reported no usage"; a model that never reports usage cannot
trip the ceiling, and the log says so once. A model with no price and no
reported cost is named once in the log and in run_summary.spend
(unpriced_models), and its calls are not counted: the figure compared is what
the job spent at least. Nothing is guessed.
Before each call — every model call the platform makes, each tool-loop turn,
revision, mediation, verdict, report section and function summary — its output
cap is held to what the spend it may use pays for at its model's output price,
after its prompt priced as uncached input, both at the highest rate in force
between now and the call's own deadline — the deadline its caller gives it,
else the whole-call deadline its request is sent with (so a call sent across a
window's edge never settles above its reservation). The call is refused only when that is
below the smallest answer it can give, measured per group: for a tool-loop turn
the largest turn (reasoning and answer together) this job has measured of that
model, for any other call the largest single-shot or verdict/report answer
measured of it, and with none of its group measured the call's own configured
output cap — there is no fixed floor; a report call's is the answer planned
for it (below). The judge's mediation turns are held as an analyst's turns
are: the model is bound to its tools before the loop starts and each turn's
held cap is set on that binding. A call that cannot be handed a cap of its own
(a model that takes no output cap per call, the structured technique question
and mediation extraction) is made only at its whole cap, and the refusal says
so. A call the ceiling refuses is not sent: a revision leaves
the analyst's answer in force, the mediator's fast path leaves no reasoning
(no agreement), the extraction falls back to reading the text, a summary keeps
the raw text, a report section is recorded as not written, and a verdict the
ceiling refuses takes the judge node's fallback for a verdict call that fails:
a conservative Suspicious verdict written by the pipeline, not a model, with
the run marked degraded and the report saying why. Each hold and each refusal is logged with its numbers and
listed in run_summary.spend.held_calls. Every admitted call reserves its
worst case — its prompt and its held cap — until it returns and is settled at
what it was charged, so calls running at the same time never spend the same
remainder. A tool loop's turn also keeps room for the loop's closing answer:
it is sent only when what is left after it still pays for the smallest answer.
The reserve for the verdict and the report. A job plans its verdict call and its report calls (one per section the composer writes, and the narrative round) and keeps aside, for each of them, what its admission will demand: its prompt as uncached input and the answer planned for it, at the rates in force. The reserve is therefore never below what the tail needs to be made, and it is sized from this job's own calls rather than from the window, with no fixed fraction:
- a planned call's prompt is the prompt it will be sent: the largest prompt of its own kind once one was sent; before that the largest single-shot prompt this job has sent (a revision's prompt carries the same reports), else the largest opening prompt of a conversation (a tool loop's first turn); each bounded by what the window accounting allows that kind of call (its model's window less its report-stage output budget). The allowance alone is used only while no prompt has been sent at all. A tool loop's conversation is never used;
- the verdict's answer is what its admission demands: its configured cap until a single-shot answer is measured, then the largest such answer;
- a report call's answer is the mean answer of the report calls measured of its model once one returned, and before that the mean answer of the model's other single-shot calls (the verdict's is left out, so the verdict does not move the report's plan), else its largest tool-loop turn. It is also the smallest answer a report call's admission demands, so what is kept for a report call is what makes it. With no answer measured the reserve is not sized;
- each kind also keeps its validation retries (the correction turn a
validation loop sends when an answer breaks a rule): one per planned call
until a call of the kind is made, then
(retries + 1) / (calls + 1)of this job's own count for the kind, so a call whose retry has not been asked yet is never planned without one. A retry is planned with the answer it corrects in its prompt. The rows say how many retries are kept and from what (retries,retries_from,retry_prompt_tokens).
Before anything is measured the reserve is at its largest. Until a
single-shot answer has been measured, the verdict is kept at its whole
configured output cap, and until a verdict call has been made, with one retry
at that cap as well: the verdict's cap is kept twice. On deepseek-flash
(a 393,216-token cap) that is about 0.47 USD at the off-peak rate and about
0.94 USD at the peak rate, before any report call is counted. Early tool-loop
turns spend only above it, so under a low ceiling at peak rates the first
turns are held short or refused until the first single-shot answer (a
revision, a chunk) is measured; the reserve then shrinks to what this job's
calls measure. A ceiling well above twice the verdict's cap at peak rates, or
an operator-set llm.judge_max_tokens that caps the verdict, avoids it. With
no ceiling none of this applies.
Each row of run_summary.spend.reserve also states its expected charge
(expected_usd): the first call of a kind with its prompt uncached, and the
ones after it, which share its prefix, at this job's cache-hit share for the
model — cached input tokens over input tokens, measured only over calls that
were not the first of their conversation (a tool loop's turns after its
first), since a conversation's first call has nothing cached to read — and at
the model's cached rate while no such share is measured. The reserve does not
count on that share, because each call's admission prices its prompt uncached.
Tool-loop turns, revisions, negotiation rounds, asks and summaries spend only
above the whole reserve; a verdict or report call spends above the reserve of
the planned calls after it, so the verdict cannot take the report's share and
an unplanned retry spends only what is left above the plan.
run_summary.spend.reserve shows the derivation, row by row, as it stood when
the summary was written.
A refusal is not exhaustion. A call that does not fit only because other
calls in flight hold their worst case — a parallel analyst's turn held to what
was left — waits for them to settle, since they usually settle far below what
they reserved, for as long as its own deadline allows; a job cancelled while a
call waits ends that call as cancelled, not as a refusal. A call the ceiling
refuses is not sent, its refusal is logged with its numbers and counted
(run_summary.spend.refused_calls), and its caller takes its salvage path: a
tool loop whose next turn is refused ends its tool phase and its agent writes
its answer from what it gathered, and the budget record says which call was
refused. The job goes on while a call of any kind made since the latest stage
began — each at the smallest prompt it was sent with, with its own cap — would
still be admitted: a mediation turn refused at its whole cap does not stop the
revisions after it. A kind the stage will make and has not made yet is asked
about on the model of the call this job made last. A refusal made while other
calls are in flight never exhausts the spend; the question is asked again once none is. The spend is
exhausted when the ceiling is reached, or when, with no call in flight, a
refusal leaves nothing else that fits. From then on every
gate reads it, run_summary.spend says exhausted with when and why, no
further negotiation round, chunk or tool loop is started, an ask is refused,
every running tool loop ends its tool phase and its agent writes its answer
from what it gathered, and only the verdict, the report and a loop's closing
answer are made, each where it fits. A revision that is not made leaves the
analyst's answer in force standing. A report section whose call is refused is
recorded as not written, with the refusal's numbers. The run summary's
degradation reasons say the ceiling ended the tool phases.
An agent's own ceiling. An llm.agents.<key> entry may set
max_spend_usd, what that agent's priced calls may cost in US dollars, and
max_unpriced_tokens, the input and output tokens its calls of a model with
no known price (a local or unpriced one) may use. Neither has a default: an
entry that sets none has none, and its models are built exactly as before. An
entry may hold only its ceilings, with no provider or model: the agent then
runs on the model it runs on with no entry, built exactly as before, and the
entry only bounds it. Calls are priced as the job's ceiling prices them, and
every model of the entry's list counts toward the same agent, fallbacks
included, as does a failed attempt its provider bills; a call that reported no
usage counts at the stated estimate it carried, and the reason says how many
did. The mediator's ceiling is its own, apart from the judge's, although the
token block and the call feed file the mediator's calls under judge.
The ceiling is checked before each call, not reserved: the call that crosses it
runs and its answer is kept, and so does any call of the agent already in
flight beside it (the reporter's sections are written one after another while
the reporter sets a ceiling, so none is). From then on the agent makes no
further model call — not a tool-loop turn, a nudge, a salvage or a closing
answer; the verdict is not exempt. An analyst stopped mid-loop keeps the latest
of its own turns that is a report, as written, or has no answer and is named as
stopped before writing one, with the evidence it gathered kept. A judge stopped
before its verdict gets this pipeline's fallback verdict, as a timed-out one
does (verdict.agent_ceiling); a mediator stopped mid-round leaves the round
unmediated with the ceiling's sentence as its note. The job's spend is not
exhausted by any of it, and no other agent is affected. The call feed records
the stop once (call: "agent ceiling", with the agent and the ceiling),
run_summary.spend.agent_ceilings gives each agent's ceilings, what it used
and whether it stopped (one line each in the summary's spend section), and a
degradation reason names the agent, the ceiling and what it had spent. The
agent editor offers both fields on every entry.
A report section's call is held to the smallest of three limits, and the log line for the call names the one that applied: the section's output budget (and how it was derived), what the model's window leaves after the prompt, or the spend ceiling's hold. A section whose answer is cut at that limit with no text written — a reasoning model that spent the whole allowance thinking — is asked again only when the second call would have more room; otherwise it is recorded as not written, with the limit and where it came from. The platform does not lower a model's reasoning effort to make an answer fit.
What the judge and the pack are shown. The judge's verdict prompt and its
technique question show every analyst's report, the evidence summary (every
technique and every source), the negotiation history and each remembered
case's summary whole when the judge's window, less its output cap, holds them.
When it does not, the largest parts are shortened first to one shared width,
each ending in …, the prompt says which parts and to what width, and the run
records it as a degradation reason. The triage pack is rendered whole when it
fits its room (the upstream bound, core.reporting.upstream_findings_max_chars,
derived from the window by default); when it does not, the detail every line
shows — names listed, characters of a text, decoded strings and detection
labels — is derived from that room, the most at which the pack fits, and each
line says what it left out and that the rest is a tool call away.
An analyst's input. core.max_token_limit (was 128,000 tokens) is
empty by default: an analyst's input text may take the room its window holds
before the reply, less the system prompt, the pack and the run-state block
around it. With no window learned the input goes whole; a number you set wins.
Input over the room is shortened as a document — a JSON input keeps every key
and loses elements off its largest lists, text keeps its head and ends in
… — the text the model reads begins with a note saying so, and the run
records a degradation reason ("The static analyst's input was shortened: …").
The function summariser's prompts are held to the same window room and say
when they were shortened; the PE loader's markdown lists every import, export
and string, and the generic MCP provider's prompt names every tool.
An analyst's chunks. An input is split into chunks at that same room —
which also keeps the tool definitions out of it and one tool answer's share
(an eighth) free — so a chunk reaches the model whole:
core.chunking.max_tokens_per_chunk (was 20,000 tokens) is empty by default.
Every chunk is a full tool loop, run after the one before it, so an input that
fits is one chunk: the agent's sources (the sample context and a sandbox slice)
are joined into one when they fit the room together, and the agent runs one
loop over all of them, with the prompt a single-chunk input has always had;
where one source is split, the whole sources next to each other that fit
together are still joined, leaving room for the header and the count of earlier
calls a later chunk carries. The pieces of one split source never fit one chunk
together, so they are never joined. An input that does not fit is chunked at
the room. The sample path and the upstream findings, which go into the first
chunk afterwards, are kept out of that first chunk only; every later chunk is
the whole room, so a large upstream block adds at most one chunk. In a list of
more than one chunk each is sent behind a one-line header, which every chunk
leaves room for, and a later chunk behind the calls the earlier ones made.
The chunks run one after another, so a later chunk is sized when it is sent:
the loops are those the shortest form of the earlier calls, their count, leaves
room for, with what is left of the source spread evenly over them and cut at a
line break only where that adds no loop (a line longer than the room is cut
inside the line), so every prompt stays within the room and a list never adds
a loop. The slack each chunk's share leaves goes to the most informative form
that fits it: the full list; the same calls with their arguments but not what
they returned, calls with identical arguments sharing a line with their
entries as ranges (decompile_function("0x3c7c") → ev_0012–ev_0015), arguments
longer than the list could ever hold said as their size and the call that holds
them; the count with each tool's calls and the span of their entries
(38 tool call(s) (decompile_function ×34, list_strings ×4)); or the count
alone. Such a chunk's header says more may follow, since a carry can add a
chunk. An identical call is still answered from its entry. The overlap between
chunks is held below three quarters of a chunk, and the hold is logged. A JSON
head that fits the room on its own stays whole when it is the agent's only
source (cut, it would lose the sample path; the single chunk's prompt shortens
it as a document and says so) and is split like any other when another source
follows. Where the block alone fills the room, the first chunk is cut at the
room like the rest. A source of nothing but whitespace adds no chunk, so no
loop runs over nothing. A revision whose agent cannot be built to measure its
room chunks at the size its first analysis measured. A number you set wins, for
splitting and joining: chunks of max_tokens_per_chunk go as they are set,
with the full list of earlier calls. With no window learned nothing is derived:
a chunk is the 20,000 tokens the setting shipped with, and sources are joined
only within it, as a tool answer keeps its old 6,000 characters; a run whose
input that size split says so in its run summary and its log, with the setting
that would size it from the window. With no room measured the prompts are
built as before; two chunk boundaries differ on purpose: whole sources beside a
split one are joined, and a packed bin counts the overlap it carries.
What the report leaves out, it says. No table cell of the report's body
is cut: a long value is printed whole in its row and the page wraps it (the
HTML report breaks a cell wherever it must, and the PDF lays its tables out
fixed, so a kilobyte-long cell costs no more to print than a short one). The markdown report keeps its
other layout bounds — Appendix A cuts a long one-line value, a long list shows
its first rows — and every one says what it left out and where the whole is:
a cut value ends in … and the section says where it is whole; a list shown in part ends with "N more …
not shown here; the JSON report carries every one." The evidence sections do
the same ("N more rows not shown here; the evidence endpoint carries every one
under ev_…", a long text "Cut here at … characters"), a figure's legend counts
what it did not draw, a drafted YARA or Suricata rule says in a comment how
many published indicators it does not match on, and the STIX export carries
every process root.
The job timeout
core.job_timeout (Agents → Limits, "Seconds per job") is how long one
analysis job may run before the worker stops it. It is empty by default, and
empty is no limit: a job ends when its pipeline does, when you cancel it, or
when its worker shuts down. arq imposes no deadline of its own on the analysis
worker.
When a change takes effect: the value is read from the job's own settings when the job starts. A value you save applies to every job started after it, with no worker restart; a job already running keeps the value it started with.
What it measures: the time since the worker started the job — its database reads, the sample download and the pipeline all count — on one monotonic clock, the same one the stop note's "N s into the run" is read from. The worker checks it on the same 15-second poll it reads the cancel flag on, which runs while the pipeline runs, so a job is stopped within one poll of its limit (at the pipeline's first poll if setup alone took longer). Storing a finished run's report after the pipeline returns is not interrupted.
A job that is stopped — by the job timeout, by your cancel, or by its worker shutting down — keeps what it produced:
- The run summary, as of the stop: the judge's when the judge ran, otherwise one built from the run's state, with the token usage per agent and model, the spend, the validation record and the stage rollup read at the stop, so the report stage's calls after the judge are counted.
- A partial report, served by the usual report endpoints: the report the
report node built, or the one it was building (the deterministic report, the
narrative if that round ended, and every composer section already written),
or the deterministic report built from the state when the run stopped before
its report stage. A stage that did not run is not filled in: no summary is
written and the report says why, and with no judge no verdict is reached —
the stored verdict is
Unknownand the report says its verdict field is a placeholder. The report'sincomplete_reasonand its degradation reasons say why the run stopped, how far into it and where. - The job status:
failedfor the job timeout and a worker shutdown,cancelledfor your cancel, witherror_messagegiving the same reason.
On a worker shutdown, keeping the stopped run is held to a 10 s grace (it
measured about 0.1 s of work on a 342-entry ledger and a 2.1 MB report, plus
one commit). The compose file gives the analysis worker a stop_grace_period
of 110 s, the sum of its shutdown bounds (pipeline stop 10 s, keeping the
stopped run 10 s, teardown 60 s, two connections 10 s each, exit guard 10 s),
so docker stop does not kill it before the stop is recorded. A deployment
that runs the worker some other way should give it the same grace.
What a node that was still running had gathered is not in the state yet and is not in the partial report; its tool calls and messages are in the run's feed.
The orphan sweep never marks a live job failed, however long it runs: it reads the per-job owner heartbeat (refreshed every 30 s, 90 s life), never the job's age. arq's own in-progress claim on the job is refreshed with the heartbeat, so a killed worker's claim expires within 90 s instead of lingering.
The evidence budget
reporting.reference_urls lists the sample lookups linked under References
on every report, first: each a URL template (url: https://, naming
{sha256}, {md5} or {sha1}), the name it is linked under (source), its
note, and whether the Sigma draft lists it among its references (sigma). It ships with VirusTotal's and MalwareBazaar's lookups, the two
every report has linked (VirusTotal's with sigma, as the draft has always
referenced it); remove one and it is no longer linked. A tool
server's own lookups (its declared reference_urls) are linked besides, only
on a run that asked that server. A report keeps its own links live in the
Markdown.
reporting.evidence_budget_bytes (512 KiB by default) is how many bytes of
tool output one agent may keep in the evidence ledger. Entries past it still
record the call — the tool, the arguments, the outcome and the timing — and
carry no output, and the report states how many were trimmed. Raise it for a
deep reversing loop whose decompilation is the evidence; set it to 0 to keep
every output, which is a supportable choice on a machine with room for it and
a way to fill a JSONB column and a context window on one that has not.
Half a megabyte holds one loop's whole tool output up to a served window of
about 183,000 tokens, which follows from the cap above: a loop's answers come
to at most (window − reply reserve) × 3 characters. evidence_corpus_bytes
(16 MB, the run-wide grounding corpus) holds six such loops at 131,072 tokens
and about five and a half at a million. Neither is scaled with the window on
purpose — they bound the worker's memory and a database column rather than the
model's context, and a machine that also runs the model cannot answer a bigger
window by holding a proportionally bigger corpus. Past either, what happens is
what always happened: the ledger entry keeps the call and drops the output and
the report says how many, and the corpus reports itself incomplete so that an
absence measured against it is advisory.
The live conversation
Two settings under Live events govern the feed the console draws a running analysis from. Neither turns the feed off: the events are what the console shows, and a run that published none would be a spinner again.
events.stream_deltas (on) publishes an agent's text while its loop is still
running, as agent_message_delta. What is published is one model turn's text
as the loop produces it, not a token at a time — the loop reads its graph as a
stream of whole states and that is the smallest thing there is to publish.
Turn it off on a deployment whose browsers are on a thin link; every finished
message is unaffected.
events.retention_days (30) is how long a finished run's feed stays in
job_events before the worker's nightly sweep removes it. The analysis worker
runs the sweep at 03:17 and once each time it starts, so a worker that is not
running at that minute (a stack started for a run and stopped after it) still
sweeps. A sweep that cannot read the stored setting, or finds it refused,
removes nothing rather than sweeping by the default, and a retention so long
that it reaches past the earliest date there is keeps every row (the worker says
so once). The feed is the
moment-by-moment record of a run, and what a reader wants while a run is
fresh; a stopped run also keeps a partial report (see "The job timeout").
Each model call's usage is in it as a model_usage event (agent, model, call
and the figures the provider reported), so what a run spent is known even when
its worker was killed before the run summary was built. What is guaranteed:
the row is committed as soon as the worker's loop runs the publish, normally
milliseconds after the call is recorded, not with the next batch; a worker
killed inside that window loses that one call's row. It is a record, not
progress, so it is kept off the live socket and the 1,000-entry replay stream
and read back through GET /jobs/{id}/events, which merges it into a page the
stream answers, in seq order, so it is there while the stream lives as well
as after. While the job runs, a usage event numbered past the last progress
event a page holds comes with the next page, so a resume never skips a progress
event still on its way. GET /jobs/{id}/usage totals the job's model_usage events: tokens
in and out, cache reads and writes, reasoning and priced spend, per agent,
built by the run summary's own code. Each record carries what the job's spend
meter charged it when it was settled (priced_usd), failed attempts included,
and the total sums those, so cost is readable during a run and after a worker
that died, and a price changed later does not move it; a record from before
records carried their charge is priced at the current llm.model_prices and
counted in repriced_calls. A failed attempt is charged as the provider bills
it: an Anthropic stream that breaks mid-answer is charged the input and cache
reads and writes its opening event reported, and the output its closing usage
reported or, where none arrived, an estimate from the characters that streamed;
its record's estimated_part says output, and estimated_usd is that part's
cost. A stream that broke before anything arrived, with no usage reported, is
charged nothing. It lives exactly as long as the
rest of the feed: events.retention_days (30) after the run. The run summary,
which a completed or stopped run stores with its report, keeps the totals for
good; the model_usage rows matter for a run whose worker died before it
stored one, and are gone with its feed after that period.
The transcript, the agent findings and the evidence ledger are kept by the report and the job and are not touched by the sweep, so shortening this does not shorten how long a finished analysis is readable.
Read-only deployment group
The Deployment group shows the bootstrap values the process is running with —
debug, the auth bypass, CORS origins, the database, Redis and MinIO endpoints,
the cookie flag and the two sample paths. They are not editable from the
console (editable=false, reason "set in the deployment environment; restart
required"), and DSNs are redacted before they are shown. Change them by
redeploying with a different environment.
Tools, and the measurement baseline
Four tool servers are enabled out of the box, and a fifth (virustotal)
ships ready to enable. What each offers is in
architecture.md; what an operator changes here is the
binding, the exposure (tools) and whether the server runs at all (enabled).
There are two ways a server reaches an agent, and the built-ins use both.
network and threatintel are bound by role (MCPServerConfig.agents), as
they always have been. analysis and knowledge carry agents: [] and are
bound only by the ToolRefs in the agent definitions — analysis and
knowledge on the static analyst, knowledge on the dynamic and network
analysts and on the judge. That is what makes a definition's tool list
authoritative: a clone of the static analyst with the analysis reference
removed really runs without the analysis tools, which could not be true if the
server also bound itself to the role.
A server's agents list holds definition keys, checked against the seeded
agents and the definition map in force after the same save: one request may
add an agent and bind a server to it, and a request that removes an agent may
not keep a server bound to it. The console's server editor offers one box per
agent of that map (lead, reverser and triage among them), an agent added
but not yet applied included and one whose removal is staged left out, from
what the settings schema serves (agent_keys and builtin_agent_keys on the
server map's entry). A stored key the map does not hold is drawn, marked as not
a known agent, and kept by every edit; only unticking it removes it, and a save
that keeps it is refused with the key named.
The per-format tools of analysis rest on optional libraries. Install them
with uv sync --extra tools (the backend image already does); without them
apk_info falls back to the zip-level facts and macho_info, the OLE2 half of
document_info and the 7z half of archive_list answer
{"error": "<module> is not installed"}. Nothing else changes, and the server
starts either way. With androguard installed, an APK whose manifest it cannot
parse still gets the zip-level facts: each fact androguard could not read is
answered as no: <reason> (the manifest could not be parsed, with the
exception's type only), and the answer's degraded note names them. A dex
file androguard's reader refuses is listed under dex_strings_unread, beside
the strings of the files it read. The triage pack leaves out only those
reasons; a package name the manifest wrote is printed as written, even one
that starts with no: .
floss, the emulating string decoder, runs FLOSS (Apache-2.0) as FLARE's
pinned standalone Linux build, v3.1.1 (zip sha256
40c05a869f34f7e2417b17ca290cc54bd3671ee1f0a2d9bd5103284c01a54666), outside the
Python environment. The backend image installs it at /usr/local/bin/floss in
a checksum-verified build stage; on a host, scripts/install_floss.sh installs
it at ~/.local/share/maljan/tools/floss-3.1.1/floss. To use a
build elsewhere, set MALJAN_FLOSS_PATH in the analysis server's env. The
tool runs only the executable whose sha256 is the pinned one. Without it the
capability manifest marks floss unavailable with the reason and the remedy and
the model is not offered the tool; with it, the first call on a sample emulates
for up to ten minutes (the tool's declared timeout_s) within 4 GiB of address
space, and later pages of the answer come from the kept result. The triage pack
runs the same function once on every PE, reading MALJAN_FLOSS_PATH and
MALJAN_STAGING_DIR from this process's environment with the decoder's
server's env over it. That server is the decoded-strings step's own server
when it names an enabled one, else the server whose tool declarations name the
step's tool (floss), else one declaring a tool of that tool's kind. The
pack's UPX carve picks its server apart, by the same rule applied to the unpack
tool (unpack_upx, kind unpack), and writes under that server's
MALJAN_STAGING_DIR, so the carved_path it states is one that server's file
tools open even when the decoded-strings step is moved to another server. The
in-process steps run whether the server is switched on or not, so a
switched-off server still lends its env; among several candidates, enabled
servers come first (by name, then by kind, in mcp.servers order) and
switched-off ones are the last resort. The run logs the choice once when
several qualify or the one read is switched off. As shipped, both are the
analysis server. Name a build in the server's env and the pack
and the tool find the same one; a MALJAN_FLOSS_PATH set only in the worker's own
environment reaches the pack and not the tool, whose child environment carries
only the server's env and its allowed keys. Without a build the pack's entry
says so and names the remedy.
Five teams ship built in; they are listed under Teams below. default is
the three analysts with their tools.
measurement is the same three analysts with exclude_servers: ["*"],
exclude_sandbox_tools on and static_provider forced to none — the
baseline for measuring what the ensemble contributes without any tool. Select
it from Settings → Agents and pipeline like any other team; a run under it
resolves each analyst to a prompt, a model and no tools at all.
The wildcard is deliberate. A fixed list of the four built-in keys would still
hand the baseline any server an operator added afterwards, and a measurement
claim that quietly acquires tools is worse than no baseline. exclude_servers
is also the one field an operator may edit on a built-in profile, so a
deployment that needs a variant of the baseline can write one without cloning
it; every other field stays locked.
A team's three fields are honoured in agents/composition.resolve_agent and
in the analysts' own attach path, so they apply to a custom team too:
exclude_servers withholds servers by key or "*" for all,
exclude_sandbox_tools withholds the in-process sandbox tool set, and
static_provider overrides every member's provider at once. A single stage can
withhold every built-in server from its own agents with
builtin_tools: false, which stacks on top of whatever the team excludes.
Where r2mcp is looked for
core.static.r2.binary_path defaults to the bare name r2mcp. r2pm -ci r2mcp installs it under radare2's own prefix and does not touch PATH, so the
provider resolves the name when it starts: a value with a directory in it is
used as it is; a bare name is looked up on the worker's PATH, then in
$R2PM_BINDIR, $R2PM_PREFIX/bin and radare2/prefix/bin under the user's
data directory ($XDG_DATA_HOME, else ~/.local/share) — where r2pm puts it.
Nothing past those is guessed. Not found, r2 degrades as it always has: the
run goes on without it, the log names every place looked, and the run summary
carries static provider 'r2' unavailable: … with the remedy (install it with
r2pm -ci r2mcp, or set binary_path to the executable's absolute path). The
connection test resolves the same way and names the same places.
Running Ghidra lighter
The ghidra-mcp service reads three variables from docker/.env, each
defaulting to the value it has always had: GHIDRA_JAVA_OPTS (the JVM's
options, -Xmx4g -XX:+UseG1GC), GHIDRA_MEM_LIMIT (the container's memory
and swap limit together, 6g) and GHIDRA_RESTART (unless-stopped). Keep
the memory limit about 2g above the heap: Ghidra's database is memory-mapped
and its direct buffers live outside the heap. A host short of memory runs it
with GHIDRA_JAVA_OPTS="-Xmx2g -XX:+UseG1GC" and GHIDRA_MEM_LIMIT=4g, and
GHIDRA_RESTART=no keeps a stopped container stopped across a reboot, for a
host that starts Ghidra only for the runs that need it
(docker compose -f docker/docker-compose.yml up -d ghidra-mcp, then stop).
A binary larger than the lighter heap can analyse fails its Ghidra calls
rather than the host.
VirusTotal's own MCP server
virustotal is a fifth built-in and the only one that is not a process of
this deployment: it is VirusTotal's server, reached over streamable-HTTP at
https://ai.virustotal.com/mcp. Nothing is installed for it and no VirusTotal
API key is involved. It ships disabled, because it needs a credential that
only a registration produces.
Register from Settings → Setup guides → Add a tool server → Connect
VirusTotal MCP. The step is the server's declared credential_bootstrap
(see Obtaining a server's credential): the
button calls POST /api/v1/settings/servers/virustotal/credential, which asks
VirusTotal for an agent token, stores it as this server's auth_token (its own
encrypted row, like every other tool-server credential), turns the server on
and answers with the masked state and the public handle VirusTotal now knows
this deployment by. Registering again replaces the token. Until a token is
stored, the Test button answers "no credential" rather than dialling out.
The lookups are ticked by default and are read-only:
get_file_report, get_url_report, get_domain_report, get_ip_report,
get_analysis and get_submission. submit_file is advertised and stays
unticked: uploading a sample publishes it to VirusTotal, which is a
disclosure an operator opts into, so it takes a deliberate tick in the Tools
step. See security.md for what that changes.
The token is subject to VirusTotal's published quotas. Over quota, the server
answers the tool call with a 429 carrying Retry-After; the analyst records
that answer and carries on without it, exactly as it does for any tool that
declines.
virustotal is referenced by every agent that reads the file or weighs the
run: the static and network analysts, the judge, and the seeded triage,
android_static and reverser agents. Their prompts say what to do with an
answer — look the hash up once, cite it like any other tool result, and treat
a reputation label as one source rather than as the verdict. The mediator
runs its tool loop every round; when it holds a reputation server (virustotal
or threatintel) and no entry in the run's evidence ledger came from one, its
prompt also says no analyst has consulted one and asks it to look the sample up
once, so the identity question is put even on a run nobody disagreed about.
What decides that sentence is the ledger rather than the analysts' prose: a
sentence saying no family could be determined is what an analyst writes when
it consulted nothing. A disabled server contributes no tools and no
degradation reason, so every one of those references costs nothing until the
server is registered.
services/threatintel-mcp is unchanged: it still offers VirusTotal and
AbuseIPDB lookups over their REST APIs with VIRUSTOTAL_API_KEY and
ABUSEIPDB_API_KEY. Where both are on, virustotal supersedes its VirusTotal
half — it is VirusTotal's own server, richer and maintained by them — while
the AbuseIPDB half stays the only source for IP abuse reports. A deployment
with an API key and no agent token keeps working exactly as before.
Both names reach that sidecar through its env_allow, and both are defaults
rather than fixtures: an operator who clears that list in the Configuration tab
stops the keys reaching the child — for an engagement where the sample must not
be looked up, or to stay inside a rate-limit budget — and they stay cleared
until the operator puts them back. The server then answers from its mock, as it
does on a host that never held a key. See
which directories a sidecar may read
for the three names that a built-in does keep whatever the stored registry
says.
The stdio alternative. The same server runs locally as vt-mcp, reading
the same agent token from VTAI_TOKEN, and that form offers one tool the
remote one cannot: submit_local_file, which uploads by path. The remote
server has no view of this host's filesystem, so it offers submit_file
(the bytes, base64) instead. Operators who want the local-path upload install
it as described in deployment.md and add a second server
entry with transport stdio.
The identification servers
die, ilspy and goresym are built-in blocks for three containers of this
deployment, one command-line program each: Detect It Easy (die_scan), ILSpy
(dotnet_types, three decompilers, dotnet_resources,
dotnet_metadata_table) and GoReSym (go_symbols, go_types,
go_strings). Each ships disabled, over streamable-HTTP at the address
the shipped compose file gives its container under its profile
(http://die-mcp:8090/mcp, http://ilspy-mcp:8091/mcp,
http://goresym-mcp:8092/mcp), bound to the static and reverser agents,
with sample_mount /data/samples and reads_staged_sample, every tool
declared read-only and running nothing. Switching one on takes its container
(its profile and its token in docker/.env) and the same token as the
block's auth_token; then its tools are offered and its pack step runs. See
Identification servers.
The rule and document servers
yarax, oletools and webcrack are built-in blocks for three more
containers of this deployment, held and switched on the same way: YARA-X
(yarax_scan, over the operator's own rule files under data/yara-x-rules
or YARA_X_RULES_DIR), oletools (ole_vba, ole_macro_risk,
ole_indicators, rtf_objects, office_dde) and webcrack
(js_deobfuscate). Each ships disabled, over streamable-HTTP at
http://yarax-mcp:8093/mcp, http://oletools-mcp:8094/mcp and
http://webcrack-mcp:8095/mcp, bound to the static and reverser agents,
with sample_mount /data/samples and reads_staged_sample, every tool
declared read-only and running nothing. yarax_scan is declared as
yara_scan is (kind signature_rules, source yara): its matches count as
signature rule hits and corroborate as the one YARA layer. See
Rule and document servers.
The container and Android servers
unblob and jadx are built-in blocks for two more containers of this
deployment, held and switched on the same way: unblob (container_scan,
container_extract, and get_artefact, the delivery primitive the artefact
store reads a written file back with, never offered to a model) and JADX
(android_classes, android_decompile_class). Each ships disabled, over
streamable-HTTP at http://unblob-mcp:8096/mcp and
http://jadx-mcp:8097/mcp, with sample_mount /data/samples and
reads_staged_sample, every tool declared read-only and running nothing.
unblob is bound to the static and reverser agents, jadx to the
android_static and reverser agents. unblob writes what it extracts into a
directory of its own in its container, never beside the sample, and stops at
the larger of the sample's size and the sample upload bound (UPLOAD_MAX_BYTES
in docker/.env, 100 MiB unless set) or past UNBLOB_MAX_ENTRIES entries,
in a /tmp tmpfs sized below its memory ceiling (UNBLOB_TMPFS_SIZE). See
Container and Android servers.
Teams and stages
A team (core.agents.profiles.<key>, edited under Settings → Agents and
pipeline → Teams) is an ordered list of stages. Each stage is:
| Field | Meaning |
|---|---|
key | Slug, unique in the team. Names the stage everywhere it is reported. |
label | Display name; empty means the key. |
kind | triage, analysis, debate, verdict or report. |
agents | Definition keys this stage runs. Empty on a triage or debate stage. |
depends_on | Earlier stage keys this one runs after. |
when | Condition deciding whether it runs. Empty means always. |
mode | parallel or sequential, for an analysis stage. Unset (the default), the stage follows the job's analyst mode (below). |
inject_upstream | none, findings (default) or full. |
debate | Round limit, consensus threshold and sycophancy check, for a debate stage. |
builtin_tools | false withholds every server the platform ships (analysis, knowledge, network, threatintel, virustotal and the container sidecars die, ilspy, goresym, yarax, oletools, webcrack, unblob, jadx) from this stage's agents. A server is shipped by the mark its seed carries, which no operator can set or post. A shipped container sidecar an operator switches on is attached after the definition's own servers, as a server of theirs is, so every tool keeps the name it had. |
How an analysis stage runs its agents
core.llm.parallel_analysts is auto (the default), true or false. It
decides how the agents of every analysis stage whose own mode is unset run —
all at once, or one after another. A stage whose mode is set keeps it,
whatever this key says. A debate's revision round runs the way the stages it
revises run: in parallel only when every one of them does.
-
truealways runs them in parallel;falsealways runs them one after another. -
autois decided per job from the endpoints of the models the analysts call (every model an analyst may call, its fallbacks included):- a model served by Ollama is a runtime on one machine, taken to serve one request at a time;
- an OpenAI-compatible endpoint's host is resolved. A loopback, private,
link-local or shared-range (
100.64.0.0/10) address — written as one, or what the name resolves to (a compose service name, a Tailscale name) — and a name only a local resolver answers (localhost,*.local,*.internal,host.docker.internal,host.containers.internal) is a local server. The address rule is the one the disclosure block uses (core.host_locality): an IPv4-mapped, 6to4, Teredo or NAT64 address is read as the IPv4 address it carries, so one carrying a public address is hosted and one carrying a private address is local (the well-known NAT64 prefix included, the one case decided less strictly thanipaddress.is_privatedecided it), and a documentation, benchmarking or other reserved address is hosted, since none of them is this machine. Its/propsis asked fortotal_slots(the request the context-window probe makes anyway): more than one slot runs the analysts in parallel; one slot, or no answer, one after another; - a host that resolves only to public addresses is a hosted API and runs
them in parallel, unless its
/propsreports exactly one slot; - a host that does not resolve runs them one after another: it could not be told whether the endpoint is hosted;
- the Anthropic and Gemini APIs are hosted.
One model taken to serve one request at a time makes the whole job sequential, because concurrent analysts on a single-slot server clobber each other's per-slot state and every step re-processes its prompt. The worker resolves the mode on a thread before it builds the job. A mock job runs its analysts one after another.
The job logs the mode and the fact that decided it (Analysts run in parallel by default for this job: …), then one line per analysis stage and per
revision round saying what it runs in and why: set on the stage, the job's
mode, a debate hands over to it (an unset stage a debate hands over to stays
one node, because a parallel stage of two agents is two nodes), or the mode of
the stages it revises. run_summary.profile.analyst_mode carries mode,
setting and reason, and stages: {stage, round, mode, from} per
analysis stage (round: analysis) and per debate (round: revision).
The same mode decides whether the judge's asks in one turn run at once. In a
job whose analysts run in parallel, an ask of an analyst runs beside the
judge's other asks, and an ask of an agent in no stage does when the auto
rule, applied to its own models whatever this key says, finds them serving
concurrent requests (no server is probed for it); otherwise, and for
a lead's asks, they run one after the other. The answers come back in the
order of the calls and the asked agents' ledger entries in ledger-id order,
however the asks finish.
The same key decides how the report composer writes its sections, asked of
every model the reporter may call (its agent entry and fallbacks, or the
global judge model). false writes them one after another. Otherwise each
model serves what its server can: a hosted API every section at once, with no
limit; a server whose /props reported its slot count that many at a time,
whatever the key says (one slot is one after another); Ollama, a local server
with no slot count, or a host that does not resolve, one after another. The
smallest of them holds, since any model of the list may answer. A section
waiting for a slot has not started its wait: its wait starts when it is
served. Under a spend ceiling (core.llm.max_spend_usd_per_job) they are
always written one after another: the meter holds each report call at what
the calls before it left, and admits report calls one at a time. Either way
each section is sent the request it would be
sent on its own (no section reads another's answer), keeps its own retry and
validation, and a section that fails or times out leaves the others running;
the answers, their validation findings, the report's degradation sentences
and its flagged sentences are applied in the sections' fixed order, so the
report reads exactly as one written section by section. A section's wait is
sized from the pace measured when it starts and, while sections run at once,
sized again from the pace measured since before it ends, never shorter. On
Claude, within each group of sections whose requests share one prefix, the
first is sent alone and the rest of the group follow once its answer has
begun, because Anthropic makes a cached prefix readable only from then on
(see The composer's shared head is cached once per shared prefix below);
every other section is sent at once. The report node logs which way it ran and why (report_node: the composer's sections run at once: …). The narrative round still runs before
the sections: its answer sizes their waits where the reporter's pace was not
measured earlier in the job.
The composer's shared head is cached once per shared prefix
Every composer section's prompt opens with the same head: the run-state
block, the triage pack and the techniques the report publishes. Anthropic
caches a prefix in the order tools, system prompt, messages, so only requests
identical up to the end of the head can share a cache entry. On the
structured path, the default on the anthropic provider, each section sends
its own answer schema as a tool: only the seven prose subsections, which
answer with one schema, share a prefix. On the manual path (no tool) every
section shares one, and a section whose structured answer holds nothing to
read is asked again there with its head noted for that group, whatever schema
its structured request sent. A structured answer that writes the object as
text, or calls the tool with arguments the schema does not take, is not asked
again: that text is the section's answer, read and checked as a manual-path
answer is, and a retry its findings need is sent with the same tool, so the
request only appends to the one answered and its thinking blocks stay valid.
An answer that is nothing but the call goes back as it came, and the
correction is the call's reply, marked as an error, so the model reads the
arguments the findings name. Only such a retry, which offers the tool again,
is sent a call; it is the section's last request, so it carries no cache
breakpoint beyond the shared head (an entry written there would never be
read). A retry that never offered the tool (the manual path, the judge's
verdict) keeps its shape: the answer is left out and the correction is asked
at the end of the user turn. A parser that hands back a plain object is read
through the section's schema the same way: one that does not fit is the
section's first answer, not a reason to ask again from the start. A
retry that has to be held under an output bound goes by the plain model, with
no tool; there the kept answer's thinking blocks no longer match the request
and are dropped, with a WARNING (the Preserved thinking recovery), and a call
is said as text: its arguments as the answer and the correction after them,
the call's thinking left out with it. Text with no
object in it — a preamble, a one-line refusal — is not an answer to read, and
the section is asked again as before. Each structured answer is recorded on
the run's ledger once, where it is read, and a model list's switch carried on
it is announced there once, a kept answer's included.
On haiku4 every one of the 16 structured answers was of this kind and was
asked again, 16 calls and 0.86 USD of the composer's spend. The narrative
round, the mediator's verdict extraction and the judge's technique question
read their structured answers by the same rule (one reader,
token_ledger.read_structured_reply): the narrative round's answer in text is
its first answer, a mediation verdict written as text is read rather than
asked up to three times again, and the technique question already read its
answer's text. For each
such group, the head is sent as its own text
block with a cache breakpoint (llm.anthropic.prompt_cache_ttl) and the rest
of the prompt as a second block; joined, the two are the prompt's text
character for character. The group's first section writes the entry and the
others read it. A section that shares its prefix with no other is sent as
before, with no breakpoint, and pays no cache write. On DeepSeek and other OpenAI-compatible endpoints prefix
caching is automatic and the request is unchanged: the head is already the
same leading text on every section.
A schema offered beside read_evidence is asked for in text where the model writes it so
No tool choice is sent for a schema on Anthropic, and a model that thinks by
default cannot be made to call one: offered beside read_evidence, Claude
Haiku 5.5 wrote the JSON as text in every structured call of haiku7 and
haiku8. The provider says so for such a model (read from the vendored table's
thinks_by_default rows, unless the model is built with thinking off), and
says it asks for no
JSON-schema answer natively (the pinned langchain-anthropic's json_schema
method replaces a model's bound tools and cannot read a reply that reads).
The composer's sections, the narrative round and the judge's technique
question are then asked for their JSON in text from the first call, with
read_evidence and no schema tool, and the model says so once in the log; the
answer is read by the same parser that read it from the text before. Every
section so sends the same tools, system prompt and head, one group whose
first section is sent alone and whose others follow once its answer has
begun, so they read the head it cached. A schema offered alone (the
mediation's extraction) and every other provider keep the tool.
The judge's tool loop is cached as an analyst's is
Each turn of the mediator's tool loop after its first tool round is noted as a loop turn, as every analyst's turn is: on Anthropic that turn sends the prompt's first user turn as one block with a breakpoint, which writes the cache entry, and every later turn of the loop reads that prefix and writes only what it added. The loop's first request is not noted: most mediations and every verdict answer on it, and a call that ends there pays no write premium for a prefix nothing would read. The judge's single-shot calls (the verdict, the technique question, the mediation's extraction) carry no breakpoint: no later request repeats their tools and system prompt within the cache's lifetime, so a breakpoint would only add the write premium.
The mediation's turn leads with the triage pack and the sample's identity, then the run state, then the reports and the history: every round sends the same tools, system prompt and head, and the head is noted as standing facts, so its own breakpoint (below) is written once and read in each later round. With the run state first, haiku8's mediations shared about 920 characters. The order is the same on every provider; only Anthropic places a breakpoint on the head. The verdict and the technique question keep the run state first.
The triage pack has a breakpoint of its own
An agent's debate rounds send the same tools, system prompt and triage pack
again, often more than five minutes apart (twelve to thirty-two in haiku8),
so each round's first request wrote the whole prefix and read none of it. On
Anthropic the pack at the head of a conversation's first user turn, noted
with its length where the
platform puts it there, is sent as its own text block with a breakpoint that
lives llm.anthropic.pack_cache_ttl (1h by default; 5m, or off for
none); joined, the two blocks are the turn's text. Every other breakpoint keeps
llm.anthropic.prompt_cache_ttl, the pack's is never shorter and always
first, as the API requires (a breakpoint asking for an hour after one asking
for five minutes is sent asking for five, with a WARNING), and a request
carries at most four. A first turn once split is remembered by its whole text,
and the same turn sent again without its note is split at the same place, so
the history a thinking block was written after never changes; a turn never
noted is never split. A call is admitted at the hour's write price while the
pack's entry may live an hour, and a streamed answer's writes are settled by
lifetime from the split its opening event reports. Other providers send the
request as before.
A stored true or false (a JSON boolean) keeps its meaning. Stages used to
be written with sequential whether or not anybody chose it; the database
revision 20261002000000 takes that word off every analysis stage, so the
stage follows the job, except where core.llm.parallel_analysts is stored
true (there a sequential stage was running one after another while the key
said parallel, and it is left as written). It logs each team and stage it
rewrote at WARNING. A team still derived from its analyst list is rebuilt from
the key on every load and is not touched. Pick sequential on a stage card to
pin a stage.
Checking a team before it is saved
The team editor checks every team as it is edited. Once typing pauses, the
console sends the staged teams, the staged agent map and the staged active
team to POST /api/v1/settings/lint-teams, which stores nothing and answers
with every finding and each team's layout. Beside each team the console draws
the stage graph that layout describes — stages as boxes in run order,
depends_on as arrows, a conditional stage dashed — and marks each stage that
has a finding; the same findings are listed in words under the graph and on
each stage card.
An error is a team apply refuses, and the lint reports it in the words
apply refuses it with: the lint reads its errors off the same functions the
settings model raises from (maljan.core.team_lint), and the apply path
refuses from the lint. A test holds the two together: every refusal the
settings model makes about a team is a lint error, word for word. The errors:
- a team with no stages;
- a stage key declared twice, or a key that is not a slug;
- a stage that depends on itself, on a stage that does not exist, or on a stage declared after it — and, named as such, a loop of stages that depend on each other;
- an agent in two analysis stages; an analysis stage with no agent;
- a triage stage that names an agent, or is keyed like a node the pipeline names itself;
- a debate with no analysis stage upstream of it, one that names an agent, or one that hands over to more than one node;
- a stage naming an agent that does not exist, is disabled (a built-in team may keep a disabled member while it is not the active team), or has the judge or reporter role in an analysis stage;
- not exactly one verdict stage, or a verdict stage not run by the judge;
- more than one report stage, a report stage not run by the reporter, or one that is not last;
- a
whenthe condition grammar refuses, reported by the parser that runs it; - a field of the wrong type or value, in pydantic's own words;
- a built-in team edited beyond its debate options, its built-in-tool switch and its excluded servers;
- a team key that is not a slug.
A warning is a team that saves and runs and will not do what it looks like it does. A warning never blocks apply, and each is decided from the team as written, never from a guess about the sample:
- a stage the verdict stage does not run after (the judge does not wait for it), or one that runs after the verdict, other than the report;
- a condition that names no field and is false, so the stage never runs;
- a condition reading
stages.<key>for a stage the team does not have, or for a stage this one does not run after; - an enabled agent that no team names and no agent asks;
- an agent reading the static provider's tools while the static provider is
none(the note apply already returns).
The layout the graph uses is maljan.core.team_layout: a stage one row below
the lowest stage it runs after, stages that share a row side by side in the
order they are written. The team diagrams on the architecture page are drawn
with the same layout.
The teams that ship
| Team | Stages | What it is for |
|---|---|---|
default | triage_pack → analysis (static, dynamic, network) → debate → verdict → report | The general case. |
measurement | The four after the pack, with every tool server withheld and no pack | What the ensemble contributes with nothing to call and nothing established. |
mobile | triage_pack → triage → android_static → dynamic → debate → verdict → report | An APK or a DEX. |
deep_static | triage_pack → triage → static → reversing → network → debate → verdict → report | Reading the code. |
team_lead | triage_pack → lead (lead) → verdict → report | One lead agent gives the specialists their work; see Delegation below. The one seeded team with no debate: a debate over a single analyst costs a second full loop and cannot change a position. |
triage_pack is a stage of kind triage: the pipeline itself running the
deterministic tools over the sample and writing each result to the evidence
ledger before any analyst starts (see The triage pack in
architecture.md). Every team but measurement ships with
it first, a team written by hand may leave it out, and a stored team gains it
on upgrade (make migrate). Its six settings sit in the Analysis layers
group: triage.enabled (off leaves the stage in place and makes it decline
with that reason), triage.strings_head (how many printable runs the strings
entry keeps; 300), triage.reputation (auto asks the enabled reputation
server once for the sample hash — VirusTotal's own server when enabled, else
the threat-intel sidecar, never one the team lists in exclude_servers — and
off records a skipped entry instead), triage.budget_seconds (1200; a
step that would start after the budget is spent is recorded as not run),
triage.memory_floor_mb (10,240: what the host must still have available
after the peaks of a step that starts beside the others and of the steps it
runs beside — as shipped, capa's measured peak and FLOSS's 4 GiB bound — for
them to run together; 0 checks only that they fit, and the worker's cgroup
limit is always checked) and triage.steps (below). The
pack runs the real tools in mock mode too, so a local observation run with a
reputation server enabled makes that one outbound call; a team that withholds
the server, or triage.reputation = off, keeps such a run offline.
The pack's steps
The pack is a list of steps (maljan.core.pack_steps), each declared as
data. The shipped list is the pack described in
architecture.md, in that order; triage.steps holds your
own, laid over it (the console edits them as a form, see Adding a step in the
console below):
| field | meaning |
|---|---|
id | the step's name, unique in the list (lowercase letters, digits, _, the whole id read) |
tool, server | what the step calls. No server: one of the platform's own steps, named by its shipped id. A server: that tool server's tool, called through the tool server registry as an agent's call is, to the path that server opens the sample at, and recorded under that server |
arguments | a server step's argument template; a string at any depth (inside objects and lists too) may hold {sample_path}, {sha256} and {file_type}, each filled once, and a file of the job's artefact store (Files passed between servers, below) at the path the step's server opens it at: {capture_path} (the sandbox's packet capture) and {sandbox_report_path} (the sandbox report as JSON), with which the step runs only where the job has one, and, on a step that reads, {artefact_path} and {artefact_sha256}. None: the argument the tool declares for the path role takes the sample's path (on a step that reads, the file's) and the one for hash its sha256 (the file's). The platform's own steps compute their own and refuse one |
formats | the routed formats the step runs for; empty is every format |
when | the values earlier steps' answers must hold for the step to run, each {"step": <an earlier step>, "value": <a value its tool declares>, "equals": [<texts>]}: the step runs when a row of that step's answer holds the value (tool_facts.<tool>.values) equal to one of the texts, compared whole (a text as it stands, any other value as JSON writes it: true, 3). Every condition must hold. One that does not, an earlier step that did not run or gave no answer, and a tool that declares no such value are a not run: entry saying which, never a failure. Any step can have one, the platform's own included; a step that starts beside the pack starts only once what it runs on has answered, and otherwise runs in turn, saying so |
outputs | a server step's answer path (a declared field path, /written/*) that names each file its tool wrote, as its server names it; each is taken into the job's artefact store, and the entry's pack_step names what was (wrote) and what was not, with why (not_taken) |
reads | an earlier server step with outputs: this server step runs once for each file that step wrote and the store took in, in the order they were named, each call's entry naming the file it read (pack_step.read); with none, one not run: entry says why |
budget_seconds | the step's wall clock; none takes the tool's declared timeout_seconds, else what is left of triage.budget_seconds. Of the platform's own steps only those whose shipped declaration says their in-process code honours a clock take one: capa (else static.capa.timeout_seconds), floss and yara_scan |
peak_mb | the memory the step takes at its peak, where this worker does not measure it; only on a step that starts beside the pack or that another step runs beside |
starts_after, beside | a step that starts on a thread of its own right after starts_after, when the host has room for it beside the steps beside names (their peaks and its own against triage.memory_floor_mb); its entry keeps its place in the list, its thread is let go when its work is done, and triage_facts.<id> and the run summary's triage.<id> say beside … or in turn: <why>; a step named as one of the state's own keys (entries, has_signature, hits, …) runs as any other and says it under _held_schedules.<id> in both instead, so no fact or count is replaced. Any server's tool can; of the platform's own steps only floss |
renderer | how its pack line shows the answer: one of the pack's renderers (core.pack_steps.PACK_RENDERERS). capability_rules reads the answer of a tool declared of kind capability_rules (or a step whose kind says so) through its declaration: each declared row that is an object and has a readable name a rule, by its values.rule, at its places.addresses, with its techniques when the tool asserts them; it is refused on a tool declared another kind or none. Every other renderer reads only the answer of the platform's own step it was written for. None: a tool of kind capability_rules is shown as that renderer shows it, any other by the generic line, the count of the declared rows that are objects, each by its declared rule, text, name or value (a declaration naming none of these gives the row's other declared values as name=value) at its first declared place |
kind | the kind of answer the step records, laid over the tool's declared kind on its entry |
display_name | what the pack's line calls the step (and, for a server's tool, what other lines name its answer by) |
enabled | false drops the step |
after | where a new step goes: after the step it names, else after every shipped step |
A declaration under a shipped id changes that step in its place; one that
names another tool or server replaces it there, keeping only the id; a new
id adds a step. A list that cannot run as declared is refused when it is
saved: an id twice, a placeholder no template may name ({artefact_path} or
{artefact_sha256} on a step that reads nothing among them), a when or
reads naming no step or one that does not come before it, a reads of a
step with no outputs or with a starts_after, outputs or reads on one
of the platform's own steps, a step with no server that names a tool and is
not one of the platform's own, a declaration with neither under an id no
configured server's tool gives, on an enabled step a server
mcp.servers does not hold or a tool outside its tools list (in the words an
agent's reference is refused in, and removing a server a step uses is refused
too), an after,
starts_after or beside naming no step or the step itself, a starts_after
or beside on a step that runs in turn, a beside with no starts_after, a
peak_mb nothing reads, and a renderer that cannot read the step's answer.
Stored steps are read leniently: a stored step this release cannot run (its
server gone, a field it no longer reads, a renderer its tool's kind cannot
feed) is left out with one warning in the log, the shipped step it changed is
in force again, a step placed after or beside it goes with it, and every other
setting and step stands. Whatever left a step out (also a reset of the server
it calls, or a shipped declaration that changed its tool's kind), the settings
schema names it with why (pack_steps_left_out, also where other stored
settings do not build) and the console marks it on the steps. Where the stored
entry of the server a step names in its server field is refused, the step is
listed as not checked or run until that entry is fixed or reset, never as
calling an unknown server, and a save leaves it to that entry's own refusal;
only the step's server is matched to a refused entry, never another value it
holds. A save that changes the steps is held to the rules above, and a
refused save names its step problems beside every other refusal; a stale
stored step sent back exactly as stored, wherever it stands, blocks no other
save, and no new or changed item rides on it. A server step whose server the team withholds or that is not
configured is a not run: entry, not a failure; a call that fails is a
failure of its tool. Every entry an operator's step writes carries the step
(pack_step), so the pack reads it back without the settings.
The steps of a server's tool. The shipped list names no server and no
server's tool. Every server in mcp.servers gives the pack a step for each of
its tools whose declaration in force (the block's tool_facts, over what the
server declares of itself) says all of this
(core.pack_steps.server_steps): it answers one of the pack's server-step
kinds about the whole file (core.pack_steps.SERVER_STEP_KINDS:
identification, type_list, function_list, signature_rules,
vba_macros, dde_links, embedded_objects, deobfuscated_script,
extracted_files), it is read_only, it takes the file in the argument it
declares for the path role, and its effects say it runs none of the
sample (executes: no), reaches no network (network: false) and sends
nothing to a third party; one that discloses the sample, or leaves any of
these undeclared, is never a step the pack takes on its own (any agent may
still call it, and a step of yours in triage.steps may name it). A server
on another host that mounts none of the samples (a remote transport and no
sample_mount) gives no step: the sample would be sent to it, which only a
step of yours may ask for; its steps are off, saying so. A server whose block
trusts what it declares of its tools and that has not been listed yet gives
none either, and is off saying its tools are not known; it is listed before
the next run (providers.served_listing). What such a block keeps of its
server's declaration is kept for one release: another release lists the
server again before reading its tools. Two releases run side by side against
one store (a rolling upgrade) each forget what the other kept, and list the
server again, until only one is left. Nothing here reads a server's or a
tool's name: a server you add whose tools declare the same gets the same
steps, whatever its tools are called.
A step of yours that calls the same tool on the same server (and reads no
other step's files) is that tool's step: it keeps its id and every field you
set, the rest are read from the tool as above, and it runs where the server's
step runs.
The step's id is its tool's name as a step name (lowercase, other characters
_), with its server's key before it where another step has it; its line
calls it by its tool's display name and the words of its kind (GoReSym · functions, GoReSym · types). It runs on
the routed formats its tool declares (routed_formats; none: every format),
writes the files its tool declares (outputs), and a tool that reads one code
runtime alone (runtimes) runs only where the first identification step whose
tool declares a runtime value names one of them; with no such step it is
off. The identification steps come first, then the servers in their order and
each one's tools by name; each runs right after the last step of the
platform's own that answers its kind (YARA-X's yarax_scan right after
yara_scan), else right after the format readers (archive_list), after any
step of yours placed there, so every id the shipped list issues before it
keeps its value. A step is off while its server is switched off or does
not offer its tool (core.pack_steps.steps_on_servers): it makes no call and
writes no entry, and the pack's result and the worker's log say why. A run
that calls no tool server has none. Switched on, a step whose condition does
not hold is a not run: entry, as any condition is. Every entry such a step
writes carries it (pack_step).
With the shipped servers switched on that is: die_scan (Detect It Easy's, on
every sample); dotnet_types (ILSpy's, on pe, where die_scan's runtime
is dotnet); go_symbols and go_types (GoReSym's, on pe, elf and
mach-o, where it is go); office_dde (on ole2, ooxml and rtf),
ole_vba (on ole2 and ooxml) and rtf_objects (on rtf), oletools';
js_deobfuscate (webcrack's, on js, hta and wsf); container_extract
(unblob's, on every sample, with outputs /files/*/path, so each file it
wrote is taken into the job's artefact store for a step of yours that reads
it); android_classes (JADX's, on apk and dex); and yarax_scan.
A declaration in triage.steps that names no server and no tool, under
an id no shipped step has, is laid over the server's step of that id where
the run's servers give one, field by field (its after places it; enabled: false drops it, and nothing then runs on what it would have answered). A
save knows the steps the configured servers' tools give, off or not
(core.pack_steps.server_step_ids): such a declaration under an id none of
them gives, and an after, starts_after, beside, when or reads naming
a step that is neither in the list nor one of them, is refused. A stored list
is read without the servers: such a name is kept, and the run resolves it. A
step placed after a server's step runs right after it, with every step placed
after that one, and after one the run does not have, last. A save that knows
the servers also refuses a when or reads on a step the run places at or
after the step naming it; at run time such a step says it runs after it. A
condition on, or a reads of, a step the run does not have is a not run:
entry saying it is no step of this run; a declaration
over a server's step the run does not have is off, saying so.
Files passed between servers. A file one server's tool writes (a payload
carved out of the sample, an archive member, a decoded script) reaches
another server's tool through the job's artefact store
(maljan.pipeline.artefacts). A step that declares outputs has each file
its answer names taken in: copied under the job's own staging directory
(<MALJAN_STAGING_DIR>/<job>/artefacts/<sha256>/<name>, every directory
0o700 and the file 0o600, removed with the job's staging) under a stable id,
art_<sha256>; its name is kept to its last part in safe characters, never
deciding where it lands. The bytes come over the producing server's
get_artefact tool where it offers one (see
MCP servers), whatever its
transport, and otherwise, for a server on the worker's filesystem (stdio, no
sample_mount), from inside that server's own job staging directory only,
opened component by component beneath it with no link followed anywhere (a
link, a .., a name outside it, anything but a regular file and a file with
more than one hard link, which a path outside it may name too, are refused). A
step that reads it, or names {capture_path} or {sandbox_report_path}
(which the pack takes into the store the first time a step names them; the
capture only from the directory the worker fetched this job's captures into,
opened the same way, so a path the report names anywhere else is refused and
never read), hands its server each file as the
sample is handed: a server on the worker's filesystem a copy under its own
job staging directory, a server with a sample_mount the worker's copy under
its mount (as read-only as the deployment made it, released with the job's
other copies), a server over HTTP an upload over its put_sample tools,
listed with the run's other uploads in the disclosure block. A server none of
these reaches is not called: the step's entry says not run: and why, and no
path only the worker can open is ever handed over. A file is streamed once,
with no size limit, and opened as bytes only.
Adding a step in the console. Triage pack steps (Analysis layers, under
Triage pack) draws your steps as a form, one card per step. Type an id under
New step id and press Add step: a shipped step's id (offered as you
type) changes that step in its place, any other id adds a step of your own; an
id that does not read whole is said there. A step named as one of the pack
state's own keys that starts beside the pack carries a note that its schedule
is recorded apart, and blocks nothing. Each card says how its step stands, by the rule the save
lays a declaration over the shipped list with (served as replacing_fields
with the shipped values they compare against): a step of your own, one that
changes its shipped step field by field, or one that replaces it at its place
because it names another server or tool, which keeps nothing of the
shipped step but its id. What a card shows for a field you have not set is
what the save puts in force: the shipped value on a step that changes its
shipped step, the field's default otherwise. Pick the Server from the tool
servers in force (none: one of the platform's own steps) and the Tool: the
box offers only tools the save accepts, from the server's tools list, the
tools it declares and what List tools reads from its manifest (the
connection test the server editor runs, under the same administrator rights; a
tool its capability manifest marks unavailable says why). A listed or declared
tool outside the server's tools list is not offered, and the card names it
with the save's reason; a step with no server is offered only the tool its own
shipped step calls; a typed name the save refuses is marked with its words,
and one no source names is kept and marked as not seen. On a disabled step,
which the save does not check against its server, a tool outside the server's
tools list is noted as refused once the step is enabled. A listing is dropped
once that server's staged settings change. Arguments are name and value
rows. A value may name
{sample_path}, {sha256}, {file_type}, {capture_path},
{sandbox_report_path}, {artefact_path} and {artefact_sha256}, and a placeholder outside the
list is flagged on its row before Apply, by the pattern and the list the
settings schema serves on the entry (pack_step_vocabulary, built by
core.pack_steps.step_vocabulary from the step model), which is the rule the
save applies; a row with no name, or a name given twice, holds Apply until it
is put right. Budget (seconds) and Formats (a box per format the
router can give a sample, extractors.sample_identity.routed_formats; a
stored format outside it is drawn, marked and kept) follow, and More fields
draws every other field by the type the schema serves, the renderer picked
from the pack's renderers (outputs and reads as text boxes; when, a list
of conditions, is shown as its JSON and edited in the setting's JSON view). An empty value over a shipped one is a value of its
own (renderer: "" is the generic line, display_name: "" the tool's own
name, beside: [] nothing to run beside, formats: [] every format): such a
field is drawn as set to empty, in place of the shipped value, and Unset
puts the shipped value back. Where a shipped value stands, Set to empty
(or Every format) writes the empty value, and emptying a box writes it
too, so emptying never brings the shipped value back unseen; on a step of your
own, emptying a box leaves the field unset, as you would write it by hand.
Up, Down and Drop reorder and remove a card. A field the step model does not know is shown on its card and kept until
you remove it (a save refuses it, naming it), and a field holding a value of
another type than the schema serves is named on its card rather than drawn
into a control. The form writes only the fields you set, key for key in the
model's order, so a step saved through it is the step written by hand:
{"id": "acme_scan", "tool": "scan_file", "server": "acmescan"}.
Edit as JSON (advanced) shows the same list as JSON; both views edit one
value, and JSON that does not read holds Apply. The save stays the judge, and
each refusal it makes is shown beside the steps.
A later step reads an earlier answer by its declared kind, never by its tool:
the function starts the hash resolution, the string decoder, the constant
scan, the anti-analysis pass and the function index are handed come from any
function_list or capability_rules answer that declares function_starts
(or a function_list answer's start place); the index's rules from any
capability_rules answer and its call-site strings from any
emulated_strings answer or decoded_strings answer that names a
call_site; an input no answer fills is stated by its kind
(capability_rules: no: the pack holds no entry of it). So swapping capa for
your own matcher means declaring its tool capability_rules with rows,
places.addresses, values.rule and function_starts, and naming it under
the capa id:
[{"id": "capa", "server": "acmescan", "tool": "match_rules"}]Its line counts and names its own rules, each where it matched, read through
its declaration. The index, the constant scan's agreements and the pack's text
name it by its declared display_name, and the answers a model reads name no
tool that did not run: a function index row holds the rules under capa only
when the shipped capa answered them, else under rules (with rule_sources
naming each rule entry's tool), the constant scan's and anti-analysis pass's
agreements likewise, and decode_string_blobs compares under
floss_rows_compared, also_recovered_by_floss and a row's floss only with
the shipped FLOSS's strings, else under decoder_rows_compared,
also_recovered_by_decoder and decoder. Where no answer of the kind came
back (the tool failed), the shipped names are kept only while the step list
still runs the shipped tool. A function claim an analyst writes is checked
against the rules of either field.
The technique check's one heuristic part has five settings in the same
group: validation.alignment_gate (auto runs the alignment gate only on a
worker whose ATT&CK index is already built; off never),
validation.alignment_gate_build (false; true lets the first run that wants
the gate build the index once, on a thread, and go without it),
validation.weak_alignment (false — the ranking is recorded on the claim and
shown to the judge, and nothing is asked again; true lets it question a claim,
at one correction turn per batch), validation.alignment_threshold (0.05, the
paper's gate) and validation.alignment_margin (0.20, how far a candidate from
the sample's own domain and another tactic must beat the claimed id before it
is questioned). The measurement behind the default off is in The technique
check in architecture.md.
validation.claim_repeat_margin (empty) is how many claims an analyst's
answer may write again before it is asked once for a whole answer
(isr.claims_repeated), with the answer shown back up to its first repeated
claim. Empty derives the margin from the answer itself: the number of distinct
claims it wrote. A whole answer that does not repeat replaces the repeating
one; otherwise the answer stands as written. On a streamed path (llama.cpp,
Ollama, DeepSeek) the same margin ends the answer while it streams.
mobile and deep_static are built from three seeded generic agent
definitions — triage, android_static and reverser — whose prompts live in
src/maljan/agents/prompts/. A generic agent has no class: it is a definition,
a prompt and a tool list, which is what makes a team of your own something to
write rather than something to build. Clone one of these as the starting point.
Their conditions are the interesting part. android_static runs on
file_type in ("apk", "dex") and dynamic on has_sandbox_report, so
submitting a PE under mobile produces a run where the Android stage is a row
that declined with the condition it failed printed beside it — the team was
applied and the console shows what it chose not to do, rather than showing
nothing. deep_static's network stage runs on
has_pcap or has_sandbox_report, and its reversing stage depends on static
with inject_upstream: findings, so the reverser is handed each static finding
and asked to confirm or refute it at function level.
reverser takes ToolRef(kind="provider") rather than a named server, which
means the tools of whichever static provider this deployment configured. On a
deployment with static.provider = none that reference resolves to nothing:
the stage still runs, and its prompt still asks it to open a decompiler, so
what comes out is a confident ungrounded answer rather than a visible failure.
Saving such a team is allowed and says so. A team validated against a runtime
provider setting could not be saved before the provider was configured, and the
order those two happen in is the operator's — so the settings API answers a
successful write with a warning on that stage instead of refusing it, and
the console draws it on the stage card: "reverser reads the static provider's
tools, and this deployment's static provider is 'none'." A stage whose agents
have no other tools at all is named as running with nothing to call; one that
also holds a tool server, as deep_static's reverser does, is named as running
without the decompiler.
Like every built-in team, all five are editable only in their debate options,
their builtin_tools switches and exclude_servers. Everything else means
cloning the team, which the console does in one click.
The reverser is handed addresses to start from. The triage pack every agent
reads names each decoded string with the routine that produced it and its call
site, and each capa rule with the places it matched, all as offsets from the
image base; the seeded prompt tells it to go there first, to confirm or refute
each upstream finding at function level, and then to look for what only
reading the code shows: command dispatch, environment checks, persistence and
cleanup, the logic that decides when and how it contacts a remote host, and
the routines that decode its data. An agent on a static provider that reads
a call graph — Ghidra over http, the reverser given
static_provider: "ghidra" included — also gets the sink-reachability
pre-pass's priority functions on its first turn, and the sample is mirrored for
its provider even when that is not the deployment's global one.
An all-tools team
docs/examples/profiles/all-tools.json is a team to import rather than one
that ships: the triage pack and triage; one static stage of three analysts
on three tools — static (the analysis, knowledge and VirusTotal servers, on
the global provider), all_tools_static_r2 (static_provider: "r2" with the analysis
and knowledge servers) and all_tools_qu1cksc0pe (a generic agent on the
qu1cksc0pe server); reversing with all_tools_reverser_ghidra, the seeded reverser
prompt on static_provider: "ghidra"; dynamic when there is a sandbox
report; network when there is a capture or a sandbox report; then debate,
verdict and report. The later stages depend on every earlier analysis
stage and read their findings (inject_upstream: findings).
It is a settings import document (maljan-settings/1) holding
core.agents.definitions and core.agents.profiles. Each of those is one
setting holding a whole map, and an import replaces what it names, so merge the
document into your own export first. The document's keys are all all_tools_*
(the team is all_tools), so it adds entries and replaces none of yours:
curl -s http://localhost:8000/api/v1/settings/export \
-H "Authorization: Bearer $TOKEN" > current.json
jq -s '{format: "maljan-settings/1", values: {
"core.agents.definitions": ((.[0].values["core.agents.definitions"] // {})
+ .[1].values["core.agents.definitions"]),
"core.agents.profiles": ((.[0].values["core.agents.profiles"] // {})
+ .[1].values["core.agents.profiles"])}}' \
current.json docs/examples/profiles/all-tools.json > merged.json
curl -s -X POST http://localhost:8000/api/v1/settings/import \
-H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d @merged.jsonWhat the team needs besides the document: an enabled qu1cksc0pe entry under
core.mcp.servers (the import refuses a reference to a server that does not
exist); Ghidra with core.static.ghidra.enabled true, transport set to
http explicitly (its shipped value is stdio), url the address the worker
reaches it at — http://localhost:8089 for a worker on the host,
http://ghidra-mcp:8089 for one inside the compose network — and auth_token
the container's GHIDRA_MCP_AUTH_TOKEN (see A team that needs Ghidra waits
for it above); core.static.r2.enabled on and r2mcp findable by the worker —
setting core.static.r2.binary_path to its absolute path is the sure way, since
where r2pm put it depends on the environment r2pm ran in (see Where r2mcp is
looked for; switched off or not found, r2 attaches nothing and the clone runs
on its two servers alone); and, so that each static analyst reads a
tool of its own, core.static.provider set to none — with the global
provider on Ghidra the static analyst opens Ghidra as well. Run it by naming it on the job ({"config": {"profile": "all_tools"}}) or by making it core.agents.profile. A test
(tests/api/test_the_all_tools_team_document.py) loads the document through
the import's validation and resolves every agent against stub servers.
What a clone is given when it names no tools
A definition that is not a built-in and has no tools key takes the tool list
of its role's seed: a clone written as {"role": "static", "static_provider": "r2"} — by a script, an import or a hand-edited export — keeps the analysis,
knowledge and VirusTotal servers the static seed reads, as a clone the
console copies does. A tools key that is present is the operator's, and an
empty list means no server at all. The roles with a seed of their own are
static, dynamic, network, report and lead; a generic definition
has none to inherit and keeps what it wrote. The settings API stores the list
it resolved, so what the console shows is what the run reads.
Delegation
A definition's tools list takes five kinds of reference. mcp names a
server (and optionally one tool of it), provider means the agent's own static
provider's tools, sandbox the in-process tools over the job's sandbox report,
capability names a kind of tool rather than a server, and agent names
another definition.
{"kind": "capability", "capability": "reputation"}A capability reference stands for every built-in server that says it serves
that capability: its block's roles name it, or a tool it declares (its
block's, or what it served when last listed) is of that kind. Whether its own
agents list binds it to some agent does not matter, and because the role is
read from the block, a server whose tools were never listed (a fresh or cloned
database keeps nothing) still counts. Each is referenced exactly as an mcp
reference by key: a disabled server gives nothing, and its tick list and
allow-list narrow what it gives. A server you add is never reached through a
capability: bind it by its own agents list or by an mcp reference. The
built-in agents that look the sample's hash up (static, network, the judge,
triage, android_static, reverser) reach reputation this way: with the
shipped servers that is the threat-intel sidecar and VirusTotal's own server; a
built-in row stored with the earlier list that named VirusTotal by key loads as
the seed. A save is never refused for a capability reference while an enabled
built-in server has not been listed yet, since it may serve it once listed: the
save is answered with a warning naming each capability that resolves to no
server yet and the servers still unlisted, and a run records "capability '…'
resolves to no server" among the agent's degradation reasons. An agent
reference:
{"kind": "agent", "agent": "static"}Bound to an agent, that reference is a tool named ask_static in its toolbox:
the model hands the static analyst a task (and, when it helps, a context),
the static analyst works on it with its own tools under the same job, and its
answer — its claims with the ledger ids they cite — comes back as the tool
result, unedited. The console's agent editor offers every other analyst-role
definition under Ask another agent in the Tools tree; the settings model
refuses a reference to an agent that does not exist, to the definition itself,
to the judge or the reporter, and any such reference on the judge or the
reporter. A disabled callee is refused when it is asked, by name, so a built-in
team may keep a disabled member while another team runs, and so is an ask that
would come back with a server the asking stage withholds — a stage with
Built-in tools off cannot reach them through a colleague that no stage
narrows. A provider reference is valid on a generic or a lead definition,
and on either it opens the agent's static provider and binds its tools the same
way; every other role opens its provider itself. A sandbox reference is valid on
any role; the agent editor's Tools tree offers it as the job's sandbox report
tools.
The seeded lead definition (role lead, prompt in
src/maljan/agents/prompts/lead.md) references static, dynamic,
network, reverser and triage, and keeps the knowledge server; the
team_lead team runs it as its one analysis stage. A team of your own may put
a reference on any analyst: a static clone that asks the network analyst is as
legal as a lead.
Three settings govern it, in the Agents group. agents.delegation_depth (2)
bounds the nesting: a stage's agent asking a specialist is depth 1, that
specialist asking another is depth 2, and an ask that would go deeper is
refused with a message the model reads. It bounds the nesting, never the
number of asks.
agents.delegation_steps and agents.delegation_timeout_seconds are what
one ask gets, and both are empty — no limit — unless you set them. They are
the delegation's own budget, not a share of the caller's: an ask carries them
whole, whatever the caller has spent, and the caller's own step budget is not
reduced by what its specialists do. The one thing the two really share is the
wall clock: where the caller's loop has a time limit, the caller waits inside
it, so an ask is cut to what the caller has left and refused only when that is
below the floor a first model turn needs. A caller with no time limit waits
for a busy callee until it frees up, unless that callee is itself waiting on
the caller, which is refused in words the model reads. A callee that reaches a
step limit writes up what it gathered, the way an analyst at its own does.
A budget is part of the definition, so a clone of a team carries the budget
its agents need; the console draws the two as Steps per loop and Seconds
per loop on the agent's card, and a blank box inherits the deployment's
react_agent_max_steps / react_agent_timeout — both empty, no limit, by
default. No built-in definition carries a budget: the seeded lead has none
either. The ask_<key> tool's description says what an ask gets ("no step
limit and no time limit of its own", or the numbers you set) and, where both
the caller and the ask have a time limit, how many asks fit
(delegation._asks_that_fit). The two react_agent_*_overrides maps are
deprecated and operator-only: they ship empty, and an entry you write is still
read for an agent whose definition sets neither, before the deployment's
value; a definition's own value wins over them. A map entry that is not a
whole number of at least one is dropped with a warning when the settings are
built, the same bound the definition's own fields carry. See Delegation in architecture.md for
what the ledger and the transcript record.
A name a later release takes
Seeding a built-in takes a name. triage, android_static, reverser,
lead, mobile, deep_static and team_lead were all legal names for an
operator's own agent or team before they were seeded, and a stored entry under
one of them would otherwise be refused as tampering with a built-in — on every
read, which is to say at boot.
So a stored entry under one of those seven names that is not the seed is renamed
out of the way on load: reverser becomes reverser_custom, and every
reference to it moves with it — the teams that named it, the per-agent model
entry under llm.agents, each server's agents binding and both
react_*_overrides maps. The rename is logged once at warning level, and
alembic upgrade head writes it into the stored document so the console shows
the new name rather than renaming the same document on every read.
A seeded definition whose prompt a later release rewrote is not such an entry.
A save stores the whole definition map, seeds included, so a database holds
each seeded row with the prompt it had on the day of its last save; a prompt
the seed itself shipped with before (FORMER_SEED_PROMPT_DIGESTS in
maljan.core.config, by SHA-256 of the exact text) is read as the seed's, and
the row loads as the seed rather than being renamed. A release that rewrites a
seeded prompt adds the one it replaced to that list.
This applies only to names a release newly reserved. static, dynamic,
network, judge, reporter, default and measurement have been reserved
for as long as there has been a settings store, so a stored document that edits
one of those is still refused, and a name typed into the console after the seed
exists is still refused per field while it is being typed.
A team needs exactly one verdict stage and at most one report stage, which
is always last. A stage may only depend on a stage declared above it, which
makes the card order the run order and a cycle impossible to write down rather
than merely detected. A debate stage needs an analysis stage upstream of it,
and an analysis stage needs at least one agent. An agent belongs to one
analysis stage.
A debate stage hands over to exactly one node. It leaves through a conditional edge, and a conditional edge has one destination per branch, so a debate may not feed two stages — and may not feed a parallel analysis stage with more than one agent, which is two nodes even though it is one stage. A sequential stage of any size is one node and is fine. This is refused when the team is saved, not when the first job builds its graph. A stage with no mode of its own is one node when the team is checked, and runs one after another wherever a debate hands over to it, whatever the job's analyst mode.
A team stored as a plain list of analysts — every team written before stages
existed — is read as the four stages that list has always meant: analysis
(those analysts, in the mode llm.parallel_analysts gives — unset on auto, so
the job decides) → debate (with the round limit and threshold from
negotiation.*) → verdict (the judge) → report
(the reporter). The stored analysts list is kept alongside the stages it
produced; the model reads the stages.
Such a team is marked derived_from_analysts, and while the mark is set and
its stages are still the plain conversion of its analyst list, they are rebuilt
from that list and those two global keys on every load. The mark is checked
rather than believed: it travels in the stored document, so it also arrives
from an import, a script's PATCH or a hand-edited export, and a team whose
stages someone has written is left as written and the mark cleared. That is what keeps a team nobody has opened following
llm.parallel_analysts: an operator who moves from a hosted API back to the
single-slot local model changes one setting and the team follows, instead of
running analysts in parallel forever because it happened to be migrated on a
day when parallel was on. The console clears the mark on the first stage edit
— from then the stages are the operator's, and nothing rewrites them.
Long agent keys in the conversation
An agent key is a slug of at most 32 characters. Everything the live feed
publishes as prose is scrubbed by the publisher, and a run of 24 or more
letters, digits, _ and - is the shape a credential has. A run made of words
— two or more pieces split on _, - or /, each only letters written the
way a word is (all lower case, all capitals, or one capital in front) and each
shorter than 24 — is read as words and travels whole:
"windows_pe_static_reverse_engineer failed" reaches a reader as written. A
key of 24 or more characters with a digit in one of its pieces does not have
that shape and is redacted to *** inside a sentence —
"windows_pe_static_analyst2 failed" reaches a reader as "*** failed".
What the word rule costs, stated: no shape tells a passphrase
(three-plain-words-together) from a hyphenated phrase, nor a letters-only
grouped code (a base32 recovery code, a product key in letter groups) from
words in capitals, so a secret of that shape is published as written unless
something else catches it. Two things do. A run that begins with a known
vendor prefix (glpat-, xoxp-, xoxb-, xapp-, ghs_, ghp_, gho_,
github_pat_, hf_, rk_live_, sk_live_, pk_live_, npm_, gocspx- and
the rest of events._PREFIXED_KEY_FORMATS) with at least 20 characters after
it is a key whatever its body reads like. Mailgun's key- begins ordinary
phrases, so a key- run is a key only when its body is not words: a 32-hex
body is masked, key-derivation-function-parameters is not. And every secret
value the platform holds is masked by exact value wherever the scrub runs,
whatever its shape (events.remember_secret_values, filled from
settings_catalog.configured_secret_values):
- what is collected is a secret value: a
SecretStror a setting the catalogue marks secret, the password inside a service URL, and an entry of a mapping (a tool server'senvor headers, the REST sandbox'ssubmit_fieldsandextra_fields) whose key's last word names a credential —password,passwd,passphrase,pass,secret,key,apikey,token,pat,credential(s),authorization,bearer, optionally followed byvalue(VT_API_KEY,DB_PASSWORD,GITLAB_PAT). A key ending in anything else is a setting about a credential and is not read (AUTH_MODE,TOKEN_TTL,PASSWORD_POLICY,SECRET_MANAGER, andPWD, the shell's working directory); nor is a value that is all digits or a switch word (true, false, yes, no, on, off, none, null), nor the REST sandbox's JSONPath maps (field_names,channels).SECRET_KEY_BASEis missed by this rule, which costs less than masking a setting's word in every report; - a value is masked only where no letter, digit or underscore touches it —
the end of an escape sequence (
\n,\t,\u00a0) in JSON text counts as a boundary:minioadminconfigured leavesminioadministratoras written, and a passphrase after\nin a tool answer is masked; - the values are held per scope, and a scope registered again replaces what it
held. The worker registers its own database, Redis and object-store
credentials and its starting settings under
processwhen it starts, and each job's settings underjobwhen the job installs them, so a secret removed from the settings is not masked in the next job. The command line registers its settings when it builds the app (app); the worker's app leaves that to the job's registration; - a configured value shorter than 8 characters is not masked by value, because masking it everywhere would take the word it spells out of every sentence.
A secret the platform does not hold — one a sample carries, or one a tool answer quotes from elsewhere — of passphrase shape is the remaining cost.
A Windows function name is a name, not a key, though a long one has the
length rule's shape (ZwSetInformationJobObject, InternetGetSecurityInfoByURL).
So are hash-algorithm ids joined by a slash (ror13_module_add/fnv1a32_lower).
Three sets of names travel as written:
- every name the vendored export-name catalogue
(
data/windows_export_names_v1.json) holds; - every hash-algorithm id of the vendored algorithm catalogue
(
data/api_hash_algorithms_v1.json); - every name this job's hash resolution read. The resolution is the pack's
resolve_api_hashescall or the analysis server's tool; another server's tool of that name adds nothing. A name is taken only when it is an identifier with both cases in it, and the set is forgotten when the next job installs its settings.
A name travels alone, or joined to other such names by /, |, + or &. A
module in front of a name (kernel32.dll!Name) is split off at the !, so the
module and the name are read apart.
What a tool or a model writes beside those names travels as written too, each by its exact form:
- in a list joined by
/,|,+or&: a module name the export-name catalogue lists, in any case (kernel32/WinINet/SHELL32), an address or a decimal number of at most eight digits (0x1a20/0x1a28, a list of process ids), and an abbreviation that completes the name before it to a catalogue name (FindFirstFileA/W,NtQueryInformationProcess/Thread); - a stretch of 24 or more characters of the standard or URL-safe base64 alphabet itself, the table a decoder is built from, when it is the whole run or the whole stretch: beside other pieces it is read as before, so a key next to it is masked with it.
Hex is read as it always was: a run with eleven or more hex digits in a row is
never one of these forms. A hex dump is masked in the events wherever it
stands, a tool's own "hex" field (the bytes a memory or byte-range read
returned) included, and so is a digest inside a list or a name that ends in
long hex (a decompiler's in_stack_… variable); a digest alone travels, as
before.
A piece that is none of these keeps the whole run a key, so a key inside such a list is still masked with its stretch.
A run that a character outside base64 splits (hello.<key>, name@<key>)
is also read stretch by stretch for two shapes no length alone decides: a
vendor prefix with a key's body after it, and a lowercase key body (24 or more
lowercase letters and digits with at least one digit and four letters past
f, so no hex run is one). Either opens the run to the rules above, and the
key's stretch is masked; a separator line, a dump's file name or a scanner's
label behind a dot is read as before. A token behind other text in one run
(x_<token>) is masked when its header opens with a quoted key ({"), so a
dotted name whose head merely decodes to { is no token.
The events and the stored transcript stay stricter than the report on purpose: they mask every credential shape above and every configured value, while report text (finding rows and the run record) masks the configured values and a URL's credentials only, and keeps a credential-shaped string the evidence holds as the evidence wrote it. Some names still read as keys in the events (a family-name list with a digit or an all-capitals piece, a long identifier no catalogue lists, a hex dump); that is the accepted cost.
A key is still a key:
- after a module and a
!; - joined to any text by
/,|,+or&, or with a token inside the run: the key is masked together with the whole stretch of base64 and base64url characters (A-Za-z0-9+/_-) around it, so none of its fragments travels. Any other character (a dot,%,|,&,!,:, a space) ends the stretch:example.com/gate/<key>/x.phpreadsexample.***.php, andhost.example|<key>|x.phpreadshost.example|***|x.php; - followed by base64 padding: the stretch of base64 characters the run ends
with, when it is 24 characters or more and no name the scrub keeps, is
masked, even when the key's own
/and+cut it into short fragments or it begins with a slash as a path does. Padding is one or two=followed by the end of the text or by a character no value starts with: whitespace, a closing quote, bracket, brace or tag,,,;,:,., or a joiner (/,|,+,&,!). One or two=followed by anything else (a letter, a digit,-,_, an opening quote) are an assignment, not padding, unless the stretch and its=signs together are a multiple of 4 characters long, as a base64 value is; - after a word, in the shape of a MIME type. Only a registered top-level type
(
application,text,imageand the rest, or anx-type) with a subtype that is no key is kept as a MIME type; - after a vendor prefix, which is still asked first;
- as a configured value, which is still masked by value before any rule is read.
None of these sets lets a credential through. Two costs follow from masking the stretch, and both are accepted, because the events and the transcript are not what the analysts read and a key fragment costs more than a directory name:
- the readable text in the same stretch as a key is masked with it:
samples/extracted/<key>/payload.binreads***.bin, and a directory or host label next to the key goes too; - a readable segment of 24 or more characters that is no word and no catalogue name (a random directory or file name, a long method name) reads as a key by its shape, and is masked with its stretch.
A run that reads as a key as a whole is masked whole, as before.
The names the scrub keeps (words, catalogue names, the platform's own variable
names, digests, identifiers and MIME types, alone or joined by /; catalogue
names also joined by |, + or &) are asked before the padding rule, so a
kept name stays readable in front of an assignment: ZwSetInformationJobObject=1,
GHIDRA_CONTAINER_SAMPLES_PATH=/x and an argument summary's
anti_debugging_techniques_seen=3 travel as written. Words are never joined by
+, so a key that + cuts into letter-only pieces is still masked.
A path's shape is not asked before the padding rule, because a key can begin
with a slash; a path is still kept when no padding follows it.
Nothing is lost but the name in that sentence. The identity fields a line is
filed under — speaker, agent, stage, label, display_name — are
exempt by name in the publisher and travel whole, so the console still files
the line under the right participant and still draws it with the label you
gave it. No shipped key is anywhere near the floor; the longest is
android_static, at fourteen.
The scrubber is deliberately not told the roster. It is one pure function shared by every job on the worker, and a rule that depended on which run was publishing would be a rule whose answer changed with the configuration — which is the property a redaction rule cannot have. A shorter key costs nothing and reads better in the conversation.
Conditions
when is an expression in a small language evaluated on the worker. It is
Python's own grammar with an allow-list on top: comparisons (==, !=, in,
not in, <, <=, >, >=), and, or, not, literals, and tuples or
lists of literals. There are no function calls, no arithmetic, no
comprehensions and no attribute access except into stages and triage. A condition that
does not parse is refused when the team is saved; one that fails at run time
skips its stage with the reason recorded rather than failing the job. The
console checks each condition box against the same parser as it loses focus
(POST /api/v1/settings/validate-condition), so a typo is answered next to the
box it was typed into rather than when the whole team is applied.
The names it may use:
| Name | Meaning |
|---|---|
file_type | The detected type, e.g. PE32 executable. |
platform | The canonical platform, e.g. windows, linux, android. |
mime | The sandbox report's media type, when there is one. |
size | Size in bytes, when the sandbox report carries it. |
extension | The submitted file name's extension, lowercased, without the dot. |
sandbox_available | Whether a sandbox report reached this run. |
has_sandbox_report | The same fact, named for readability. |
has_pcap | Whether the report carries a non-empty network block. |
stages.<key>.<field> | A stage result: ran, reason, claim_count, technique_ids, finding_count, agents. |
triage.<field> | What the triage pack and the run so far established: has_signature, reputation_malicious (a count, or None when no lookup answered with one), rule_hits (how many rules matched in every answer of a rule-matching kind recorded so far — capability_rules, signature_rules, event_rules — the pack's and earlier stages' alike, whichever server answered; an undeclared tool's matches read by their shape never count here), and yara_hits and capa_hits, kept for the teams that read them: the pack's own signature-rule and capability-rule counts. |
triage.hits["<key>"] | The same count for one key: an answer's declared kind (signature_rules), the name its tool declares for corroboration (source, such as yara), or the key of the tool server that answered (acmescan). rule_matches_by_shape alone counts the distinct rules a tool that declares no kind names as matched by its answer's shape: a shape names no kind and reads an indicator list or a technique catalogue as it reads rules, so these count under no other key and not toward rule_hits. A key nothing counted reads as 0; the key is written as a literal. |
stages["triage"].ran is the same lookup as stages.triage.ran. A stage the
run never reached reads as one that did not run, so naming a stage that was
itself skipped is not an error. A team without a triage pack reads triage
as nothing established: false, None, and every count 0.
Examples:
platform == "windows"
extension in ("apk", "dex")
has_pcap and stages.triage.claim_count > 0
"T1055" in stages.static.technique_ids
not stages.detonate.ran
size > 10485760
triage.rule_hits > 0
triage.hits["signature_rules"] > 0 or triage.hits["capability_rules"] > 0
triage.hits["acmescan"] > 0
not triage.has_signature and triage.reputation_malicious != None and triage.reputation_malicious > 0What a stage reads
inject_upstream decides what a stage is told about the stages it depends on.
none tells it nothing, which is what the default team uses — its analysts
have never seen each other's work before the debate. findings gives it each
upstream agent's claims with their technique, confidence and evidence id.
full adds each upstream agent's prose report. Both are capped by
core.reporting.upstream_findings_max_chars, and a block that is cut says so.
It ships at 0, which derives the cap from the window this job's models
serve the way a tool answer's cap is: the share one answer may take (an
eighth) of what the window leaves after the reply room, at three characters a
token — 294,912 characters on a 1,048,576-token window with a 262,144-token
reply room. A window nothing reported derives nothing, and the documented
6,000 applies. A positive value is the operator's, used whatever the window.
The block arrives as an upstream_findings field inside the stage's first
chunk when that chunk is a JSON document, and in front of it when it is not. A
static or generic agent's first chunk is JSON with a contract on it — the
container-visible analysis_file_path is read back out of it, and putting
prose in front would leave the agent inventing a path again.
Injection never changes whether a stage has data. An agent whose loaders produced nothing but a "no data available" placeholder is still skipped, with or without a block to read.
Two blocks every agent reads regardless of inject_upstream: the triage pack,
one line per fact with its ledger id, at the head of the agent's first human
turn under Facts established before analysis, cut at the same
core.reporting.upstream_findings_max_chars; and the run-state block in the
system turn — sample, identity, signature, reputation, stages run or skipped,
ledger count, failed tools, remaining steps and seconds — regenerated on every
turn. Neither is a setting of the stage: a team without a triage stage has no
pack and its agents see only the run state. See The triage pack in
architecture.md.
Which slice of the job an agent reads is its own setting,
agents.definitions.<key>.data_sources. Empty means the slice the agent's
role has always read: the parsed sample for a static analyst without a
sandbox report and the report's target block with one, the behaviour log for
a dynamic analyst, the network block for a network analyst, and the sample plus
the whole report for a generic one. A non-empty list is taken literally and in
order, which is the point — a clone of the static analyst can be pointed at the
network block without becoming a network analyst:
| Source | What it contributes |
|---|---|
sample.path | The container-visible path the agent's tools open the sample at. |
sample.chunks | The parsed sample profile. |
sandbox.target | The sandbox report's target block. |
sandbox.behavior | The behaviour log, through the dynamic parser. |
sandbox.network | The network block, through the network parser. |
sandbox.full | The whole sandbox report. |
The console's agent editor shows the list as Input data, one box per
source the settings schema serves (data_source_choices on the agent
definitions' entry). A ticked source is appended, so the list is read in the
order the boxes were ticked, and the editor prints that order; a list written
by a JSON import keeps its order and every source through any other edit.
A slice handed over as JSON (sandbox.target, sandbox.full, and the network
or behaviour block where no parser reads it) is compact JSON — the same
document without indentation; the report the run stores is unchanged.
Where a source is cut when it does not fit one chunk is declared per source in
agents.definitions.<key>.chunk_boundary, a map from a source in data_sources
to a regular expression for the start of a unit (a function's header, a
process's banner). The source is cut before each match and whole units are
packed into a chunk, as the built-in boundaries of the static, dynamic and
network domains do; a unit larger than a chunk is cut by a window. A pattern
that RE2 cannot compile (a lookaround, a back-reference), that matches the empty
text, or that names a source the agent does not list is refused. A source with no entry is cut as before, and a
definition with none stores and dumps as it always did.
A source with nothing behind it contributes nothing — an agent that asked for the network block on a sample nobody detonated has no network block, and the analyst node reports that once as a no-data stage rather than once per source.
Whether an agent is asked for an ATT&CK technique line is its own setting too,
agents.definitions.<key>.attck_techniques, on by default. On is the claim
format every analyst has always been given: each claim ends with
TECHNIQUE: <T-ID, several separated by commas, or NONE>. A generic or
lead definition whose work is not malware behaviour (a code reviewer, a
configuration checker, a domain checker) may turn it off. Every text that asks
the agent for a technique then leaves the request out and asks for the CLAIM,
EVIDENCE and CONFIDENCE lines only: its claim instruction, its negotiation round,
a task a lead hands it, the validation questions about its answer, and the view
and tier prompts that replace its prompt when llm.view_decomposition_views is 2
or more (either llm.view_decomposition_mode). An agent reading a static provider is also sent the provider's text
without the platform's sentence naming techniques to look for, and the
priority-functions hint without its clause about picking a technique. Off is
refused on every other role, which keeps its own claim format, and the built-in
definitions keep it on.
Texts that name a technique without asking for one stay as they are: a question about a technique id the agent wrote, a provider's verification discipline that limits when a technique may be claimed, a provider's own text worded in its own words, and the evidence itself.
Turning the line off never hides a technique the agent states. A TECHNIQUE line
it writes anyway is read and checked exactly as with the line on: an unreadable
line is asked about and an unknown id is flagged. A claim with no TECHNIQUE line
raises no question either way. Corroboration, the judge and the capability
matrix are unchanged; an agent with the line off contributes no asserted
technique unless it writes one. A definition that keeps the line on is stored
and exported without the field, as every definition written before it was. The
console's agent editor shows the switch as Ask for an ATT&CK technique line on
each claim, locked on every built-in definition (the served
builtin_agent_keys).
Tool servers on another host
Every path-taking tool assumes the server can open the path the worker hands it. That holds for a local stdio sidecar and for nothing else. There are two ways to make it hold elsewhere.
A shared volume. Mount the worker's samples directory into the server, and
declare where on the server's block: sample_mount is the path inside the
server it is mounted at, and work_dir the directory under it the worker
copies the sample into (0o700 directory, 0o600 file, removed when the job
ends). The server is handed <sample_mount>/<work_dir>/<file>. Nothing is
uploaded; the mount in your compose file and the declared sample_mount have
to agree.
The put_sample convention. A server reached over HTTP advertises
put_sample on its manifest, and Malspark uploads the sample to it before the
agent's first tool call:
| tool | arguments | returns |
|---|---|---|
put_sample | filename, content_b64, sha256 | {"path": ...} |
put_sample_begin | filename, sha256, size | {"upload_id": ...} |
put_sample_chunk | upload_id, seq, content_b64 | {"seq": ...} |
put_sample_finish | upload_id | {"path": ...} |
Samples over 8 MiB go through the three chunked calls when the manifest carries
all of them, and through the single call otherwise. Chunks are keyed by seq
rather than streamed, so a transport that retries one cannot corrupt the file.
The returned path is what that server's tools are then called with —
agents.tool_pinning.pin_paths substitutes per server, so one agent can hold a
local sidecar's tools and a remote server's at the same time and each gets the
path it can open. Uploads are cached per (server, sha256) for half an hour.
The transport decides, not the manifest. Staging runs for http,
streamable-http and sse transports only. A stdio sidecar is a child process
of the worker reading the same filesystem, so it is handed the path: uploading
to it would write a second copy of the sample — a full read plus base64 in the
worker's memory and malware bytes accumulating on disk — to tell the server
about a file it can already open.
Staging never fails a run. An upload that goes wrong is recorded as
sample staging failed for '<server>': <reason> on the run's degradation
reasons, and the server is called with the local path exactly as before. The
paths that were used are recorded on the run under remote_sample_paths, which
stays empty on a default install because every built-in server is stdio.
The path substitution is per server and matches three spellings of the sample:
the worker path in full, its basename, and the basename of the staged copy. A
server that stored the sample under a name of its own — the analysis sidecar
prefixes the digest — is therefore still corrected when the model repeats the
name the prompt showed it.
The built-in analysis sidecar implements put_sample* even though it ships as
a stdio server, because an operator may run that same file behind an HTTP
transport on another host. Two environment variables configure that, and with
MALJAN_SAMPLE_ROOTS — defined once in the next section, and seen by the
network sidecar as well — they are the only ones it is allowed to see:
| variable | default | meaning |
|---|---|---|
MALJAN_STAGING_DIR | a maljan-analysis-mcp directory under the system temp dir | the base uploads land under |
MALJAN_STAGING_TTL_HOURS | 24 | how long a staged sample, and a payload carved out of one, is kept; 0 disables pruning |
MALJAN_STAGING_DIR is the base, and each job writes into one directory of
its own inside it — job-<the job's id>, holding that job's uploads and its
carved/<sha256>/ trees. The name is composed by the process that starts the
sidecar and handed over as a third variable, MALJAN_STAGING_JOB; it is a
single directory name, never a path, so an operator's MALJAN_STAGING_DIR is
the base whatever else is configured. That variable is not read from the
environment and is not something to set: a sidecar started without one writes
into the base itself, which is what a settings probe and a server run by hand
do.
Both the base and the job directory are created with mode 0o700 and refused if
what is already at that path is a symlink or belongs to another user — the
default name is predictable and the system temp directory is shared. Each file
is created with O_CREAT|O_EXCL|O_NOFOLLOW at 0o600 rather than written and
then chmodded, and every put_sample* call prunes past the TTL: the files of a
job directory still in use, the whole directory of one whose newest file is
past the cutoff, and the flat files an older release left in the base. So a
long-lived server accumulates neither samples nor job directories, and an
upgrade has nothing to migrate.
A job directory is pruned whole only once the newest file anywhere inside it is past the cutoff, and a running job keeps its own directory current while it refreshes its owner heartbeat — so a run longer than the TTL does not lose its carved payloads to a second worker's sidecar sweeping the same base. Set the TTL below the longest run this deployment can have and that marker is the only thing standing between a live job and its own directory; there is no reason to.
The sandbox capture a job fetches lands in a captures/ child of the same
directory and obeys every rule above: 0700, files 0600 from their first byte,
removed with the job, swept by the same TTL, and unreachable from another job.
The directory an earlier release used, maljan-cape-pcap under the system temp
directory, is swept as well and is no longer written.
One configuration would widen this if nothing else stopped it: a
MALJAN_SAMPLE_ROOTS entry containing the staging base — the deployment's
whole samples directory, say, with the base inside it — which names every
job's directory as one a sidecar may read. confined_to_this_job refuses it
anyway: a path argument resolving under the base but outside this job's own
directory is refused whatever the roots say, so the job directory is the
boundary and the roots cannot loosen it.
Which directories a sidecar may read
A sample is adversary-authored content and the analyst model reads it, so the
path a tool is asked for is a path the sample's author may have written. Both
file-reading sidecars — analysis and network — therefore resolve every
path, pcap_path and ruleset argument (symlinks followed) and refuse
anything that lands outside the directories they were given:
- this job's staging directory, where their own uploads land, and
- every directory in
MALJAN_SAMPLE_ROOTS.
This job's, not the base: a path that resolves into another job's staging directory is refused even when a sample root happens to contain the base, so the job directory is the boundary whatever the roots are configured as.
A capture is named relative to the job's own directory wherever a model sees
it (captures/<file>), never by host path. When the job has exactly one
capture, pcap_path is hidden from the schema the built-in servers' tools are
bound with and filled in by the platform, the way the sample's own path is
(agents.tool_pinning); with several it stays the model's to give, a relative
value is read inside the job's directory, and a refusal lists the job's
captures by those names — or says there is none — instead of advising a caller
to leave out an argument the tool requires. A filled-in capture the server
cannot read comes back as that failure, naming the capture by its job-relative
name and saying the platform filled it in. Every capture tool reads the whole
capture as a stream and states the packets it read and the packets in the
capture; packet_limit has no default and applies only when a caller passes
it, and read_pcap_summary with none answers the capture's facts rather than
a line per packet.
carved_path on the analysis sidecar is narrower than both, because it is
the one file argument a model chooses rather than the platform: it is held to
<staging>/job-<id>/carved/<sha256 of the file the call is pinned to>/ and to
that file itself, so a run reaches the payloads it carved and nothing another
run carved or uploaded — two jobs on the same sample carve into two directories
and neither can name the other's. The resolved value must be a regular file; a
directory, a FIFO, a device or a socket is refused.
| variable | default | meaning | seen by |
|---|---|---|---|
MALJAN_SAMPLE_ROOTS | empty | the directories a path, pcap_path or delivered sample may be read in, separated by : | analysis, network |
MALJAN_STAGING_DIR | a maljan-analysis-mcp directory under the system temp dir | where a delivered sample lands, and a root for both | analysis, network |
Those two, plus MALJAN_STAGING_TTL_HOURS above, are the whole of what the
analysis sidecar's env_allow carries; the network sidecar's carries the
two in this table and nothing else. Neither is started with a credential in its
environment. That holds for the environment alone: each runs as the worker's
own user, in the worker's container, on its filesystem and its network, so what
that user can read there, a sidecar can read too.
How the variable reaches a sidecar. A stdio child is started with a built
environment rather than the worker's own: the general-purpose names (PATH,
HOME, the locale and temp ones), then exactly the names that server's
env_allow lists, then its env map. MALJAN_SAMPLE_ROOTS is therefore named
on the two file-reading built-ins and on no others — a variable that says where
this host keeps malware is not something every child Malspark starts has any
business reading. A tool server an operator adds receives it only when they
put the name in its own env_allow, which is the same switch a server of
theirs that takes paths would need anyway.
The environment is copied into the child when it is spawned, so the roots have to be complete before a job's first sidecar starts — and they are: the worker exports its download directory and sample mirrors at startup, the mirror step and a sandbox capture fetch name theirs while the run is still assembling its inputs, and a run that was handed a sample path names that file's directory before the pipeline builds. A job's servers are attached after all of it, and a sidecar held over from an earlier job is closed and started again for the new job, so nothing has to be restarted mid-run for a root to take effect.
The platform's own variables never reach a server. The bootstrap variables
that open the platform's stores — the database and Redis URLs, the object
store's keys, the settings key and the JWT signing keys, and the names the
shipped Compose file writes those values under (the list is in
The bootstrap contract) — are never among a child's
names. A server block that names one in env_allow or as a key of env, in
any case, is refused when it is saved, with the variable and what it opens and
never its value; a stored one fails every job that loads it, naming the
setting, before any server starts; the connection test reports it and starts
nothing. The spawn drops such a name whatever a block says, for a block that
reached it some other way, and logs a warning naming the variable and the
server. A model provider's key is not one of these: it is an application
setting, and an operator may list it in the env_allow of a server that needs
it.
The names a built-in always gets. Three names are not an operator's to
take away: MALJAN_SAMPLE_ROOTS and MALJAN_STAGING_DIR on analysis and
network, and MALJAN_STAGING_TTL_HOURS on analysis. They are what a
sidecar cannot work out for itself — which directories it may read, where a
delivered sample lands and how long it is kept — so they are put back on load,
on save, in the Configuration tab's own view and in the connection test. The
tab draws them above the box as names that are always passed, so a deletion is
never accepted and then quietly undone; what stays editable there is the rest
of the list. The registry is stored as one row holding every server, written
whole whenever
anything in it is saved, so without that floor a deployment that had configured
its servers
before a sidecar gained a variable would keep starting that sidecar without
it — which for MALJAN_SAMPLE_ROOTS means every tool call on the run's own
sample refused with path_outside_roots.
Every other name a built-in ships with is a default rather than a floor:
threatintel's VIRUSTOTAL_API_KEY and ABUSEIPDB_API_KEY are the
deployment's own credentials, and an env_allow an admin empties stays empty
everywhere that list is read. A name an admin adds to a built-in is kept, after
the required ones. A server an operator added is left exactly as they typed it,
required names and all: it reads the sample roots only when its own env_allow
names them.
A refusal is the ordinary structured error with the code path_outside_roots
and a remedy, and it names no host path.
The worker fills MALJAN_SAMPLE_ROOTS in for itself: its download directory
(UPLOAD_TEMP_DIR), the sample mirrors under SAMPLES_DIR and the directory a
sandbox capture is fetched to are exported before any sidecar starts, so a
default deployment needs no configuration. Set the variable when a sample lives
somewhere the worker did not put it — a corpus directory an operator points the
CLI at, or an HTTP sidecar on another host that is handed paths rather than
uploads. ruleset is held to the rule corpora instead: the repository's data
tree and whatever MALJAN_YARA_RULES_DIR and MALJAN_SIGMA_RULES_DIR name.
A server whose open resets its analysis
Some analysis servers reset what they have analysed when their "open the
sample" tool is called again: Ghidra's load_program imports the file afresh in
the place of the analysed program, and a disassembler's open may drop its
analysis the same way. A model that opens the sample before each question then
reads a program with no call or code reference. Each server's settings block,
built-in or added, may declare which of its tools opens a sample under
sample_open, and nothing in the platform names a server or a tool:
| key | meaning |
|---|---|
open_tool | the tool that opens a sample; empty declares nothing |
path_argument | the argument the path goes in (required with open_tool) |
name_field | the field of the open's JSON answer that names what was opened; empty means the path |
select_tool | the tool that selects a sample already open (optional) |
select_argument | the argument the opened name goes in |
analyse_tool | the tool that runs or awaits the server's analysis after the first open (optional) |
analyse_argument | the argument the opened name goes in; empty calls it with none |
program_arguments | further arguments a tool names an opened program by (a diff's two programs, say) |
open_path_arguments | further arguments that name a file a tool opens, beside path_argument |
job_safe_tools | tools that take no program or path argument and read nothing a job opened |
shared_withheld_tools | tools withheld on a shared server whatever they take (ones that run code or change the whole project) |
close_tool | the tool that closes or deletes an opened program; declared, each program the job opened is closed when the job ends, otherwise it stays on the server |
close_argument | the argument the program's name goes in (declared together with close_tool) |
On a server other jobs share (http, streamable-http, sse), the server's
current program is every job's, so each call is held to the job's own. Every
program argument a call carries (select_argument, analyse_argument and
program_arguments) must name a program this job opened there, and one the
tool takes that the call leaves out is given the program the job opened or
selected last; every path argument (path_argument and open_path_arguments)
may only name the job's own copy of the sample. Anything else is refused before
the server is asked (foreign_program), as is a call naming no program before
the job opened one (no_job_program). A tool whose input schema takes none of
these arguments, or cannot be read, is withheld with its reason on the run's
degraded record unless job_safe_tools lists it, and a tool
shared_withheld_tools lists is withheld whatever it takes. A tool withheld
by either list is the operator's configuration rather than a loss: it is
listed under "Not configured or withheld by choice" (operator_choices) and
never draws the degraded-run banner. One withheld because its schema could
not be read stays a degradation. The arguments
checked are the ones a call carries, so one its schema does not declare is held
to the job too. On a server no other job shares, the tools are offered as they
are.
With open_tool set, the platform keeps a note per job and per server of each
path once it has been opened, and analysed where analyse_tool is named. A
later open of the same path, by a model or by one of the platform's own passes,
is answered from the note with the first open's own answer and logged as such;
where select_tool is named it is sent first, and only a select the server
refuses (it restarted, the program is gone) opens and analyses the path afresh
and writes the note again. An open the server refused, or an analysis it did
not confirm, is not noted, so the next open tries again; a select refused right
after a fresh open is handed on as a failed open, the select's refusal with the
open's own answer beside it (on Ghidra, the loop stops with a sentence saying
Ghidra refused to make the loaded sample current, and the next loop loads it
again: nothing is remembered for the job). A lock per path makes two first opens at once
import once. The select and analysis tools are called even when the model is
not offered them, and a declaration naming a tool the server does not offer is
logged and leaves the open tool as it is.
The note belongs to one job and is never shared with another. Within the job it
is kept per server, not per handle: every handle to a remote server (http,
streamable-http, sse) shares it, the server named by its scheme, host, port
and path with the host lower-cased and the default port written, so two
spellings of one address are one server. localhost, 127.0.0.1 and ::1 are
kept apart, since three listeners may answer them (at most one more open). What
may select another backend behind one address (the query, the userinfo, the
credential sent) adds an opaque number to the key, never the value or a digest
of it, since keys reach the log; a local
(stdio) server is a process per handle and keeps its own, each process named
by a number never reused. A note is also keyed by the open tool's other
arguments, so an open with other options (a loader, a base address) is another
note. When an open is noted under a name the server gave, any other note of the
same server under that name is dropped: a server that names programs by file
name replaced it. A forwarded open the server answered drops the server's notes
under the name its answer gives, or all of the server's notes when it gives
none, for the same reason.
A path is read once. The opener reads the path it is asked to open in one
way, and that one spelling is both the note's key and the argument every
declared tool is sent. Nothing is tidied that the server might read
differently, and nothing is refused either: a path with a backslash, a percent
sign (an encoding), a control character, surrounding blanks, a doubled
separator, a . or .. segment or a trailing slash, or a relative path, is
forwarded to the server exactly as written and never noted, so no note can be
borrowed or poisoned through it. The job's own copy of the sample on that
server (the mirror a static provider pins) is platform data: it is compared
exactly, whatever characters it holds, a relative name is read against it only
where the job holds one, and any other path (another file, a carved payload) is
forwarded and not noted. Nothing read from a tool's answer, the name_field
included, decides which note answers: it is what the select and analysis tools
are sent, and the name under which an older note of the same server is dropped.
The platform's own copies of a sample are named by its SHA-256 plus the
upload's suffix only when that suffix is plain ([A-Za-z0-9._-]), so every
path a tool is handed is plain.
The declared open and analysis tools are held as long-running, exactly as a
tool the server's own capabilities manifest marks long_running: an import
of a large file and its analysis take as long as they take, so they get no
client deadline unless mcp.breaker.call_timeout_seconds is set. A deadline
would only cut a call the server is still working on, the open would not be
noted, and every later open would import and analyse again; an operator who
sets that budget accepts that a cut analysis is opened again by the next open.
Any agent given the open tool is told to open first. A server that declares
sample_open (declared, accepted from a proposal, or bound to an agent by the
operator through agents) marks the open tool it offers with its declaration.
A generic agent offered that tool is told, on the request that carries the
sample path, LOAD THIS BINARY FIRST: call open_tool(path_argument="<path>"),
the line the static role gets from its provider; the static role reads an
operator-bound server's mark when its own provider declares none. An agent
whose provider tools open the sample before the loop is not told twice. A
connection test proposes sample_open (low confidence, open_tool and
path_argument only) when exactly one tool takes a path as its only
required argument and shows it opens the sample for analysis: either the
server marks it read-only (readOnlyHint true) and its name or description
says it opens, loads, imports or analyses, or the server gives no annotations
and its description (never its name alone) says it opens, loads or imports a
binary, an executable, a sample or an image, or a program or a file for
analysis. A tool marked destructive or not read-only, one whose path argument
names an output, and one whose name, description or annotation title names a
changing verb (write, delete, export, save, patch, apply, update, edit,
install, run, …), a directory, or another object (a configuration, settings,
a plugin, a URL) is never proposed. At least two other tools
share a required argument that is neither a path nor a place in code;
accepting it in the console fills only the fields the server leaves empty.
Declared tools must refuse in a structured way. A refusal is read by the
platform's one rule for failed tool answers: an MCP isError result, a
{"error": …} answer, or an HTTP error. A refusal written as plain text
("Program not found") reads as success, so a refused select would be answered
from the note while the server holds nothing.
Ghidra's block ships declaring load_program (path argument file, name field
program), switch_program (argument program) and run_analysis (argument
program); its client applies the same long-running rule to them. An
analyse_tool may be named with no open_tool: the tool the server's analysis
runs through, which the platform then never calls itself (the agent does); it
is read as the server's analysis step (its kind, and the block's
analyse_effects), and nothing else of sample_open applies. radare2's block
ships sample_open: {analyse_tool: analyze} (config.R2_SAMPLE_OPEN), its
provider opening the sample itself. A
stored Ghidra block that never set sample_open carries that declaration; one
an operator set, empty included, is kept as set. The fields are in the settings
schema and catalogue, and in the server editor's Declaration section.
What a server declares about its tools
sample_open is one part of a server's declaration: the facts the platform
needs about a server and its tools, written as data on the server's own block
rather than in code against a tool's name, so a server you plug in is read
exactly like a built-in one. Every block has the same fields: an mcp.servers
entry, the static.ghidra and static.r2 provider blocks and the CAPE MCP
block (sandbox.cape2.mcp). In the console they are the Declaration section
of a server card (closed by default), and in the catalogue the advanced
Declaration subgroup of each provider block. The consumers move onto it one
by one, each with the same output for the built-ins. Read today: sample_open;
the worker reads sample_mount and work_dir (below); error_patterns, for
every server; prompt_fragment by the prompts (a static provider's block after
its claim guidance, any other server after the sentence about tools);
reads_staged_sample by the path guard, which hides the sample argument of
such a server's tools from the model and fills it, and fills its capture
argument when the job has one capture; each tool's long_running and
timeout_seconds by the server guard of a registry server (an mcp.servers
entry), as a manifest's own (the larger budget wins); a static provider's own
client does not read them;
auto_name_prefixes wherever a function's auto-name is read as its address,
and with data_label_prefixes wherever the report reads a claim's code
locations;
reference_urls by the report's references; and each tool's kind, reader,
asserts_techniques, source, techniques, rows, places, values,
words, display_name, category and arguments by the evidence summary, the
evidence roots, validation, the function map, the triage pack's function index
and rule check, the stage conditions, the MCP client's description tags and the
run summary's rule layers (see Who reads a tool's kind below).
Where a server sees the sample. The worker copies the job's sample into
work_dir under SAMPLES_DIR (the shared .work when a block names none;
radare2's block ships r2-work, as it may not be hidden) and hands the server the copy under
its sample_mount: <sample_mount>/<work_dir>/<sha256><suffix>, or the copy's
host path when sample_mount is empty. Each server has its own, so two
containerised servers mounted at different paths are each handed their own
path. A server that declares a sample_mount is copied the sample whether a
static provider drives it or an agent of the job (one the profile's analysts
can reach, or the judge or reporter) binds it by its agents list or a tool
reference, and the agent's path arguments are held to that copy; a server the
sample was uploaded to (put_sample) keeps the path its upload answered. A
server no agent of the job binds is not copied. Two jobs on one sample share
the copy's name, so the first job to end removes it while the second job's
server may still be reading it, as with a static provider's copy. Ghidra's block ships /data/samples, the mount the shipped compose file
writes. A generic_mcp server reached over http, sse or streamable-http that
declares no mount is taken to see the samples directory at /data/samples as
well; one over stdio reads the worker's filesystem. The directories the worker
sweeps of stale copies, the ones a sample's delete clears and the sample roots
it hands the sidecars are the shipped mirror directories (.work, r2-work)
and the mirror directory of each static provider the settings select and
switch on, so a server you add is cleaned up like a built-in. A directory is
used only when it is a plain child of the samples directory, never a symbolic
link, that the worker created (it writes a .maljan-mirror marker into it) or
a shipped one: a work_dir naming a folder of your own corpus is refused, the
job's copy for that server is not made, and nothing in the folder is changed.
Only files named as the worker names a copy (the SHA-256 and a plain suffix)
are ever removed.
| key | meaning |
|---|---|
sample_open | which tool opens a sample, and which select and analyse it (above) |
sample_mount | the absolute path inside the server that the worker's copy of the sample is mounted at (no ..; written plainly); empty when the server reads the worker's own filesystem |
work_dir | the directory under the worker's samples directory the sample is copied into for this server (one name); empty when it makes no copy of its own |
error_patterns | how the server's answers say "error" (below) |
roles | what the server is for: reputation, decompiler, disassembler, sandbox, rule_scanner, strings, reference, capture, or any word of your own |
prompt_fragment | the workflow text shown to an agent only while the server's tools are in its request: after the sentence about tools for a server of the registry, and, for a static provider's block, after the provider's opening sentence and the claim guidance every static analyst is given (the built-in Ghidra and radare2 blocks and the CAPE MCP block ship their workflows here). A sentence naming another server's tool is written {{attached:<server key>|<text while that server's tools are in the request>|<text otherwise>}}, in any block's fragment (a registry server's, a static provider's or the CAPE MCP block's): the key is a server's in mcp.servers (never provider, sandbox or team), neither text holds |, { or }, and no part sits inside another; a fragment whose part does not read is refused when it is saved. Ghidra's names the analysis server's crypto-constant tool so. A Ghidra or radare2 block that stored the whole fragment of an earlier release (opening with the provider's old lead sentence), which no prompt read, is read as not set |
required_env_allow | the environment names the server cannot run without |
variable_names | environment variable names of the server's that are not secrets; those a shipped declaration or a registered provider class (variable_names) names are kept whole in published text, read from those so the redaction does not depend on a run's settings |
reads_staged_sample | whether the server's path-taking tools read the sample the platform staged |
platform_owned | whether the server is the platform's own, whose answers are the platform's code |
trusts_served_declaration | whether what the server declares of its own tools when it lists them (each tool's _meta at maljan/tool_facts) is read as this block's declaration, under what the block itself says; off, only its cautious claims are read, as the server's own statement, and the rest is proposed (see Writing a tool server). Shipped on for the platform's own sidecars, and that shipped trust holds only while the block still starts or reaches the server it shipped with (its transport, command, arguments, working directory and address); a block pointed at another server trusts it only where you set the flag yourself, so set it when you point a built-in block at that sidecar's own container: the platform ships no table of its sidecars' tools, and an untrusted block reads only their cautious claims. The console's server editor shows the trust in force (a built-in pointed elsewhere shows it off), and a tick there is recorded with the launch it was given for (trust_granted_for, a digest of the transport, command, arguments, working directory and address as written, so a release changes nothing of it): the trust holds while the block points there, and pointing the block at another server withdraws it on save, which the editor says before you save. A shipped value sent back is never read as your tick |
auto_name_prefixes | the prefixes its decompiler writes before an address when it names a function itself (FUN_, fcn.); every server's join the shipped ones (FUN_, fcn., sub_, func., loc., LAB_, SUB_, sub.) wherever a function's auto-name is read as its address |
data_label_prefixes | the prefixes its decompiler writes before an address when it names a datum itself (DAT_); every server's join the shipped one (DAT_, which the Ghidra block also declares) wherever the report reads a claim's code locations (claims not discussed), and are never read as a function's name |
reference_urls | sample lookup URL templates, https:// and naming {sha256}, {md5} or {sha1}; a report links them, under the name its lookup's record gives (else the server's label), only when the run recorded an entry from this server. reporting.reference_urls holds the lookups linked on every report (shipped with VirusTotal's and MalwareBazaar's) |
tool_facts | per tool name, what its answer is and where its facts sit (below) |
credential_bootstrap | how the server's credential is obtained by one request (below); empty when its token is typed in |
launcher_depth | how many levels at the top of the process tree the worker starts for the server are a launcher that does not read the sample (1: the started process alone); the report's isolation part holds them only to carrying none of the platform's own variables and every process under them to every check (Where the sample was read). Empty declares no launcher |
default_effects | what every tool of the server does beyond answering, unless the tool says otherwise: an effects value (executes, network, third_party, when), laid under the own effects of each tool its server listed, sub-field by sub-field (a tool nothing has listed has not said whether it does otherwise, so the statement says nothing of it). A tool that says otherwise keeps what it says: one read as emulating, as running whatever it is given (why_undeclared), or as sending the file. A block whose default_effects say what its tools send (third_party) and whose roles hold a program role drops the reading that a tool sends the file it opens (see "A decompiler's or disassembler's tools read from what it lists"); it never lifts that reading off another server's tools. The shipped Ghidra and radare2 blocks say their tools run nothing, reach no network and send nothing (executes: no, network: false, third_party: no); empty says nothing |
analyse_effects | what the tool sample_open.analyse_tool names does beyond answering, over default_effects: the server's analysis step as a whole, never a table of tools. radare2's block names analyze (with no open tool) and says it emulates the sample's code when the analysis level is 2 or above, where r2mcp runs aaa, whose type matching (aaft) emulates each function with ESIL; Ghidra's run_analysis gains nothing of it; empty says nothing |
prepass_tools | which of the server's tools the platform calls itself, once per job, when it is the server the generic_mcp provider drives (static.generic.server, or radare2's own block): call_graph (the program's whole call graph, read for the priority-functions pre-pass), function_hashes (a hash per function, read for function-hash attribution and the triage pack's function-hash step) and anti_analysis (the triage pack's anti-analysis scan). Empty, or a name left empty, declares nothing, and that pass is not made, as before (below) |
Tools the platform calls itself
A Ghidra block's call graph, function hashes and anti-analysis scan are made
through Ghidra's REST API. Any other static server gets the same passes by
naming its tools in prepass_tools; each answer is read by that tool's own
declaration in tool_facts, never by its name:
arguments: rolepathis given the path the server sees the job's copy under, roleprogramthe name the block'ssample_opengave its open of it (the sample is opened throughsample_openfirst when it is declared, and an open the server refuses fails the pass with the server's words). A tool that declares neither is called with no argument, after the same open, for a server that keeps the opened sample as its session; a tool that declares onlypathopens nothing first. A role declared with no value to give (nosample_open, so no program name) makes no call. A pass that attached the server itself keeps its open in a note of its own, which goes when it lets the server go, so a later attach opens the sample afresh.rowsis where the answer's rows sit (the answer itself when it is a list androwsis empty), andvaluesandplacesthe path inside a row of each part a pass reads; a part named in neither is read from the row's key of the same name. A call graph readscaller,calleeand the caller'saddress(a row may also be a two-item list), or, answered as text, oneCALLER -> CALLEEline per call; a function named with one of the block'sauto_name_prefixesbefore a hexadecimal address gives that address. Function hashes readhashand the function asname, elsefunction, elseaddress. The scan readscategory,technique,instruction,offsetandfunction, in the rows' vocabulary of the scan the pack states (suspicious_instruction,peb_teb_access), and only its exact part is stated, by the same rule as Ghidra's; it reads the formats the tool'srouted_formatsnames, every routed format where it names none.
A pass made before an analyst attached the server (the triage pack's steps) attaches it for the call and lets it go after; one made while it is attached uses that attach. A refused or failed call is said: no priority hint, a hash step that says why no hash was read, a scan entry with the server's words.
Obtaining a server's credential
A server whose credential a service issues on request declares
credential_bootstrap: url (https only, never taken from the caller), the
JSON body posted (each {family} replaced by family, the name this
deployment registers under), token_json_path (where the token sits in the
answer), token_prefix (what a real token starts with), facts (the answer's
other values the operator is shown and audited under facts, by path, first
non-empty one first; a fact may not read the token's path, and one that carries
the token is masked) and
note (shown beside the button). The tool-server guide then opens with a
Connect step for that server, and POST /api/v1/settings/servers/<key>/credential (admin only) sends the request,
stores the token as the server's auth_token in its own encrypted row, turns
the server on and answers with the masked state and the facts. The token is in
no log line, audit row or answer; a refusal carries the service's own sentence,
with a token it still holds masked. Until a token is stored, the server's Test
button answers "no credential" without dialling. VirusTotal's agent-token
registration is the built-in virustotal server's, written in its catalog
entry beside its address (core.virustotal), as its reputation role, its
reputation_service and its reference URL are. Only a tool
server of mcp.servers takes one: a provider's own block (static.ghidra,
static.r2, sandbox.cape2.mcp) refuses it on save.
Each entry of tool_facts describes one tool, under its own name on its own
server. A name may hold __ (get__thing); a name spelled as the server's own
renamed tool, <server key>__<tool>, is refused, since a registry renames a
tool two servers both offer to that spelling. A renamed tool is read by taking
off its own server's key and __ only, so a tool whose name holds __ keeps
it:
| key | meaning |
|---|---|
kind | what kind of fact the answer is (required) |
asserts_techniques | the tool's technique id is a statement about this sample (a rule that matched it); needs techniques |
source | the name a corroboration row gives an asserting tool (capa, yara, sigma …) |
long_running | the tool's work grows with the sample and has no wall clock of its own |
timeout_seconds | the budget the tool itself keeps, where it has one |
memory_peak_mb | the tool's measured memory peak, where known |
discloses_sample | the tool sends the sample, or its bytes, out of the deployment |
effects | what a call does beyond answering, each part optional and said to be undeclared when left out: executes (no, native, emulation or sandbox: whether the call runs the sample's code, and how), network (true or false: whether the tool itself reaches a network service beyond its own server; or one of three named ways it reaches one only: on_refresh, only to fetch or refresh reference data it keeps, such as the ATT&CK bundle once stale or embedding weights on first use, never with anything of the sample; case_store, the configured case-memory store, which the disclosure's case-memory row places; operator_sandbox, the sandbox service its server fronts, at the address the operator configured; whether that is the operator's own, and so not a third party, is never taken from the declaration: the report derives it from where the disclosure places the call, and says it cannot be told where the disclosure cannot place it), third_party (no, hash or bytes: what of the sample it sends to a third party; bytes and discloses_sample say the same, and either one makes the call an upload in the disclosure block) why_undeclared (why a part is left out, printed with it) and when (the condition under which a declared effect happens, printed with it). A tick of discloses_sample lifts the tool's effects to third_party: bytes and a network that carries them, whatever the shipped effects say, so ticking it on a shipped tool (put_sample on a remote analysis server) saves as before; a save whose own effects send the sample or its hash with network false, on_refresh or case_store is refused. An operator's effects is merged with the shipped one sub-field by sub-field (tool_declarations.SUBFIELD_MERGED) and stored as the sub-fields that differ, so a later release's correction of another sub-field still reaches the tool; effects: null clears it. Printed per tool called in the report's Tool effects block; an operator declares a third-party server's tools here as a built-in's are. Every built-in tool declares all three (builtin_declarations._with_effects, held by a test that sweeps every shipped block): the CAPE sidecar's status, search, task status, report, IOC and configuration tools reach operator_sandbox and its submit_file hands the sample to it to run (executes: sandbox); radare2's analysis step (its block's analyse_effects, for the analyze its sample_open names) is executes: emulation when the analysis level is 2 or above, where r2mcp runs aaa, whose type matching (aaft) emulates each function with ESIL; VirusTotal's submit_file and submit_local_file are read from its listing as handing the sample to a third party's sandbox (executes: sandbox, network: true, discloses_sample; see "A reputation service read from what it lists"), since the service may run what it is sent; the ATT&CK knowledge tools, family_lookup and similar_cases are on_refresh (the bundle once stale, the embedding weights on first use), and the triage pack's function_matches is case_store. Ghidra's and radare2's tools are read from what each lists, under their blocks' default_effects (see "A decompiler's or disassembler's tools read from what it lists"): a tool that requires an argument naming code, a script or a command (Ghidra's run_ghidra_script and run_script_inline, radare2's run_command) runs whatever it is given and declares none, with that as its why_undeclared, and an emulate verb emulates. A tool map an earlier release stored whole keeps the shipped effects of each shipped tool it leaves them out of, merged per field as a stored change is (tool_declarations.FIELDS_AFTER_WHOLE_MAPS): a value the operator set wins |
read_only | the tool changes nothing and answers an identical call the same (read-only or idempotent): a revision that repeats an identical call of an earlier round is answered from that round's entry, as a tool its server marks readOnlyHint is, while no call that is not read-only has run on the same server since that entry was recorded; any other tool runs again. The platform's own sidecars declare their reading tools themselves; Ghidra's and radare2's are read from what each lists: a listing, read or code listing that takes no file path and, where its block's sample_open names the argument a program is named by, takes it, read-only only while the block's default_effects say its tools run nothing and reach no network (see "A decompiler's or disassembler's tools read from what it lists") |
holds_strings | the answer holds the sample's strings as data though its kind is not a strings kind (a reverser's tool of another kind whose name says strings, such as Ghidra's batch_string_anchor_report and find_undocumented_by_string); a function claim's quoted value is looked for in it |
reads_delay_imports | the tool's import rows cover the imports the format binds late: the PE delay-load table, or the one import table ELF keeps them in (the shipped pe_info and elf_info). A format answer read whole is the binary's complete import list only when its tool declares this or the answer carries a delay_imports list; otherwise the report's binary facts call the list partial |
routed_formats | the routed sample formats a format-specific tool serves, each a known format the router gives a sample (extractors.sample_identity.routed_formats(): pe, elf, mach-o, apk, dex, ole2, ooxml, pdf …); a name it never gives, such as macos or exe, is refused on save, since it would drop the tool's reason for every sample, and so is unknown, the label of a sample of no known format, which no tool serves. When the tool cannot answer on its server, a sample of another known format loses nothing by it, so its unavailability is not recorded as a degradation of that run; a sample of an unknown format, and a tool that declares none, keep the reason. The shipped analysis server declares its parsers (builtin_declarations.ANALYSIS_ROUTED_FORMATS: pe_info pe, elf_info elf, macho_info mach-o, apk_info apk and dex, document_info ole2, ooxml and pdf); a map of the analysis server stored whole before the field existed reads those as shipped |
runtimes | the code runtimes a runtime-specific tool reads (dotnet, go), each a vocabulary word, in the words an identification answer gives as its runtime value. A pack step the platform takes of the tool by its kind (The steps of a server's tool, below) runs only where an identification step's answer names one of them, and is off where no identification step's tool declares a runtime value. The shipped ILSpy and GoReSym servers declare dotnet and go of their tools, and Detect It Easy's die_scan the runtime value of each detection |
outputs | the path of the answer that names each file the tool wrote, as its server names it (unblob's container_extract: /files/*/path). A pack step the platform takes of the tool by its kind declares it as its own outputs, so each file is taken into the job's artefact store for a step that reads it. Read through the kind |
function_starts | the path of the answer's list of function start addresses (offsets from the image base), read for a function_list or capability_rules answer by the triage pack's function index and the readings of an image with no function table of its own |
rows | the path of the answer's rows |
places | per kind of place (offset, rva, address, addresses, call_site, function, pid, command_line …), the path inside a row that holds one |
values | per kind of value (text, rule, namespace, value …), the path inside a row that holds one |
techniques | where the answer names ATT&CK ids: a list of {path, format}, format being attck (an id or a decorated label) or sigma_tag (attack.t1055.012) |
reputation | for a reputation lookup (kind: reputation): service (the name it is attributed to; empty uses display_name, then the tool server's label, then the server key; the label is read when the entry is written, so a stored run keeps the name it recorded), the paths stats, malicious, suspicious, total, labels, detections, engines, summary, the path lists first_seen, last_seen and link, text_patterns and rank (below) |
arguments | per role (path, hash, address, program, command_lines …), the argument a call gives it in |
reader | the answer shape the paths cannot express (below); empty reads the paths |
report_sections | the report sections whose writer is shown this tool's answers, beside the sections its kind reaches (below) |
words | words a statement may use to name this tool's entry |
display_name | the name published text gives the tool: a rule table's title, the tool a recovered value names, the decoder a decoded value names; empty uses the tool's name |
category | the tag its description is shown with (analyze, list, modify …); an undeclared tool is tagged by its name's first word |
aggregate_of | the tools of the same server this tool is the aggregate (combined) form of: one call answers what many calls of those would (a decompilation of many functions at once, a whole-program graph that combines per-function ones). Its description, as the model is shown it, ends with "Aggregate form of <tool>: one call answers what many calls of it would.", written once the names a request offers are settled: each aggregated tool is named as the request offers it (<server>__<tool> where another server took the name), and a tool the request does not offer (withheld, or not picked by a reference) is left out of the note. The model can then choose the one call over the many; nothing else reads it, a server's own words quoted for it never carry the note, and a tool that declares none is shown and called as before. Each name is a tool the server declares in tool_facts (an aggregated tool declared with only its category is enough) or that its tools allow-list names; it is not the tool itself, nor a tool that declares itself the aggregate form of this one. A save that adds or changes such a declaration is refused naming the field (<tool>.aggregate_of, "declare each in tool_facts (e.g. its category)"), checked against the allow-list in force (sent, else stored, else shipped), and a stored one that names a tool the server no longer has is read without that name, with a warning. A reverser's aggregate tools are read from what it lists (core.program_reading): a batch, bulk or full tool, or one whose object is the plural of another tool's and whose arguments name it, aggregates each listed tool of its verb whose object is its own in the singular, or whose object an argument of its names (batch_decompile over functions of decompile_function), or else the one tool whose object holds its own. Ghidra's read so: batch_decompile, batch_analyze_completeness, get_bulk_function_hashes, get_full_call_graph, batch_create_labels, batch_delete_labels, batch_set_comments, rename_variables; get_bulk_xrefs fits several tools no argument tells apart and aggregates none, nor does bulk_fuzzy_match, whose verb is no other tool's |
shows, does_not_show | what this tool's answer shows and what it does not show, each one line in the operator's own words (a reputation lookup shows what others report of a hash, not this run's own analysis of the sample; a strings dump shows runs as they sit in the file, not strings that are encoded, encrypted or built at run time). Shown in the server card's Tool declarations box and here; the model is shown them only while core.mcp.answer_scope_notes is on (off by default, so every prompt is as it was): the tool's description then ends with This answer shows: <shows>. It does not show: <does_not_show>., after its aggregate note, and a tool that declares neither is shown as before. A line break or another control character is refused on save (<tool>.shows); a stored one is read as a space, with a warning; no length is imposed. The shipped declarations: the analysis server's strings (shows printable ASCII and UTF-16LE runs as they sit in the file, with their offsets; does not show strings that are encoded, encrypted, compressed or built at run time, nor runs shorter than min_len, or past this page (read next_offset)) and capa (shows the capabilities whose capa rules match features of the file, its code and its structure; does not show whether the matched code runs, or a capability no rule describes); the threat-intel server's check_hash, check_ip_reputation and check_domain_reputation (what VirusTotal, and AbuseIPDB for an address, report; not this run's own analysis, nor whether the sample contacts an address or a domain); a reputation service's lookup read from its listing (what the service holds for the hash; not this run's own analysis, and an unknown hash or no detections is no evidence that the file is clean). Every other built-in tool declares neither |
Paths are JSON-pointer-like: /strings/*/offset is the offset of every
item of strings; * is every item of a list or every value of an object at
that level, a number is one item, ~1 writes a / inside a key, ~0 a ~
and ~2 a *. ** is any depth (/**/attck is every attck key wherever it
sits) and **N at most N levels (/**6/attck), at most once in a path and
never in a path read inside a row (places, values), so every read is linear
in the size of the answer. A path that names nothing reads as absent.
Error patterns. Each is {match, code, remediation, remediation_with_path, tools, applies_to, multiline, message_group}, applied in list order:
- the text held is the tool's answer, or, where
applies_tonamesprotocol_error, the message of the MCP protocol error the client reports in its place (applies_todefaults toanswer), stripped of surrounding blanks; - a text with a line break never matches a pattern that is not
multiline; matchis held against the whole text, never a line inside one;- the error message is the text, or the group
message_groupnames, each line stripped and blank lines dropped; codeis the error code (tool_failed,bad_argument…); the remediation isremediation_with_pathwith{sample_path}replaced by the path the server can open for the job's sample where that is known, elseremediation;toolslimits it to some tools (empty: all).
Every server's answers are held against its patterns by one function, the
registry's servers and the static providers' alike, so an answer a pattern
matches is filed as a failed call with the pattern's code and remediation, the
answers of a declared sample_open tool and of the select and analyse calls it
makes included. A server that declares no pattern has its answers passed on as
they are, with one exception for a tool its source declares nothing for (an
operator's tool declared without a kind included): an answer with an error's
form is filed as a failed call. That is a traceback (its last line is the
message), or one line of prose of at most 400 characters that opens with an
error label followed by a message (Error: …, Fatal error: …, [ERROR] …,
[Errno 2] …, Error executing …: …), an exception class's shape
(ValueError: …) or a failure phrase with its verb (Failed to …, Unable to …, Cannot …, Could not …, Permission denied). A line that merely opens
with an error word is data (Error handling routine at …, Invalid signature, Not found in …), as is a label followed by a count (errors: 0,
Failed logins: 0) and an answer holding any data.
remediation_with_path names the job's sample where the server's
path for it is known: for the static providers' servers (the job's mirror);
an mcp.servers entry gets remediation.
Patterns run in linear time. Every operator regular expression (error
patterns and reputation text_patterns) is compiled and run by RE2, whose
matching time is linear in the text whatever the pattern, so a sample's
strings in an answer cannot stall a run. A pattern RE2 cannot compile, a
back-reference or a lookaround, is refused when it is saved, with the reason.
RE2's shorthand classes are ASCII only: \s is space, tab and the ASCII line
breaks, \d is 0-9, \w is [0-9A-Za-z_], and \b is a boundary between
those. A Python pattern ported as it is therefore no longer matches a
non-breaking space with \s or a non-ASCII digit with \d, and \b sees a
boundary before a non-ASCII letter. Where Unicode is meant, write the class out:
\p{Z} (Unicode separators) or an explicit list such as
[\s\x{a0}\x{2000}-\x{200a}\x{3000}] for whitespace, \pN for digits and \pL
for letters.
text_patterns are searched for anywhere in a reputation answer written in
prose, the first pattern that finds a match giving the counts; each names a
malicious group and optionally total and suspicious ones.
Reputation lookups. Any server whose tool declares kind: reputation is a
reputation source, whatever the tool is called and however it answers:
- the triage pack asks one such tool about the sample's sha256, when it also
declares the argument it takes a hash in (
arguments: {hash: <name>}) and, where its server's block names atoolslist, is ticked in it (an unticked tool is never the pack's call; an agent's own use of tools is unchanged): the first enabled one, lowestreputation.rankfirst (default 100; the built-in VirusTotal server's lookups take rank 10 from its block'sreputation_service, the threat-intel sidecar declares rank 20), lookups of one rank in the order their servers are configured; - its answer is read once, when its entry is written, through the declared
paths into one record (the service,
malicious,suspicious,total,engines, the counts object,labels,detections,first_seen,last_seen,link,summary); a count no path gives is looked for in the text bytext_patterns; a date is kept as the ISO-8601 text it was written in, or converted from whole seconds since the epoch; a link is kept only as anhttp(s)URL; a prose answer with nosummarypath is its own summary; - the record is stored on the entry and carried in the report
(
MalwareReport.reputation), and the pack's count condition (reputation_malicious), the run-state block, the report's reputation line and the console's reputation section read it, named by the declared service, so a re-render reads what the worker read without the operator's settings; - the judge holds a lookup when a tool of this kind is attached to it or a server its definition references declares one, and counts the identity question as asked once the ledger holds a call to one;
- a failed lookup is the pack's reputation step failing
(
triage.reputation_failed, read to the judge as one sentence); - an agent reaches a reputation server you add the way it reaches any server
of yours: by the server's
agentslist or anmcpreference to it (the built-in agents' capability reference reaches the shipped servers whose role is reputation; see Delegation).
A tool named like a built-in lookup that declares another kind, or nothing, gets none of this. A report stored before the records is read the way it always was.
A reputation service read from what it lists. The platform ships no
declaration of a reputation service's tools, VirusTotal's included. For a
server whose block's roles hold reputation (the catalog sets it on the
built-in virustotal entry; set it on a reputation server of yours), each
tool that neither the block nor the server's own _meta declares is read from
its MCP definition when the server is listed (core.derived_declarations),
by the same rules for every server:
- a tool that sends the file (below) is a submission (
submission,discloses_sample,executes: sandbox); so is a write tool, one not markedreadOnlyHint: trueor markeddestructiveHint: true(MCP's defaults make an unannotated tool a write tool), so the triage pack never calls one; - only a tool marked read-only and not destructive is read as asking: one that
takes the id of an analysis, a submission, a task or a request is that
submission's status (
submission_status,third_party: hash); one that requires an argument read as the sample's hash is a reputation lookup (kind: reputation, that argument asarguments.hash,reader: reputation_shape,effectsexecutes: no,network: true,third_party: hash); one that requires a domain, an address or a URL is a report on it (indicator_report,third_party: no, with the argument it requires each indicator in underarguments); network: trueunless the server marks the toolopenWorldHint: false;- what the block's
reputation_servicesays of the service as a whole is laid under each of its lookups: itsnameas the lookup'sdisplay_nameandreputation.service, itsrankasreputation.rank, each only where the lookup's declaration (the block's, an operator's row) says none; a lookup that names no rank is asked at the service's rank before the default. The built-invirustotalentry's is{name: VirusTotal, rank: 10}. Itsrate_limit_secondsandmalicious_whenare read only by post-verdict enrichment, when it asks the server as a source of its own (Enrichment sources): the pause after each question and the counts that give its records a verdict.
A lookup that names the reputation_shape reader and declares no path is
read by its answer's shape. Only such a lookup is: a lookup whose declaration
declares a path or a pattern of its reputation, or a reader of its own,
never takes the reading's reader, so its declared paths are read (one that
names only a service or a rank keeps the shape reader), and a stored entry
whose facts name no reader reads as it always did.
A stored change of such a tool that names no kind of its own (an operator's
get_file_report: {reputation: {rank: 30}} on the built-in VirusTotal server,
saved over the table a release once shipped) is not dropped with its
kind-dependent fields: they are kept with the block and laid over the
reading once it puts a kind in force, a reputation change field by field
(tool_declarations.kind_free_changes). Until the server is first listed, or
after its block is pointed at another server and before that one is listed,
the change waits; it applies from the listing on. It is kept with the block
through every dump of the settings (the settings crossing a process as JSON, a
save that sends the block back), written as it was stored. A new edit of such a
change is refused, as a kind-dependent field with no kind always is; to rank or
name a reputation server's lookups, set the server's reputation_service
(rank, name) in the server's Declaration panel instead. The console's tool
declarations box shows such a change in place and names it below the box as
kept as stored: left exactly as it is, it does not stop an edit of the other
tools, and an edit of it is checked in full. The counts are the object
whose malicious (or positives) is a whole number, that holds at least three
whole numbers and sits under a key that does not say vote (an engine tally,
never a vote); of several, the one counting the most verdicts (malicious,
suspicious, harmless, undetected and the like), then the most whole
numbers; the service, where the lookup names none, is the source, service,
provider or vendor text in or beside it; the engine count the first whole number under a key that says engines;
the detections the first list of texts under a key that says detections;
the labels the texts under a key that says label or family, then threat names,
then threat categories (a list of objects read by each one's value, name
or label); first seen the earliest date under a key that says first and
seen, submitted, analysed or scanned, last seen the latest under one that says
last (or names an analysis or scan date alone); the link the first http(s)
text under a key that says link or permalink, or a report's, page's or
interface's URL. It looks eight levels deep and at 20,000 members at most.
Whether a tool sends the sample is read for every server, whatever its
roles, and for every tool, one its server declares under _meta included: a
tool that takes the file's content as bytes (an argument named for content,
bytes, base64, b64, a blob or a payload, or for data or a body beside a file's
word such as file_data or sample_body, or one whose schema says format
or contentEncoding byte, binary or base64; a name only when its
schema does not type it a number or a flag, which cannot carry the file:
max_scan_bytes: integer does not) is discloses_sample; only
a server whose block trusts its declaration (trusts_served_declaration) can
lift that by marking the tool both readOnlyHint: true and
openWorldHint: false, an untrusted server's own hints never (such a tool of
a trusted reputation service is then read as the read-only tool it is); a tool
that
takes a path or a file (an argument named for a path or a file name, or only
for files or samples, such as file, files, local_file or paths, or one
whose schema or array items say format: path) is unless it is marked
openWorldHint: false. A tool whose name or description says it submits or
uploads (its name read by its words, so submitFile and upload-sample
count) and that takes either sends it whatever its hints say; one that is not
marked readOnlyHint: true and says it submits or uploads now (its name's
word submit or upload, or its description's submit, submits, upload
or uploads; a past submitted or uploaded names what others did and does
not count) sends it whatever it takes, since it may hand the sample over by
an id or a hash. A definition that cannot be read
is read as sending the sample; a listing that cannot be read keeps what was
read of the server before. A path's word or format wins
over a hash's (hash_file_path is a path). Each argument
is read in every schema form it may take: a local $ref (#/$defs/…,
#/definitions/…) followed from the root schema, each reference once (a cycle
ends there), the members of anyOf, oneOf and allOf, array items, and
the arguments of nested objects one level down. readOnlyHint says a tool
changes nothing, not that it sends nothing, so it never excuses sending.
When the definition leaves it open, the reading is that it sends: a tool of
an undeclared server that takes a path and is not marked
openWorldHint: false is shown in the run's disclosure as an upload of the
sample, whatever it does with the file. To show a local tool that only reads
the file as reading it, mark it closed: its server lists it with
annotations: {openWorldHint: false}, or you declare what it sends in the
server's tool_facts (<tool>: {effects: {network: false, third_party: no}}), which is in force over the reading. The reading is dropped only where
the block itself (an operator's declaration, or what the
platform ships) says what the tool sends (discloses_sample or
effects.third_party), which is how an operator allows a tool to send
nothing, or where a server the block trusts declares the tool; never on an
untrusted server's word. Whatever else a block or a server declares of a tool
is in force over the reading field by field.
What is read is kept with the server's own declaration, under the same launch
identity and past the process, and a job that runs for real lists each enabled
server whose roles hold reputation and of which nothing is kept, as it
lists a server whose block trusts its declaration.
A decompiler's or disassembler's tools read from what it lists. The
platform ships no table of Ghidra's or radare2's tools. For a server whose
block's roles hold decompiler or disassembler (the Ghidra and radare2
blocks ship both; set one on a reverser of yours), each tool its server does
not declare under _meta is read from its MCP definition when the server is
listed (core.program_reading), by the same rules for every server and never
by its name:
- its kind, by its verb and its object: the first word of its name (the
word after
batchorbulkfor a batch tool; camelCase is read as words) and the words after it, before anyby. A verb that changes the program (set,rename,create,delete,apply,clear,save…) isprogram_edit, soset_decompiler_commentis never a decompiler (forceorrefreshof a decompilation isdecompile_refresh);openandloadareopen_sample(read by the open-program readers, so the file itspathargument names is a program the agent named), of a project aloneproject_navigation, of a program from the project's storeproject_load;switchandselectselect_sample;closesample_close;emulateemulation;runrun;importof a filefile_import; a name whose last word isstatusstatus, whose answer moves. Where the verb itself decompiles or disassembles, the code listing its name says (decompile,decompile_batch,function_disassembly,disassembly); a name that says cross-references (before anyby)xrefs. Any other reading verb's object names the kind a platform consumer reads (core.program_reading.OBJECT_KINDS), each word read anywhere in the object by its stem or singular: a listing or read (list,get,read,show,fetch…, never asearch) of importsimport_listing, of exportsexport_listing, of segments or sectionssegments, of exactlyfunctionsfunction_list, ofmethodsmethod_listand ofstringslisting_strings; asearchofstringsstring_search; any verb on a program's, a binary's or a file'sinfoprogram_info, on callerscallersand calleescallees,call chainscall_chains,crypto constantscrypto_constants,dataflow, iocsiocs_with_context, behaviorsbehaviour_scan,anti analysisanti_analysis_scan,documentation completenessdocumentation_completeness(as is an analysis or check ofcompleteness); an analysis of onefunctionwhose arguments name it (by its address or its name)function_analysis. A batch's answer is many answers, so its object names none. Anopenof a program or file from a project isproject_load, of a project aloneproject_navigation; a verb on a decompilation that does not itself decompile (force_decompile), orreand the code verb, isdecompile_refresh. Else a check ischeck, an analysis (analyze,detect,find,compare…)analysis, a listing or read (list,get,search,read)listing, any other verbutility; a tool whose name says strings and whose kind is no strings kind holds them (holds_strings). The tools the block'ssample_opennames are read as opening, selecting, analysing (analyse_sample) and closing a program. The object words are the consumers' own, the same for every server, never one server's tools: a server whose tools say the same with another verb of the same category or another spelling reads the same; - its arguments, each by an exact role word:
programfor the argument namedprogram;addressfor the first read as a code or memory address of the binary (a struct's, a range end's, a base's, a diff'saddress_aand a hash routine'shash_function_addressnever are; a range'sstart_addressis);pathfor the argument that names a file, of a tool that does not change the program (a category's or a folder's path never is); - whether it reads only: a listing, read or code listing that takes no
file path, whose name says nothing of a state that moves (
current,status) unless it names the program it reads, isread_onlywhere the block'sdefault_effectssay its tools run nothing and reach no network and, where itssample_opennames the arguments a program is named by (select_argument,analyse_argument,program_arguments), the tool takes one. A tool of a shared server that names no program reads what every job shares (the programs open on it, its scripts, its project), which another job may change, so it never answers from an earlier call; - what it does beyond answering, only where its definition says it: an
emulateverb emulates; a tool with an argument whose name holds a word for code to run (code,script,command,cmd,python,js,lua), whatever else it holds (user_script,hook_code,asm_code), unless its other words only name a property of the code (style,type,kind,mode,format,language,id,count,limit,offset,page,size:code_style,command_count); an expression evaluated as a condition or a filter (condition_expression); or, for a verb that runs or evaluates (run,exec,eval…), asource,queryorexpressionwith the words that carry them (name,text,body,inline,string), runs whatever it is given and declares no effect (why_undeclared), its block's own analysis step included (a search'squeryand asource_addressname something else); arunverb (orexec,eval,invoke…) with no such argument runs what its definition does not say, unless the block'ssample_opennames it as its analysis step. A tool that takes the file's content, by the one rule the disclosure reads it by (below), sends its bytes (third_party: bytes), and so does one that says it submits, uploads or sends now: unless it is marked read-only, and whatever its server's hints say when it takes a path, the file's content, an id or a hash. Everything else is what the block'sdefault_effectssay, and the analysis tool's what itsanalyse_effectssay over them; - its aggregate, for a batch tool (see
aggregate_of).
The reading of whether a tool sends the file it opens is dropped for such a
block's tools where its default_effects say what its tools send, as for a
tool the block declares, only for a tool it reads as opening or reading the
file where it runs: never for a tool read as sending its bytes, never for one
of the utility or run kind or that runs what it is given, and never on
another server's word. A stored
removal of such a tool (<tool>: null in the block's changes) stays a
removal, kept through a dump and its validation: nothing its server lists is
read of it, but that it sends the sample, where its listing or its server
says so (the removal takes what the block says away, never a disclosure). A stored change of a field read only through a kind
(extract_iocs_with_context: {places: …}) is laid over the kind the reading
puts in force, inside the rows that kind's own reader reads. Ghidra's HTTP transport serves no MCP listing:
its /mcp/schema is read as one (each entry's name, description and
parameters, the required ones marked), so Ghidra over HTTP and over stdio read
the same 165 tools alike. The schema's HTTP method is not read: the server
opens, selects and saves programs by GET and reads by POST, so it says
nothing of whether a call changes the program. A run stored before keeps the
facts its entries stored, and one stored before entries recorded facts reads
the frozen snapshot (core.legacy_tool_facts).
Readers are named by the answer shape they read, never by a tool, so a tool of yours that answers in that shape may name one:
| reader | the answer it reads |
|---|---|
pe_header | a PE header answer: sections and the entry point, read as file layout |
emulated_strings_with_call_sites | rows of recovered strings, each with the call site that produced it and the routine around it, joined to the encoded blobs a static decoder read at that site |
function_table | a function index: per function its start, range and what it references |
byte_transform | the result of byte transforms over a range of the file, with its origin |
unpacked_image | an unpacked program written out as a file of its own |
flow_table | flows and DNS queries with their endpoints, from a capture or a sandbox |
dns_query_lines | a text answer holding one DNS query per line |
packet_lines | a text answer holding one captured packet per line, with its endpoints |
report_items | sandbox report items, each with an id and the fields that place it |
reputation_shape | a reputation answer read by its shape: its engine tally, labels, detections, dates and report link (named by a lookup read from a reputation service's listing) |
Vocabulary. kind, roles, source, reader, category, a technique
format, a place or value name and an error code are open: any lowercase
word of letters, digits and underscores is accepted, so a new kind of tool
needs no release to be declared. The built-ins' kinds are as fine as the tables
the platform reads tools by today, so each table is exactly a set of kinds
(CONSUMER_KINDS in maljan.core.builtin_declarations, held tool by tool by a
test): for example strings (the analysis server's printable strings),
listing_strings (a disassembler's string listing), emulated_strings,
decoded_strings, file_iocs, text_iocs, iocs_with_context,
capability_rules, signature_rules, event_rules, command_match,
reputation, indicator_report,
indicator_reputation, program_info, file_info, load_sample,
open_sample, open_project_program, decompile, function_disassembly,
disassembly, function_list, method_list, xrefs, callers, callees,
and one sandbox_<section> kind per sandbox report tool.
Which source a tool is read under. Every ledger entry records, when it is
written, where its tool's declaration lives: mcp:<server key> for a tool
server's tool, provider:<id> for a static provider's own tool (Ghidra,
radare2, the evidence-only capa/YARA provider) or the CAPE provider's, sandbox
for the sandbox report tools and pack for the triage pack. The generic static
provider drives a tool server of the registry, so its tools record that
server's mcp:<key>, the server it drove when the entry was written: pointing
static.generic.server elsewhere later does not change how a stored entry is
read. A tool is looked up by that recorded source and its name together, never
by its name alone, an agent's name or today's settings; a tool of yours that
shares a built-in tool's name never takes the built-in's facts. An entry
written before sources were recorded is read the way it always was.
The facts an entry stores. When an entry is written, the facts its tool
declares under its recorded source are stored with it (facts on the ledger
entry): {"v": 1, "kind": …} and, where set, reader, asserts_techniques,
source, display_name, rows, places, values, techniques,
reputation, words, report_sections, arguments, holds_strings,
reads_delay_imports, function_starts, discloses_sample and effects; {"v": 1} alone when the
source declares nothing for the tool. Beside them, under hints, the record
keeps the MCP annotations the tool's own server gave it when it was attached
(readOnlyHint, destructiveHint, idempotentHint, openWorldHint, each
only where given as a yes or no). They are not a declaration: only the tool
effects read them, openWorldHint alone, for a tool whose declaration does
not say whether it reaches the network. Everything read from a run later (a later stage,
a re-render, the API) reads these stored facts, never the reading process's
settings, so a run reads the same wherever it is opened, and changing a
declaration later changes only new runs. A record of a newer version is read
for the fields this release knows, and a record that no longer validates is
read as declaring nothing, each with a warning. An entry that recorded a source
but no facts is read as an undeclared tool; an entry written before sources
were recorded is read the way it always was, from a frozen, digest-pinned
snapshot of the facts the built-in tools declared then
(data/legacy_tool_facts_v1.json), never from the declarations a release
ships now, so a later change to a built-in declaration leaves a stored run
reading as before.
Who reads a tool's kind. A tool of yours declared with a built-in's kind is treated as the built-in of that kind, and its answer is read in that kind's shape:
- a tool with
asserts_techniquesnames the techniques itstechniquespaths hold as statements about the sample, under itssource(its own name when empty), in the judge's evidence summary, the corroboration rows and the run summary's per-source attribution; - the evidence roots place an entry by its kind or
reader(a whole-file fact, a reference lookup, a disassembler's import listing, a capability or signature rule match, recovered strings …); a kind with no reading of its own, or a tool whoserowsandplacesdiffer from the shipped tool of its kind, is placed by itsrowsandplaces(offseta file offset,rva,call_siteorfunctionan offset from the image base,addressoraddressesan address,pida sandbox process), else by the address its call was given; - validation reads a report section by the kind of the tool that opened it
(
capability_rulesandsignature_rulesare rule matches, whose words ground no capability;strings,file_iocs,emulated_stringsthe sample's own values;api_cataloguea reference table), names a rule matcher in a sentence by itsdisplay_name, and names an entry in a statement by its tool's name andwords; - the function map and the triage pack's function index take
api_hash_resolution,decoded_strings,emulated_stringsandfunction_indexanswers by kind, the index and the pack's readings of the bytes their function starts from anyfunction_listorcapability_rulesanswer'sfunction_startsand their rules from anycapability_rulesanswer (The pack's steps), and the report step does not run an evidence-only static provider when the pack already recorded an answer of a kind that provider declares; - a tool's role is its kind:
decompile,decompile_batchanddecompile_refreshlist decompiled code,disassembly,byte_disassemblyandfunction_disassemblylist instructions (the last a whole function),xrefsreads cross-references, andstrings,emulated_strings,decoded_strings,listing_strings,listing_all_stringsandstring_searchhold the sample's strings, as does any tool declaringholds_strings. The decompiled-function list, the function map, a function claim's check, a repeated decompile and the static turn's tool line read it so, and a call gives its address in the argumentarguments.addressnames, then in the shared list (address,addr,function_address,ea,va,start_address,entry_address), then, for a tool whose answer is a code listing, in any of the call's own arguments whose name has a code address's shape: address words alone (address,rva) or paired with words for the place of code (func_addr,target_address,instruction_address,name_or_address). A network or identity address (ip_address,email_address,remote_addr,address_family), a bound, a size or a base (end_address,base_address,address_count) andoffset(a page's first row as often as a place) are never read as one. Only a tool that declares nothing is read by the words of its name (decompil,disassembl,xref,string), in one place (core.undeclared_tools), and the report reads such a tool's code listing by the same words: adecompil…tool is drawn as a code figure, both it and a whole-functiondisassembl…tool are listed under "Functions examined" and shown to the composer's code sections, as the function map reads them; - a network turn offers the capture tools by kind (
packet_overview,dns,http), under the names the request offers them by; with none of those kinds and a tool that declares no kind in the request, it says that such a tool may read the capture rather than that there is no packet tool; - a stage condition counts the rows of every
capability_rules,signature_rulesandevent_rulesanswer as matched rules (triage.rule_hits,triage.hits[...]).
How the report reads a tool. The report reads an entry by its tool's declared kind, from whichever server declares it, and never by the tool's name:
- the section a tool's answer is drawn as:
identity,signing,format_info,strings,file_iocs/text_iocs, the three rule kinds, thesandbox_<section>kinds,pcapand the reputation kinds each get a purpose-built section; any other kind, and a tool of yours whose answer the purpose-built section cannot read, gets the generic block, one per tool, since its answer is in that tool's own shape; - the three rule kinds share one reader: every
signature_rules,event_rulesorcapability_rulestool is drawn in its kind's one table, under its kind's key whichever tool filled it (yara_matches,sigma_matchesandcapa_capabilities, the keys stored reports and the console already read), titled by itsdisplay_name. The rows are read at its declaredrows; each column at the value it declares under the column's name (values.rule,values.title,values.namespace,values.tags,values.where,values.level,values.matched_fields,values.mbc), else at the field the kind's answers share (rule,tags,level,attck,mbc…), and the technique column at its declaredtechniquesor the row's ATT&CK tags. A table that holds the rows of more than one tool gains aToolcolumn naming each row's tool, and its title names every tool. A server you add whose tool declares one of these kinds and answers in the shape the built-in does gets the built-in's table, rows and run-comparison rows as they are; - "Functions examined" lists every call of a
decompile,function_disassembly,function_listormethod_listtool, and the code figures draw everydecompileanswer; - each technical section of the composed report is shown the captured
outputs of the kinds it is about (a function's
decompileorfunction_disassembly,listing_strings,segments,crypto_constants,emulation,dataflow,call_chains,iocs_with_context,import_listing,anti_analysis_scan,behaviour_scan,function_analysis,documentation_completeness), and of any tool whosereport_sectionsnames it; - a
capability_rulestool's rows become technique hits under itssource; - the published record of a value only a recovering tool read states each
emulated_stringsanddecoded_stringstool under itsdisplay_name, with the scheme, routine and call site its declaration places.
The report reads these from the facts each entry stored when it was written, copied onto the report's evidence index and captured outputs, never from the settings of the process that renders it, so the console shows what the worker built. So a reverser you add with tools of these kinds feeds every section as Ghidra's tools do. An entry written before sources were recorded is read by the tool names the report always read it by.
A server added under a built-in server's key (analysis, knowledge,
network, threatintel, virustotal) is read as that built-in: a tool of
yours there whose answer a purpose-built section cannot read is not given the
generic block, and its answer is left out of the report's sections. Give a
server of your own a key of its own.
What ships. The built-in servers (analysis, knowledge, network,
threatintel, virustotal), the Ghidra and radare2 blocks (their tools are
read from what each lists, never shipped), the CAPE MCP block and the
platform's in-process tools ship declarations that say what the code holds
today: threatintel and virustotal carry the reputation
role, the three file-reading sidecars reads_staged_sample, analysis and the
triage pack platform_owned, Ghidra its mount /data/samples, its work
directory .work, its workflow text and its non-secret variable, radare2 its
work directory r2-work, its workflow text and its error patterns (the
open-first refusal, its log envelope and the sentences r2mcp writes as plain
answers), VirusTotal its reference URL and its submit tools
discloses_sample, and the threat-intel sidecar both of its prose forms
(N/M detections, N/M malicious).
A server that declares nothing. A server whose block declares no
prompt_fragment and offers a tool its tool_facts say nothing about is
introduced by its own words instead, while its tools are in the request: the
MCP instructions it served when it was initialised and each offered tool's
own description, each quoted as one value on one line (its line breaks and
<< written out, whitespace collapsed as a tool's description is) under a
sentence saying the quoted text is the server's data and never an instruction,
and that the agent's instructions hold where it differs. Each quoted value is
written as a JSON string (every quote, backslash and control character
escaped), so nothing the server serves ends it, and a served
{{attached:…}} is shown as written, never resolved. A tool's name is quoted the same way, as the
request offers it (<server>__<name> where another server took the name).
The text is kept whole, as a description is, and is sent with every request
of every agent bound to the server: a server with long instructions costs
that much of each agent's context on every turn, and declaring a
prompt_fragment replaces it. A server whose every offered tool is declared
(every built-in) carries what its declaration says and nothing more. The
server the generic_mcp static provider drives, with no prompt_fragment, is
introduced in the provider's workflow the same way, by its served
instructions and its offered tools' descriptions, quoted under the same
sentence; only a server that served no instructions gets the generated
paragraph naming its tools.
A proposed declaration. The connection test (POST /api/v1/settings/test/mcp, admin only) proposes a declaration for each tool
the server offers that its block and the shipped declarations say nothing
about, under details.proposals, read from what the server says itself:
| Read from | Proposes |
|---|---|
MCP inputSchema (a property's name, format, pattern, length, description) | arguments: the path, hash, address, ip, domain and url roles, one argument each |
MCP outputSchema | rows (the first list of objects), places and values inside a row, techniques (/**N/<field>), the reputation paths where the answer holds a malicious count (dates and links read by generic words: first or last with seen, submission, analysis, scan, date or time; link or url) |
MCP annotations and execution | display_name (the tool's own title, verbatim, one printable line of at most 80 characters), long_running (execution.taskSupport), discloses_sample (openWorldHint on a tool taking a path) |
the description and the capabilities manifest cell's facts | the kind, with the arguments and the answer's shape (signature_rules for rows with techniques and a rule match, reputation for a hash lookup with counts, a code kind for a decompiler, from the tool's name, or from the description's words only when the probe answer holds code); no kind when nothing names one |
the capabilities manifest cell | timeout_seconds, long_running |
Each field carries its evidence (the schema location or answer it was read
from; evidence never quotes the server, though a proposed display_name is
the server's title and a proposed error pattern its answer's line) and a
confidence: high when the server named it outright (a format, the
manifest's own budget), medium when a property's name or the schema's shape
says it, low for a reading an operator should check (asserts_techniques
always is). requires names the fields it needs: a field read only through a
kind (rows, places, values, techniques, reputation, reader,
report_sections, asserts_techniques, source, holds_strings,
reads_delay_imports, function_starts) requires
the kind and is offered only with one, accepting it stages both or neither,
and a row field also needs rows. A tool declaration names a kind or states
only the fields read without one (display_name, category, words,
timeout_seconds, long_running, memory_peak_mb, discloses_sample,
read_only, arguments, routed_formats, effects, aggregate_of, shows,
does_not_show;
core.tool_declarations.KIND_FREE_FIELDS): a declaration with an
unknown key ({"knd": …}), an empty one, or one of no kind that sets a field
read only through a kind is refused on save, by the API and the console's box
alike: the box reads the field split, each field's default and the no-kind word
from the served settings schema (tool_declaration_fields), and a field counts
as set only when it differs from its default, as the API counts it (a yes-or-no
field as the model reads it, so 0, "false", "no" and "off" are its
default, false), so a declaration stored with every default spelled out reads
as it was saved. A
declaration of no kind keeps every reading an undeclared tool gets —
the composer's citation routing below, the code-listing reading of its name,
its server's own words, the network line — until a kind is accepted for it;
unclassified is read as no kind only when the declaration sets no field a
kind reads (with one, it is an ordinary kind word, read as it always was, and
only a declaration with no kind at all loses such fields); a declaration's
display_name and words still name
the tool (a section built from an operator's tool is titled by its declared
name, with a kind or without, and a statement may name the entry by its
words; a stored record's display_name, words and arguments are read
whatever its kind, so a report argument declared without a kind still keeps
the tool off the "looked at the file" lines). These rules hold for what an operator types now: a save checks only the
declarations it adds or changes, against what is stored and what ships, so a
declaration left as it was never blocks a save. Everything already accepted —
stored settings, shipped blocks, a runtime provider's declaration, an import —
is read leniently: a key this version does not read, and on a declaration of no
kind a field only a kind makes readable, is left out with one warning naming it
(sanitised, once per process), and the rest is kept; an import says what it
left out in its warnings (every warning about one tool, in order), and an import of an export from another version is
never refused for it.
Schemas are walked once, without recursion, to 16 levels below the answer's
root, and a name that is not a plain identifier is never made a path or an
argument. A tool whose schema cannot be read loses only its own proposal
(details.proposals.failed says which and why), and no proposal ever changes
whether the connection test passed.
The proposals also list planned harmless calls: one per undeclared tool that
its own server marks readOnlyHint: true (and not destructiveHint) and whose
required arguments are all a path, a hash or an indicator, or that takes none.
An unannotated tool is never called. Each planned call names its arguments (a
path under a directory that does not exist, the digest of empty input,
192.0.2.1, example.invalid; never a sample), whether the tool may reach a
service of its own (its server does not mark it openWorldHint: false) and
what the call is made without (withheld). Nothing is called until the admin
confirms the list in the server editor; the confirmed request names the tools
(confirmed_calls in the body), only tools still planned are called, once
each, five seconds each, on an attach of their own that uses the server
exactly as it is configured (its URL, arguments and files) and withholds only
its auth token and the environment values it does not declare non-secret
(variable_names), each named in withheld; a server whose declaration says
it needs a credential (credential_bootstrap or required_env_allow) is sent
it, which the list says (credential). The connection test's verdict is the
configured server's own handshake. details.proposals.probed says per tool
whether it was called and what it was made without; a server that does not
answer without the withheld values (a child that dies at start included), and
a call that answers with an error, say so plainly; a cancelled probe is still
cancelled. An answer
proposes the row shape when the tool has no outputSchema or one that only
wraps a text (a single string field, the form an MCP server writes for a tool
that returns a string) and the answer's text is JSON that is not an error
report (an isError reply, {"error": …}, an object of only a message), the auto-name
prefixes it writes twice beyond the defaults, and, for a one-line prose answer
to the missing path that the platform does not already read as an error and
that is no longer than a captured output is kept (6,000 characters), an
error_patterns entry matching that line with the path as any text (low
confidence when the answer does not name the path, since the same line may be
the tool's answer to a file it found nothing in). The server editor shows each
proposal with an accept per field and per tool (a tool proposed nothing is
listed on one line); accepting stages the field, with the fields it requires
that the declaration does not hold yet and never a kind it did not ask for,
into the server's tool_facts (or adds the patterns and prefixes beside the
server's own), and Apply saves it like a typed declaration. Nothing is applied
without an accept, and no run reads a proposal. Accepting on a built-in server
stores only the tools you accept (see Saving below); every other tool keeps
following what the server ships.
A live sandbox server's tools are proposed sandbox kinds by the shape of
their rows, read from the outputSchema or from a confirmed harmless call's
answer, only where the server's declaration names it a sandbox source (its
roles hold sandbox): the same rows are what a memory image, a capture, a
live resolver or an intelligence lookup answers with, and a sandbox kind makes
them observed run facts. Set the role on the server's block before its
connection test. A tool asked about an address, a domain or a URL (an argument
read as one) describes that subject rather than a run, and is proposed no
sandbox kind on any server. Rows naming a process id, its parent and a name propose
sandbox_processes, with the process id as the pid place and the name and
command line as values; rows naming a destination and a port, or a DNS
question, propose sandbox_network (with the flow_table reader where the
answer lays its rows out as one, under tcp, udp, conversations or
dns); rows naming a digest, a name and a size propose
sandbox_dropped_files. A DNS question needs DNS-specific evidence: a key
only DNS rows write (qname, qtype, rrtype ...), question types that are
DNS record types (where a confirmed call's answer shows them), or a question,
its type and its answers under answers; a list or tool name saying DNS alone
reads with low confidence. A question under request or query needs a
second DNS signal beside it (one of those, a name saying dns, lookups or
resolutions, or answer values shaped like host names), and rows carrying an
HTTP request's or an endpoint's keys (method, url, status, port ...)
read as no DNS question unless a key only DNS rows write or record types say
otherwise. Question values read as host names when most do (three in four
shaped like one, half of those dotted); rows whose values are no host names
read as no DNS question unless a key only DNS rows write says otherwise, or
record types do beside answers under answers or a name saying DNS. A
connection test of a server without the role whose tools' rows read like a
sandbox run's says so in the proposals' notes, naming the tools, and
proposes no sandbox kind. A name with its addresses (addresses, ips, resolved) is what
interfaces, hosts files, name servers and embedded configurations hold, and is
no DNS question; dropped files need a list or tool name saying so: a name and
a type, or a digest, a name and a size, are what function, section, rule and
archive listings hold. The shape
is asked only where nothing else the server said names a kind, so a tool its
words or roles name (a decompiler, a rule scanner, a reputation lookup) keeps
that kind. An accepted sandbox kind gives the tool's answers what
the built-in sandbox tools' answers get: a run root per process or flow, a
place among the sandbox's entries (observed steps) and the built-in section
drawn from the declared rows.
Where an undeclared tool's answers go. The composer shows a technical
section the captured output of a tool its source declares no kind for when a
claim the section is given cites the output's entry (the entry the report's
evidence index holds under that call, with the same tool and agent; an output
that carries no call number is not routed), beside the outputs of the kinds it
reads; the grounding a section is checked against is built from the report,
the answers and the ledger, so a citation adds none. An agent definition
declares the console tab its tools' sections are drawn on (tab: identity,
static, dynamic, network or persistence; the built-ins declare theirs — the
three analysts their own tabs, the judge identity — a generic or lead agent
none until an operator sets one, a clone copies its source's, and a definition
written before the field reads its role's). Every ledger entry records the
calling agent's declared role and tab (agent_role, agent_tab; none for the
pack), and the evidence index copies them. The served report (GET /api/v1/reports/{id}) names under section_tabs the console tab of each
section built from an operator's tool: by the kind its tool declared (a sandbox
kind on DYNAMIC, a capture on NETWORK, a reputation lookup on IDENTITY, a code
listing, rule match, strings or import listing on STATIC), else by the stored
tab of the agent whose calls it cites, read from the entry and never from
settings or an agent's name. A section of a built-in tool, a section of a
stored run written before tabs, and a section neither reading places, keeps
its place by key.
A tool's answer that the server gives as structuredContent beside a prose
text is kept as the entry's structure (structured, marked
structured_from_server); the text is what the model read, as before. It is
kept only where it adds something and costs nothing:
- not when the text is JSON (the SDK's servers, every built-in among them,
answer a dict both ways), and not when the object only wraps the text again
(
{"result": <the text>}); - not when its JSON is longer than the guardrail let the model be shown of that call (the same limit the text was held to, counted as the guardrail counts, in characters; the evidence budget below counts bytes);
- never for an answer the guardrail cut, the conversation had no room for, or a repeat;
- in the evidence budget, every text of every loop is decided first exactly as with no structure, and kept structures are the first thing evicted: when a later loop's texts take room, the structures the agent still holds from its earlier loops are dropped largest first until everything fits, and a new structure is kept only while the room left holds it. A structure never costs a text anything.
Last, before the run's ledger is saved and the report is built from it, one
pass over the whole ledger drops kept structures, largest first, per agent
(the agent whose call it was, a callee's for the callee), until each agent's
structures fit in what its reporting.evidence_budget_bytes leaves after its
texts (none, when its texts alone fill it).
That covers an agent whose first record was already collected and that ran
again later (a revision or debate round, a judge round, a teammate asking it
again), whose earlier structures it could no longer drop itself. The pass
never touches a text and does nothing for a ledger with no such structure
(every built-in's). So in the saved ledger and the report:
- every text is what it would be with no structure kept;
- a structure never takes an agent over its budget: its texts alone may still exceed a budget exactly as they would with no structure (the judge, for one, keeps its answers with no budget while the run goes on), and then the agent keeps no structure;
- no entry holds a structure longer, in characters, than the model could have been shown of the call, though the structure may hold rows the prose text only summarised.
While the run is still going, the ledger in the run's state may hold, for a time, structures that the last pass will drop. A fact read from such a structure during the run (a reputation record, a technique a declared path names, a line of the judge's evidence) can stay on the record after the last pass drops the structure, as dev's budget already leaves on the record what was read from a text it later blanked.
A provider registered at runtime. A static or sandbox provider class
registered with register_static_provider or register_sandbox_provider
declares its own tools by answering declaration(cfg) with the declaration
fields of a server block (an object or a mapping); they are read under
provider:<id>, wherever a built-in provider's block is read, and switched on
while the settings select the provider. A class whose settings hold a server
block of its own names that block's dotted path in declaration_block
instead: the block joins the provider blocks, is read, snapshotted and
switched on and off as a built-in's is, and leaves them when the provider is
taken out. The built-in providers' blocks (static.ghidra, static.r2,
sandbox.cape2.mcp) are read as before, and no class may name another path
for one of them. Registration refuses, with the reason, a path where the
settings model holds no server block (a block with tool_facts) and a path
that is, holds or sits inside another provider's block, shipped or registered,
so two providers never read one block; a tool server's block in
mcp.servers is read under its own server's source and is refused too. A
third-party provider's own block has its place in the settings: see
Third-party provider blocks. A sandbox provider says where a submission sends the
sample in its submission_address() method: the host its submission URL
reaches, resolved as its HTTP client resolves it (the REST provider merges
sandbox.rest.submit.path with base_url as httpx does, so an absolute
submit.path names its own host, which is stated rather than refused);
None for one whose submission never leaves the worker, as the mock's
recorded fixtures and an attached report do; and "", the default, for one
that does not say. A run's disclosure block records that host when the
sample is submitted, and says it cannot tell where a submission went when
the provider does not say
(What left this machine).
Saving. A built-in's stored block takes the shipped value of any field it
never set, and one the operator set, empty included, is kept. A save stores
only the fields that differ from the shipped value at the time of the save, so
a value written equal to the shipped one is not recorded and follows later
releases. A server you add starts with no declaration. A run's settings
snapshot records only the declaration fields an operator set and names the
rest by a digest per source (declaration_digest.<source>) and one of the
shipped data (declaration_digest.shipped).
Tool declarations are saved one tool at a time. On a built-in block (a
built-in entry of mcp.servers, and the static.ghidra, static.r2 and
sandbox.cape2.mcp blocks' tool_facts), what is stored is each tool's change
over the shipped declaration of that tool:
| Stored for a tool | Means |
|---|---|
| nothing | the tool is declared as shipped, and follows later releases |
{"timeout_seconds": 900} | only these fields differ from the shipped declaration; the rest follow later releases |
null | a shipped tool you removed; it stays removed |
| a whole declaration | a tool the block does not ship (yours) |
On load the shipped declarations are merged with what is stored field by
field, so a correction a later release ships for any tool you did not change
reaches the run, and a tool shipped later appears. The declaration box shows
the declarations in force (shipped, with your changes over them), lists each
tool that carries a change (changed, removed, or yours) and has a Reset to
shipped per tool; a provider block's tool declarations are edited the same
way under its Declaration heading. A field holding a map or a list
(arguments, places, values, techniques, reputation, words,
report_sections) is one field: changing one key of it stores the whole field,
which then no longer takes a key a later release adds to it. A server you add
has nothing shipped, so its stored map is its whole map.
A map stored as changes carries a mark, "": "changes" (the empty key, which
no tool can be named by), and GET /api/v1/settings and the export serve every
stored map in this marked form. A map without the mark was stored whole by an
earlier release and is read whole, exactly as that release read it: a shipped
tool it does not name stays removed. Nothing tells the two forms apart by their
shape.
A map sent through the API is read by the same mark. On every path that
takes one (a save through PATCH /api/v1/settings, a connection test, a
probe's candidate settings, and an import in either format), a marked map is
the block's changes, so a tool it leaves out is not removed and null removes
one; a map without the mark is the whole map, as earlier releases read every
map, so a shipped tool it leaves out is removed. A maljan-settings/1 import
reads the same way: its maps carry no mark, so they are whole, and a map marked
as changes (only /2 writes the mark, so such a file was relabelled from one)
is its changes there too, never a whole map that would remove every shipped
tool. A map sent back exactly as it is stored keeps the
reading it is stored with, since it says what the stored one says; the console
always sends the marked form it was served.
When a later release changes a shipped declaration so that your stored change
of that tool no longer validates over it (a field you set needs one the release
no longer ships), or stops shipping a tool you changed (so your change alone
is now its whole declaration), the change is left out of what is in force with
one warning in the log: the tool runs as shipped, or not at all where nothing
ships for it, and the rest of the settings is built. No stored change stops the
settings from being built. The declaration box marks that tool as conflicting
with its stored change (the settings schema lists it under
tool_facts_conflicts), with Reset (Reset to shipped for a shipped
tool), which removes the stored change, or you can edit it. Until then a save
that sends the change back as stored, also one that edits another tool of the
same block, keeps it exactly as stored. A stored tool named as its own
server's renamed tool (<server>__<tool>) is left out the same way.
A stored value of a form no release writes (a block's tool declarations that
are not a map, or a map whose "" key holds anything but "changes"; only a
hand edit of the database makes one) is read as absent: what ships is in
force, one warning in the log says so, the settings schema lists the "" key
under tool_facts_conflicts, and the settings serve the block as no change,
so saving it writes a marked map anew. A save or an import that names a tool
by an empty or padded name (also with the value null), names a server's tool
as that server's renamed tool, or sends a value that is not a map, is refused
with the key named. Declaration data nested at any depth is read without
failing: the settings build, and a save of such data is refused with the key
named.
Settings saved by an earlier release stored a built-in's tool declarations
whole. The database migration (revision 20261012000000) rewrites each such
map once: against the declarations of the release that wrote it (inferred from
a frozen record of what every earlier release shipped), a tool stored as that
release shipped it is dropped and follows today's declaration, an edited tool
keeps the fields you changed, and a missing shipped tool stays removed. Where
the writing release cannot be told apart, the stored map is kept whole as
yours. A server you added ships nothing, so its whole map is its changes: it is
marked and reads as before. A provider block's tool declarations that no
earlier release shipped any declaration for (the sandbox.cape2.mcp block
today) are left as stored and read whole until a save of this release writes
them marked. Each rewritten map is marked; a marked map, one this release saved
also before the migration ran, is left alone, so running it again changes
nothing. It logs one line when it rewrites anything. Run the migration
(make migrate) before starting this release: until it has run, a map an
earlier release stored reads as that release read it, and the API logs one
warning at startup while the database is behind its latest migration (another
when it is ahead of it, migrated by a later release). A downgrade writes every
marked map whole again.
A declaration for a server you add. docs/examples/profiles/qu1cksc0pe.declaration.json
holds the declaration fields of a Qu1cksc0pe server, its six tools' kinds,
path argument and budgets. It is not a settings import document (an import of
core.mcp.servers replaces the whole server map): paste its tool_facts value
into the server card's Tool declarations box and its other fields into the
boxes of the same names, and save.
What each tool's answer shows, told to the model
core.mcp.answer_scope_notes (Tool servers → Tool descriptions, off by default) decides whether a tool's description, as the model is shown it, ends with what its declaration says its answer shows and does not show (shows, does_not_show). Off, the declarations are shown in the console and on this page only, and every prompt and stored run reads as before; it stays off until an A/B measures what it changes. The notes after a description (this one and aggregate_of's) are written once over the agent's whole tool list, grouped by the source each tool is declared under, so a server's tool reached through one reference names the tools of the same server another reference or binding offered, and a note that names a tool the agent is not offered is taken off.
A tool server that keeps failing is rested
Per job and per tool server, three settings under Tool servers → Resilience:
| Setting | Default | Where the default came from |
|---|---|---|
core.mcp.breaker.failures_to_open | 3 | The number of attempts the platform already gives a model call that drops its connection before calling it a failure. No recorded live run had a tool server fail at the transport, so it is a judgement, not a measurement. |
core.mcp.breaker.cooldown_seconds | 60 | A judgement: long enough for a sidecar being restarted to come back, short against the analysts' own loop budgets. |
core.mcp.breaker.call_timeout_seconds | 0 (per server) | Set, the budget of every call to every server. Zero gives each server the call_timeout_seconds its own block declares, and a server that declares none (every built-in) core.mcp.breaker.default_call_timeout_seconds. A tool whose server declares a longer budget in its manifest or its declaration (tool_facts.<tool>.timeout_seconds) gets that, a long-running tool (by either) none unless this is set, and thirty seconds of grace are added either way. |
<server block>.call_timeout_seconds | empty | Each server's own budget per call (an mcp.servers entry, static.ghidra, static.r2, sandbox.cape2.mcp), read when core.mcp.breaker.call_timeout_seconds is zero; zero sends its calls with no deadline of their own, empty takes core.mcp.breaker.default_call_timeout_seconds. Ghidra's REST requests (open, call graph, anti-analysis scan) and radare2's open before the loop wait it too. |
core.mcp.breaker.default_call_timeout_seconds | 300 | The budget per call of every server whose block declares none (every built-in), read alike whatever the server or its tools are called; zero sends their calls with no deadline of their own. 300 is the floor every server got before servers declared their own, the longer of the two static timeouts as they ship. It used to be derived from core.static.capa.timeout_seconds and core.static.yara.timeout_seconds, two blocks named for one provider's tools; the upgrade wrote a deployment's own floor here (900 on a slow host still fits), so no server's budget changed, and importing a maljan-settings/2 or /1 export gives it the floor that export carried. |
core.mcp.breaker.max_concurrent_calls | 4 | A judgement: the shipped teams run their analysts one after another, and four lets one analyst's parallel tool calls through while bounding a team that fans out. 0 leaves the calls uncapped, as every server was before. |
An unanswered call is either a transport failure — a timeout, a refused or dropped connection, the server's process gone — or a call that did not finish within its caller's own budget (a loop's or an ask's) while it waited on the server. Every call is sent with a deadline (above), so a server that hangs times out and is counted; a call its caller's budget cut short is counted too, under its own reason, rather than let go. After that many in a row the server rests: a call is not sent, and the model is answered with a tool error in the structured shape —
{"error": {"code": "server_resting",
"message": "tool server 'analysis' is resting after 3 calls in a row it did not answer; it will be tried again in 60 s",
"remediation": "this server did not answer several calls in a row and is not being called for now; use another tool, or call this one again after the time the message names"},
"tool": "pe_info"}— which the ledger records as a failed call like any other. After the cooldown
one call is let through (the others are told that one call is trying the server
again and nothing more is sent until it answers); a success ends the rest and
its own failure to answer starts another. The guard covers every server the job's
registry attaches; the Ghidra and CAPE providers' own toolkits are outside it. A tool that answers with its own error (a bad argument, a file
that is not there) has answered, and never counts. Each rest is published as a
tool_server_rested event, drawn in the conversation, and kept in
run_summary.server_rests, which the report and the console's "What the run
spent" print. The event and the record name the server by its key (server)
and, where its block has a label other than its key, by that label too
(label); the sentence printed names it Label (key), else by the key, so a
server with no label of its own reads as before. The call cap queues calls per event loop: a handle is opened per
loop, and for a stdio server that is one child process per loop.
Writing a tool server
Any MCP server works: Malspark reads its manifest and calls its tools. Three
optional conventions make a server a better citizen of a run, and the four
built-in sidecars follow all three; every other sidecar under services/
declares its tools too.
The capability manifest. A tool named capabilities, taking no argument,
answers what the server can do on the host it runs on:
{"server": "analysis", "version": "1.0.0",
"tools": [{"name": "document_info", "optional_dependency": "olefile",
"available": false, "reason": "olefile is not installed",
"timeout_s": null,
"remediation": "install the optional tool libraries on the host that runs this server: uv sync --extra tools",
"without": "the PDF and OOXML halves"}]}Compute it when the server starts, by probing — an import, a which, an
environment variable — never by asserting; maljan.tools.capabilities.manifest
does that for a list of ToolNeeds and is what the sidecars use. The registry
reads the manifest once per job when it attaches the server and keeps it on
the server's entry; the connection test (POST /api/v1/settings/test/mcp)
returns it under details.capabilities, and the console's server card lists
the unavailable tools with their reason before any run. When an analysis
stage's agent starts, each tool it binds that its server's manifest marks
unavailable is recorded once as
server.<key>.<tool>_unavailable(<reason>); still answers <without>; <remediation>
in the run's degradation reasons, instead of being discovered by a failed call
mid-run. A tool whose declaration names the routed formats it serves
(routed_formats) is not recorded for a sample of another known format: a
missing Mach-O library costs a PE sample nothing. timeout_s is the tool's real timeout, taken from the constant the
tool itself uses, so a manifest cannot say a call has none when it gives up
after fifteen seconds. A tool missing its library or binary makes the run degraded, as
does a server that could not be attached and any triage pack tool that failed: each is
evidence the run would otherwise have had, and the degraded-run banner names each tool the
run went without; one decision (pipeline.operator_choice.is_loss) sets both, so a
degraded run always names a lost tool or a loss that is no tool. A tool cost the sample
nothing, and neither degrades the run nor is named, when it serves only other formats
than the sample's known one (its declared routed_formats, or its pack step's formats:
a Mach-O parser on a PE), or when it still answers part of what it would (without, or a
pack answer marked degraded) and the part it lacks is not the sample's: the tool
declares the routed formats its missing part reads (ToolNeeds.without_reads, 7z for
archive_list without py7zr; the job's own manifest first, the shipped declaration
otherwise), the sample's format is outside them, and the tool did not answer the run with
its missing-dependency error. On one of those formats, after such an error, for a tool
that declares none, or when the run holds no format, it is a loss.
A tool whose manifest cell carries the code not_configured is the operator's choice
only when the claim holds against what the platform knows of that server: the cell names
at least one setting (optional_dependency), each is one the platform can hand the
server (a key of its env or a name in its env_allow), none holds a value on the
platform's side, and the reason is exactly the probe's <NAME> is not configured for
each. A manifest speaks only for the server that answered it: one naming another
server's key is read under its own. Such a tool, and a referenced tool of a server
switched off, cost the run no evidence its configuration would have given: they are
listed apart as the operator's choices (operator_choices in the report and the run
summary, §13 of the report, the console's run record, the rehearsal gate's output), not
among the degradation reasons. Anything else is a loss, whatever its words say. The run
summary keeps what the degraded flag was decided on (loss_facts), and the header names
the lost tools from it.
Data a tool reads. A ToolNeeds entry may also declare a data path
(maljan.tools.capabilities.data): the default path the tool reads, the core
setting whose value a caller that reads it hands the tool instead, the server
environment variable that replaces the default, and the file pattern a directory
must hold. The built-in sidecars declare the corpora that are fetched rather than
tracked: capa's rules (*.yml; static.capa.rules_dir, which the analysis
server's capa tool reads as MALJAN_CAPA_RULES_DIR) and the Sigma corpus
(*.yml or *.yaml; MALJAN_SIGMA_RULES_DIR, else data/sigma_rules).
A data path declared with both a setting and a server variable is read in one
place by every reading of the tool: when the setting names a path other than the
default, a server of this repository launched over stdio (services/<dir>/server.py)
is started with the variable set to the setting's value, resolved against the
project root to an absolute path, so the worker's own
reading, the static provider and the agents' calls to the server read one rule
set. A server block whose env sets the variable keeps its own value; a server
reached at an address or launched as another program (a container) keeps its own
environment; under the default nothing is added to a launch.
POST /jobs and the system status look at each declared path where the tool
would read it: for a tool a server runs, the setting its launch is handed, else
its override or default; for the worker's own reading, the setting. A tool whose path is missing, an empty
file, or a directory holding no matching file is named as unavailable: the reason
gives what named the path, and the path relative to the project root
(data/capa-rules is missing or empty (named by the default); a path outside the
project is not shown), and the remedy says how to put it there. A server whose
block sets PYTHONPATH has neither its modules nor its data checked. A server
does not probe a data path in its manifest. A tool that declares no data is
checked as before.
Errors that name their remedy. A tool that cannot answer returns, never raises:
{"error": {"code": "missing_dependency",
"message": "olefile is not installed",
"remediation": "install the optional tool libraries: uv sync --extra tools"},
"tool": "document_info"}The codes are missing_dependency, timeout, bad_argument, no_such_file,
unsupported_format, not_configured and tool_failed; a code of your own is
kept as written. maljan.tools.errors.tool_error(code, message, tool=...)
builds the shape with the authored remediation for the code, and each sidecar's
guard maps an exception it catches to a code. The older flat shape,
{"error": "<text>"}, is still read: the sidecars rewrite it into the
structured one on the way out, inferring the code from the text, and the
ledger reads either. A returned error is a failed ledger entry: ok false,
error the message, remediation the hint — the report header lists each
distinct failure once with its remedy, the console's evidence row shows both,
and GET /api/v1/jobs/{id}/evidence serves them.
Tools that declare themselves. A server may say what each of its tools is
when it lists them, so it is read at full strength with nothing stored for it.
Each tool's MCP annotations give its title and the hints readOnlyHint,
idempotentHint, destructiveHint and openWorldHint (kept on each of the
tool's ledger entries as the server's hints), and the tool's _meta holds,
under the key maljan/tool_facts, its declaration in the vocabulary of
tool_facts (below: kind, rows, places, values, techniques,
routed_formats, shows, does_not_show, effects, long_running,
timeout_seconds, reader, arguments, read_only and the rest):
{"name": "scan_file", "annotations": {"title": "Scan a file", "readOnlyHint": true,
"idempotentHint": true, "destructiveHint": false, "openWorldHint": false},
"_meta": {"maljan/tool_facts": {"kind": "identification", "rows": "/detections/*",
"values": {"name": "/name"}, "arguments": {"path": "path"}, "read_only": true,
"effects": {"executes": "no", "network": false, "third_party": "no"}}}}The client reads it for every server alike, whenever it attaches one,
leniently (a field this version does not know is left out with a warning; a
declaration the model refuses declares nothing, with a warning), and keeps it
under the launch identity of the block that reached the server (its transport,
command, arguments, working directory and address; a command is read as the
file it runs, so bin/python and its link bin/python3 in one directory are
one launch), so a block pointed at another server reads none of the last
one's. What it keeps outlives the process: each server's last declaration is
kept in the database (served_tool_declarations, one row per source), read
by the worker at the start of each job and by the API before it answers what
depends on it, and written back after each job and connection test. A block
whose launch identity changed forgets what its server declared. How much of
it is read is the block's choice, trusts_served_declaration (in the table
above):
- A block that trusts it (the platform's own sidecars ship it set; set it for a server you vouch for) reads it as a declaration, under what the block puts in force, per tool and field by field: a field an operator stored for the tool (or the platform ships for a built-in server's tool) is in force over the server's, the server's fills what neither says, and a tool nothing declares stays undeclared. A built-in tool an operator removed stays removed whatever its server says.
- Any other block reads only the claims that make the platform more
careful: the tool sends the sample (
discloses_sample), sends its hash or bytes to a third party, reaches the network (true, never a named way such asoperator_sandbox), runs something, or is not read-only (said outright). Each field such a claim sets that the block does not is stored with the tool's entries as the server's own statement (servedin the entry's facts): the tool-effects part says it as the server's, the disclosure counts the call where the block's own declaration places it (an undeclared tool stays undeclared), never moving it to an upload, lookup or server row, and no reader of a stored entry reads it as declared. Its kind is never put in force, since a kind routes a tool, names its report sections and tags it: a kind whose answer moves or that reaches outside the deployment (never a reputation kind) is stored apart as the server's claim (served_claims), said as the server's, and read only to keep an identical call from being answered from an earlier entry when the kind's answer moves. The rest of the declaration, its kind included, is what the connection test proposes, each field with medium confidence as the server's own claim (high only for a block that trusts it), for you to accept; it never offers a field less careful than what the tool's hints already say (a sample sent, a network reached, a tool that may change something).
No block takes a kind of what a sandbox observed (sandbox_…) from a
server's claim unless the block's roles hold sandbox. Each entry stores
the facts in force when it was written, so a stored run never reads a
server's declaration again; a run's recorded declaration digests are of the
blocks alone, and what each server declared is recorded by its own digest
beside the name and version it gave (the run fingerprint's server versions).
What reads a server's declaration: everything that reads a tool's
declaration, before any server attaches too: the facts each entry stores,
every reading of the live declarations (core.declared_reading.declarations),
the call guard's per-tool budgets and long-running flags, the description
tags of its tools, the tool names the prompts know, the static pre-pass's
tools (prepass_tools), the triage pack's choice of the server whose
environment an in-process step reads and of the server its readings run on,
the reputation lookups it asks and their order, the formats a tool serves,
the servers a capability reference binds and the check of a pack step's
kind. In FastMCP, maljan.tools.declared_facts.tool_declaration writes both
parts for @mcp.tool(...); every built-in sidecar declares its tools so, and
a test holds what each declares equal, field by field, to what the platform
read for it before. The platform ships no table of its own sidecars' tools:
what it reads of them is what each serves.
Before a server's first listing. On a fresh install nothing is kept yet, and a server is read as any undeclared server is: nothing about it is refused, and its tools are offered and called as an undeclared server's are. So that no run is weaker for it, a job that runs for real (never a mock run) lists, before it starts, each enabled server whose block trusts its declaration and for which nothing is kept (the handshake a connection test makes; no tool is called), at most four at a time within thirty seconds, and keeps what each declares; a server that cannot be reached then, or whose listing declares nothing, is read as undeclared until it attaches, is never refused, and is not reached early again for ten minutes. A server whose block does not trust it is not reached early: only its cautious claims would be read, and its own attach reads them. A row listed through the launch a server ships with is read under the shipped launch of the host that reads it, so an API on the host and a worker in a container read the same row. A connection test keeps what it listed only where the entry tested is the saved block, as saved; a staged or unsaved entry's listing is not kept.
The budget meter needs nothing from a server. The tool loop emits
budget_tick every five steps and once more when it ends (steps used against
the cap, seconds against the limit, prompt characters, the characters of the
tool definitions sent with every request, ledger entries so far)
and stage_ended_at_cap when a cap ended the work — steps, time,
repeats or, for the triage pack, budget_seconds; run_summary.budget sums
the spend per agent with the caps it hit and keeps the largest
tool_definition_chars of its loops, and the console's pipeline panel
says beside the step which cap ended it.
Provider plugins
A provider that is not an MCP tool server, such as a model API that is not OpenAI-compatible, a sandbox with its own client or a decompiler with a library of its own, can come from an installed Python package. The package declares it through an entry point in the group of its kind:
| Kind | Entry-point group | The entry point names |
|---|---|---|
llm | maljan.llm_providers | a subclass of maljan.llm.provider_base.LLMProvider with a build_model method |
static | maljan.static_providers | a subclass of maljan.providers.base.StaticProvider, its abstract methods written |
sandbox | maljan.sandbox_providers | a subclass of maljan.providers.base.SandboxProvider, its abstract methods written |
memory | maljan.memory_backends | a callable that calls register_memory_backend(<name>, builder, ...) |
The entry point's name is the provider id the plugin adds: it is what
llm.provider, static.provider, sandbox.provider, memory.backend, an
agent's static_provider or a job's provider override names. It is
lowercase letters, digits, _ and -, and starts with a letter. A package's
pyproject.toml declares one like this:
[project.entry-points."maljan.static_providers"]
acme = "acme_maljan.provider:AcmeProvider"The classes are the ones the built-in providers derive from, so a plugin
answers the same interface and declares the same facts about itself (its
console title, its settings model, its probe). An LLM provider class may leave
its name empty or set it to the entry point's name; a memory plugin's
callable registers exactly the entry point's name.
Third-party provider blocks
core.providers (Settings → Plugins → Third-party provider blocks, a JSON
editor) holds one server block per third-party provider, keyed by the
provider's id: the fields of a tool server's entry in mcp.servers, saying how
the provider's server is reached and what it declares about its tools. Empty by
default, and settings stored before it existed read exactly as before.
{"mydecomp": {"enabled": true, "transport": "streamable-http",
"url": "http://mydecomp-mcp:9000/mcp", "env_allow": ["MYDECOMP_TOKEN"],
"tool_facts": {"list_strings": {"kind": "strings", "rows": "/strings/*"}}}}The block declares the provider's tools under provider:<id>, read wherever a
built-in provider's block is read, with no code of the platform's naming the
provider. It comes before the provider class's declaration(cfg) hook, which
answers only while no block is set, and the run's settings snapshot records
its declaration fields as it does a built-in's. The provider reads how its
server is reached from the same block
(maljan.core.tool_declarations.provider_block(settings, "<id>")), and a class
may name providers.<id> as its declaration_block; another provider's entry,
the whole map or a path inside a tool server's block is refused at
registration. A provider whose class names another block keeps it: its entry
here is not read, which is logged once, and a save that writes one is refused.
Each block validates as a tool server's entry does, and a save refuses with the
reason, at core.providers.<id>: an id that is not a provider id; the id of a
built-in provider, which keeps its own block or declaration; a tool declared
under the provider's renamed spelling (<id>__<tool>); and a tool declaration
the save adds or changes that breaks the strict rules (one sent back as stored
is not checked again). The map is stored as one clear row, so a block holds no
auth_token, no env value and no url with a user name or password, and
takes no credential_bootstrap: a
credential reaches the provider's server by name, through env_allow, from
the worker's environment.
Allowing a plugin
No installed plugin is loaded until it is allowed. plugins.allowed (Settings
→ Configuration → Platform → Provider plugins, admin only) holds one entry per
plugin, written <kind>:<name> (static:acme); it is empty by default. It
holds names only, never a module or a path. The page lists every installed
plugin per kind, read from the package metadata, with its package, its version
and its status: Allowed, Not allowed, or Refused with the reason the metadata
gives (a built-in id, a name two packages declare, a name that is no provider
id). Ticking Allowed stages the entry, and Apply saves it. The API column
is the API process's own load, where the API ran that plugin's code; whether a
plugin loads in the analysis worker is written to the worker's log, each load
with its package and version and each failure with its reason, and the page
never shows the API's load as the worker's. An entry no installed package
declares is listed as such, to be removed. GET /api/v1/settings/plugins
answers the same list.
When a plugin is loaded
Reading a registry loads no plugin. The analysis worker loads the allowed plugins at start, so a failure is in its log before any job, and again as each job starts, by the list the job's own stored settings carry, before those settings are built: a list saved while a job runs applies to the next job, and a running job keeps the plugins it started with. The enrichment worker loads the allowed memory plugins before it builds the memory store. The command line loads the plugins its settings allow, then reads its settings again, so a plugin's own block is read with its model, as the worker does.
The API imports no plugin to list plugins, to judge a saved provider id, to
offer the provider choices or to serve the settings pages, and none at start
or for a save, import or reset that writes no plugin's own settings. It loads
an allowed plugin only where a request runs that plugin's own code, and that
plugin alone: a save or import that changes a key of an LLM or memory
plugin's own settings block (llm.<name>.*, memory.<name>.*) and does not
drop that plugin from the list, a connection test of the plugin (for the LLM
test, of the provider it would ask; for an agent's
test, of the model, static and sandbox providers that agent runs on), and the
memory purge and the deep health check (/health?deep=true) when the selected
backend is a plugin.
An installed plugin's block the API has not loaded is shown on the settings
page from its stored rows, typed from their values, under the heading
<kind>:<name> (as stored); a credential among them is shown as set, never
its value. Changing one loads that plugin to read it while the list allows it;
clearing one (a null) or resetting it deletes the stored row without loading
anything. A plugin's own fields join
the page once the API has loaded it: after its connection test
(POST /api/v1/settings/test/<name>; for an LLM plugin, the LLM test with it
selected), after the deep health check or a purge while it is the selected
memory backend, or after a save or import that writes a key of its block once
the saved list allows it.
Those loads run on one thread of their own, never on the API's event loop. A request waits for one at most a connection test's budget (10 s) or, for the health check, that check's own (3 s), then goes on without the plugin, and once a load has run longer than a request's budget no request waits on it. Loading a plugin is logged with its package and version. A plugin that fails to import, or whose entry point names something that is not a provider of its kind, is logged with the error and the process goes on without it; a job that names it fails with that reason. A plugin the list no longer names is withdrawn from its registry at that process's next load; its code stays imported in that process until the process restarts.
Every shipped provider registers before any plugin is imported. The loader
never registers a plugin under a shipped id: a plugin named like one (ghidra,
cape2, openai, qdrant) is refused and never imported, and a registration
a plugin makes while it loads, under a shipped id or under any name but its
own, is refused with the reason. Two installed packages that declare the same
name are both refused. These are the loader's guarantees, not a sandbox: once
allowed, a plugin's own code runs with the process's full access and could
change a registry outside its load.
When a plugin hangs or ends the process
- It hangs as the API imports it. The request goes on without it after its budget, but the API's one loading thread stays with the hung import: every later plugin load in the API (another block, a test, the memory purge and deep health check of a plugin backend) waits until the import ends or the API restarts, and stopping the API waits for that thread until it is killed. Remove the plugin from the list (Provider plugins, Apply, with a new value for any setting that selects it), then restart the API: removing it alone does not free the thread.
- It hangs as the worker imports it. At start the worker then takes no
job; as a job starts, that job never goes on (the job timeout,
core.job_timeout, none by default, watches only the run that follows the load), and the next job's load waits behind it. The console stays reachable: remove the plugin from the list there (with a new value for any setting that selects it), then restart the worker. - It ends the whole process as it is imported, rather than raising. The
process that imports it goes down: the worker at start or as a job starts,
the API on the requests above. The settings pages, the listing, and a save,
import or reset that writes no plugin's own block never import it, and
neither does a save or import that drops it or sends its block back as
stored, so it can be removed: save the list without it, together with a new
value for every setting that selects it (a save that drops a plugin a
setting still selects is refused, naming those settings); or export the
settings, edit the list and those settings in the file, and import it; or
reset the Provider plugins group or call
DELETE /api/v1/settings/core.plugins.allowed, either of which removes the whole list once no setting selects a plugin on it.
Saving a provider id
A provider id is saved when it is a shipped one, or when it is an installed
plugin of its kind that the list the save leaves names, read from the package
metadata without importing the plugin and never from what the API process has
loaded; the worker judges the stored settings the same way. So one save may
allow a plugin and select it, but may not drop it and select it. A save, or a
reset of the list, that drops a plugin a provider id still selects
(llm.provider, static.provider, sandbox.provider, memory.backend, an
agent's static_provider, a per-agent model or one of its fallbacks) is
refused, naming each key that selects it. Any other refusal gives the cause:
the plugin's own refusal, a list that does not name it, an entry no package
declares, or how a plugin is accepted. Changing the list needs an
administrator, and each change is logged with who made it and kept in the
audit trail.
The checks that read an LLM or memory plugin's own code run only in a process
that has loaded it, and the worker's, as a job starts, is final. Where the API
has not loaded the plugin, they wait for the worker: a missing llm.<name>
block that names the role models (accepted when a job is submitted, refused
when it starts), the selected LLM provider's block at a save, a per-agent
base_url or reasoning effort for an LLM plugin, and the validation of an LLM
or memory plugin's own block against its settings model. A plugin's block is
also known only once the API has loaded the plugin, so a save or import that
allows a plugin and writes its block at once is refused for those keys, with
the reason: save the list first, then the block. A save or import that drops a
plugin never loads it, so a change to its block in the same save is refused
for those keys, with the reason. A block key a save or import sends with the
value stored for it, compared exactly as JSON (1, 1.0 and true differ;
key order does not), writes nothing: it is left out without a word and loads
no plugin, whether the plugin is allowed or not, so an export imported back
unedited, or with only its list and selections edited, loads no plugin. A
secret is never compared: one sent is always a write, judged as any secret is,
and no answer depends on what a stored secret holds (an export omits secrets,
and the console never sends a stored one back). A changed value in a block
whose plugin the change does not load (dropped, not allowed, or failed to load)
is refused with the reason, and a null that clears a key deletes its stored
row without loading the plugin.
Memory plugins written to the older advice
Memory backends were loaded from installed packages on their own before the
list existed. Revision 20261016000000 adds the one memory plugin the stored
memory.backend names, when it is installed in the image that runs the
migration, to the list once, as memory:<name>, so a deployment that used one
keeps it; nothing else is added. A memory plugin's entry point must now be named
after the backend id it registers, in lowercase; the older advice only
recommended it. When the stored backend is not shipped and no installed memory
entry point carries its name, the revision says so in its output and its log,
naming the installed memory entry points, and the backend does not load; the
health check and every job then name those entry points and the rule. To keep
it: install a release of the plugin whose entry point in the
maljan.memory_backends group is named after the backend id, then allow
memory:<id>; or select a shipped backend.
What allowing a plugin trusts
Allowing a plugin runs its package's code in the workers and the command line, and in the API where it runs that plugin's own code, with their access: the settings and the credentials they hold, the samples, the database and the network. Allow only a package you would install into the image yourself, from a source you trust; the list decides what is loaded, not what is safe.
Export and import
GET /api/v1/settings/export (admin) returns the configuration as JSON and
sets Content-Disposition: attachment; filename=maljan-settings.json:
{
"format": "maljan-settings/3",
"exported_at": "2026-09-12T00:00:00Z",
"values": { "core.llm.provider": "openai" },
"secrets_omitted": ["core.llm.openai.api_key"]
}- Only keys the store actually holds are exported — values still on their catalog default are left out, as are read-only entries.
- No credential is ever written to the document. Secret entries are skipped,
masked values nested inside a composite (an MCP server's
auth_token, a frontier arm'sapi_key) are stripped at any depth, and every value in a server'senvmap is masked while its variable names stay. secrets_omittednames each credential the document does not carry, so an operator can see what has to be re-entered on the other side. Those paths are informational, not catalog keys.- A stored row nested deeper than an answer can be written (the bound is the
JSON writers' own, never a depth Malspark sets) is left out and named under
unreadablewith why, which the console names beside the download; an import ignores that field.GET /api/v1/settingsserves the same row with no value and its reason inunreadable, which the console shows on the row, and a warning is logged, so one such row never fails the settings page or the export.
POST /api/v1/settings/import (admin) accepts the same document. In
maljan-settings/3, as in /2, a built-in block's tool declarations are stored as their
changes over the shipped ones, one tool at a time (see Tool declarations are
saved one tool at a time), so an export and import round trip keeps exactly
those changes. A map in a /3 or /2 document without the mark is read whole, as
a stored one is. A document in maljan-settings/2, written before a server's
call budget had a setting of its own, is read as /3 is, except that where it
carries core.static.capa.timeout_seconds or core.static.yara.timeout_seconds
and no core.mcp.breaker.default_call_timeout_seconds, the budget is given the
floor that document gave every server (the longer of the two, each at its
shipped default where it is absent), and its YARA timeout is the check budget
core.reporting.detection_rule_check_seconds; a /3 document's static
timeouts are their own. A document in maljan-settings/1, written before, is still
accepted: a built-in block's tool declarations there are its whole map, so a
shipped tool it does not name stays removed and a tool it names reads as
written; a map marked as changes reads as it does in /2, and its call budget as in /2. A format other than these three is rejected with 422; unknown or read-only keys
are rejected with 422 and a per-key error, and nothing is applied when any key
fails. A key of a setting retired with no effect — core.preprocessing.summarizer_provider,
core.preprocessing.summarizer_model, core.reporting.narrative_max_tokens —
is refused the same way, and its error says it is retired, what replaced it and
to delete that key from the file and import again; an export written before the
retirement carries it only if the setting had been saved. A successful import
writes one audit row (settings.import) naming the applied keys. The console previews the document against the current
configuration before sending it.
Secrets
Secret settings are encrypted with Fernet under SETTINGS_ENCRYPTION_KEY and
stored as enc:v1:<token> (src/maljan/core/settings_secrets.py). The API and
the worker both read the key from their own environment, so both can open the
same rows.
Credentials nested inside composite settings are never kept inside the
composite row. An MCP server's auth_token and a frontier arm's api_key are
split into their own encrypted rows and merged back when the value is read, and
a startup repair moves any that a previous version left inline
(apps/api/app/services/composite_secrets.py). The console never echoes a
stored credential: it shows a mask and a short hint.
Rotation caveat. There is no re-encryption step and no multi-key reader. If
SETTINGS_ENCRYPTION_KEY changes, existing secret rows can no longer be
opened: the service logs a warning per row and the setting falls back to its
default, while the console still reports the secret as set. Re-enter each
secret after a key change, or restore the previous key.