Integrations
HTTP API
Router groups, the evidence endpoint, the run-summary fields, the stage events on the WebSocket, authentication and pagination.
Every route is mounted under /api/v1, except the health endpoints and the
WebSocket. This document describes the router groups and the conventions that
hold across them; the exact request and response models are in the generated
schema rather than repeated here.
Schema
FastAPI serves /docs, /redoc and /openapi.json only when DEBUG is true.
A production deployment publishes no schema, so read it from a development
instance, or from the routers under apps/api/app/api/v1/.
Authentication
Send either an access token or an API key:
Authorization: Bearer <access token>
X-API-Key: <key>A request carrying X-API-Key is judged on that key, except that changing the
password, minting or revoking API keys, reading or exporting the settings, and
every settings write (a save, a reset, an import, a server credential's
bootstrap, a connection test, accepting a sandbox report format) take a bearer
session and refuse a key with 403. A save, import or reset that changes the
sign-in policy (the rate limiter, the login lockout, the trusted proxies, the
token lifetimes)
also needs current_password. The refresh token is an HttpOnly cookie scoped to /api/v1/auth
and is never sent by hand. See security.md.
Router groups
| Prefix | Purpose |
|---|---|
/auth | Register, log in, refresh, log out, and read or update the current user. |
/samples | Upload a sample, list and read sample metadata, delete a sample. |
/samples/{sample_id}/sandbox-reports | Attach, list and delete sandbox reports produced elsewhere, for the upload sandbox provider. With unread in sandbox.upload.allowed_formats, a report no format recognises is accepted as unread, and the answer carries a proposal (a format declaration with each field's evidence and confidence); .../{report_id}/proposal reads it again. |
/sandbox-report-formats | Every declared sandbox report format, built-in and your own, in the order a report is tried against them: how each is recognised, where its task id and target hash sit, its reader, its signature scale and whether uploads accept it. POST /sandbox-report-formats/accept (admin) saves an accepted declaration for an unread upload, only where it recognises that report, and reads the upload as it. |
/jobs | Create an analysis job, list and read jobs, read a job's event history and its evidence ledger, ask a finished job a follow-up question, cancel a job. |
/reports | Everything a finished analysis produces: the report itself, its renderings, its indicators, its signatures, its timeline, and post-hoc enrichment. |
/dashboard | Aggregate counts for the console's landing page, and GET /dashboard/tools?limit= — the per-tool call counts of the caller's last limit completed runs (default 20, at most 100), read from each run's run_summary.evidence.by_tool. |
/audit | The audit trail, admin only (/audit/logs), and API keys (/audit/api-keys): any signed-in user mints, lists and revokes their own keys, and a key acts as the account that minted it. |
/settings | The settings catalog, values, patches, resets, export, import and the connection probes. Admin only. |
/system | Non-secret pipeline-mode flags for dashboards, and long-term-memory maintenance. |
Outside the prefix: GET /health and GET /healthz (see
deployment.md), and the WebSocket at
/ws/analysis/{job_id}, which streams one analysis run's events and
negotiates the maljan.v1 subprotocol when the client asks for it.
Jobs
GET /jobs/{job_id}/evidence is the analysis's evidence ledger: one entry per
tool call, in the order the calls were made. An entry carries the citation id
the report quotes (entry_id, e.g. ev_0007), the stage, the agent, the tool
server (null for an in-process tool), the tool, the arguments, whether the call
succeeded, its duration, the result text and the parsed result when the tool
answered JSON. Beside those it carries every other field the run recorded on
the entry: where the tool's declaration lives (source), what that
declaration said about the answer (facts), the reputation record
(reputation), the operator's triage pack step (pack_step), whether
structured is the server's own structured answer (structured_from_server,
present only when true), and the calling agent's declared role and console
tab (agent_role, agent_tab). Each of these seven is left out where the
entry carried none, and on every entry stored before they were kept: absent,
never false or empty. stage, agent and tool narrow it — the three grains the
console groups by — and page and page_size (up to 200) page it. Ordering is
by the sequence the ids were issued in across the whole run, so paging walks
the analysis rather than one agent at a time, and the page an id is on is
therefore arithmetic: ev_0051 is the first entry of page two at the default
page size.
Ownership is the job's own — the same rule the job's report endpoint applies —
and an entry whose output was dropped to the per-agent byte budget comes back
with an empty output and the rest of its record intact.
An entry the run took a part of its answer from also carries spilled: one
reference per part kept whole in object storage — answer (the tool's answer
before the output guardrail cut it), structured_content (the server's
structured answer the entry did not keep), output and structured (what the
byte budget or the structure eviction took) — each with key, bytes and
sha256, or with error and no key when the part was not written
(upload_failed (<class>), upload_failed (store unreachable),
upload_failed (store busy), upload_failed (not finished), or
exceeds upload_max_bytes (<n> bytes)). It is absent where nothing was taken
and on entries stored before it was kept.
GET /jobs/{job_id}/evidence/{entry_id}/objects/{part} streams the part's
bytes (text/plain for a text, application/json for a structure), with the
recorded SHA-256 in X-Content-SHA256 and the recorded length in
Content-Length, under the same ownership rule. The bytes are hashed as they
go and the last chunk is sent only once the whole object matches the recorded
digest, so the body is complete only when it matches; a mismatch ends the
answer short. It is a 404 when the part was never kept, stating the reason its
reference records, and a 502 when the store cannot hand it back.
Each entry of a sealed run also carries digest, the SHA-256 of every column
its row persists in canonical form, and chain, the job's keyed chain value
after it in seq order; both are absent on entries stored before sealing.
GET /jobs/{job_id}/evidence/verify recomputes the chain from the rows and
checks it against the seal, under the same ownership rule, and writes nothing.
It reads only that job's rows and its own audit rows, so its cost follows the
job. It answers status (verified; broken; deleted when the report was
deleted by its owner and the ledger still verifies; other_key when the seal
names a key the platform cannot derive; unverifiable when something it needs
could not be checked, such as an audit append that did not land or a key
rotation the audit chain cannot yet vouch for; not_sealed for a run stored
before sealing),
reason, first_broken_seq, entries, the recomputed head, the seal,
the number of amendments, and the checks that held and notes on what was
not checked. The seal is also served in the run summary and the full report as
ledger_seal, with any ledger_amendments an enrichment chained onto it
(see Operations, Sealed evidence ledger).
GET /jobs/{job_id}/package downloads the job's verifiable package, a
deterministic .tar.gz (application/gzip, served as
maljan-job-<job_id>.tar.gz), under the same ownership rule: another
account's job is not found. It holds the stored run in the replay export's
layout under run/ (the report row and document, the evidence ledger, the run
summary, the judge's bundle, the events, the job row with the sample's hash
and name), report/report.md, export/stix-bundle.json,
ledger/spilled.json with each spilled part's bytes under
ledger/objects/<sha256> where they were read back and verified, seal.json
(the seal and its verification at build time) and manifest.json (every
file's SHA-256 and size, each part's path, and each absent part with why),
plus manifest.sig when the platform holds a settings encryption key. The
X-Package-Seal-Status header is the build-time verification's status and
X-Package-Signed says whether the manifest is signed. The rows and the
seal's verification are read in one snapshot. The sample file itself is not
included; spilled tool answers may contain bytes from it. A file above
upload_max_bytes is left out and listed in the manifest's oversized and
absent with its SHA-256, its size and the reason "larger than the package
bound". Check one with python -m scripts.package.verify <file> (see
Operations, Job package).
GET /jobs/{job_id}/usage is what the job's model calls used and cost so far,
totalled from its model_usage events (one per call, committed as the call is
recorded): calls, and retries and failed_attempts, which are records but
not calls; tokens, the run summary's own block (input_tokens,
output_tokens, cached_input_tokens, cache_write_input_tokens,
reasoning_tokens, a provider-reported cost, each part with the calls that
reported it, and per_agent); and spend, the sum of what the job's spend
meter charged each call and failed attempt when it was settled, as each record
carries it (priced_usd, price_source, estimated_usd for a charge by stated
estimate), so a price changed later does not change it. A record written before
records carried their charge is priced again by the meter's code at the current
llm.model_prices over the vendored table and counted in repriced_calls
(repriced_usd). spend also gives spent_usd, prices_from,
estimated_calls, estimated_usd, unreported_calls, unpriced_models,
spent_is_at_least where a call could not be priced, and per_agent. It
answers during a run and after a worker that died before its run summary. Only the job's owner reads it, as for the events endpoint: another
account's job is not found, to an admin as to anyone else.
POST /jobs/{job_id}/follow-ups ({"question": "..."}) asks a finished job
a new question. The report's question round answers it from what the job
stored: its report, its evidence ledger and the run state rebuilt from both,
with the same prompt and the same checks as a question asked at submit time
(every cited id must be one the run issued, established needs an entry of a
call that answered, limited_negative needs the calls that looked and never
rests on an analyst's open question) and its one retry. No tool is called and
nothing is analysed again, so a question the stored evidence cannot settle is
not_determinable with the reason. The model is the reporter's model the
run's settings snapshot recorded (the provider, model, effort and temperature
of its llm.agents.reporter entry and of each model it falls back to, else the
judge role's provider and model). No endpoint is taken from the snapshot: each
model is served where its provider is served now (the current reporter entry's
own address for that provider, else the provider's configured one), with the
current credential, and the current entry's ceilings are kept. A model the run
called at another place than its provider is served now, whose provider is no
longer registered, that the snapshot hides, or that no longer builds settings
is not asked; the current report model is, and model_source says why. A job
that ran as a mock is answered as one: it had no report model, so the answer
is not_determinable. It is built as the report stage builds it, with the
same output cap and the job's spend ceiling, and its calls are recorded as
model_usage events of the job under the call name follow-up, numbered from
the job's event counter like every event of its feed (from the table, and said
in notes, when Redis cannot be reached), so GET /jobs/{job_id}/usage
counts them. A stored ledger row that keeps part of its answer in object
storage is read as the row holds it, and notes says so. The answer comes back once the model has answered (201): question,
outcome, answer (the question round's answer: the model's words,
evidence_ids, looked_at, open_questions, answered and checks where a
check refused its word), model, model_source, report_sha256 (the digest
of the stored report it was answered from), record and notes (what could
not be read back or recorded). The stored report is never changed: the answer
is a record of its own, and with a settings encryption key a keyed record of
it (the report digest, the digest of the report's seal and of its last
amendment, the SHA-256 of the question and answer) is written onto the audit
chain as a report.follow_up row in the same transaction; without one record
is null and notes says so. GET /jobs/{job_id}/evidence/verify checks each
follow-up against that record: its question, answer and model are digested
again, the record must verify under its key and name the row, and the chained
audit row must hold the same record; a follow-up that does not match makes the
answer broken, as does a chained follow-up whose row is gone while the report
it names is stored (one gone with its report is noted). No transaction is held while the model answers. A
job still pending or running, or one that stored no report, is a 409; a
report that predates the structured payload or does not read is a 422.
GET /jobs/{job_id}/follow-ups lists the job's follow-ups, oldest first. Both
are the job owner's alone, checked as the evidence page checks it: another
account's job is not found.
The run summary
A finished report carries run_summary, and five of its keys describe the run
rather than the sample:
| Key | What it says |
|---|---|
evidence | entries, ok, failed, trimmed and by_tool — how many calls the run made, how many worked, and how many lost their output to the per-agent byte budget. |
sections_without_evidence | Report sections that can name neither a ledger entry nor the finding they came from. Not zero is a defect. |
validation | retries, by_code and unresolved — what the feedback loops cost and what stayed wrong, with the agent that owns each. |
corroboration | Per technique id, the sources that named it. A count of distinct sources, not a combined confidence; the judge is listed as a source but does not corroborate, because it read the analysts. |
stages | Every stage of the active team in declaration order: key, kind, ran, reason, agents, duration_ms. A stage that declined is a row saying so, not an absent row. |
The analysis WebSocket
/ws/analysis/{job_id} streams one run. Alongside status_change,
phase_change, agent_progress, agent_message, completed, error and
cancelled, the team announces each of its stages exactly once:
| Event | Payload | When |
|---|---|---|
stage_started | stage, kind, agents | The stage's first node runs. |
stage_skipped | stage, kind, reason | That node instead, when the stage's when condition is false. |
stage_finished | stage, kind, ran, reason, agents, duration_ms | The one node that runs after everything in the stage is done. |
A stage that was skipped is never finished — the skip was its terminator — and
no stage is announced twice, whatever shape it has. Events are also mirrored
into a bounded Redis stream and served by GET /jobs/{job_id}/events for a tab
that opens mid-run; the stream expires after 24 hours, after which
run_summary.stages is the record.
The socket is held to the same account checks an HTTP route is. The handshake reads the account the token names and closes with 1008 when it is missing or deactivated, before it says anything about the job; while the socket streams, it reads that account again every minute and closes with 1008 the moment it is deactivated, rather than letting the feed run until the access token expires. Every string on the feed is scrubbed by the publisher — credential shapes replaced, URLs cut to scheme and host, host paths cut to file names — and a published failure names the kind of exception it was, never its message.
Reports
One report is reachable by its own id or by the job that produced it, and the
same content is offered in several renderings under
/reports/{report_id}/...: markdown, html, pdf, stix, mitre,
full, iocs, signatures/{kind} and timeline. Every rendering is made on
request from the stored report, so there is one source of truth and no second
copy to fall behind it. POST /reports/{report_id}/enrich queues
threat-intelligence enrichment and answers 202 — the lookups run as their own
job so they never delay a verdict.
A job that failed after its report was built has that report, served by the
same routes. Its incomplete_reason is one sentence saying where the run
failed, the exception's class and the error id, and the same sentence is among
its degradation reasons; the job itself stays failed. A report of a run that
completed has incomplete_reason: null. GET /reports carries the same field
on each list item. The stix, iocs, signatures/{kind} and mitre routes
serve a kept report's content as it was built, with nothing added to the
exported bundle: the mark is on the report, its degradation reasons and the
list item. The dashboard's verdict_distribution counts completed jobs only,
like its average duration and its tool usage; a failed job that kept its
report is counted under jobs_by_status.failed and not by its verdict.
stix serves the exported bundle. stix?source=judge serves the judge's own
bundle beside it, as {"bundle": …, "labels": {…}}: the bundle as the pipeline
read it and the map from each id the judge wrote to the id it was published
under. It is the bundle every export decline row says an object "is unchanged
in". A report stored before it was kept answers {"kept": false, "reason": …}
— the report exists and has none — and only a report that does not exist, or
is not the caller's, answers 404. A kept record carries "kept": true, and a
label the judge gave two objects maps to the list of ids it named.
iocs is a feed another system acts on, and it answers accordingly. kind
narrows to one of hash, domain, ip, url, user_agent, ja3, ja3s,
and, for a value the exported STIX bundle carries from the judge's own
indicators, email, path, registry, mutex or command.
include decides what is returned:
include | what comes back |
|---|---|
published (default) | only what the platform's publish rule would publish — the same rule the exported STIX bundle is built with |
unpublished | only the rows it withholds, for a reader who is triaging rather than acting |
all | both |
Every row carries kind, value, is_suspicious, notes, source —
sandbox for something the sample resolved, reached or requested, analyst
for something an agent put in an artefact, strings for a run of bytes in the
file that has the shape of one, identity for the sample's own hashes,
judge for a value the judge's own indicators name — published, and
publish_answer: the publish rule's answer as the report's IOC table
prints it, yes: and why the row is published, or no: and why not.
A domain, address or URL the sample hid also carries recovered_by: the
tool that recovered it (floss by emulation, decode_string_blobs from the
file's bytes), its ledger entry and where in the file, as the report's IOC
table states it; every other row omits it (null).
A judge value is asked the same publish rule as every other row, with the
judge not counting as a second source; the exported bundle carries it only
when the rule publishes it, and this feed says the same. A name only the sample's own byte image knows is not an
observation of infrastructure, so it is withheld from the default feed and
labelled in the wider ones rather than shipped looking like one the sandbox
watched. A sandbox address is the sample's observation only when the
sandbox report attributes a flow to it to the sample's process tree; one it
attributes elsewhere, or does not attribute — every address a CAPE, REST or
mock report records, since those carry no process on a flow — and a
well-known benign name the guest resolved, is published only when the judge
keeps it as an indicator after it was asked once with the sandbox's fact
(stix.indicator_unattributed_flow, recorded answered in
run_summary.validation); a judge's indicator it was never asked about, an
analyst's artifact listing it and a claim that only mentions it keep nothing. The host of a URL the feed
publishes is a domain row with the URL's decision, added when the report has
no row for it, unless it is a well-known benign host no model kept.
What changed between two runs
GET /reports/diff?a=<id>&b=<id> compares two stored runs; by=job names
them by job id instead of report id. Both must be runs the caller may read,
and one that is not answers 404 exactly as a single report read does. The
two runs may be of different samples: the answer's same_sample is true,
false, or null when either SHA-256 is not recorded, and
sample_statement says which in a sentence. GET /jobs?sample_id=<id>
lists one sample's runs, which is how a run finds the others to compare with.
The diff is made on request from the two stored records — the report
columns, the MalwareReport document, the run summary, the exported STIX
bundle and the per-agent findings — and nothing else. It states what each
record says; it does not say which run is right, does not merge them and
writes into neither. The same function, maljan.reporting.run_diff.diff_runs,
can be called on any two records; it is linear in their sizes, and the route
runs it and encodes its answer off the event loop.
The answer carries a and b (report id, job id, time, SHA-256, file name),
totals, and sections in a fixed order. Each section has a match_key
saying what its rows are paired by, counts per status, recorded saying
whether each run's record holds the source the section reads, notes, and
rows, plus section_evidence: the ledger ids a record cites for the
section as a whole (the rule-match sections and the capability profile cite
their entries that way, not per row). A row has its status, both sides'
fields as recorded (a, b, null where the run has no such row), changes
naming each differing field with both values, and the evidence-ledger ids
each run's record cites for that row. Ids are read only from fields that hold
ids — family_evidence_ids, a key finding's evidence_ids, a configuration
item's, a command's and a C2 channel's evidence_refs, a persistence
mechanism's and a claim's evidence_ref, a STIX object's
x_maljan_evidence_refs — and never out of a quote, a note, a title or a
sample's own strings. A row whose record holds no such field cites none; that
is every ATT&CK mapping and every indicator.
A stated_by names who stated a value only where the record shows it. A
confidence is the judge's when the verdict reading is stated. A family's is
its family_source. A severity is the judge's only on a report that carries
verdict_reading: a report stored before that carries a rating the builder
computed, and its severity row has no stated_by at all. Who stated the
confidence and the severity follows from the verdict reading, which the
Verdict row compares, so on those two rows stated_by is shown and not
compared: a run pair that differs only in it counts that difference once, on
the Verdict row, and the confidence and severity rows say so in a note.
| Section | Paired by |
|---|---|
verdict | the field: verdict (with verdict_reading), confidence, severity, family (with family_source and its evidence ids), category |
attack | ttp_mappings.technique_id; the confidence source comes from the capability matrix |
indicators | consolidated_iocs kind and value. A row stored without a kind (a report stored before the column existed, whose network values may be defanged) is keyed by its type and value as stored and pairs only with rows of that shape; against the other shape both runs' rows are listed by run, with a note. A report without the table is read from its network block, which records no publish decision, and says so |
key_findings | exact text only |
analysts | agent_findings.agent_name |
persistence | kind and target |
configuration | the configuration key |
commands | the command id, or the name when it has none |
c2_channels | the channel name |
capability_profile | the behaviour category of static.api_capabilities |
detection | engine and rule name from the yara_matches, sigma_matches and capa_capabilities sections |
stix | STIX type and an identifying property: the ATT&CK id, an indicator's pattern, a name (as written, and compared), a value, a file's SHA-256 or name, a registry key, a directory path; a relationship by its type and both ends' keys, a sighting by what it sights. Any other object pairs only with an identical object, id included |
run | the fact: profile, analysts, models per agent, token figures, wall time, job duration, degraded |
tools | the tool name of run_summary.evidence.by_tool |
degradation | exact text only |
A status is added or removed (present in B only or A only by key),
changed, unchanged, or only_in_a / only_in_b. The last two are rows
the record does not key stably — a key finding's prose, a degradation
sentence, a STIX object with no identifying property such as a report, a
note or a process, a relationship to one — and indicators of two storage
shapes. They are listed by run and never paired by resemblance. Object ids
are not a key: the platform mints some ids per run, so two runs of one sample
carry different ids for the same content.
Rows under one key pair as a multiset. Identical rows pair first, as
unchanged. What is left pairs as changed only when exactly one row is left
on each side; any other remainder is listed as removed and added with a
note that the key repeats and the record states no correspondence between
the rows. The answer does not depend on the order either run stored its rows
in, and a record compared with itself is unchanged in every row.
One case rule applies to indicator values and STIX keys alike: a value is compared without regard to case only where it is case-insensitive by definition — a domain name, an IP address, a hash, a MAC address and a Windows registry key; an e-mail address in its domain part only. A URL, a path, a mutex name and a malware or tool name are compared as written.
Settings
GET /settings/schema returns the catalog: every entry with its type, bounds,
choices, group, title, description and when it takes effect, and
tool_declaration_fields: the word naming no kind and each kind-free and
kind-dependent tool-declaration field with its default. GET /settings
returns the current values with their source. PATCH /settings applies a set
of changes in one write; DELETE /settings/{key} and DELETE /settings remove
one override or a group's. GET /settings/export and POST /settings/import
carry configuration between instances, and POST /settings/test/{probe} (plus
/test/mcp and /test/agent, which take a body) run the connection probes.
/test/mcp?server=<key> answers the server's tools, its capabilities
manifest and, under details.proposals, a proposed declaration for each
undeclared tool with each field's evidence and confidence, and the harmless
calls it would make (planned: only tools their server marks read-only, each
with its arguments, whether it may reach its own service and what it would be
made without). A body naming some of them in
confirmed_calls calls those once each, never with a sample (see the
configuration guide's "A proposed declaration"). Nothing it proposes is
saved.
POST /settings/validate-condition checks one stage's when expression
against the parser that will run it and answers {"valid": …, "problems": […]}; it stores nothing, and the console calls it as each condition box loses
focus so a typo is answered next to the box rather than at apply time.
POST /settings/lint-teams takes {"profiles": …, "definitions": …, "profile": …} — each optional, read from the store when left out — and
answers {"findings": […], "graphs": {…}}: every team finding with its
severity, code, message, team, stage, field and the dotted path a refusal
would be keyed by, errors first, and each team laid out as nodes with a row
and a column and edges between them. It stores nothing; its errors are the
refusals PATCH /settings makes, in the same words. See
configuration.md.
GET /settings/plugins (admin) answers every installed provider plugin per
kind (llm, static, sandbox, memory) with its entry-point group: each
plugin's name, its entry as the allow-list writes it, its distribution and
version, whether the stored core.plugins.allowed names it, its state from
the metadata (refused with the error, else allowed or not_allowed),
and the API process's own load result where the API loaded it to run its code
(api_state: loaded, failed or refused, with api_error; empty when it
never did; a worker's own result is in that worker's log), and per kind the
allowed names no installed package declares (allowed_absent). It reads
package metadata only; no plugin is imported to be listed.
The ATT&CK cache is asked through the worker that holds it (admin):
GET /settings/attck answers, per domain (enterprise, mobile, ics), the
cached release, its date, when the file was written and how old it is, the
next automatic check, and the last check and update with who ran them.
POST /settings/attck/check compares the cache with MITRE's release index and
downloads nothing; its outcome is up_to_date, outdated (each domain with
its cached and latest release and the release date, and the next automatic
check) or failed with the reason, never up to date when the index could not
be read. POST /settings/attck/update queues the update and answers 202
with a job_id; GET /settings/attck/update/{job_id} answers queued,
running, done with the result, or unknown; a result's outcome is
updated, up_to_date, failed, busy (with who holds the cache), or
stopped (the worker gave up on it; kept_previous is then null and the
status says what the cache holds). Each answers 503 when the
worker does not answer in time. Neither takes an address: the upstream URL is
the platform's.
Conventions
- Pagination — list endpoints take
page(from 1) andpage_size(1 to 100, default 20) and answer with{"items": [...], "total": n, "page": p, "page_size": s}.GET /jobsadditionally acceptsstatus. - Identifiers — samples, jobs and reports are UUIDs.
- Validation — a value that does not fit its field comes back as 422 with the offending key named, which is also how a settings patch or import reports a bad value. Nothing in a rejected patch is applied.
- Probes — a connection test answers 200 whether the connection worked or not; the outcome is in the body. A failed test is an answer, not an error.
- Errors — refusals are JSON with a
detailstring. 401 means the credential is missing, invalid or expired; 403 means the account is deactivated or lacks the role, and names the role the caller actually has. - Correlation — send
X-Request-IDto tie your call to the server's log lines; the API echoes it back and generates one when it is absent. - Rate limits — requests are throttled per client address and path from the settings store; see operations.md.