Malspark

Advanced

Deployment

Compose services and healthchecks, required secrets, health endpoints, Kubernetes and systemd notes, upgrades and migrations.

How the stack is assembled, what it refuses to start without, how to check that it is healthy, and how to upgrade it. For the meaning of individual settings see configuration.md.

The compose stack

docker/docker-compose.yml is the production shape. docker/docker-compose.dev.yml is an overlay that swaps the frontend to next dev and supervises the worker so source edits take effect; make dev-up applies both.

ServiceImagePurpose
postgrespostgres:16-alpineRelational store. Healthchecked with pg_isready.
redisredis:7-alpineQueue, events, rate-limit counters. Password-protected; the healthcheck asserts an actual PONG.
qdrantqdrant/qdrant:v1.18.2Vector store, pinned to match the client version. Runs under the qdrant profile, which the shipped docker/.env turns on; without QDRANT_API_KEY it restarts, logging the variable to set.
miniominio/minio:RELEASE.2025-09-07T16-13-09ZSample storage, with its own console.
ghidra-mcpbuilt from external/ghidra-mcpA static analyser. Runs (and is built) under the ghidra profile, which the shipped docker/.env turns on; without GHIDRA_MCP_AUTH_TOKEN it refuses to bind and restarts. Capped at 6 GB memory and swap, because the JVM's own limit does not bound the container. GHIDRA_JAVA_OPTS, GHIDRA_MEM_LIMIT and GHIDRA_RESTART in docker/.env change the JVM options, the cap and the restart policy; see Running Ghidra lighter in configuration.md.
ghidra-portalpine/socat (pinned by digest)Publishes Ghidra's port on BIND_ADDRESS for a worker run on the host, under the ghidra profile; ghidra-mcp itself is on an internal network only. See The container that opens the sample below.
die-mcp, ilspy-mcp, goresym-mcpbuilt from services/<name>/DockerfileThe identification servers (Detect It Easy, ILSpy, GoReSym), each under its own profile (die, ilspy, goresym), which the shipped docker/.env does not turn on; each refuses to start without its token. Capped at 2 GB, 4 GB and 2 GB of memory and swap (DIE_MEM_LIMIT, ILSPY_MEM_LIMIT, GORESYM_MEM_LIMIT). See Identification servers.
die-port, ilspy-port, goresym-portalpine/socat (pinned by digest)Each publishes its server's port (8090, 8091, 8092) on BIND_ADDRESS for a worker run on the host, under the server's profile, as ghidra-port does.
yarax-mcp, oletools-mcp, webcrack-mcpbuilt from services/<name>/DockerfileThe rule and document servers (YARA-X, oletools, webcrack), each under its own profile (yarax, oletools, webcrack), off in the shipped docker/.env; each refuses to start without its token. Capped at 2 GB of memory and swap each (YARAX_MEM_LIMIT, OLETOOLS_MEM_LIMIT, WEBCRACK_MEM_LIMIT). yarax-mcp reads the operator's rule files from data/yara-x-rules (or YARA_X_RULES_DIR), mounted read-only. See Rule and document servers.
yarax-port, oletools-port, webcrack-portalpine/socat (pinned by digest)Each publishes its server's port (8093, 8094, 8095) on BIND_ADDRESS for a worker run on the host, under the server's profile.
unblob-mcp, jadx-mcpbuilt from services/<name>/DockerfileThe container and Android servers (unblob, JADX), each under its own profile (unblob, jadx), off in the shipped docker/.env; each refuses to start without its token. Capped at 2 GB and 4 GB of memory and swap (UNBLOB_MEM_LIMIT, JADX_MEM_LIMIT). Each has a /tmp tmpfs of its own, sized below its ceiling (UNBLOB_TMPFS_SIZE, JADX_TMPFS_SIZE). unblob-mcp writes what it extracts there, never beside the sample, and is handed UPLOAD_MAX_BYTES, UNBLOB_MAX_ENTRIES and CARVED_KEEP_SECONDS. See Container and Android servers.
unblob-port, jadx-portalpine/socat (pinned by digest)Each publishes its server's port (8096, 8097) on BIND_ADDRESS for a worker run on the host, under the server's profile.
analysis-mcp, network-mcpmaljan-backendThe platform's own analysis and network servers, each in a container of its own under its profile (analysis, network), off in the shipped docker/.env; each refuses to listen without its token (ANALYSIS_MCP_AUTH_TOKEN, NETWORK_MCP_AUTH_TOKEN). They run the worker's image and source read-only, with no platform credential. With a server's built-in block pointed at its container (transport streamable-http, url http://analysis-mcp:8098/mcp or http://network-mcp:8099/mcp, the token, sample_mount /data/samples), the triage pack's own readings of the sample run there. The worker still reads the sample for its hashes and routing, and for the few readings tests/unit/test_sample_reading_guard.py lists as left for a later change. Each job's staging directory in the container's /tmp (capa's and FLOSS's scratch, the unpacked program, a capture handed over) is removed when the job ends, the worker calling the server's release_job with the job's id; a release that fails leaves it to the server's own sweep MALJAN_STAGING_TTL_HOURS (24 h unless set) after it last changed, and it counts against the tmpfs and the memory ceiling until then. Capped at 8 GB and 2 GB of memory and swap (ANALYSIS_MEM_LIMIT, NETWORK_MEM_LIMIT), with a /tmp tmpfs for each job's staging (ANALYSIS_TMPFS_SIZE 2g, NETWORK_TMPFS_SIZE 512m).
analysis-port, network-portalpine/socat (pinned by digest)Each publishes its server's port (8098, 8099) on BIND_ADDRESS for a worker run on the host, under the server's profile.
migratemaljan-backendOne-shot alembic upgrade head. Runs to completion before the API and the worker start; on an upgrade, migrate before restarting either, since a stored built-in team written before a seeding revision loads under a renamed key until the revision has run.
backend-apimaljan-backendThe FastAPI service.
backend-workermaljan-backendThe arq worker. Capped at 8 GB memory and swap; WORKER_RSS_RESTART_MB makes it exit between jobs before it gets there, and restart: unless-stopped brings it back. It runs one analysis at a time (WorkerSettings.max_jobs = 1): a job's settings, the values the event scrub masks and the API names its hash resolution read are held per process, so any other max_jobs is refused at startup with that reason. More analyses at once take more worker processes.
frontendbuilt from docker/Dockerfile.frontendThe console.

The three backend services share one build and one image tag, so Compose builds maljan-backend once rather than once per service.

Ordering is explicit: migrate waits for a healthy Postgres, the API waits for migrate to exit successfully and for Postgres and Redis to be healthy, and the worker waits for the API's own healthcheck on top of that. Nothing waits for a tool server or a vector store: a job whose team needs a static provider that is not ready is refused with the agent and the address, and a memory backend that is unavailable is reported and the run goes on without it.

Running without Ghidra or Qdrant

ghidra-mcp and qdrant run under the Compose profiles ghidra and qdrant. The shipped docker/.env.example turns both on (COMPOSE_PROFILES=ghidra,qdrant), so a default install runs them exactly as before; a docker/.env copied before this line existed needs it added, or neither starts on the next up. make docker-down and make dev-down stop every profile's services whatever the line says.

To run without one, take its name out of COMPOSE_PROFILES (an empty value runs neither); its token is then not needed, and nothing else waits for it. The stack starts and runs a job once two settings follow: select the static analyser you do run (radare2, capa and YARA, a tool server of your own through generic_mcp, or none) as core.static.provider, and, without Qdrant, set core.memory.backend to memory. The shipped backend is qdrant, so until then every run reports long-term memory unavailable and goes on without it.

Required secrets

Every secret the core services need is declared with :?, so the stack refuses to start — with the variable named — while any is missing. There is no baked-in default for any of them. Ghidra's and Qdrant's tokens are not guarded that way, since Compose reads every service's variables whether its profile is on or not and a guard would stop a stack that runs neither: with its profile on and its token missing, that one container exits, restarts under its restart policy (docker ps shows it "Restarting") and logs the variable to set, and no other service is stopped by it.

VariableUsed by
POSTGRES_PASSWORDThe database, and the DATABASE_URL Compose assembles.
REDIS_PASSWORD--requirepass on Redis, and the API and worker URLs.
MINIO_ROOT_USER / MINIO_ROOT_PASSWORDMinIO, and the API's MinIO credentials.
JWT_SECRET_KEYSigns API session tokens.
SETTINGS_ENCRYPTION_KEYEncrypts secrets in the settings store.
Variable, with its profile onUsed by
QDRANT_API_KEY (qdrant)QDRANT__SERVICE__API_KEY on Qdrant; without one it exits and restarts.
GHIDRA_MCP_AUTH_TOKEN (ghidra)The bearer token the Ghidra MCP container requires; without one it refuses to bind and restarts.

Optional knobs with defaults: BIND_ADDRESS (127.0.0.1), the published ports, RUN_MIGRATIONS_ON_STARTUP, CORS_ORIGINS, COOKIE_SECURE, WORKER_RSS_RESTART_MB, MALJAN_EMBED_THREADS and MALJAN_EMBED_BATCH.

Every published port binds to BIND_ADDRESS, which the example file sets to 127.0.0.1. The stack is unreachable from the network until that is changed deliberately, behind a firewall or a reverse proxy you control.

The API, the workers and the migration mount src/, apps/, tests/ and services/ read-only, and the API runs without --reload. A process in those containers opens hostile bytes, so it must not be able to plant code that the API or a later worker would run, or rewrite the host's working tree. data/ is read-only too; only data/samples/ and data/uploads/ under it are writable, for the sample mirrors and the uploads. A source edit reaches a production container only when it restarts. The development overlay (make dev-up) mounts the sources writable and reloads the API on an edit.

The API, the workers and the migration run as an unprivileged account (uid and gid 10001 by default) with every Linux capability dropped and no-new-privileges. The image's virtual environment and sources are owned by root, so that account can only read them. Set MALJAN_UID and MALJAN_GID in docker/.env to run them under another id; on a machine where a login user also works in this checkout's data/, use that user's ids (id -u, id -g) so the files stay theirs. make docker-up and make dev-up set both to the invoking user's ids (under sudo, the user who ran it) when neither the environment nor docker/.env sets them, and to 10001 when that user is root.

Volume ownership

Before the backend ran unprivileged it ran as root, so a deployment created then has entries in data/samples/, data/uploads/ and the attck_cache volume owned by root. The one-shot volume-init service hands the root-owned ones to MALJAN_UID:MALJAN_GID before the API and the worker start, on every up:

  • It never takes a file from anyone else. An entry owned by a uid or gid that is neither root nor the target stops it before it has changed anything, with the id to set, for example volume-init: data/samples is owned by uid 1000; set MALJAN_UID/MALJAN_GID=1000 in docker/.env. The API and the worker then do not start.
  • It changes ownership only (chown -h), never a file's content or mode, never through a symbolic link, never past the mount it was given, and executes nothing it finds.
  • It touches only root-owned entries, so after the first run it changes nothing.
  • It runs as root with only the CHOWN and DAC_READ_SEARCH capabilities (the second lets it enter the 0700 work directories), with no network, and exits. It refuses a uid or gid that is 0 or not a number.

Nothing needs doing by hand: the first docker compose up -d after the upgrade runs it. To run it alone, docker compose run --rm volume-init. The ATT&CK cache now mounts at /home/maljan/.cache/maljan (the volume keeps its name), so its contents carry over.

The worker writes its sample mirrors 0700 and 0600 as MALJAN_UID. The ghidra-mcp container runs as that same uid and gid, so it reads them through its read-only mount; it shares nothing else with the worker. A Ghidra that runs as another user needs a group it shares with the worker, and mirrors written 0750 and 0640 for that group.

The container that opens the sample

ghidra-mcp opens the sample, so it is kept from the platform: it runs as MALJAN_UID:MALJAN_GID (volume-init hands the ghidra_projects volume over first), with no Linux capability, no way to gain one and a process limit (GHIDRA_PIDS_LIMIT, 2048 by default). Its only secret is its own GHIDRA_MCP_AUTH_TOKEN. Its one network is ghidra, internal (no gateway), shared only with backend-api and backend-worker, which reach it at http://ghidra-mcp:8089, and with ghidra-port. It publishes no port and has no route to the internet, the host, Postgres, Redis, MinIO or Qdrant. A deployment that points it at a shared Ghidra server elsewhere needs a network with a route to that server. The JVM's home is /tmp (JAVA_TOOL_OPTIONS), since the uid has none in the image.

ghidra-port (same ghidra profile) publishes Ghidra's port on BIND_ADDRESS for a worker run on the host (http://localhost:8089): a relay that forwards bytes to ghidra-mcp:8089 and nothing else. It mounts nothing, holds no variable, runs as 65534:65534 with no capability, no way to gain one and a read-only root, and its other network, ghidra-publish, carries nothing else.

GHIDRA_USER is empty by default: the platform imports each sample without a Ghidra project, so the name Ghidra runs as owns nothing it opens. A project in ghidra_projects created while the image ran as root is owned by root, and Ghidra refuses a project owned by another name; an operator who opens such projects sets GHIDRA_USER=root in docker/.env.

The identification servers (die-mcp, ilspy-mcp, goresym-mcp) open the sample too, and are held the same way: MALJAN_UID:MALJAN_GID, no capability, no way to gain one, a process limit, and its own token as its only secret. Each also runs with a read-only root whose only writable place is a tmpfs at /tmp, mounts the samples directory read-only at /data/samples and nothing else, and is on an internal network of its own (die, ilspy, goresym) shared only with backend-api, backend-worker and its relay, whose other network (die-publish …) carries nothing else. Each has a memory ceiling the kernel enforces, with swap pinned to it, as Ghidra's: a machine-safety limit, not a bound on the analysis, since a program's output is read whole. A file past it fails its call and the run records the failure; DIE_MEM_LIMIT (2g), ILSPY_MEM_LIMIT (4g) and GORESYM_MEM_LIMIT (2g) in docker/.env change them.

The rule and document servers (yarax-mcp, oletools-mcp, webcrack-mcp) are held exactly so, on networks of their own (yarax, oletools, webcrack) with their relays' (yarax-publish …), with 2g each (YARAX_MEM_LIMIT, OLETOOLS_MEM_LIMIT, WEBCRACK_MEM_LIMIT). yarax-mcp mounts one directory besides the samples, the operator's YARA rule files, read-only at /data/rules.

The container and Android servers (unblob-mcp, jadx-mcp) are held exactly so, on networks of their own (unblob, jadx) with their relays' (unblob-publish, jadx-publish), with 2g and 4g (UNBLOB_MEM_LIMIT, JADX_MEM_LIMIT), and each has a /tmp of its own, a tmpfs sized below that ceiling (UNBLOB_TMPFS_SIZE, JADX_TMPFS_SIZE, 1536m each), so a run that writes too much fills the file system rather than the container's memory. Keep each below its memory ceiling and change the two together: unblob's at about three quarters of UNBLOB_MEM_LIMIT, JADX's at JADX_MEM_LIMIT less its heap (half of it) and about 0.5 GiB more for the runtime. unblob-mcp writes the files it extracts under /tmp/carved; it stops a run past the larger of the sample's size and UPLOAD_MAX_BYTES (100 MiB unless set; set it to the console's upload_max_bytes where you raised that) or past UNBLOB_MAX_ENTRIES entries (100000 unless set), no one file it writes may pass that byte bound, and it removes each extraction CARVED_KEEP_SECONDS after it was written (3600 unless set), the oldest first when a new run would not fit.

Where a tool server sees the sample

The worker copies each sample into a directory under data/samples/ on the host, one per server that needs a copy (.work by default), and a containerised server mounts ../data/samples inside its own container. Each server's block declares that mount as its sample_mount; the worker hands the server its copy under it. Ghidra's ships /data/samples, the mount the ghidra-mcp service writes (read-only), whether or not the worker itself runs in Compose. A server on the worker's own filesystem (a stdio sidecar) declares none and is handed the host path. See What a server declares about its tools.

Set to a path the container does not have, Ghidra answers every load with File not found: ..., the Ghidra agent stops before its first model turn with "Ghidra could not open the job's sample: ...; check core.static.ghidra.sample_mount / the container mount", and the rest of the team runs without it. The worker's first lines say, for each static provider the settings select, where its copy goes and the path its server is handed:

Static provider 'ghidra' mirrors a sample to data/samples/.work/<sha256> and is handed it as /data/samples/.work/<sha256>.

GHIDRA_CONTAINER_SAMPLES_PATH is retired. The migration that retired it read a value other than /data/samples once into core.static.ghidra.sample_mount, where the store held none; a worker that still has it set warns at start and names that setting. MALJAN_ATTCK_MAX_AGE_DAYS is retired the same way: a whole number other than 7 is read once into core.validation.attck_refresh_days (0 or less as 0, never check), which the console edits under Analysis layers → ATT&CK data. The revision reads the environment of the process that runs it; in compose that is the migrate service, which is handed the variable from docker/.env.

Rotating the JWT signing secret

Three steps and one wait. The wait is the part a rotation is usually forgotten in, so the end of it is something you write down.

  1. Move the current JWT_SECRET_KEY to JWT_PREVIOUS_SECRET_KEY and JWT_KEY_ID to JWT_PREVIOUS_KEY_ID, put the new secret in JWT_SECRET_KEY and a new id in JWT_KEY_ID, and set JWT_PREVIOUS_SECRET_NOT_AFTER to the moment the old secret stops being accepted — an ISO-8601 timestamp, read as UTC when it carries no offset. Make it a little longer than the refresh-token lifetime (jwt_refresh_token_expire_days), because that is the oldest token still in circulation. Restart the API and the worker. The moment is optional and a previous secret without one is accepted with no end — the startup check warns about that at every start until you set it or clear the previous secret — but a value that does not read as an ISO-8601 moment is refused at startup rather than taken as "no end".
  2. Wait. New tokens are signed with the new secret and carry the new kid; tokens already issued keep working until they expire. GET /api/v1/system/status reports the rotation to an admin caller under jwt_grace_secret — the previous kid, when it lapses, whether it is still accepted and whether an end was written down at all (bounded) — and the API logs the same line at every start.
  3. After the moment passes, a token signed with the old secret is refused and jwt_grace_secret.accepted reads false; the startup check then warns until you clear JWT_PREVIOUS_SECRET_KEY and JWT_PREVIOUS_SECRET_NOT_AFTER.

Nothing rotates on its own. The point of the timestamp is that a secret you forget stops being honoured rather than being honoured for the life of the deployment. See security.md for where signing sits among the rest of the auth posture.

Health

GET /health and GET /healthz are the same endpoint under two paths, so both a bare and a Kubernetes-style liveness probe work without extra configuration.

  • Without parameters it performs no I/O: it returns status, the service name and version, and a config block reporting bootstrap and encryption, both of which the process already satisfied before it could serve anything. A liveness probe must not restart the API because Postgres blinked.
  • GET /health?deep=true probes Postgres, Redis, MinIO and the configured long-term-memory backend (components.memory, which names the backend it probed) concurrently, each with a three-second budget, and reports every result as data. An in-process backend answers without I/O; a backend id no registered builder knows is reported with ok: null and detail: "not configured", never as a failure, and one whose package failed to load with ok: false, detail: "unavailable" and the reason in error. status becomes degraded when Postgres or Redis is unreachable and degraded_optional when only MinIO or the memory store is, so use the deep form for readiness and the bare form for liveness.

Operational detail — throttle state and dropped audit rows — lives on GET /api/v1/system/status, where those fields are returned to admin callers only; the health endpoint publishes only the public readiness bit.

Outside Compose

The API and the worker need nothing but the bootstrap contract in their process environment, so any orchestrator that can inject environment variables can run them.

  • API: uv run uvicorn app.main:app --host 0.0.0.0 --port 8000 from the repository root, without --reload.
  • Worker: uv run arq app.worker.analysis_worker.WorkerSettings.
  • Kubernetes: put the contract in a Secret and a ConfigMap and inject it as environment variables. SETTINGS_ENCRYPTION_KEY must be identical for the API and the worker deployments, or the worker cannot open the credentials the console stored. Point the liveness probe at /healthz and the readiness probe at /health?deep=true. Run migrations as a Job before the rollout and leave RUN_MIGRATIONS_ON_STARTUP false, so replicas do not race the same upgrade.
  • systemd: an EnvironmentFile= pointing at a root-owned copy of bootstrap.env is enough for both units; nothing else is read from disk.

Behind a reverse proxy, set CORS_ORIGINS to the console's real origin, keep COOKIE_SECURE true, and populate the trusted_proxy_ips setting so the rate limiter honours X-Forwarded-For only from your proxy.

VirusTotal over stdio (optional)

The virustotal built-in needs nothing installed: it is VirusTotal's own server over HTTP, and registering an agent token from Settings → Setup guides is the whole setup. Install the package only for submit_local_file, the upload-by-path tool that exists solely when the server runs on the host that holds the sample:

uv tool install --python 3.12 vt-mcp==0.8.4

uvx --python 3.12 vt-mcp==0.8.4 runs the same release without installing it. The command is stdio-only and takes no flags; it reads the agent token from VTAI_TOKEN (or VTAI_TOKEN_FILE) in its own environment. Add it as a second tool server with transport stdio and put the variable name in env_allow, so the token reaches the child from the worker's environment rather than being written into a setting the console echoes back.

A server added this way is started with exactly the names it lists. A built-in is the one case where that field has a floor: analysis and network are always passed the sample roots and the staging directory, whatever the stored registry holds, because a child without them refuses the run's own sample. See configuration.md.

Upgrades and migrations

The API does not migrate on startup unless RUN_MIGRATIONS_ON_STARTUP is set, which it should not be in production.

  • On the compose stack the migrate service reruns on the next up and exits immediately when there is nothing to apply: docker compose up migrate.
  • Elsewhere, make migrate from the repository root. It sources bootstrap.env when present and runs alembic upgrade head from apps/api; DATABASE_URL must be reachable from where you run it, so the compose-internal postgres:5432 will not do.

After a pull that adds a revision, migrate before restarting the services. The production stack bakes the frontend into an image and runs the worker under plain arq, so neither picks up a source edit on its own: make fe-rebuild rebuilds and replaces the frontend container, and make worker-restart restarts the worker.

There is no automatic import of a previous .env deployment. An instance upgrading from one enters its settings once in Settings → Configuration, or imports a JSON export from another instance.

Edit on GitHub

On this page