Malspark

Benchmark

Latrodectus score: all-tools team, local Qwen3.6-35B

The all-tools team on a local Qwen3.6-35B, scored key by key against the human report.

The reference sample run under the importable all-tools team with a real sandbox, on a local model. dev @ 4d80c62e, team all_tools (three static analysts, a Ghidra reverser, then the dynamic and network stages; see An all-tools team), model Qwen3.6-35B-A3B on ik_llama.cpp with thinking off, a 131,072-token window probed from the server, output caps of 32,768 tokens for every analyst, the judge and the report model, no spend ceiling. The server had one slot, so the analysts ran one after another. Sandbox: Hatching Triage, a 600 s run. The job completed in 8,373 s.

The method is the benchmark's, with the all-tools series' scoring surface: the 57 core items of K1–K10 (K1.1 split into family and loader), scored on the report body, the STIX bundle and the /iocs feed.

  • partly (body): the body states part of the item, or states it with the wrong purpose.
  • partly (appendix): the value is printed only in the report's appendix.
  • A behaviour that a string only hints at counts as missed.
  • wrong: the body states something the human reports contradict.

Human sources: B Bitsight (names this hash), E Elastic, P Proofpoint.

Totals

GroupItemsFoundPartly (body)Partly (appendix)MissedWrong
K1 identity532000
K2 execution flow612030
K3 anti-analysis730040
K4 identifiers302100
K5 persistence301110
K6 C2722030
K9 IOCs821500
Main (K1–K6, K9)3911107110
K7 commands1103080
K8 discovery110000
K10 ATT&CK631020
Depth (K7, K8, K10)18440100
All core5715147210

15 found / 21 partly (14 body, 7 appendix) / 21 missed / 0 wrong, and 12 facts beyond the human reports (below).

  • The best local found count of the all-tools series, still with nothing wrong, and far below the hosted run's 45.
  • The gains are the C2 channel and the task. The POST to /live/, the fixed User-Agent, and T1053.005, published from the sandbox's "Uses Task Scheduler COM API" signature.
  • The losses are strings the report did not repeat. Custom_update, Update_%x and Updater are in the appendix only, because no claim named them.
  • The analysis is not deeper. The reverser decompiled 42 distinct functions, including every function the decoded strings of the key items are used in, and described none of them in a claim, even when asked once.

What the evidence had and the report did not

The triage pack gave every analyst FLOSS, resolve_api_hashes and decode_string_blobs, which ties each decoded string to the function that uses it. The reverser decompiled each of these functions; no claim describes any of them, so these key items stayed missed or appendix-only:

Key itemDecoded stringFunctionDecompiledIn a claim
K2.3 / K9 mutexrunnung0x3868yesno
K3.7 self-deletion:wtfbbq0x3a24yesno
K5.2 / K9 taskUpdater0x33acyesno
K5.1 / K9 install folderCustom_update0x30d0yesno
K4.2 / K4.3 groupLittlehw0x463cyesno
K6.7 response tagsURLS0x541c, 0xcf14yesno
K4.5 RC4 key123450x5750, 0xb670yesno
K7.18 bp.datfiles/bp.dat0x4110yesno
K2.6 / K5.3\update_data.dat0xcea4nono

The platform asked once about the decompiled functions no claim described; the kept answer still named 41 of 42 in no claim, and the report says so.

Item by item

#ScoreReport (quoted or condensed)
K1.1a familyfound"Family: Latrodectus (high confidence, 0.98, stated by the judge)"
K1.1b loaderfound"Category: loader"; "Latrodectus/IcedId loader DLL"
K1.2 IcedID linkpartly (body)"Latrodectus/IcedId/Ulise family" (labels only)
K1.4 x64 DLL; arch checkpartly (body)"pe, x86-64, DLL". No architecture check
K1.5 four exports, one addressfound"All 4 exports share one address, 0x3ce4."
K2.1 APIs by hashfound"resolves 164 API names … using CRC32"; "parses the PE export table … at offset 0x869c"
K2.2 abort on failed checkpartly (body)"Performs sandbox evasion checks including process enumeration and hook installation". Checks stated, from sandbox signatures; no abort
K2.3 mutexmissedOnly CreateMutexW in the measured import table
K2.4 install, then persistmissedPersistence given as Active Setup and the Startup folder
K2.5 register, then looppartly (body)The beacon template as "Exfiltration Format"; host data sent by POST. No loop, no command reception
K2.6 update_data.dat read firstmissedString only in the appendix
K3.1 PEB BeingDebuggedfound"accesses the Process Environment Block (PEB) for anti-debugging", with capa's Debugger Detection
K3.2 process count 75/50missed—
K3.3 architecture / WOW64missed—
K3.4 MAC checkmissedGetAdaptersInfo only as "network interface enumeration"
K3.5 PEB walk + CRC32 namesfoundExport parsing at 0x869c; CRC32; 164 names; the library list
K3.6 string encryptionfound"XOR-based string obfuscation … decoded at runtime by the routine at 0xae78"
K3.7 self-deletionmissedOnly an unpublished rule-match row for T1070.004
K4.1 bot IDpartly (appendix)guid=%s in the template; the format string in the appendix only
K4.2 group → FNV-1apartly (body)"FNV hashing for integrity checks"; group=%lu in the template. Not joined
K4.4 two encrypted C2spartly (body)Both URLs in the IOC section, decoded by 0xae78 inside 0x6988; the prose names only the first as C2
K5.1 Custom_update\Update_%x.dllpartly (appendix)Appendix only
K5.2 task Updater, COM, logonpartly (body)T1053.005 published: "uses Task Scheduler COM API to schedule tasks for persistence or delayed execution". No name, no logon trigger
K5.3 update_data.dat holds C2smissedAppendix only
K6.1 HTTPS POST /live/found"constructs HTTP POST requests to the domain skinny…[.]com (specifically the path /live/)"
K6.2 User-Agentfound"a custom User-Agent string (Mozilla/4.0 compatible; MSIE 7.0; Windows NT 5.1; Tob 1.1)" (opening parenthesis misplaced)
K6.3 RC4 then base64partly (body)"communication channel is encrypted using RC4 PRGA"; no base64, no key
K6.4 beacon intervalmissed"Exfiltration Interval · 1000 · decrypted" — the value is in no cited evidence entry
K6.5 beacon formatpartly (body)Base template exact; no &mac, &domain
K6.6 beacon types 1–5missed—
K6.7 URLS/CLEARURL/COMMAND/ERRORmissed—
K9 own hashesfoundpublished
K9 /live/foundboth URLs published with provenance
K9 UApartly (body)In the body once, inside a statement, not as an indicator
K9 mutex runnungpartly (appendix)Appendix only
K9 Custom_update\partly (appendix)Appendix only
K9 Update_<hex>.dllpartly (appendix)Update_%x in the appendix only
K9 update_data.datpartly (appendix)Appendix only
K9 task Updaterpartly (appendix)Appendix only; the body names a task, not its name
K7 2 desktop linksmissedDesktop appears only as a persistence folder
K7 3 process listpartly (body)Toolhelp32 enumeration; "exfiltrates … process lists". No command
K7 4 sysinfopartly (body)The reconnaissance set, exfiltrated; no command id
K7 12 download + run EXEpartly (body)"Downloads additional payloads using URLDownloadToFile". No execution, no command
K7 13 DLL via rundll32missed—
K7 14 shellcodemissed—
K7 15 updatemissed—
K7 17 uninstallmissed—
K7 18 bp.datmissedAppendix only
K7 19 longer intervalmissed—
K7 20 reset countermissed—
K8 discoveryfoundipconfig /all, systeminfo, nltest /domain_trusts /all_trusts, net group "Domain Admins" /domain, net view /all [/domain], whoami /groups, net config workstation, the SecurityCenter2 AntiVirusProduct query
K10 T1027foundpublished (capa RC4)
K10 T1218.011foundpublished, corroborated
K10 T1055partly (body)Published on remote injection from resolved names and a sandbox signature; not command 14
K10 T1053.005foundpublished, kept by the judge when asked
K10 T1070.004missedRule match only, not published
K10 T1059.003missedParent T1059 claimed and dropped by the judge

Build and later items (consistency checks, not scored): the first C2 domain, stated as C2, is on B's list of 47; the second is published but never called a C2. The RC4 key 12345, the group Littlehw and the stealer field &stiller= are in the appendix only.

Beyond the human reports

Each fact below is stated in the report body and is held by no item of the human reports' key (K1–K11), and none of B, E or P states it for this hash. The run's own evidence proves each one; the evidence id and the tool follow each line.

  1. Compile TimeDateStamp 2024-03-25 15:54:25 UTC — ev_0004 pe_info.
  2. Export-directory name UpdaterTag.dll — ev_0004 pe_info; ev_0035 r2 list_exports.
  3. Section layout: five sections, highest entropy .data 6.48, no packer signatures — ev_0004 pe_info.
  4. The static import table is exactly five imports (PeekNamedPipe, GetLastError, CreateMutexW, MessageBeep, MessageBoxA) — ev_0004 pe_info; ev_0034 r2 list_imports.
  5. No Authenticode signature — ev_0003 signing_info.
  6. The string decoder is the routine at 0xae78 — ev_0019 floss (every decoded string names it); ev_0073 Ghidra decompile_function.
  7. The export-table resolver is at 0x869c — ev_0008 capa ("resolve function by parsing PE exports" at 0x869c); ev_0086 Ghidra decompile_function.
  8. The runtime API surface: 164 names resolved from CRC32 hashes across kernel32, user32, wininet, shell32, advapi32, urlmon, shlwapi and iphlpapi, with the WinINet, URLDownloadToFileW, Toolhelp32 and SHGetFolderPathW names stated — ev_0020 resolve_api_hashes.
  9. This hash's C2 pair, both on /live/, decoded by 0xae78 inside 0x6988 at 0x69b3 and 0x6a1f from file offsets 0xe050 and 0xe078 — ev_0019 floss; ev_0021 decode_string_blobs.
  10. Where each decoded C2 URL goes next: the decoder's output slot [rsp+0x40] becomes argument 1 of the call to 0xbc1c, at 0x6a4e for the first URL and 0x69e2 for the second — ev_0021 decode_string_blobs.
  11. VirusTotal: 52 of 75 engines, analysis of 2026-08-23 — ev_0018 get_file_report.
  12. The sandbox record: two rundll32.exe <sample>.dll,#1 processes, i.e. the load is by ordinal 1 — ev_0012 sandbox_processes.

12 facts beyond the human reports.

Not counted: wrong, or not proven by the evidence.

  • Persistence by Active Setup and the Startup folder (T1547.001 and T1547.014 published): contradicted by B and E, whose only persistence is the Updater task; it rests on a sandbox signature.
  • 63 "host identifier" rows naming registry keys that no evidence entry holds.
  • A sandbox address (104[.]18[.]…) stated as the C2's infrastructure: the sandbox attributes no flow to the sample, and the IOC feed does not publish it.
  • Remote-process injection, privilege escalation, Volume Shadow Copy deletion and IE-settings changes: generic sandbox signatures with no call site.
  • "Exfiltration Interval 1000": not in the cited evidence entry.
  • "Six dropped files": sandbox memory dumps, not the sample's drops.
  • The sandbox's SCSI-registry signature as the sample's own check: its process is not stated, so it is not counted, though it may be right.
  • T1000 and T1042 (not valid ATT&CK ids), T1048, T1134, T1490, T1112 and T1083 as stated.

Reverse direction: claims no human report supports

ClaimRuling
Persistence by Active Setup registry keys and the Startup foldercontradicted (B, E)
63 host-identifier rows built by joining PE strings, string fragments and a tool's answer onto a registry pathwrong; in no evidence entry; not published
The sample connects to a CDN address on port 443 that "corresponds to" the first C2 domainunsupported: the sandbox attributes no flow to the sample
Process injection via VirtualAllocEx / WriteProcessMemory (T1055)unsupported: resolved names and a generic sandbox signature, no call site
T1134, T1490, T1112, T1048unsupported or wrong ids, from generic sandbox signatures
"Installer Name · UpdaterTag.dll"; "Exfiltration Interval · 1000"; "Payload Extension · .dat"wrong purpose, or not in the cited evidence
"The sample dropped six files"sandbox capture artefacts, not the sample's drops
Family, loader, four exports at 0x3ce4, CRC32 resolution with 164 names at 0x869c, the decoder 0xae78, the reconnaissance set, RC4 PRGA, the beacon template, POST to /live/, the User-Agent, both C2 URLs, T1218.011, T1027, T1053.005correct

The run

Duration8,373 s
Debate1 round; no dissent, no revision
Published techniques21
Published network indicators2 domains, 2 URLs, no IP address
Model calls149; 7.60 M input tokens (5.71 M cached), 0.146 M output; 39.2 tokens/s generation, 499 tokens/s prompt
Tool calls112 (1 failed): pipeline 22, triage 6, static 0, static_r2 30, qu1cksc0pe 1, reverser 53, dynamic 0, network 0

Three analysts — static, dynamic and network — answered from the triage pack without calling a tool even after being asked to. One analyst's answer ran to its 32,768-token cap repeating claims, which the platform caught after the answer and asked about once. This is a single run.

Edit on GitHub

On this page