{"updated": "2026-10-26T03:33:37Z", "n": 200, "kinds": {"yesno": 9, "date": 35, "negative": 15, "reasoning": 29, "fact": 112}, "rules": ["Every question was written by hand against a real dossier, in the phrasing an analyst would use; none is a training template.", "Each question carries machine-checkable expectations: `must` strings that have to appear in the answer, `any` groups where one of each group must appear, `expect_yes` / `expect_no` for yes-no and refusal questions, `min_distinct` for 'name three'.", "Before a question is admitted, a check proves every expected string is present in the dossier the model reads, and every `absent` string is not; an expectation can never be something the model could not have read.", "Judging is string matching after folding typographic hyphens, spaces and quotes to ASCII; it is applied identically to every model and every run.", "The model sees exactly what a user of the console sees: the same dossier, the same focus and the same token budget. Scores are per run, on the live graph of that day; the graph moves, so a question can outgrow its expectation and is then fixed or retired in the open.", "A miss is a miss. The page lists every one with the model's answer and the rule it failed."], "sets": [{"file": "handwritten_v1.jsonl", "n": 100, "written": "2026-10-25 morning, 25 dossiers"}, {"file": "handwritten_v2.jsonl", "n": 100, "written": "2026-10-25 afternoon, 27 more entities"}], "runs": [{"tag": "120b", "label": "120B MoE, medium effort, strict line (live reader)", "stamp": "20261025-2242", "pass": 195, "n": 200, "by_kind": {"date": "33/35", "fact": "110/112", "negative": "15/15", "reasoning": "28/29", "yesno": "9/9"}, "mean_s": 1.69, "fails": [38, 63, 98, 127, 200]}, {"tag": "30b", "label": "30B MoE, strict line (rented 32 GB card)", "stamp": "20261025-2217", "pass": 185, "n": 200, "by_kind": {"date": "31/35", "fact": "106/112", "negative": "15/15", "reasoning": "24/29", "yesno": "9/9"}, "mean_s": 0.35, "fails": [38, 63, 76, 81, 98, 107, 111, 118, 122, 127, 133, 138, 162, 181, 193]}, {"tag": "cpu", "label": "3B adapter v4, 6-bit, four CPU cores (fallback; nightly run)", "stamp": "20261026-0202", "pass": 168, "n": 200, "by_kind": {"date": "34/35", "fact": "92/112", "negative": "15/15", "reasoning": "19/29", "yesno": "8/9"}, "mean_s": 6.81, "fails": [8, 26, 27, 33, 39, 65, 67, 71, 74, 78, 85, 91, 98, 100, 103, 107, 118, 124, 134, 137, 138, 150, 151, 155, 161, 169, 171, 177, 181, 185, 193, 197]}], "rows": [{"id": 1, "entity": "Helix", "kind": "yesno", "question": "Did Helix claim Uber as a victim?", "gold": "Yes.", "expect": {"must": ["Uber"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. “Helix (RansomwareGroup) claimed victim Uber (Organization).” “Uber (RansomwareVictim) claimed by Helix (RansomwareGroup, ransomware-group).”", "s": 1.07}, "30b": {"ok": true, "why": "ok", "pred": "Yes. Helix (RansomwareGroup) claimed victim Uber (Organization).", "s": 0.32}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. Helix (RansomwareGroup, ransomware-group) claimed victim Uber (Organization).", "s": 8.1}}}, {"id": 2, "entity": "Helix", "kind": "date", "question": "Which two victims did Helix post on 21 October 2026?", "gold": "Westrock Coffee and Hypertherm.", "expect": {"must": ["Westrock Coffee", "Hypertherm"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Westrock Coffee and Hypertherm; “On 2026-10-21, Helix (RansomwareGroup, ransomware-group) claimed victim Westrock Coffee (Organization).” “On 2026-10-21, Helix (RansomwareGroup, ransomware-group) claimed victim Hypertherm (Organization).”", "s": 1.83}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-21, Helix (RansomwareGroup, ransomware-group) claimed victim Westrock Coffee (Organization).  \nOn 2026-10-21, Helix (RansomwareGroup, ransomware-group) claimed victim Hypertherm (Organization).", "s": 0.46}, "cpu": {"ok": true, "why": "ok", "pred": "victim Westrock Coffee, victim Hypertherm.", "s": 7.43}}}, {"id": 3, "entity": "Helix", "kind": "date", "question": "Name a victim Helix claimed on 18 October 2026.", "gold": "Southwire or Quikrete.", "expect": {"any": [["Southwire", "Quikrete"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Southwire (Organization) – “On 2026-10-18, Helix (RansomwareGroup, ransomware-group) claimed victim Southwire (Organization).”", "s": 1.77}, "30b": {"ok": true, "why": "ok", "pred": "Southwire (RansomwareVictim) claimed by Helix (RansomwareGroup, ransomware-group).", "s": 0.3}, "cpu": {"ok": true, "why": "ok", "pred": "victim Southwire, victim Quikrete.", "s": 7.36}}}, {"id": 4, "entity": "Helix", "kind": "negative", "question": "Does the dossier say Helix uses PowerShell?", "gold": "No. The dossier does not mention PowerShell.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.67}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.24}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention PowerShell.", "s": 7.4}}}, {"id": 5, "entity": "suncrypt", "kind": "date", "question": "When did suncrypt claim Atlas Copco?", "gold": "2022-04-14.", "expect": {"any": [["2022-04-14", "14 April 2022", "April 14, 2022"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2022-04-14 – On 2022-04-14, suncrypt (RansomwareGroup, ransomware-group) claimed victim Atlas Copco (Organization).", "s": 0.98}, "30b": {"ok": true, "why": "ok", "pred": "On 2022-04-14, suncrypt (RansomwareGroup, ransomware-group) claimed victim Atlas Copco (Organization).", "s": 0.33}, "cpu": {"ok": true, "why": "ok", "pred": "On 2022-04-14, suncrypt (RansomwareGroup, ransomware-group) claimed victim Atlas Copco (Organization).", "s": 10.7}}}, {"id": 6, "entity": "suncrypt", "kind": "reasoning", "question": "Which Dutch municipality appears among suncrypt's victims?", "gold": "Gemeente Buren.", "expect": {"must": ["Buren"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Gemeente Buren — On 2022‑04‑14, suncrypt (RansomwareGroup, ransomware‑group) claimed victim Gemeente Buren (Organization).", "s": 0.94}, "30b": {"ok": true, "why": "ok", "pred": "Gemeente Buren (Organization).", "s": 0.26}, "cpu": {"ok": true, "why": "ok", "pred": "Gemeente Buren.", "s": 8.9}}}, {"id": 7, "entity": "suncrypt", "kind": "reasoning", "question": "Which suncrypt victim is a healthcare clinic?", "gold": "Oklahoma City Indian Clinic.", "expect": {"must": ["Oklahoma City Indian Clinic"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Oklahoma City Indian Clinic – “On 2022‑03‑28, suncrypt (RansomwareGroup, ransomware‑group) claimed victim Oklahoma City Indian Clinic (Organization).”", "s": 1.58}, "30b": {"ok": true, "why": "ok", "pred": "Oklahoma City Indian Clinic (Organization).", "s": 0.25}, "cpu": {"ok": true, "why": "ok", "pred": "Oklahoma City Indian Clinic.", "s": 8.75}}}, {"id": 8, "entity": "suncrypt", "kind": "reasoning", "question": "Was Migros claimed before or after FitFlop?", "gold": "Before: Migros on 2022-03-16, FitFlop on 2022-03-17.", "expect": {"must": ["before"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Migros was claimed before FitFlop. (On 2022-03-16, suncrypt (RansomwareGroup, ransomware-group) claimed victim Migros (Organization).; On 2022-03-17, suncrypt (RansomwareGroup, ransomware-group) claimed victim FitFlop Ltd. (Organization).)", "s": 2.0}, "30b": {"ok": true, "why": "ok", "pred": "On 2022-03-16, suncrypt (RansomwareGroup, ransomware-group) claimed victim Migros (Organization).\nOn 2022-03-17, suncrypt (RansomwareGroup, ransomware-group) claimed victim FitFlop Ltd. (Organization).\nMigros was claimed before FitFlop.", "s": 0.48}, "cpu": {"ok": false, "why": "missing 'before'", "pred": "On 2022-03-16, suncrypt (RansomwareGroup, ransomware-group) claimed victim Migros (Organization).", "s": 10.48}}}, {"id": 9, "entity": "rhysida", "kind": "fact", "question": "How does Rhysida get initial access, according to the dossier?", "gold": "Spearphishing Attachment.", "expect": {"must": ["Spearphishing"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes – Spearphishing Attachment (Technique) 【Rhysida (RansomwareGroup, ransomware-group) uses Spearphishing Attachment (Technique).】", "s": 1.71}, "30b": {"ok": true, "why": "ok", "pred": "Rhysida (RansomwareGroup, ransomware-group) uses Spearphishing Attachment (Technique).", "s": 0.73}, "cpu": {"ok": true, "why": "ok", "pred": "uses Inhibit System Recovery, uses Spearphishing Attachment, uses Data Encrypted for Impact, uses PowerShell, uses SMB/Windows Admin Shares.", "s": 9.14}}}, {"id": 10, "entity": "rhysida", "kind": "fact", "question": "Name one .onion site Rhysida operates.", "gold": "rhysidafc6lm7qa2mkiukbezh7zuth3i4wof4mh2audkymscjm6yegad.onion or rhysidafohrhyy2aszi7bm32tnjat5xri65fopcxkdfxhi4tidsg7cad.onion.", "expect": {"any": [["rhysidafc6lm7qa2mkiukbezh7zuth3i4wof4mh2audkymscjm6yegad", "rhysidafohrhyy2aszi7bm32tnjat5xri65fopcxkdfxhi4tidsg7cad"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "rhysidafc6lm7qa2mkiukbezh7zuth3i4wof4mh2audkymscjm6yegad.onion (Infrastructure). 【Rhysida (RansomwareGroup, ransomware-group) operates site rhysidafc6lm7qa2mkiukbezh7zuth3i4wof4mh2audkymscjm6yegad.onion (Infrastructure).】", "s": 2.24}, "30b": {"ok": true, "why": "ok", "pred": "rhysidafc6lm7qa2mkiukbezh7zuth3i4wof4mh2audkymscjm6yegad.onion (Infrastructure).", "s": 0.89}, "cpu": {"ok": true, "why": "ok", "pred": "rhysidafc6lm7qa2mkiukbezh7zuth3i4wof4mh2audkymscjm6yegad.onion.", "s": 10.64}}}, {"id": 11, "entity": "rhysida", "kind": "date", "question": "Which German company did Rhysida claim on 22 October 2026?", "gold": "DR.SCHNELL GmbH & Co. KGaA.", "expect": {"must": ["SCHNELL"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "DR.SCHNELL GmbH & Co. KGaA – On 2026-10-22, rhysida (RansomwareGroup, ransomware-group) claimed victim DR.SCHNELL GmbH & Co. KGaA (Organization).", "s": 2.3}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-22, rhysida (RansomwareGroup, ransomware-group) claimed victim DR.SCHNELL GmbH & Co. KGaA (Organization).", "s": 1.04}, "cpu": {"ok": true, "why": "ok", "pred": "DR.SCHNELL GmbH & Co. KGaA.", "s": 8.93}}}, {"id": 12, "entity": "rhysida", "kind": "fact", "question": "Which technique does Rhysida use to stop victims recovering their systems?", "gold": "Inhibit System Recovery.", "expect": {"must": ["Inhibit System Recovery"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Inhibit System Recovery (Technique) – “Rhysida (RansomwareGroup, ransomware-group) uses Inhibit System Recovery (Technique).” (Dossier line)", "s": 1.57}, "30b": {"ok": true, "why": "ok", "pred": "Rhysida (RansomwareGroup, ransomware-group) uses Inhibit System Recovery (Technique).", "s": 0.83}, "cpu": {"ok": true, "why": "ok", "pred": "Inhibit System Recovery.", "s": 7.68}}}, {"id": 13, "entity": "rhysida", "kind": "date", "question": "Which college did Rhysida claim in September 2023?", "gold": "Ort Harmelin College of Engineering.", "expect": {"must": ["Ort Harmelin"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Ort Harmelin College of Engineering – On 2023-09-23, rhysida (RansomwareGroup, ransomware-group) claimed victim Ort Harmelin College of Engineering (Organization).", "s": 1.74}, "30b": {"ok": true, "why": "ok", "pred": "On 2023-09-23, rhysida (RansomwareGroup, ransomware-group) claimed victim Ort Harmelin College of Engineering (Organization).", "s": 0.9}, "cpu": {"ok": true, "why": "ok", "pred": "victim Ort Harmelin College of Engineering.", "s": 8.21}}}, {"id": 14, "entity": "leaktheanalyst", "kind": "date", "question": "Which leaktheanalyst victim entries are dated 29 March 2022?", "gold": "18 Build your own drone and 19 You Must Never Forget LeakTheAnalyst.", "expect": {"any": [["Build your own drone", "Never Forget"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "On 2022-03-29, leaktheanalyst (RansomwareGroup, ransomware-group) claimed victim 18 Build your own drone (Organization).; On 2022-03-29, leaktheanalyst (RansomwareGroup, ransomware-group) claimed victim 19 You Must Never Forget LeakTheAnalyst (Organization).", "s": 1.68}, "30b": {"ok": true, "why": "ok", "pred": "On 2022-03-29, leaktheanalyst (RansomwareGroup, ransomware-group) claimed victim 18 Build your own drone (Organization).  \nOn 2022-03-29, leaktheanalyst (RansomwareGroup, ransomware-group) claimed victim 19 You Must Never Forget LeakTheAnalyst (Organization).", "s": 0.54}, "cpu": {"ok": true, "why": "ok", "pred": "victim 18 Build your own drone, victim 19 You Must Never Forget LeakTheAnalyst.", "s": 10.02}}}, {"id": 15, "entity": "leaktheanalyst", "kind": "reasoning", "question": "What single date carries most of leaktheanalyst's victim posts?", "gold": "2022-01-01.", "expect": {"any": [["2022-01-01", "1 January 2022", "January 1, 2022"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2022-01-01 — “On 2022-01-01, leaktheanalyst (RansomwareGroup, ransomware-group) claimed victim 13 We are kids ! (Organization).”; “On 2022-01-01, leaktheanalyst (RansomwareGroup, ransomware-group) claimed victim 2 Breaking News (Organization).”", "s": 1.96}, "30b": {"ok": true, "why": "ok", "pred": "2022-01-01", "s": 0.21}, "cpu": {"ok": true, "why": "ok", "pred": "2022-01-01.", "s": 9.39}}}, {"id": 16, "entity": "ZaWoo", "kind": "reasoning", "question": "Which ZaWoo victims have French domains?", "gold": "www.francare.com and agiliance.fr.", "expect": {"any": [["agiliance.fr", "francare"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "agiliance.fr – “On 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim agiliance.fr (Organization).” and “On 2026-09-24, agiliance.fr (RansomwareVictim) claimed by ZaWoo (RansomwareGroup, ransomware-group).”", "s": 2.93}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim agiliance.fr (Organization).\nOn 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim www.francare.com (Organization).\nOn 2026-09-19, ZaWoo (RansomwareGroup, ransomware-group) claimed victim FRANCARETRAD (Organization).", "s": 0.53}, "cpu": {"ok": true, "why": "ok", "pred": "www.francare.com, agiliance.fr.", "s": 7.22}}}, {"id": 17, "entity": "ZaWoo", "kind": "reasoning", "question": "Which ZaWoo victim is Czech?", "gold": "kdynium.cz or BOTEC-CZ.", "expect": {"any": [["kdynium.cz", "BOTEC-CZ"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "kdynium.cz and BOTEC-CZ; lines: “ZaWoo (RansomwareGroup, ransomware-group) claimed victim kdynium.cz (Organization).” and “ZaWoo (RansomwareGroup, ransomware-group) claimed victim BOTEC-CZ (Organization).”", "s": 1.26}, "30b": {"ok": true, "why": "ok", "pred": "ZaWoo (RansomwareGroup, ransomware-group) claimed victim kdynium.cz (Organization).", "s": 0.19}, "cpu": {"ok": true, "why": "ok", "pred": "kdynium.cz.", "s": 2.92}}}, {"id": 18, "entity": "ZaWoo", "kind": "date", "question": "What happened on 24 September 2026 in ZaWoo's dossier?", "gold": "ZaWoo claimed www.francare.com, agiliance.fr and amb-pvc.com.", "expect": {"any": [["francare", "agiliance", "amb-pvc"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "On 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim www.francare.com (Organization); On 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim agiliance.fr (Organization); On 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim amb-pvc.com (Organization); On 2026-09-24, ZaWoo (RansomwareGroup, ransomware-grou…", "s": 4.55}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim www.francare.com (Organization).  \nOn 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim agiliance.fr (Organization).  \nOn 2026-09-24, ZaWoo (RansomwareGroup, ransomware-group) claimed victim amb-pvc.com (Organization).  \nOn 2026-09-24, ZaWoo (RansomwareGroup, ransomwar…", "s": 0.69}, "cpu": {"ok": true, "why": "ok", "pred": "ZaWoo (RansomwareGroup, ransomware-group) claimed victim www.francare.com (Organization).", "s": 7.74}}}, {"id": 19, "entity": "ZaWoo", "kind": "negative", "question": "Does the dossier give any ATT&CK techniques for ZaWoo?", "gold": "No. The dossier lists no techniques for ZaWoo.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.57}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.16}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention ATT&CK techniques.", "s": 3.67}}}, {"id": 20, "entity": "ZaWoo", "kind": "reasoning", "question": "Which ZaWoo victims are German?", "gold": "winterdienst-berlin.com, ng-engineering.de, vectorsoft.de.", "expect": {"any": [["winterdienst-berlin", "ng-engineering", "vectorsoft"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "German victims: ng‑engineering.de, vectorsoft.de, hd‑werkstaetten.de, rsk‑immobilien.de, berghotel‑oberhof.de, montronix.de. Sources: “ZaWoo (RansomwareGroup, ransomware-group) claimed victim ng‑engineering.de (Organization).”, “ZaWoo (RansomwareGroup, ransomware-group) claimed victim vectorsoft.de (Organization).”, “ZaWoo (RansomwareGroup, ransomware-group)…", "s": 5.52}, "30b": {"ok": true, "why": "ok", "pred": "ZaWoo (RansomwareGroup, ransomware-group) claimed victim ng-engineering.de (Organization).\nZaWoo (RansomwareGroup, ransomware-group) claimed victim vectorsoft.de (Organization).\nZaWoo (RansomwareGroup, ransomware-group) claimed victim BOTEC-CZ (Organization).\nZaWoo (RansomwareGroup, ransomware-group) claimed victim hd-werkstaetten.de (Organization).\nZaWoo (R…", "s": 0.57}, "cpu": {"ok": true, "why": "ok", "pred": "www.francare.com, agiliance.fr, amb-pvc.com, kdynium.cz, vectorsoft.de.", "s": 2.14}}}, {"id": 21, "entity": "monti", "kind": "reasoning", "question": "How many days passed between Monti's press release and the full leak for American Eagle Logistics?", "gold": "Seven days: press release 2025-05-03, full leak 2025-05-10.", "expect": {"any": [["7", "seven"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "7 days (On 2025-05-03, monti (RansomwareGroup, ransomware-group) claimed victim American Eagle Logistics - Press Release (Organization); On 2025-05-10, monti (RansomwareGroup, ransomware-group) claimed victim American Eagle Logistics - Full Leak (Organization)).", "s": 1.47}, "30b": {"ok": true, "why": "ok", "pred": "7 days. On 2025-05-03, monti (RansomwareGroup, ransomware-group) claimed victim American Eagle Logistics - Press Release (Organization). On 2025-05-10, monti (RansomwareGroup, ransomware-group) claimed victim American Eagle Logistics - Full Leak (Organization).", "s": 0.5}, "cpu": {"ok": true, "why": "ok", "pred": "7 days.", "s": 9.17}}}, {"id": 22, "entity": "monti", "kind": "fact", "question": "Which ATT&CK technique is attributed to Monti?", "gold": "Data Encrypted for Impact.", "expect": {"must": ["Data Encrypted for Impact"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Monti (RansomwareGroup, ransomware-group) uses Data Encrypted for Impact (Technique).", "s": 1.71}, "30b": {"ok": true, "why": "ok", "pred": "Monti (RansomwareGroup, ransomware-group) uses Data Encrypted for Impact (Technique).", "s": 0.34}, "cpu": {"ok": true, "why": "ok", "pred": "Data Encrypted for Impact.", "s": 9.06}}}, {"id": 23, "entity": "monti", "kind": "date", "question": "Which victim did Monti claim on 7 May 2025?", "gold": "Amtech Software.", "expect": {"must": ["Amtech"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Amtech Software (Organization) – from “On 2025-05-07, monti (RansomwareGroup, ransomware-group) claimed victim Amtech Software (Organization).”", "s": 1.34}, "30b": {"ok": true, "why": "ok", "pred": "On 2025-05-07, monti (RansomwareGroup, ransomware-group) claimed victim Amtech Software (Organization).", "s": 0.37}, "cpu": {"ok": true, "why": "ok", "pred": "victim Amtech Software.", "s": 8.94}}}, {"id": 24, "entity": "monti", "kind": "fact", "question": "Name one .onion site Monti operates.", "gold": "4s4lnfeujzo67fy2jebz2dxskez2gsqj2jeb35m75ktufxensdicqxad.onion or mblogci3rudehaagbryjznltdp33ojwzkq6hn2pckvjq33rycmzczpid.onion.", "expect": {"any": [["4s4lnfeujzo67fy2jebz2dxskez2gsqj2jeb35m75ktufxensdicqxad", "mblogci3rudehaagbryjznltdp33ojwzkq6hn2pckvjq33rycmzczpid"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "4s4lnfeujzo67fy2jebz2dxskez2gsqj2jeb35m75ktufxensdicqxad.onion (Infrastructure). 【Monti (RansomwareGroup, ransomware-group) operates site 4s4lnfeujzo67fy2jebz2dxskez2gsqj2jeb35m75ktufxensdicqxad.onion (Infrastructure).】", "s": 2.25}, "30b": {"ok": true, "why": "ok", "pred": "monti (RansomwareGroup, ransomware-group) operates site 4s4lnfeujzo67fy2jebz2dxskez2gsqj2jeb35m75ktufxensdicqxad.onion (Infrastructure).", "s": 0.45}, "cpu": {"ok": true, "why": "ok", "pred": "site 4s4lnfeujzo67fy2jebz2dxskez2gsqj2jeb35m75ktufxensdicqxad.onion.", "s": 11.36}}}, {"id": 25, "entity": "monti", "kind": "date", "question": "Which victim did Monti claim on 6 March 2025?", "gold": "agi.net.", "expect": {"must": ["agi.net"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "agi.net (Organization) – On 2025-03-06, monti (RansomwareGroup, ransomware-group) claimed victim agi.net (Organization).", "s": 1.45}, "30b": {"ok": true, "why": "ok", "pred": "On 2025-03-06, monti (RansomwareGroup, ransomware-group) claimed victim agi.net (Organization).", "s": 0.43}, "cpu": {"ok": true, "why": "ok", "pred": "victim agi.net.", "s": 8.91}}}, {"id": 26, "entity": "CVE-2026-53132", "kind": "fact", "question": "Which kernel subsystem does CVE-2026-53132 concern?", "gold": "vsock/virtio (virtio transport for vsock).", "expect": {"any": [["vsock", "virtio"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "vsock/virtio (line: “vsock/virtio: fix potential unbounded skb queue”)", "s": 0.86}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-53132 concerns the vsock/virtio kernel subsystem.", "s": 0.2}, "cpu": {"ok": false, "why": "none of ['vsock', 'virtio']", "pred": "In the Linux kernel, the following vulnerability has been resolved.", "s": 3.11}}}, {"id": 27, "entity": "CVE-2026-53132", "kind": "fact", "question": "What kind of packets let an attacker queue an unbounded number of skbs in CVE-2026-53132?", "gold": "Packets with len == 0 and VIRTIO_VSOCK_SEQ_EOM set.", "expect": {"any": [["VIRTIO_VSOCK_SEQ_EOM", "len == 0", "zero-length", "length 0", "len 0"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Packets with len == 0 and VIRTIO_VSOCK_SEQ_EOM — “If fed with packets with len == 0 and VIRTIO_VSOCK_SEQ_EOM,”.", "s": 1.19}, "30b": {"ok": true, "why": "ok", "pred": "Packets with len == 0 and VIRTIO_VSOCK_SEQ_EOM let an attacker queue an unbounded number of skbs in CVE-2026-53132.", "s": 0.25}, "cpu": {"ok": false, "why": "none of ['VIRTIO_VSOCK_SEQ_EOM', 'len == 0', 'zero-length', 'length 0', 'len 0']", "pred": "a very large number of packets can be queued.", "s": 2.98}}}, {"id": 28, "entity": "CVE-2026-53132", "kind": "fact", "question": "How was CVE-2026-53132 fixed?", "gold": "By estimating the skb metadata size.", "expect": {"must": ["metadata"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Fix this by estimating the skb metadata size:【vsock/virtio: fix potential unbounded skb queue … Fix this by estimating the skb metadata size:】", "s": 1.52}, "30b": {"ok": true, "why": "ok", "pred": "Fix this by estimating the skb metadata size:", "s": 0.15}, "cpu": {"ok": true, "why": "ok", "pred": "Fix this by estimating the skb metadata size.", "s": 1.98}}}, {"id": 29, "entity": "CVE-2026-53132", "kind": "fact", "question": "What CVSS score and severity does CVE-2026-53132 carry?", "gold": "7.1, HIGH.", "expect": {"must": ["7.1", "high"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVSS score=7.1; severity=HIGH (cve_id=CVE-2026-53132; cvss_score=7.1; cvss_vector=CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:C/C:N/I:N/A:H; severity=HIGH; epss_score=0.00138; epss_percentile=0.02915; published_at=2026-06-25T09:16:30.437)", "s": 2.66}, "30b": {"ok": true, "why": "ok", "pred": "CVSS score: 7.1; severity: HIGH.", "s": 0.13}, "cpu": {"ok": true, "why": "ok", "pred": "cvss_score=7.1; cvss_vector=CVSS:3.1/AV:L/AC:L/PR:N/UI:N/S:C/C:N/I:N/A:H; severity=HIGH.", "s": 4.36}}}, {"id": 30, "entity": "CVE-2024-50302", "kind": "fact", "question": "What is the CVSS score and severity of CVE-2024-50302?", "gold": "5.5, MEDIUM.", "expect": {"must": ["5.5", "medium"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVSS score=5.5, severity=MEDIUM; source: cve_id=CVE-2024-50302; cvss_score=5.5; cvss_vector=CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N; severity=MEDIUM;", "s": 2.34}, "30b": {"ok": true, "why": "ok", "pred": "CVSS score: 5.5; severity: MEDIUM.", "s": 0.29}, "cpu": {"ok": true, "why": "ok", "pred": "cvss_score=5.5; cvss_vector=CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N; severity=MEDIUM.", "s": 12.03}}}, {"id": 31, "entity": "CVE-2024-50302", "kind": "reasoning", "question": "Besides the Linux kernel itself, which vendor's products are listed as affected by CVE-2024-50302?", "gold": "Siemens (SCALANCE switches).", "expect": {"any": [["Siemens", "SCALANCE"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Debian, Siemens, and Google – see:", "s": 2.19}, "30b": {"ok": true, "why": "ok", "pred": "Siemens, Google", "s": 0.29}, "cpu": {"ok": true, "why": "ok", "pred": "product debian debian linux 11.0, product siemens scalance xr528-6m, product siemens scalance xr524-8wg, product siemens scalance xr326-8eec, product siemens scalance xrm334, product siemens scalance xr524-8c, product siemens scalance xrh334, product siemens scalance xr552-12m, product siemens scalance xr526-8c, product siemens scalance xr526-8, product siem…", "s": 17.79}}}, {"id": 32, "entity": "CVE-2024-50302", "kind": "fact", "question": "What CWE weakness is behind CVE-2024-50302?", "gold": "Use of Uninitialized Resource.", "expect": {"must": ["Uninitialized Resource"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Use of Uninitialized Resource (Weakness) – On 2024-11-19, CVE-2024-50302 (Vulnerability) has weakness Use of Uninitialized Resource (Weakness).", "s": 1.46}, "30b": {"ok": true, "why": "ok", "pred": "Use of Uninitialized Resource (Weakness)", "s": 0.18}, "cpu": {"ok": true, "why": "ok", "pred": "weakness Use of Uninitialized Resource.", "s": 8.58}}}, {"id": 33, "entity": "CVE-2024-50302", "kind": "reasoning", "question": "Is CVE-2024-50302 exploitable over the network according to its CVSS vector?", "gold": "No. The attack vector is local (AV:L).", "expect": {"any": [["local", "AV:L"]], "expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "No – the CVSS vector shows AV:L (local attack) 【cvss_vector=CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N】.", "s": 0.89}, "30b": {"ok": true, "why": "ok", "pred": "No. CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N", "s": 0.18}, "cpu": {"ok": false, "why": "none of ['local', 'AV:L']", "pred": "No. The dossier does not mention network.", "s": 1.14}}}, {"id": 34, "entity": "CVE-2019-12900", "kind": "fact", "question": "Which library has the flaw in CVE-2019-12900, and in which function?", "gold": "bzip2, in BZ2_decompress.", "expect": {"must": ["bzip2", "BZ2_decompress"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "bzip2 library, function BZ2_decompress – CVE-2019-12900: BZ2_decompress in decompress.c in bzip2 through 1.0.6 has an out-of-bounds write when there are many selectors.", "s": 1.29}, "30b": {"ok": true, "why": "ok", "pred": "The flaw in CVE-2019-12900 is in the library bzip2, in the function BZ2_decompress in decompress.c.", "s": 0.29}, "cpu": {"ok": true, "why": "ok", "pred": "BZ2_decompress in decompress.c in bzip2 through 1.0.6.", "s": 4.71}}}, {"id": 35, "entity": "CVE-2019-12900", "kind": "fact", "question": "What is the EPSS percentile of CVE-2019-12900?", "gold": "0.94748.", "expect": {"any": [["0.94748", "0.947", "94.7"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "0.94754 — cve_id=CVE-2019-12900; cvss_score=9.8; cvss_vector=CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H; severity=CRITICAL; epss_score=0.07977; **epss_percentile=0.94754**; published_at=2019-06-19T23:15:09.910", "s": 1.29}, "30b": {"ok": true, "why": "ok", "pred": "The EPSS percentile of CVE-2019-12900 is 0.94754.", "s": 0.17}, "cpu": {"ok": true, "why": "ok", "pred": "epss_percentile=0.94713.", "s": 1.12}}}, {"id": 36, "entity": "CVE-2019-12900", "kind": "fact", "question": "Which Ubuntu release is listed as affected by CVE-2019-12900?", "gold": "Ubuntu 18.04.", "expect": {"must": ["18.04"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 18.04 (Product). CVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 12.04 (Product). CVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 19.04 (Product). CVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 16.04 (Product). CVE-2019-129…", "s": 2.44}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 18.04 (Product).  \nCVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 12.04 (Product).  \nCVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 19.04 (Product).  \nCVE-2019-12900 (Vulnerability) affects product canonical ubuntu linux 16.04 (Product).  \nCVE-…", "s": 0.6}, "cpu": {"ok": true, "why": "ok", "pred": "product canonical ubuntu linux 18.04, product canonical ubuntu linux 12.04, product canonical ubuntu linux 19.04, product canonical ubuntu linux 16.04, product canonical ubuntu linux 14.04.", "s": 5.64}}}, {"id": 37, "entity": "CVE-2019-12900", "kind": "fact", "question": "What severity does CVE-2019-12900 carry?", "gold": "CRITICAL (CVSS 9.8).", "expect": {"any": [["critical", "9.8"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CRITICAL (severity=CRITICAL)", "s": 0.81}, "30b": {"ok": true, "why": "ok", "pred": "CRITICAL; severity=CRITICAL", "s": 0.13}, "cpu": {"ok": true, "why": "ok", "pred": "CRITICAL.", "s": 0.54}}}, {"id": 38, "entity": "CVE-2019-12900", "kind": "fact", "question": "Which versions of bzip2 are affected by CVE-2019-12900?", "gold": "bzip2 through 1.0.6.", "expect": {"must": ["1.0.6"]}, "results": {"120b": {"ok": false, "why": "missing '1.0.6'", "pred": "The dossier does not say.", "s": 0.57}, "30b": {"ok": false, "why": "missing '1.0.6'", "pred": "CVE-2019-12900 (Vulnerability) affects product bzip bzip2 (Product).", "s": 0.2}, "cpu": {"ok": true, "why": "ok", "pred": "bzip2 through 1.0.6.", "s": 3.34}}}, {"id": 39, "entity": "CVE-2026-59989", "kind": "fact", "question": "Which PHP framework does CVE-2026-59989 affect, and which component?", "gold": "Phalcon, the Volt template compiler.", "expect": {"must": ["Phalcon"], "any": [["Volt", "compiler"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Phalcon — the high‑performance, full‑stack PHP framework (CVE-2026-59989: Phalcon is a high-performance, full-stack PHP framework.); affected component: phalcon/Mvc/View/Engine/Volt/Compiler.zep (resolveFilter in phalcon/Mvc/View/Engine/Volt/Compiler.zep builds the join filter…).", "s": 1.83}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-59989 affects the Phalcon PHP framework, specifically the Volt compiler component.", "s": 0.19}, "cpu": {"ok": false, "why": "none of ['Volt', 'compiler']", "pred": "Phalcon is a high-performance, full-stack PHP framework.", "s": 3.11}}}, {"id": 40, "entity": "CVE-2026-59989", "kind": "fact", "question": "Which versions of Phalcon are affected by CVE-2026-59989?", "gold": "5.15.0 and earlier.", "expect": {"must": ["5.15.0"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Versions 5.15.0 and earlier are affected. 【In 5.15.0 and earlier, resolveFilter in phalcon/Mvc/View/Engine/Volt/Compiler.zep builds the join filter…】", "s": 0.97}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-59989: Phalcon is a high-performance, full-stack PHP framework. In 5.15.0 and earlier, resolveFilter in phalcon/Mvc/View/Engine/Volt/Compiler.zep builds the join filter by inserting the raw separator and array token values into generated PHP without passing them through expression().", "s": 0.35}, "cpu": {"ok": true, "why": "ok", "pred": "In 5.15.0 and earlier.", "s": 2.68}}}, {"id": 41, "entity": "CVE-2026-59989", "kind": "fact", "question": "What two weakness types are attached to CVE-2026-59989?", "gold": "Code Injection and Improper Neutralization of Special Elements Used in a Template Engine.", "expect": {"must": ["Code Injection", "Template Engine"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Improper Control of Generation of Code ('Code Injection') (Weakness); Improper Neutralization of Special Elements Used in a Template Engine (Weakness) – CVE-2026-59989 (Vulnerability) has weakness Improper Control of Generation of Code ('Code Injection') (Weakness). CVE-2026-59989 (Vulnerability) has weakness Improper Neutralization of Special Elements Used …", "s": 2.72}, "30b": {"ok": true, "why": "ok", "pred": "Improper Control of Generation of Code ('Code Injection') (Weakness) and Improper Neutralization of Special Elements Used in a Template Engine (Weakness)", "s": 0.19}, "cpu": {"ok": true, "why": "ok", "pred": "weakness Improper Control of Generation of Code ('Code Injection'), weakness Improper Neutralization of Special Elements Used in a Template Engine.", "s": 2.97}}}, {"id": 42, "entity": "CVE-2026-59989", "kind": "date", "question": "When was CVE-2026-59989 published?", "gold": "2026-08-21.", "expect": {"any": [["2026-08-21", "21 August 2026", "August 21, 2026"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2026-08-21T21:17:00.713 — cve_id=CVE-2026-59989; severity=critical; epss_score=0.00543; epss_percentile=0.44869; published_at=2026-08-21T21:17:00.713.", "s": 1.06}, "30b": {"ok": true, "why": "ok", "pred": "published_at=2026-08-21T21:17:00.713", "s": 0.25}, "cpu": {"ok": true, "why": "ok", "pred": "2026-08-21T21:17:00.713.", "s": 2.94}}}, {"id": 43, "entity": "CVE-2026-97824", "kind": "fact", "question": "What kind of attacker can exploit CVE-2026-97824, and what is the impact?", "gold": "An authorized attacker; information disclosure over a network.", "expect": {"any": [["authorized", "authenticated"], ["disclos", "information"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Authorized attacker can exploit CVE-2026-97824 to disclose information over a network. (CVE-2026-97824: Out-of-bounds read in DNS Server allows an authorized attacker to disclose information over a network.)", "s": 1.41}, "30b": {"ok": true, "why": "ok", "pred": "An authorized attacker can exploit CVE-2026-97824 to disclose information over a network.", "s": 0.23}, "cpu": {"ok": true, "why": "ok", "pred": "allows an authorized attacker to disclose information over a network.", "s": 4.75}}}, {"id": 44, "entity": "CVE-2026-97824", "kind": "yesno", "question": "Does CVE-2026-97824 affect Windows 10?", "gold": "Yes: Windows 10 22H2 and 21H2 are listed.", "expect": {"any": [["22h2", "21h2", "windows 10"]], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. “CVE-2026-97824 (Vulnerability) affects product microsoft windows 10 22h2 (Product).”", "s": 0.84}, "30b": {"ok": true, "why": "ok", "pred": "Yes. CVE-2026-97824 (Vulnerability) affects product microsoft windows 10 22h2 (Product).  \nCVE-2026-97824 (Vulnerability) affects product microsoft windows 10 21h2 (Product).  \nCVE-2026-97824 (Vulnerability) affects product microsoft windows 10 1809 (Product).  \nCVE-2026-97824 (Vulnerability) affects product microsoft windows 10 1607 (Product).", "s": 0.53}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. CVE-2026-97824 (Vulnerability) affects product microsoft windows 10 22h2 (Product).", "s": 5.85}}}, {"id": 45, "entity": "CVE-2026-97824", "kind": "fact", "question": "What Windows component is vulnerable in CVE-2026-97824?", "gold": "DNS Server.", "expect": {"must": ["DNS Server"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "DNS Server — CVE-2026-97824: Out-of-bounds read in DNS Server allows an authorized attacker to disclose information over a network. (Line: “CVE-2026-97824: Out-of-bounds read in DNS Server allows an authorized attacker to disclose information over a network.”)", "s": 1.18}, "30b": {"ok": true, "why": "ok", "pred": "DNS Server", "s": 0.09}, "cpu": {"ok": true, "why": "ok", "pred": "Out-of-bounds read in DNS Server.", "s": 3.7}}}, {"id": 46, "entity": "CVE-2026-97824", "kind": "fact", "question": "What is the CVSS base score of CVE-2026-97824?", "gold": "6.5.", "expect": {"must": ["6.5"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "6.5 – cve_id=CVE-2026-97824; cvss_score=6.5; cvss_vector=CVSS:3.1/AV:N/AC:L/PR:L/UI:N/S:U/C:H/I:N/A:N; severity=MEDIUM; epss_score=0.00769; epss_percentile=0.5516; published_at=2026-10-13T19:22:21.463", "s": 2.14}, "30b": {"ok": true, "why": "ok", "pred": "The CVSS base score of CVE-2026-97824 is 6.5.", "s": 0.19}, "cpu": {"ok": true, "why": "ok", "pred": "CVSS base score 6.5 (MEDIUM).", "s": 4.16}}}, {"id": 47, "entity": "CVE-2026-97824", "kind": "fact", "question": "Which CWE weakness is recorded for CVE-2026-97824?", "gold": "Out-of-bounds Read.", "expect": {"must": ["Out-of-bounds Read"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Out-of-bounds Read (Weakness) – “CVE-2026-97824 (Vulnerability) has weakness Out-of-bounds Read (Weakness).”", "s": 0.86}, "30b": {"ok": true, "why": "ok", "pred": "CWE-125: Out-of-bounds Read (Weakness)", "s": 0.14}, "cpu": {"ok": true, "why": "ok", "pred": "weakness Out-of-bounds Read.", "s": 3.27}}}, {"id": 48, "entity": "CVE-2026-71300", "kind": "fact", "question": "Which version of Apache Camel fixes CVE-2026-71300?", "gold": "4.22.0.", "expect": {"must": ["4.22.0"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Version 4.22.0 fixes the issue. 【Users are recommended to upgrade to version 4.22.0, which fixes the issue.】", "s": 1.59}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-71300 is fixed in Apache Camel version 4.22.0, and also in 4.14.9 for the 4.14.x LTS stream and 4.18.4 for the 4.18.x stream.", "s": 0.37}, "cpu": {"ok": true, "why": "ok", "pred": "Users are recommended to upgrade to version 4.22.0, which fixes the issue.", "s": 6.04}}}, {"id": 49, "entity": "CVE-2026-71300", "kind": "reasoning", "question": "If I am on the Camel 4.14 LTS line, which release should I move to for CVE-2026-71300?", "gold": "4.14.9.", "expect": {"must": ["4.14.9"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Upgrade to 4.14.9 – “If users are on the 4.14.x LTS releases stream, then they are suggested to upgrade to 4.14.9.”", "s": 1.59}, "30b": {"ok": true, "why": "ok", "pred": "If users are on the 4.14.x LTS releases stream, then they are suggested to upgrade to 4.14.9.", "s": 0.28}, "cpu": {"ok": true, "why": "ok", "pred": "suggested to upgrade to 4.14.9.", "s": 5.62}}}, {"id": 50, "entity": "CVE-2026-71300", "kind": "fact", "question": "Which Camel component is affected by CVE-2026-71300?", "gold": "camel-atmosphere-websocket.", "expect": {"must": ["atmosphere"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Apache Camel Atmosphere Websocket component — CVE-2026-71300: Improper input validation vulnerability in Apache Camel Atmosphere Websocket component.【CVE-2026-71300: Improper input validation vulnerability in Apache Camel Atmosphere Websocket component.】", "s": 1.33}, "30b": {"ok": true, "why": "ok", "pred": "Apache Camel Atmosphere Websocket component.", "s": 0.18}, "cpu": {"ok": true, "why": "ok", "pred": "Apache Camel Atmosphere Websocket component.", "s": 5.39}}}, {"id": 51, "entity": "CVE-2026-71300", "kind": "fact", "question": "How does the injection happen in CVE-2026-71300?", "gold": "The producer selects target WebSocket peers through Exchange headers that are not filtered.", "expect": {"any": [["Exchange header", "header"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "In a route bridging an HTTP consumer into an atmosphere‑websocket producer, an external sender could supply the list header and take over the producer's dispatch decision. 【In a route bridging an HTTP consumer into an atmosphere-websocket producer, an external sender could supply the list header and take over the producer's dispatch decision.】", "s": 3.78}, "30b": {"ok": true, "why": "ok", "pred": "The injection happens through Exchange headers whose names sat outside the filtered Camel namespace: websocket.connectionKey and websocket.connectionKey.list, along with websocket.sendToAll, websocket.eventType and websocket.errorType.", "s": 0.3}, "cpu": {"ok": true, "why": "ok", "pred": "the producer selected its target peers through Exchange headers whose names sat outside the filtered Camel namespace.", "s": 5.65}}}, {"id": 52, "entity": "composer/typo3/cms-backend", "kind": "date", "question": "Which CVEs were attached to composer/typo3/cms-backend on 12 June 2026?", "gold": "CVE-2026-47351 and CVE-2026-47352.", "expect": {"must": ["CVE-2026-47351", "CVE-2026-47352"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVE-2026-47352, CVE-2026-47351 – On 2026-06-12, CVE-2026-47352 (Vulnerability) affects composer/typo3/cms-backend (Software).; On 2026-06-12, CVE-2026-47351 (Vulnerability) affects composer/typo3/cms-backend (Software).", "s": 2.03}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-47352 (Vulnerability), CVE-2026-47351 (Vulnerability)", "s": 0.24}, "cpu": {"ok": true, "why": "ok", "pred": "CVE-2026-47352, CVE-2026-47351.", "s": 3.74}}}, {"id": 53, "entity": "composer/typo3/cms-backend", "kind": "date", "question": "Which CVE was linked to composer/typo3/cms-backend on 1 September 2026?", "gold": "CVE-2026-19418.", "expect": {"must": ["CVE-2026-19418"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVE-2026-19418 (Vulnerability) – On 2026-09-01, composer/typo3/cms-backend (Software) vulnerable to CVE-2026-19418 (Vulnerability).", "s": 2.12}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-09-01, composer/typo3/cms-backend (Software) vulnerable to CVE-2026-19418 (Vulnerability).", "s": 0.24}, "cpu": {"ok": true, "why": "ok", "pred": "to CVE-2026-19418.", "s": 2.99}}}, {"id": 54, "entity": "composer/typo3/cms-backend", "kind": "yesno", "question": "Is CVE-2025-59020 listed against composer/typo3/cms-backend?", "gold": "Yes.", "expect": {"must": ["CVE-2025-59020"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. \"CVE-2025-59020 (Vulnerability) affects composer/typo3/cms-backend (Software).\"", "s": 0.82}, "30b": {"ok": true, "why": "ok", "pred": "Yes. \"CVE-2025-59020 (Vulnerability) affects composer/typo3/cms-backend (Software).\"", "s": 0.2}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. CVE-2025-59020 (Vulnerability) affects composer/typo3/cms-backend (Software).", "s": 4.01}}}, {"id": 55, "entity": "composer/typo3/cms-backend", "kind": "negative", "question": "Does the dossier list any 2024 CVE for composer/typo3/cms-backend?", "gold": "No. There is no CVE-2024 entry.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "No. Lines showing only 2025‑2026 CVEs, e.g., “On 2026‑09‑01, composer/typo3/cms‑backend (Software) vulnerable to CVE‑2026‑19418 (Vulnerability).” and “CVE‑2025‑59020 (Vulnerability) affects composer/typo3/cms‑backend (Software).” indicate no 2024 CVE listed.", "s": 1.51}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.14}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention 2024 CVE.", "s": 2.89}}}, {"id": 56, "entity": "pip/djust", "kind": "reasoning", "question": "Which advisory identifiers for pip/djust are GHSA rather than CVE?", "gold": "GHSA-xjw9-38cr-6372 and GHSA-9395-2g46-rj3f.", "expect": {"must": ["GHSA-xjw9-38cr-6372", "GHSA-9395-2g46-rj3f"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "GHSA-xjw9-38cr-6372, GHSA-9395-2g46-rj3f — On 2026-09-17, pip/djust (Software) vulnerable to GHSA-xjw9-38cr-6372 (Vulnerability).; On 2026-09-17, pip/djust (Software) vulnerable to GHSA-9395-2g46-rj3f (Vulnerability).", "s": 2.5}, "30b": {"ok": true, "why": "ok", "pred": "GHSA-xjw9-38cr-6372 (Vulnerability), GHSA-9395-2g46-rj3f (Vulnerability)", "s": 0.31}, "cpu": {"ok": true, "why": "ok", "pred": "GHSA-xjw9-38cr-6372, GHSA-9395-2g46-rj3f.", "s": 8.84}}}, {"id": 57, "entity": "pip/djust", "kind": "date", "question": "What was added to pip/djust's dossier on 21 October 2026?", "gold": "CVE-2026-113731.", "expect": {"must": ["CVE-2026-113731"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVE‑2026‑113731 vulnerability was added (On 2026‑10‑21, pip/djust (Software) vulnerable to CVE‑2026‑113731 (Vulnerability); On 2026‑10‑21, CVE‑2026‑113731 (Vulnerability) affects pip/djust (Software)).", "s": 1.84}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-21, pip/djust (Software) vulnerable to CVE-2026-113731 (Vulnerability).  \nOn 2026-10-21, CVE-2026-113731 (Vulnerability) affects pip/djust (Software).", "s": 0.32}, "cpu": {"ok": true, "why": "ok", "pred": "vulnerable to CVE-2026-113731.", "s": 1.46}}}, {"id": 58, "entity": "pip/djust", "kind": "date", "question": "Name three CVEs dated 16 September 2026 for pip/djust.", "gold": "Any three of CVE-2026-61588 … CVE-2026-61599 (twelve were attached that day).", "expect": {"min_distinct": {"pattern": "CVE-2026-615(?:8[89]|9[0-9])", "n": 3}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVE-2026-61591, CVE-2026-61592, CVE-2026-61597 (using lines: “On 2026-09-16, pip/djust (Software) vulnerable to CVE-2026-61591 (Vulnerability).”; “On 2026-09-16, pip/djust (Software) vulnerable to CVE-2026-61592 (Vulnerability).”; “On 2026-09-16, pip/djust (Software) vulnerable to CVE-2026-61597 (Vulnerability).”)", "s": 3.16}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-61591 (Vulnerability), CVE-2026-61592 (Vulnerability), CVE-2026-61597 (Vulnerability).", "s": 0.38}, "cpu": {"ok": true, "why": "ok", "pred": "to CVE-2026-61591, to CVE-2026-61592, to CVE-2026-61597.", "s": 9.08}}}, {"id": 59, "entity": "TinyTurla", "kind": "fact", "question": "Who uses TinyTurla, and in which countries have targets been hit?", "gold": "Turla; targets in the US, Germany and Afghanistan.", "expect": {"must": ["Turla", "Germany", "Afghanistan"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Turla (ThreatActor) uses TinyTurla (Software, malware) (On 2021‑12‑02, Turla (ThreatActor) uses TinyTurla (Software, malware).), Turla (EcoActor) uses TinyTurla (Malware) (On 2026‑10‑24, Turla (EcoActor) uses TinyTurla (Malware).), Turla (DwActor) dw uses TinyTurla (DwMalware) (Turla (DwActor) dw uses TinyTurla (DwMalware).); targets hit in the US, Germany, …", "s": 3.48}, "30b": {"ok": true, "why": "ok", "pred": "Turla (ThreatActor) uses TinyTurla (Software, malware), and targets have been hit in the US, Germany, and Afghanistan.", "s": 0.28}, "cpu": {"ok": true, "why": "ok", "pred": "has been used by Turla against targets in the US, Germany, and Afghanistan since at least 2020.", "s": 5.65}}}, {"id": 60, "entity": "TinyTurla", "kind": "fact", "question": "Since when has TinyTurla been in use?", "gold": "Since at least 2020.", "expect": {"must": ["2020"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Since at least 2020 – “TinyTurla is a backdoor that has been used by Turla against targets in the US, Germany, and Afghanistan since at least 2020.” (first line)", "s": 1.0}, "30b": {"ok": true, "why": "ok", "pred": "Since at least 2020.", "s": 0.19}, "cpu": {"ok": true, "why": "ok", "pred": "since at least 2020.", "s": 4.62}}}, {"id": 61, "entity": "TinyTurla", "kind": "fact", "question": "Which ATT&CK techniques does TinyTurla implement?", "gold": "Web Protocols and Data from Local System.", "expect": {"must": ["Web Protocols", "Data from Local System"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Web Protocols (Technique) – On 2021-12-02, TinyTurla (Software, malware) implements Web Protocols (Technique); Data from Local System (Technique) – On 2022-04-14, TinyTurla (Software, malware) implements Data from Local System (Technique); Fallback Channels (Technique) – On 2022-04-16, TinyTurla (Software, malware) implements Fallback Channels (Technique); Q…", "s": 5.11}, "30b": {"ok": true, "why": "ok", "pred": "TinyTurla (Software, malware) implements Web Protocols (Technique), Data from Local System (Technique), Fallback Channels (Technique), Query Registry (Technique), Scheduled Transfer (Technique), Masquerade Task or Service (Technique), Fileless Storage (Technique), Ingress Tool Transfer (Technique), Windows Command Shell (Technique), Service Execution (Techni…", "s": 0.5}, "cpu": {"ok": true, "why": "ok", "pred": "Web Protocols, Data from Local System, Fallback Channels, Query Registry, Scheduled Transfer, Masquerade Task or Service, Fileless Storage, Ingress Tool Transfer, Windows Command Shell, Service Execution, Modify Registry, Match Legitimate Resource Name or Location.", "s": 7.32}}}, {"id": 62, "entity": "TinyTurla", "kind": "negative", "question": "Is TinyTurla ransomware?", "gold": "No. It is a backdoor.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.63}, "30b": {"ok": true, "why": "ok", "pred": "No. The dossier does not say.", "s": 0.2}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention ransomware.", "s": 4.21}}}, {"id": 63, "entity": "PowerDuke", "kind": "fact", "question": "Which group used PowerDuke, and in what year?", "gold": "APT29, in 2016.", "expect": {"must": ["APT29", "2016"]}, "results": {"120b": {"ok": false, "why": "missing '2016'", "pred": "APT29 used PowerDuke in 2017 (On 2017-05-31, APT29 (ThreatActor) uses PowerDuke (Software, malware)) and in 2026 (On 2026-10-24, APT 29 (EcoActor) uses PowerDuke (Malware)).", "s": 2.14}, "30b": {"ok": false, "why": "missing '2016'", "pred": "APT29 (ThreatActor) uses PowerDuke (Software, malware) on 2017-05-31.  \nAPT 29 (DwActor) dw uses PowerDuke (DwMalware) on 2026-10-24.", "s": 0.36}, "cpu": {"ok": true, "why": "ok", "pred": "APT29 in 2016.", "s": 4.44}}}, {"id": 64, "entity": "PowerDuke", "kind": "fact", "question": "How was PowerDuke delivered to victims?", "gold": "Through Microsoft Word or Excel attachments.", "expect": {"any": [["Word", "Excel", "attachment"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "PowerDuke was primarily delivered through Microsoft Word or Excel attachments containing malicious macros. (Line: “PowerDuke: PowerDuke is a backdoor that was used by APT29 in 2016. It has primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.”)", "s": 1.26}, "30b": {"ok": true, "why": "ok", "pred": "PowerDuke was delivered through Microsoft Word or Excel attachments containing malicious macros.", "s": 0.12}, "cpu": {"ok": true, "why": "ok", "pred": "primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.", "s": 1.07}}}, {"id": 65, "entity": "PowerDuke", "kind": "fact", "question": "How does PowerDuke persist on a host?", "gold": "Registry Run Keys / Startup Folder.", "expect": {"any": [["Registry Run Keys", "Startup Folder"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "It persists via Registry Run Keys / Startup Folder. 【On 2017-12-14, PowerDuke (Software, malware) implements Registry Run Keys / Startup Folder (Technique).】", "s": 1.08}, "30b": {"ok": true, "why": "ok", "pred": "PowerDuke (Software, malware) implements Registry Run Keys / Startup Folder (Technique).", "s": 0.24}, "cpu": {"ok": false, "why": "none of ['Registry Run Keys', 'Startup Folder']", "pred": "primarily been delivered through Microsoft Word or Excel attachments containing malicious macros.", "s": 7.53}}}, {"id": 66, "entity": "PowerDuke", "kind": "negative", "question": "Does the dossier link PowerDuke to Turla?", "gold": "No. PowerDuke is linked to APT29, not Turla.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.43}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.1}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention Turla.", "s": 0.99}}}, {"id": 67, "entity": "rust/kanidm", "kind": "fact", "question": "Which advisories affect rust/kanidm?", "gold": "GHSA-gpxg-fx2g-qxj2 and GHSA-53hj-r94p-8c8f.", "expect": {"must": ["GHSA-gpxg-fx2g-qxj2", "GHSA-53hj-r94p-8c8f"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "GHSA-53hj-r94p-8c8f, GHSA-gpxg-fx2g-qxj2 (GHSA-53hj-r94p-8c8f (Vulnerability) affects rust/kanidm (Software).; GHSA-gpxg-fx2g-qxj2 (Vulnerability) affects rust/kanidm (Software).)", "s": 3.07}, "30b": {"ok": true, "why": "ok", "pred": "GHSA-53hj-r94p-8c8f (Vulnerability) affects rust/kanidm (Software).  \nGHSA-gpxg-fx2g-qxj2 (Vulnerability) affects rust/kanidm (Software).", "s": 0.3}, "cpu": {"ok": false, "why": "missing 'GHSA-gpxg-fx2g-qxj2'", "pred": "The dossier does not say.", "s": 1.32}}}, {"id": 68, "entity": "rust/kanidm", "kind": "negative", "question": "Does rust/kanidm have any CVE-numbered advisory in the dossier?", "gold": "No. Only GHSA identifiers are listed.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "No. [\"Rust/kanidm (Software) vulnerable to GHSA-gpxg-fx2g-qxj2 (Vulnerability).\", \"Rust/kanidm (Software) vulnerable to GHSA-53hj-r94p-8c8f (Vulnerability).\"]", "s": 1.21}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.12}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention CVE-numbered advisory.", "s": 1.65}}}, {"id": 69, "entity": "Baker McKenzie", "kind": "fact", "question": "Which ransomware group claimed Baker McKenzie?", "gold": "SilentRansomGroup (Silent Ransom Group); the merged dossier also carries a leakeddata dark-web claim.", "expect": {"any": [["SilentRansomGroup", "Silent Ransom Group"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "SilentRansomGroup (RansomwareGroup) — Baker McKenzie (RansomwareVictim) claimed by SilentRansomGroup (RansomwareGroup).", "s": 1.41}, "30b": {"ok": true, "why": "ok", "pred": "SilentRansomGroup claimed Baker McKenzie.", "s": 0.17}, "cpu": {"ok": true, "why": "ok", "pred": "Silent Ransom Group.", "s": 2.0}}}, {"id": 70, "entity": "Baker McKenzie", "kind": "fact", "question": "What sector and country is Baker McKenzie filed under?", "gold": "Professional Services, US.", "expect": {"must": ["Professional Services"], "any": [["US", "United States"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Professional Services, US — sector=Professional Services; country=US", "s": 2.24}, "30b": {"ok": true, "why": "ok", "pred": "Baker McKenzie (RansomwareVictim) in country US (Region).  \nBaker McKenzie (RansomwareVictim) in sector Professional Services (Sector).", "s": 0.25}, "cpu": {"ok": true, "why": "ok", "pred": "sector=Professional Services; country=US.", "s": 2.26}}}, {"id": 71, "entity": "Baker McKenzie", "kind": "fact", "question": "What does Baker McKenzie do?", "gold": "It is a global law firm headquartered in Chicago.", "expect": {"must": ["law firm"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Baker McKenzie: [AI generated] Baker McKenzie is a global law firm headquartered in Chicago, Illinois, United States. Founded in 1949, it operates in the legal services industry, providing counsel in areas such as corporate law, tax, dispute resolution, mergers and acquisitions, employment law, and intellectual property. The firm serves multinational corpora…", "s": 2.2}, "30b": {"ok": true, "why": "ok", "pred": "Baker McKenzie is a global law firm headquartered in Chicago, Illinois, United States. Founded in 1949, it operates in the legal services industry, providing counsel in areas such as corporate law, tax, dispute resolution, mergers and acquisitions, employment law, and intellectual property. The firm serves multinational corporations and organizations, with o…", "s": 0.39}, "cpu": {"ok": false, "why": "missing 'law firm'", "pred": "providing counsel in areas such as corporate law, tax, dispute resolution, mergers and acquisitions, employment law, and intellectual property.", "s": 3.23}}}, {"id": 72, "entity": "Baker McKenzie", "kind": "negative", "question": "Was Baker McKenzie claimed by LockBit?", "gold": "No. It was claimed by SilentRansomGroup.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.61}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.15}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention LockBit.", "s": 0.84}}}, {"id": 73, "entity": "Baker McKenzie", "kind": "fact", "question": "When was Baker McKenzie founded?", "gold": "1949.", "expect": {"must": ["1949"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Founded in 1949【Baker McKenzie: [AI generated] Baker McKenzie is a global law firm headquartered in Chicago, Illinois, United States. Founded in 1949, it operates…】", "s": 0.94}, "30b": {"ok": true, "why": "ok", "pred": "Baker McKenzie was founded in 1949.", "s": 0.17}, "cpu": {"ok": true, "why": "ok", "pred": "Founded in 1949.", "s": 2.06}}}, {"id": 74, "entity": "The State Bank", "kind": "fact", "question": "Which group claimed The State Bank?", "gold": "Storm.", "expect": {"must": ["Storm"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Storm (RansomwareGroup) – The State Bank (RansomwareVictim) claimed by Storm (RansomwareGroup).", "s": 1.45}, "30b": {"ok": true, "why": "ok", "pred": "The State Bank (RansomwareVictim) claimed by Storm (RansomwareGroup).", "s": 0.22}, "cpu": {"ok": false, "why": "missing 'Storm'", "pred": "The dossier does not say.", "s": 2.99}}}, {"id": 75, "entity": "The State Bank", "kind": "fact", "question": "Where is The State Bank headquartered?", "gold": "7526 Hancock Drive, Wonder Lake, Illinois.", "expect": {"must": ["Wonder Lake"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The company headquarters is located in 7526 Hancock Drive, Wonder Lake, IL 60097, United States.11-50 Employees", "s": 1.86}, "30b": {"ok": true, "why": "ok", "pred": "The State Bank headquarters is located in 7526 Hancock Drive, Wonder Lake, IL 60097, United States.", "s": 0.24}, "cpu": {"ok": true, "why": "ok", "pred": "7526 Hancock Drive, Wonder Lake, IL 60097, United States.", "s": 3.7}}}, {"id": 76, "entity": "The State Bank", "kind": "fact", "question": "How large is The State Bank by employee count?", "gold": "11-50 employees.", "expect": {"any": [["11-50", "11 to 50"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "11-50 Employees — “The State Bank (RansomwareVictim) … 11-50 Employees”.", "s": 1.34}, "30b": {"ok": false, "why": "none of ['11-50', '11 to 50']", "pred": "The company currently have a staff of more than 80 employees.", "s": 0.21}, "cpu": {"ok": true, "why": "ok", "pred": "11-50 Employees.", "s": 3.1}}}, {"id": 77, "entity": "The State Bank", "kind": "fact", "question": "Which sector is The State Bank in?", "gold": "Financial Services.", "expect": {"must": ["Financial Services"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Financial Services; used lines: `sector=Financial Services` and `The State Bank (RansomwareVictim) in sector Financial Services (Sector)`.", "s": 1.16}, "30b": {"ok": true, "why": "ok", "pred": "The State Bank is in the Financial Services sector.", "s": 0.19}, "cpu": {"ok": true, "why": "ok", "pred": "sector=Financial Services.", "s": 2.91}}}, {"id": 78, "entity": "On Demand Occupational Medicine", "kind": "fact", "question": "Who claimed On Demand Occupational Medicine?", "gold": "Wallstreet.", "expect": {"must": ["Wallstreet"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Wallstreet (RansomwareGroup) – “On Demand Occupational Medicine (RansomwareVictim) claimed by Wallstreet (RansomwareGroup).”", "s": 1.5}, "30b": {"ok": true, "why": "ok", "pred": "Wallstreet (RansomwareGroup) claimed On Demand Occupational Medicine.", "s": 0.16}, "cpu": {"ok": false, "why": "missing 'Wallstreet'", "pred": "The dossier does not say.", "s": 1.22}}}, {"id": 79, "entity": "On Demand Occupational Medicine", "kind": "fact", "question": "What services does On Demand Occupational Medicine provide?", "gold": "Occupational health, drug testing, workplace safety and employee wellness services.", "expect": {"any": [["occupational health", "drug testing", "workplace safety", "wellness"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "On Demand Occupational Medicine provides occupational health, drug testing, workplace safety, and employee wellness services in Austintown, Ohio. [Line 1]", "s": 1.07}, "30b": {"ok": true, "why": "ok", "pred": "On Demand Occupational Medicine provides occupational health, drug testing, workplace safety, and employee wellness services in Austintown, Ohio.", "s": 0.18}, "cpu": {"ok": true, "why": "ok", "pred": "occupational health, drug testing, workplace safety, and employee wellness services.", "s": 1.65}}}, {"id": 80, "entity": "On Demand Occupational Medicine", "kind": "fact", "question": "Where is On Demand Occupational Medicine based?", "gold": "Austintown, Ohio.", "expect": {"must": ["Austintown"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Austintown, Ohio – On Demand Occupational Medicine provides occupational health, drug testing, workplace safety, and employee wellness services **in Austintown, Ohio**. (Line: “On Demand Occupational Medicine provides occupational health, drug testing, workplace safety, and employee wellness services in Austintown, Ohio.”)", "s": 1.73}, "30b": {"ok": true, "why": "ok", "pred": "On Demand Occupational Medicine is based in Austintown, Ohio.", "s": 0.14}, "cpu": {"ok": true, "why": "ok", "pred": "Austintown, Ohio.", "s": 0.6}}}, {"id": 81, "entity": "On Demand Occupational Medicine", "kind": "date", "question": "On what date was the claim against On Demand Occupational Medicine published?", "gold": "2026-09-10.", "expect": {"any": [["2026-09-10", "10 September 2026", "September 10, 2026"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2026-09-10T13:56:19+00:00 — published_at=2026-09-10T13:56:19+00:00.", "s": 1.06}, "30b": {"ok": false, "why": "none of ['2026-09-10', '10 September 2026', 'September 10, 2026']", "pred": "The dossier does not say.", "s": 0.13}, "cpu": {"ok": true, "why": "ok", "pred": "2026-09-10T13:56:19+00:00.", "s": 2.4}}}, {"id": 82, "entity": "341 Malicious Clawed Skills Found by the Bot They Were Targeting", "kind": "fact", "question": "What nickname was given to the malware campaign in this report?", "gold": "ClawHavoc.", "expect": {"must": ["ClawHavoc"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "ClawHavoc — “A massive malware campaign dubbed ClawHavoc has been uncovered…” (first description line).", "s": 1.25}, "30b": {"ok": true, "why": "ok", "pred": "ClawHavoc", "s": 0.27}, "cpu": {"ok": true, "why": "ok", "pred": "A massive malware campaign dubbed ClawHavoc.", "s": 9.82}}}, {"id": 83, "entity": "341 Malicious Clawed Skills Found by the Bot They Were Targeting", "kind": "fact", "question": "Which marketplace was targeted by the ClawHavoc campaign?", "gold": "The ClawHub marketplace.", "expect": {"must": ["ClawHub"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "ClawHub marketplace — “A massive malware campaign dubbed ClawHavoc has been uncovered in the ClawHub marketplace, targeting OpenClaw bots and their users.”【Dossier line 1】", "s": 1.17}, "30b": {"ok": true, "why": "ok", "pred": "The ClawHavoc campaign targeted the ClawHub marketplace.", "s": 0.32}, "cpu": {"ok": true, "why": "ok", "pred": "A massive malware campaign dubbed ClawHavoc has been uncovered in the ClawHub marketplace.", "s": 10.51}}}, {"id": 84, "entity": "341 Malicious Clawed Skills Found by the Bot They Were Targeting", "kind": "fact", "question": "Which stealer is tagged in this campaign?", "gold": "Atomic Stealer (AMOS).", "expect": {"any": [["Atomic Stealer", "AMOS"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "atomic stealer (amos) — tags=typosquatting, crypto wallets, amos, malicious skills, **atomic stealer (amos)**, bot security, supply chain attack, clawhub, openclaw.", "s": 1.71}, "30b": {"ok": true, "why": "ok", "pred": "The dossier says: atomic stealer (amos)", "s": 0.17}, "cpu": {"ok": true, "why": "ok", "pred": "Atomic Stealer (AMOS).", "s": 0.94}}}, {"id": 85, "entity": "341 Malicious Clawed Skills Found by the Bot They Were Targeting", "kind": "reasoning", "question": "On which date did Signal record the larger velocity spike for this campaign, and what was its z-score?", "gold": "2026-08-12, z=9.9.", "expect": {"must": ["9.9"], "any": [["2026-08-12", "12 August 2026", "August 12, 2026"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2026-08-12, z=9.9 — On 2026-08-12, Velocity spike: 341 Malicious Clawed Skills Found by the Bot They Were Targeting (+1 edges, z=9.9) (EmergentSignal) involves 341 Malicious Clawed Skills Found by the Bot They Were Targeting (Campaign).", "s": 2.08}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-08-12, Velocity spike: 341 Malicious Clawed Skills Found by the Bot They Were Targeting (+1 edges, z=9.9) (EmergentSignal) involves 341 Malicious Clawed Skills Found by the Bot They Were Targeting (Campaign).", "s": 0.57}, "cpu": {"ok": false, "why": "missing '9.9'", "pred": "Velocity spike: 341 Malicious Clawed Skills Found by the Bot They Were Targeting (+1 edges, z=7.4) on 2026-09-21.", "s": 12.71}}}, {"id": 86, "entity": "Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Compilers with Binary Ninja Workflows", "kind": "fact", "question": "Which Fortinet product is involved in the EKZ Stealer campaign?", "gold": "Fortinet EMS.", "expect": {"must": ["EMS"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Fortinet EMS – “In May 2026, threat actors exploited CVE-2026-35616, an improper access control vulnerability in Fortinet EMS versions 7.4.5 through 7.4.6, to deploy EKZ Stealer…”【Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Compilers with Binary Ninja Workflows (Campaign)】", "s": 1.64}, "30b": {"ok": true, "why": "ok", "pred": "Fortinet EMS", "s": 0.28}, "cpu": {"ok": true, "why": "ok", "pred": "an improper access control vulnerability in Fortinet EMS versions 7.4.5 through 7.4.6.", "s": 10.1}}}, {"id": 87, "entity": "Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Compilers with Binary Ninja Workflows", "kind": "date", "question": "When were threat actors exploiting CVE-2026-35616 according to the report?", "gold": "In May 2026.", "expect": {"must": ["May 2026"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "May 2026 – “In May 2026, threat actors exploited CVE‑2026‑35616, an improper access control vulnerability…” (first paragraph).", "s": 1.16}, "30b": {"ok": true, "why": "ok", "pred": "In May 2026, threat actors exploited CVE-2026-35616, an improper access control vulnerability in Fortinet EMS versions 7.4.5 through 7.4.6, to deploy EKZ Stealer within an energy sector organization.", "s": 0.43}, "cpu": {"ok": true, "why": "ok", "pred": "In May 2026, threat actors exploited CVE-2026-35616.", "s": 9.84}}}, {"id": 88, "entity": "Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Compilers with Binary Ninja Workflows", "kind": "fact", "question": "Which other CVEs are mentioned in this report besides CVE-2026-35616?", "gold": "CVE-2026-18577 and CVE-2026-16232.", "expect": {"must": ["CVE-2026-18577", "CVE-2026-16232"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVE-2026-18577 and CVE-2026-16232; On 2026-08-26, CVE-2026-18577 (Vulnerability) mentioned in Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Compilers with Binary Ninja Workflows (Campaign). On 2026-08-26, CVE-2026-16232 (Vulnerability) mentioned in Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Com…", "s": 3.11}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-18577, CVE-2026-16232", "s": 0.3}, "cpu": {"ok": true, "why": "ok", "pred": "CVE-2026-16232, CVE-2026-18577.", "s": 9.98}}}, {"id": 89, "entity": "Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Compilers with Binary Ninja Workflows", "kind": "fact", "question": "Which binary analysis tool is named in the report's title?", "gold": "Binary Ninja.", "expect": {"must": ["Binary Ninja"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Binary Ninja — “Fortinet Vulnerability CVE-2026-35616 and EKZ Stealer, Attacking Obfuscating Compilers with **Binary Ninja** Workflows (Campaign)” (first line).", "s": 1.1}, "30b": {"ok": true, "why": "ok", "pred": "Binary Ninja", "s": 0.14}, "cpu": {"ok": true, "why": "ok", "pred": "Attacking Obfuscating Compilers with Binary Ninja Workflows.", "s": 1.17}}}, {"id": 90, "entity": "Akira Ransomware Attack Investigation", "kind": "fact", "question": "Which tools are tagged for exfiltration and tunnelling in the Akira investigation?", "gold": "rclone (exfiltration) and gost (tunnel).", "expect": {"must": ["rclone", "gost"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Rclone and GOST tunnel – “The adversary deployed Rclone for data exfiltration purposes and utilized a GOST tunneling tool…” (first paragraph) and “tags=… rclone … gost tunnel …” (tags line).", "s": 1.68}, "30b": {"ok": true, "why": "ok", "pred": "rclone, gost tunnel", "s": 0.23}, "cpu": {"ok": true, "why": "ok", "pred": "Rclone for data exfiltration purposes and utilized a GOST tunneling tool to maintain persistence within the network.", "s": 7.03}}}, {"id": 91, "entity": "Akira Ransomware Attack Investigation", "kind": "fact", "question": "Which threat actor is linked to the Akira Ransomware Attack Investigation?", "gold": "Akira.", "expect": {"must": ["Akira"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. Akira (ThreatActor) — “On 2026-10-21, Akira (ThreatActor) mentioned in Akira Ransomware Attack Investigation (Campaign).”", "s": 1.18}, "30b": {"ok": true, "why": "ok", "pred": "Akira (ThreatActor) is linked to the Akira Ransomware Attack Investigation.", "s": 0.24}, "cpu": {"ok": false, "why": "missing 'Akira'", "pred": "The dossier does not say.", "s": 5.13}}}, {"id": 92, "entity": "Akira Ransomware Attack Investigation", "kind": "fact", "question": "What remote access protocol is tagged in the Akira investigation?", "gold": "RDP.", "expect": {"any": [["RDP", "Remote Desktop"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "rdp – tags=data exfiltration, antivirus evasion, rclone, ransomware, **rdp**, persistence, akira, gost tunnel, gost (tags line)", "s": 1.41}, "30b": {"ok": true, "why": "ok", "pred": "The dossier says: rdp", "s": 0.15}, "cpu": {"ok": true, "why": "ok", "pred": "Remote Desktop Protocol.", "s": 0.57}}}, {"id": 93, "entity": "Akira Ransomware Attack Investigation", "kind": "date", "question": "In which month did the attack described in the Akira investigation happen?", "gold": "September.", "expect": {"must": ["September"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "September — Akira Ransomware Attack Investigation: In September, an organization experienced an Akira ransomware attack, …【Akira Ransomware Attack Investigation (Campaign)】", "s": 1.01}, "30b": {"ok": true, "why": "ok", "pred": "September", "s": 0.2}, "cpu": {"ok": true, "why": "ok", "pred": "In September, an organization experienced an Akira ransomware attack.", "s": 6.35}}}, {"id": 94, "entity": "metaencryptor", "kind": "fact", "question": "Which countries does metaencryptor attack?", "gold": "Canada, United States, Japan and South Korea.", "expect": {"must": ["Canada", "Japan", "South Korea"], "any": [["United States", "US"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Canada, United States, Japan, South Korea — Metaencryptor (DwCrew) dw attacks Canada (DwCountry).; Metaencryptor (DwCrew) dw attacks United States (DwCountry).; Metaencryptor (DwCrew) dw attacks Japan (DwCountry).; Metaencryptor (DwCrew) dw attacks South Korea (DwCountry).", "s": 1.57}, "30b": {"ok": true, "why": "ok", "pred": "Metaencryptor (DwCrew) dw attacks Canada (DwCountry).\nMetaencryptor (DwCrew) dw attacks United States (DwCountry).\nMetaencryptor (DwCrew) dw attacks Japan (DwCountry).\nMetaencryptor (DwCrew) dw attacks South Korea (DwCountry).", "s": 0.73}, "cpu": {"ok": true, "why": "ok", "pred": "dw attacks Canada, dw attacks United States, dw attacks Japan, dw attacks South Korea.", "s": 8.25}}}, {"id": 95, "entity": "metaencryptor", "kind": "reasoning", "question": "Which healthcare victim did metaencryptor claim on 12 October 2026?", "gold": "MyMichigan Health.", "expect": {"must": ["MyMichigan"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "MyMichigan Health – On 2026‑10‑12, metaencryptor (RansomwareGroup, ransomware‑group) claimed victim MyMichigan Health (Organization).【On 2026-10-12, metaencryptor (RansomwareGroup, ransomware-group) claimed victim MyMichigan Health (Organization).】", "s": 1.88}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-12, metaencryptor (RansomwareGroup, ransomware-group) claimed victim MyMichigan Health (Organization).", "s": 0.67}, "cpu": {"ok": true, "why": "ok", "pred": "VIATechnik, MyMichigan Health.", "s": 10.62}}}, {"id": 96, "entity": "metaencryptor", "kind": "negative", "question": "Does metaencryptor attack Germany according to the dossier?", "gold": "No. Germany is not listed.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 1.11}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.21}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention Germany.", "s": 0.91}}}, {"id": 97, "entity": "interlock", "kind": "reasoning", "question": "Which educational institution did interlock claim on 21 October 2026?", "gold": "Coastal Alabama Community College.", "expect": {"must": ["Coastal Alabama"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Coastal Alabama Community College — “On 2026-10-21, interlock (RansomwareGroup, ransomware-group) claimed victim Coastal Alabama Community College (Organization).”", "s": 1.47}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-21, interlock (RansomwareGroup, ransomware-group) claimed victim Coastal Alabama Community College (Organization).", "s": 0.57}, "cpu": {"ok": true, "why": "ok", "pred": "Coastal Alabama Community College.", "s": 9.21}}}, {"id": 98, "entity": "interlock", "kind": "reasoning", "question": "Name two healthcare-related victims of interlock in the dossier.", "gold": "Any two of Concord Eye Center, O2 Dental Group, Riviera Healthcare Center.", "expect": {"min_distinct": {"pattern": "Concord Eye|O2 Dental|Riviera Healthcare", "n": 2}}, "results": {"120b": {"ok": false, "why": "fewer than 2 distinct Concord Eye|O2 Dental|Riviera Healthcare", "pred": "Riviera Healthcare Center (On 2026-10-07, interlock (RansomwareGroup, ransomware-group) claimed victim Riviera Healthcare Center (Organization).) and Clearview Eye Centre (On 2026-06-25, interlock (RansomwareGroup, ransomware-group) claimed victim Clearview Eye Centre (Organization).)", "s": 2.83}, "30b": {"ok": false, "why": "fewer than 2 distinct Concord Eye|O2 Dental|Riviera Healthcare", "pred": "On 2026-10-07, interlock (RansomwareGroup, ransomware-group) claimed victim Riviera Healthcare Center (Organization).  \nOn 2026-09-15, interlock (RansomwareGroup, ransomware-group) claimed victim Springfield Public Schools (Organization).", "s": 0.66}, "cpu": {"ok": false, "why": "fewer than 2 distinct Concord Eye|O2 Dental|Riviera Healthcare", "pred": "Coastal Alabama Community College, Concord Eye Center.", "s": 9.39}}}, {"id": 99, "entity": "interlock", "kind": "fact", "question": "Which countries does interlock attack?", "gold": "Canada and the United States.", "expect": {"must": ["Canada"], "any": [["United States", "US"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Canada; United States — On 2026-10-05, interlock (Crew) attacks Canada (Country).; On 2026-10-05, interlock (Crew) attacks United States (Country).", "s": 1.64}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-05, interlock (Crew) attacks Canada (Country).\nOn 2026-10-05, interlock (Crew) attacks United States (Country).\nInterlock (DwCrew) dw attacks Canada (DwCountry).\nInterlock (DwCrew) dw attacks United States (DwCountry).", "s": 0.71}, "cpu": {"ok": true, "why": "ok", "pred": "Canada, United States.", "s": 8.89}}}, {"id": 100, "entity": "interlock", "kind": "reasoning", "question": "Was O2 Dental Group claimed before or after Valenti Management?", "gold": "Before: O2 Dental Group on 2026-10-05, Valenti Management on 2026-10-12.", "expect": {"must": ["before"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Before – On 2026-10-05, interlock (RansomwareGroup, ransomware-group) claimed victim O2 Dental Group (Organization).  \nOn 2026-10-12, interlock (RansomwareGroup, ransomware-group) claimed victim Valenti Management (Organization).", "s": 2.16}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-05, interlock (RansomwareGroup, ransomware-group) claimed victim O2 Dental Group (Organization).\nOn 2026-10-12, interlock (RansomwareGroup, ransomware-group) claimed victim Valenti Management (Organization).\nO2 Dental Group was claimed before Valenti Management.", "s": 0.69}, "cpu": {"ok": false, "why": "missing 'before'", "pred": "On 2026-10-05, interlock (RansomwareGroup, ransomware-group) claimed victim O2 Dental Group (Organization).", "s": 10.68}}}, {"id": 101, "entity": "stormous", "kind": "fact", "question": "Which ATT&CK technique does the dossier record for stormous?", "gold": "Data Encrypted for Impact.", "expect": {"must": ["Data Encrypted for Impact"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Stormous (RansomwareGroup, ransomware-group) uses Data Encrypted for Impact (Technique).", "s": 1.0}, "30b": {"ok": true, "why": "ok", "pred": "Stormous (RansomwareGroup, ransomware-group) uses Data Encrypted for Impact (Technique).", "s": 0.4}, "cpu": {"ok": true, "why": "ok", "pred": "Data Encrypted for Impact.", "s": 9.23}}}, {"id": 102, "entity": "stormous", "kind": "date", "question": "Which victim did stormous claim on 2 June 2026?", "gold": "katholiekamersfoort.nl.", "expect": {"must": ["katholiekamersfoort.nl"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "katholiekamersfoort.nl (Organization) – from “On 2026-06-02, stormous (RansomwareGroup, ransomware-group) claimed victim katholiekamersfoort.nl (Organization).”", "s": 1.73}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-06-02, stormous (RansomwareGroup, ransomware-group) claimed victim katholiekamersfoort.nl (Organization).", "s": 0.44}, "cpu": {"ok": true, "why": "ok", "pred": "victim katholiekamersfoort.nl.", "s": 9.9}}}, {"id": 103, "entity": "stormous", "kind": "reasoning", "question": "Which Tunisian retailer appears among stormous's victims?", "gold": "monoprix.tn.", "expect": {"must": ["monoprix.tn"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "monoprix.tn — On 2026‑06‑28, stormous (RansomwareGroup, ransomware‑group) claimed victim monoprix.tn (Organization).", "s": 1.23}, "30b": {"ok": true, "why": "ok", "pred": "monoprix.tn (Organization).", "s": 0.17}, "cpu": {"ok": false, "why": "missing 'monoprix.tn'", "pred": "berjayaClubs.", "s": 0.82}}}, {"id": 104, "entity": "stormous", "kind": "yesno", "question": "Did stormous claim www.nasa.gov as a victim?", "gold": "Yes, on 2024-10-06.", "expect": {"must": ["nasa.gov"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. \"On 2024-10-06, stormous (RansomwareGroup, ransomware-group) claimed victim www.nasa.gov (Organization).\"", "s": 1.01}, "30b": {"ok": true, "why": "ok", "pred": "Yes. On 2024-10-06, stormous (RansomwareGroup, ransomware-group) claimed victim www.nasa.gov (Organization).", "s": 0.42}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. On 2024-10-06, stormous (RansomwareGroup, ransomware-group) claimed victim www.nasa.gov (Organization).", "s": 10.79}}}, {"id": 105, "entity": "revil", "kind": "fact", "question": "Which Norwegian victim did revil list, and how much data was advertised?", "gold": "inocean.no, 2000 GB.", "expect": {"must": ["inocean.no"], "any": [["2000 GB", "2000GB", "2,000 GB"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes – inocean.no / 2000 GB (Organization) 【On 2021-09-09, revil (RansomwareGroup, ransomware-group) claimed victim inocean.no / 2000 GB (Organization).】", "s": 1.74}, "30b": {"ok": true, "why": "ok", "pred": "On 2021-09-09, revil (RansomwareGroup, ransomware-group) claimed victim inocean.no / 2000 GB (Organization).", "s": 0.57}, "cpu": {"ok": true, "why": "ok", "pred": "inocean.no / 2000 GB.", "s": 9.65}}}, {"id": 106, "entity": "revil", "kind": "date", "question": "When did revil claim PTT Exploration and Production?", "gold": "2021-10-15.", "expect": {"any": [["2021-10-15", "15 October 2021", "October 15, 2021"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2021-10-15 (On 2021-10-15, revil (RansomwareGroup, ransomware-group) claimed victim PTT Exploration and Production - 720GB (Organization)).", "s": 1.55}, "30b": {"ok": true, "why": "ok", "pred": "On 2021-10-15, revil (RansomwareGroup, ransomware-group) claimed victim PTT Exploration and Production - 720GB (Organization).", "s": 0.72}, "cpu": {"ok": true, "why": "ok", "pred": "2021-10-15.", "s": 9.6}}}, {"id": 107, "entity": "revil", "kind": "reasoning", "question": "Was Unicity International claimed before or after Ronmor Holdings?", "gold": "After: Ronmor Holdings on 2021-10-01, Unicity International on 2022-05-03.", "expect": {"must": ["after"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "After; On 2021-10-01, revil (RansomwareGroup, ransomware-group) claimed victim Ronmor Holdings (Organization).; On 2022-05-03, revil (RansomwareGroup, ransomware-group) claimed victim Unicity International (Organization).", "s": 2.69}, "30b": {"ok": false, "why": "missing 'after'", "pred": "On 2021-05-03, revil (RansomwareGroup, ransomware-group) claimed victim Unicity International (Organization).\nOn 2021-10-01, revil (RansomwareGroup, ransomware-group) claimed victim Ronmor Holdings (Organization).\nUnicity International was claimed before Ronmor Holdings.", "s": 0.73}, "cpu": {"ok": false, "why": "missing 'after'", "pred": "Unicity International (Organization) On 2022-05-03, revil (RansomwareGroup, ransomware-group) claimed victim Unicity International (Organization).", "s": 11.35}}}, {"id": 108, "entity": "revil", "kind": "negative", "question": "Does the dossier say revil claimed the Vatican as a victim?", "gold": "No. The dossier does not mention the Vatican.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "No. The dossier does not say.", "s": 2.86}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.48}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention the Vatican.", "s": 9.14}}}, {"id": 109, "entity": "ragnarlocker", "kind": "fact", "question": "Which Belize utility did ragnarlocker leak?", "gold": "Belize Electricity Limited.", "expect": {"must": ["Belize Electricity"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Belize Electricity Limited – On 2023-07-10, ragnarlocker (RansomwareGroup, ransomware-group) claimed victim Belize Electricity Limited - Leaked (Organization).", "s": 1.12}, "30b": {"ok": true, "why": "ok", "pred": "Belize Electricity Limited - Leaked (Organization).", "s": 0.35}, "cpu": {"ok": true, "why": "ok", "pred": "victim Belize Electricity Limited - Leaked.", "s": 9.19}}}, {"id": 110, "entity": "ragnarlocker", "kind": "date", "question": "What did ragnarlocker post on 18 May 2023?", "gold": "Autlan Metallorum, Mexican Miner Leak.", "expect": {"must": ["Autlan"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "On 2023-05-18, ragnarlocker (RansomwareGroup, ransomware-group) claimed victim Autlan Metallorum, Mexican Miner Leak (Organization).【Line: On 2023-05-18, ragnarlocker (RansomwareGroup, ransomware-group) claimed victim Autlan Metallorum, Mexican Miner Leak (Organization).】", "s": 1.36}, "30b": {"ok": true, "why": "ok", "pred": "On 2023-05-18, ragnarlocker (RansomwareGroup, ransomware-group) claimed victim Autlan Metallorum, Mexican Miner Leak (Organization).", "s": 0.38}, "cpu": {"ok": true, "why": "ok", "pred": "claimed victim Autlan Metallorum, Mexican Miner Leak.", "s": 9.27}}}, {"id": 111, "entity": "ragnarlocker", "kind": "reasoning", "question": "Which ragnarlocker victim is a Portuguese transport company?", "gold": "Scotturb.", "expect": {"must": ["Scotturb"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Portugal Scotturb — On 2023-07-05, ragnarlocker (RansomwareGroup, ransomware-group) claimed victim **Portugal Scotturb Data Leaked (Organization).**", "s": 1.07}, "30b": {"ok": false, "why": "missing 'Scotturb'", "pred": "The dossier does not say.", "s": 0.27}, "cpu": {"ok": true, "why": "ok", "pred": "Portugal Scotturb Data Leaked.", "s": 8.94}}}, {"id": 112, "entity": "ragnarlocker", "kind": "negative", "question": "Does the dossier say ragnarlocker uses PowerShell?", "gold": "No. The dossier does not mention PowerShell.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.72}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.34}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention PowerShell.", "s": 9.1}}}, {"id": 113, "entity": "cactus", "kind": "fact", "question": "Which Swedish company did cactus claim, and what revenue figure was listed?", "gold": "assaabloy.com, $14.4B.", "expect": {"must": ["assaabloy"], "any": [["14.4B", "14.4 B", "14.4 billion"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "assaabloy.com $14.4B Sweden – On 2025‑03‑17, cactus (RansomwareGroup, ransomware-group) claimed victim assaabloy.com\\$14.4B\\Sweden\\229GB\\&lt;1% DISCLOSED (Organization).", "s": 1.62}, "30b": {"ok": true, "why": "ok", "pred": "On 2025-03-17, cactus (RansomwareGroup, ransomware-group) claimed victim assaabloy.com\\$14.4B\\Sweden\\229GB\\&lt;1% DISCLOSED (Organization).", "s": 0.57}, "cpu": {"ok": true, "why": "ok", "pred": "assaabloy.com\\$14.4B\\Sweden\\229GB\\&lt;1% DISCLOSED.", "s": 11.61}}}, {"id": 114, "entity": "cactus", "kind": "reasoning", "question": "Which cactus victims are in Canada?", "gold": "regulvar.com and northernresponse.com.", "expect": {"min_distinct": {"pattern": "regulvar|northernresponse", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "regulvar.com (On 2025-03-03, cactus (RansomwareGroup, ransomware-group) claimed victim regulvar.com\\$253.5M\\Canada\\3.6TB\\100% DISCLOSED (Organization)) and northernresponse.com (On 2025-02-17, cactus (RansomwareGroup, ransomware-group) claimed victim northernresponse.com\\$17.4M\\Canada\\366GB\\100% DISCLOSED (Organization)).", "s": 2.3}, "30b": {"ok": true, "why": "ok", "pred": "On 2025-02-17, cactus (RansomwareGroup, ransomware-group) claimed victim northernresponse.com\\$17.4M\\Canada\\366GB\\100% DISCLOSED (Organization).  \nOn 2025-02-24, cactus (RansomwareGroup, ransomware-group) claimed victim regulvar.com\\$253.5M\\Canada\\3.6TB\\<1% DISCLOSED (Organization).", "s": 0.67}, "cpu": {"ok": true, "why": "ok", "pred": "regulvar.com\\$253.5M\\Canada\\3.6TB\\100% DISCLOSED, northernresponse.com\\$17.4M\\Canada\\366GB\\100% DISCLOSED, regulvar.com\\$253.5M\\Canada\\3.6TB\\&lt;1% DISCLOSED, kyb.com\\$600.5M\\USA\\1.8TB\\&lt;1% DISCLOSED, kinseysinc.com\\$74.9M\\USA\\675GB\\&lt;1% DISCLOSED, bestbrands.com\\$25.5M\\USA\\659GB\\&lt;1% DISCLOSED, ssm", "s": 19.7}}}, {"id": 115, "entity": "cactus", "kind": "date", "question": "Name two victims cactus posted on 17 March 2025.", "gold": "kyb.com, tempel.com, assaabloy.com or thermoid.com.", "expect": {"min_distinct": {"pattern": "kyb\\.com|tempel\\.com|assaabloy|thermoid", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "kyb.com (On 2025-03-17, cactus (RansomwareGroup, ransomware-group) claimed victim kyb.com\\$600.5M\\USA\\1.8TB\\&lt;1% DISCLOSED (Organization).), tempel.com (On 2025-03-17, cactus (RansomwareGroup, ransomware-group) claimed victim tempel.com\\$628.7M\\USA\\111GB\\100% DISCLOSED (Organization).)", "s": 3.07}, "30b": {"ok": true, "why": "ok", "pred": "On 2025-03-17, cactus (RansomwareGroup, ransomware-group) claimed victim kyb.com\\$600.5M\\USA\\1.8TB\\&lt;1% DISCLOSED (Organization).  \nOn 2025-03-17, cactus (RansomwareGroup, ransomware-group) claimed victim tempel.com\\$628.7M\\USA\\111GB\\100% DISCLOSED (Organization).", "s": 0.68}, "cpu": {"ok": true, "why": "ok", "pred": "victim kyb.com\\$600.5M\\USA\\1.8TB\\&lt;1% DISCLOSED, victim thermoid.com\\$183.2M\\USA\\199GB\\100% DISCLOSED.", "s": 12.74}}}, {"id": 116, "entity": "cactus", "kind": "fact", "question": "Besides encrypting data, which techniques does cactus use?", "gold": "PowerShell, SMB/Windows Admin Shares, Exploit Public-Facing Application, Disable or Modify Tools.", "expect": {"min_distinct": {"pattern": "PowerShell|SMB/Windows Admin Shares|Exploit Public-Facing Application|Disable or Modify Tools", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Cactus (RansomwareGroup, ransomware-group) uses Exploit Public-Facing Application (Technique); Cactus (RansomwareGroup, ransomware-group) uses SMB/Windows Admin Shares (Technique); Cactus (RansomwareGroup, ransomware-group) uses Disable or Modify Tools (Technique); Cactus (RansomwareGroup, ransomware-group) uses PowerShell (Technique).", "s": 4.04}, "30b": {"ok": true, "why": "ok", "pred": "Cactus (RansomwareGroup, ransomware-group) uses Exploit Public-Facing Application (Technique), SMB/Windows Admin Shares (Technique), Disable or Modify Tools (Technique), and PowerShell (Technique).", "s": 0.5}, "cpu": {"ok": true, "why": "ok", "pred": "uses Data Encrypted for Impact, uses Exploit Public-Facing Application, uses SMB/Windows Admin Shares, uses Disable or Modify Tools, uses PowerShell.", "s": 11.19}}}, {"id": 117, "entity": "clop", "kind": "date", "question": "What is clop's most recent claim in the dossier, and on what date?", "gold": "LEGAL-RECORDS.US on 2026-10-21.", "expect": {"must": ["LEGAL-RECORDS.US"], "any": [["2026-10-21", "21 October 2026", "October 21, 2026"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Legal-Records.US on 2026‑10‑21 – On 2026-10-21, clop (RansomwareGroup, ransomware-group) claimed victim LEGAL-RECORDS.US (Organization).", "s": 2.96}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-21, clop (RansomwareGroup, ransomware-group) claimed victim LEGAL-RECORDS.US (Organization).", "s": 0.57}, "cpu": {"ok": true, "why": "ok", "pred": "On 2026-10-21, clop (Crew) claimed LEGAL-RECORDS.US (DarkVictim).", "s": 10.93}}}, {"id": 118, "entity": "clop", "kind": "reasoning", "question": "How many days passed between clop's claims of ZEBRA.COM and HARLEY-DAVIDSON.COM?", "gold": "27 days (2026-08-14 to 2026-09-10).", "expect": {"must": ["27"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "27 days; using “On 2026-08-14, clop (RansomwareGroup, ransomware-group) claimed victim ZEBRA.COM (Organization).” and “On 2026-09-10, clop (RansomwareGroup, ransomware-group) claimed victim HARLEY-DAVIDSON.COM (Organization).”", "s": 2.2}, "30b": {"ok": false, "why": "missing '27'", "pred": "31 days. On 2026-08-14, clop (RansomwareGroup, ransomware-group) claimed victim ZEBRA.COM (Organization). On 2026-09-10, clop (RansomwareGroup, ransomware-group) claimed victim HARLEY-DAVIDSON.COM (Organization).", "s": 0.83}, "cpu": {"ok": false, "why": "missing '27'", "pred": "6 days.", "s": 8.58}}}, {"id": 119, "entity": "clop", "kind": "fact", "question": "Which archiving technique does clop use?", "gold": "Archive via Utility.", "expect": {"must": ["Archive via Utility"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Archive via Utility – Clop (RansomwareGroup, ransomware-group) uses Archive via Utility (Technique).【Clop (RansomwareGroup, ransomware-group) uses Archive via Utility (Technique).】", "s": 1.29}, "30b": {"ok": true, "why": "ok", "pred": "Clop (RansomwareGroup, ransomware-group) uses Archive via Utility (Technique).", "s": 0.58}, "cpu": {"ok": true, "why": "ok", "pred": "Archive via Utility.", "s": 9.12}}}, {"id": 120, "entity": "clop", "kind": "yesno", "question": "Did clop claim SHELL.COM?", "gold": "Yes, on 2026-08-12.", "expect": {"must": ["SHELL.COM"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes – \"On 2026-08-12, clop (RansomwareGroup, ransomware-group) claimed victim SHELL.COM (August 2026) (Organization).\"", "s": 1.49}, "30b": {"ok": true, "why": "ok", "pred": "Yes. On 2026-08-12, clop (RansomwareGroup, ransomware-group) claimed victim SHELL.COM (August 2026) (Organization).", "s": 0.65}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. On 2026-08-12, clop (RansomwareGroup, ransomware-group) claimed victim SHELL.COM (August 2026) (Organization).", "s": 11.45}}}, {"id": 121, "entity": "snatch", "kind": "fact", "question": "Which clearnet domains does snatch operate?", "gold": "snatch.press, filesnatchcloud.top, sn76920193ch.top, dwhyj2.top.", "expect": {"min_distinct": {"pattern": "snatch\\.press|filesnatchcloud\\.top|dwhyj2\\.top|sn76920193ch\\.top", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "filesnatchcloud.top, snatch.press, sn76920193ch.top, dwhyj2.top — Snatch (RansomwareGroup, ransomware-group) operates site filesnatchcloud.top (Infrastructure).; Snatch (RansomwareGroup, ransomware-group) operates site snatch.press (Infrastructure).; Snatch (RansomwareGroup, ransomware-group) operates site sn76920193ch.top (Infrastructure).; Snatch (Ransomwa…", "s": 2.02}, "30b": {"ok": true, "why": "ok", "pred": "snatch (RansomwareGroup, ransomware-group) operates site filesnatchcloud.top (Infrastructure).  \nsnatch (RansomwareGroup, ransomware-group) operates site snatch.press (Infrastructure).  \nsnatch (RansomwareGroup, ransomware-group) operates site sn76920193ch.top (Infrastructure).  \nsnatch (RansomwareGroup, ransomware-group) operates site dwhyj2.top (Infrastruc…", "s": 0.68}, "cpu": {"ok": true, "why": "ok", "pred": "site filesnatchcloud.top, site snatch.press, site hl66646wtlp2naoqnhattngigjp5palgqmbwixepcjyq5i534acgqyad.onion, site sn76920193ch.top, site dwhyj2.top.", "s": 13.53}}}, {"id": 122, "entity": "snatch", "kind": "date", "question": "Which German museum did snatch claim on 30 November 2023?", "gold": "Museum für Naturkunde.", "expect": {"must": ["Naturkunde"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Museum für Naturkunde (Organization) – “On 2023-11-30, snatch (RansomwareGroup, ransomware-group) claimed victim Museum für Naturkunde (Organization).”", "s": 1.15}, "30b": {"ok": false, "why": "missing 'Naturkunde'", "pred": "The dossier does not say.", "s": 0.35}, "cpu": {"ok": true, "why": "ok", "pred": "Museum für Naturkunde.", "s": 9.38}}}, {"id": 123, "entity": "snatch", "kind": "reasoning", "question": "Which snatch victim is a Maldivian port operator?", "gold": "Maldives Ports Limited.", "expect": {"must": ["Maldives Ports"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Maldives Ports Limited – On 2023-11-30, snatch (RansomwareGroup, ransomware-group) claimed victim Maldives Ports Limited (Organization).", "s": 1.16}, "30b": {"ok": true, "why": "ok", "pred": "Maldives Ports Limited (Organization).", "s": 0.36}, "cpu": {"ok": true, "why": "ok", "pred": "victim Maldives Ports Limited.", "s": 9.4}}}, {"id": 124, "entity": "snatch", "kind": "yesno", "question": "Did snatch claim both Tyson Foods and Kraft Foods?", "gold": "Yes: Tyson Foods on 2023-11-30 and Kraft Foods on 2023-12-17.", "expect": {"must": ["Tyson", "Kraft"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. \"On 2023-12-17, snatch (RansomwareGroup, ransomware-group) claimed victim Kraft Foods (Organization).\" and \"On 2023-11-30, snatch (RansomwareGroup, ransomware-group) claimed victim Tyson Foods (Organization).\"", "s": 1.6}, "30b": {"ok": true, "why": "ok", "pred": "Yes. On 2023-11-30, snatch (RansomwareGroup, ransomware-group) claimed victim Tyson Foods (Organization). On 2023-12-17, snatch (RansomwareGroup, ransomware-group) claimed victim Kraft Foods (Organization).", "s": 0.54}, "cpu": {"ok": false, "why": "missing 'Kraft'", "pred": "Yes. On 2023-11-30, snatch (RansomwareGroup, ransomware-group) claimed victim Tyson Foods (Organization).", "s": 11.16}}}, {"id": 125, "entity": "CVE-2016-5195", "kind": "fact", "question": "What CVSS score does CVE-2016-5195 carry, and is it local or network exploitable?", "gold": "7.0, local (AV:L).", "expect": {"must": ["7.0"], "any": [["AV:L", "local"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVSS score 7.0; local exploitable – from line `cve_id=CVE-2016-5195; cvss_score=7.0; cvss_vector=CVSS:3.1/AV:L/AC:H/PR:L/UI:N/S:U/C:H/I:H/A:H; severity=HIGH; epss_score=0.83524; epss_percentile=0.99687; published_at=2016-11-10T21:59:00.197; vendorProject=Linux; product=Kernel; requiredAction=Apply updates per vendor instructions.; dateAdded=2022-03-03; first…", "s": 4.1}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2016-5195 carries a CVSS score of 7.0 and is local exploitable.", "s": 0.38}, "cpu": {"ok": true, "why": "ok", "pred": "cvss_score=7.0; cvss_vector=CVSS:3.1/AV:L/AC:H/PR:L/UI:N/S:U/C:H/I:H/A:H; severity=HIGH.", "s": 12.62}}}, {"id": 126, "entity": "CVE-2016-5195", "kind": "fact", "question": "What weakness class is CVE-2016-5195?", "gold": "Race condition (Concurrent Execution using Shared Resource with Improper Synchronization).", "expect": {"any": [["Race Condition", "Concurrent Execution"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition') (Weakness) – from “On 2016-11-10, CVE-2016-5195 (Vulnerability) has weakness Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition') (Weakness).”", "s": 1.63}, "30b": {"ok": true, "why": "ok", "pred": "Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition') (Weakness)", "s": 0.27}, "cpu": {"ok": true, "why": "ok", "pred": "weakness Concurrent Execution using Shared Resource with Improper Synchronization ('Race Condition').", "s": 9.86}}}, {"id": 127, "entity": "CVE-2016-5195", "kind": "date", "question": "When was CVE-2016-5195 added to the KEV catalogue?", "gold": "2022-03-03.", "expect": {"any": [["2022-03-03", "3 March 2022", "March 3, 2022"]]}, "results": {"120b": {"ok": false, "why": "none of ['2022-03-03', '3 March 2022', 'March 3, 2022']", "pred": "2026-05-13 – On 2026-05-13, Linux Kernel Race Condition Vulnerability (KEVEntry) describes CVE-2016-5195 (Vulnerability). (Line: “On 2026-05-13, Linux Kernel Race Condition Vulnerability (KEVEntry) describes CVE-2016-5195 (Vulnerability).”)", "s": 1.25}, "30b": {"ok": false, "why": "none of ['2022-03-03', '3 March 2022', 'March 3, 2022']", "pred": "The dossier does not say.", "s": 0.26}, "cpu": {"ok": true, "why": "ok", "pred": "2022-03-03.", "s": 9.54}}}, {"id": 128, "entity": "CVE-2016-5195", "kind": "fact", "question": "Which Palo Alto Networks product is affected by CVE-2016-5195?", "gold": "PAN-OS.", "expect": {"must": ["pan-os"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "paloaltonetworks pan-os — On 2016-11-10, CVE-2016-5195 (Vulnerability) affects product paloaltonetworks pan-os (Product).", "s": 1.21}, "30b": {"ok": true, "why": "ok", "pred": "On 2016-11-10, CVE-2016-5195 (Vulnerability) affects product paloaltonetworks pan-os (Product).", "s": 0.38}, "cpu": {"ok": true, "why": "ok", "pred": "product paloaltonetworks pan-os.", "s": 9.49}}}, {"id": 129, "entity": "CVE-2016-5195", "kind": "reasoning", "question": "Which Ubuntu releases does CVE-2016-5195 affect?", "gold": "12.04, 14.04, 16.04 and 16.10.", "expect": {"min_distinct": {"pattern": "12\\.04|14\\.04|16\\.04|16\\.10", "n": 3}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Ubuntu releases affected: 16.04, 16.10, 12.04, 14.04 — “product canonical ubuntu linux 16.04 (Product).”, “product canonical ubuntu linux 16.10 (Product).”, “product canonical ubuntu linux 12.04 (Product).”, “product canonical ubuntu linux 14.04 (Product).”.", "s": 1.54}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2016-5195 affects product canonical ubuntu linux 16.04 (Product).\nCVE-2016-5195 affects product canonical ubuntu linux 16.10 (Product).\nCVE-2016-5195 affects product canonical ubuntu linux 12.04 (Product).\nCVE-2016-5195 affects product canonical ubuntu linux 14.04 (Product).", "s": 0.58}, "cpu": {"ok": true, "why": "ok", "pred": "product canonical ubuntu linux 16.04, product canonical ubuntu linux 16.10, product canonical ubuntu linux 12.04, product canonical ubuntu linux 14.04.", "s": 11.69}}}, {"id": 130, "entity": "CVE-2026-45321", "kind": "fact", "question": "What did the malicious @tanstack packages exfiltrate?", "gold": "Cloud credentials, GitHub tokens and SSH keys.", "expect": {"any": [["cloud credentials", "GitHub tokens", "SSH keys"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "They exfiltrated cloud credentials, GitHub tokens, and SSH keys — “Malware in @tanstack/* packages exfiltrates cloud credentials, GitHub tokens, and SSH keys”.", "s": 1.27}, "30b": {"ok": true, "why": "ok", "pred": "Malware in @tanstack/* packages exfiltrates cloud credentials, GitHub tokens, and SSH keys", "s": 0.36}, "cpu": {"ok": true, "why": "ok", "pred": "cloud credentials, GitHub tokens, and SSH keys.", "s": 9.92}}}, {"id": 131, "entity": "CVE-2026-45321", "kind": "fact", "question": "What severity and CVSS score does CVE-2026-45321 carry?", "gold": "Critical, 9.6.", "expect": {"must": ["9.6", "critical"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CRITICAL, 9.6 – from line: cve_id=CVE-2026-45321; cvss_score=9.6; cvss_vector=CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H; severity=CRITICAL;", "s": 2.62}, "30b": {"ok": true, "why": "ok", "pred": "CRITICAL; cvss_score=9.6", "s": 0.31}, "cpu": {"ok": true, "why": "ok", "pred": "cvss_score=9.6; cvss_vector=CVSS:3.1/AV:N/AC:L/PR:N/UI:R/S:C/C:H/I:H/A:H; severity=CRITICAL.", "s": 11.12}}}, {"id": 132, "entity": "CVE-2026-45321", "kind": "fact", "question": "What weakness is recorded for CVE-2026-45321?", "gold": "Embedded Malicious Code.", "expect": {"must": ["Embedded Malicious Code"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Embedded Malicious Code (Weakness) — On 2026-05-12, CVE-2026-45321 (Vulnerability) has weakness Embedded Malicious Code (Weakness).", "s": 1.11}, "30b": {"ok": true, "why": "ok", "pred": "Embedded Malicious Code (Weakness)", "s": 0.12}, "cpu": {"ok": true, "why": "ok", "pred": "weakness Embedded Malicious Code.", "s": 8.55}}}, {"id": 133, "entity": "CVE-2026-45321", "kind": "date", "question": "When was CVE-2026-45321 added to KEV?", "gold": "2026-05-27.", "expect": {"any": [["2026-05-27", "27 May 2026", "May 27, 2026"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2026-05-27 — dateAdded=2026-05-27.", "s": 0.75}, "30b": {"ok": false, "why": "none of ['2026-05-27', '27 May 2026', 'May 27, 2026']", "pred": "The dossier does not say.", "s": 0.12}, "cpu": {"ok": true, "why": "ok", "pred": "2026-05-27.", "s": 8.96}}}, {"id": 134, "entity": "CVE-2026-47302", "kind": "fact", "question": "Which .NET versions are affected by CVE-2026-47302?", "gold": ".NET 8, .NET 9 and .NET 10.", "expect": {"min_distinct": {"pattern": "\\.NET ?(8|9|10)", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": ".NET 8, .NET 9, and .NET 10 – “A denial of service vulnerability exists in .NET 8, .NET 9, and .NET 10 related to XML processing.” (line containing that statement)", "s": 1.33}, "30b": {"ok": true, "why": "ok", "pred": ".NET 8, .NET 9, and .NET 10", "s": 0.35}, "cpu": {"ok": false, "why": "fewer than 2 distinct \\.NET ?(8|9|10)", "pred": ">= 10.0.0, <= 10.0.9.", "s": 10.92}}}, {"id": 135, "entity": "CVE-2026-47302", "kind": "fact", "question": "What weakness is CVE-2026-47302 classed as?", "gold": "CWE-770, Allocation of Resources Without Limits or Throttling.", "expect": {"any": [["CWE-770", "Allocation of Resources Without Limits"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Allocation of Resources Without Limits or Throttling (CWE-770) — CVE-2026-47302 (Vulnerability) has weakness Allocation of Resources Without Limits or Throttling (Weakness).【On 2026-07-20, CVE-2026-47302 (Vulnerability) has weakness Allocation of Resources Without Limits or Throttling (Weakness).】【- **Weakness:** CWE-770 (Allocation of Resources Without Limi…", "s": 1.72}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-47302: ## Executive summary\n- **Weakness:** CWE-770 (Allocation of Resources Without Limits or Throttling)", "s": 0.42}, "cpu": {"ok": true, "why": "ok", "pred": "weakness Allocation of Resources Without Limits or Throttling.", "s": 10.41}}}, {"id": 136, "entity": "CVE-2026-47302", "kind": "fact", "question": "What is the patched version of System.Security.Cryptography.Xml for .NET 10?", "gold": "10.0.10.", "expect": {"must": ["10.0.10"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Patched version is 10.0.10 — source line: System.Security.Cryptography.Xml | >= 10.0.0, <= 10.0.9 | 10.0.10.", "s": 1.35}, "30b": {"ok": true, "why": "ok", "pred": "The patched version of System.Security.Cryptography.Xml for .NET 10 is 10.0.10.", "s": 0.36}, "cpu": {"ok": true, "why": "ok", "pred": "10.0.10.", "s": 10.22}}}, {"id": 137, "entity": "CVE-2026-47302", "kind": "reasoning", "question": "Does CVE-2026-47302 threaten confidentiality or availability?", "gold": "Availability: it is a denial of service (A:H, C:N).", "expect": {"any": [["availability", "denial of service", "A:H"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes, it threatens availability (CVSS vector `CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H`; denial of service vulnerability) — cvss_vector=CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H; “A denial of service vulnerability exists in .NET 8, .NET 9, and .NET 10…”.", "s": 1.85}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-47302 threatens availability.", "s": 0.32}, "cpu": {"ok": false, "why": "none of ['availability', 'denial of service', 'A:H']", "pred": "The dossier does not say.", "s": 10.06}}}, {"id": 138, "entity": "CVE-2013-4590", "kind": "fact", "question": "What kind of flaw is CVE-2013-4590, and what does it expose?", "gold": "An XML External Entity (XXE) issue exposing Tomcat internals.", "expect": {"must": ["Tomcat internals"], "any": [["XXE", "XML External Entity"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Exposure of Sensitive Information to an Unauthorized Actor (Weakness) — it allows attackers to obtain “Tomcat internals” information【CVE-2013-4590 (Vulnerability) has weakness Exposure of Sensitive Information to an Unauthorized Actor (Weakness).】【CVE-2013-4590: Apache Tomcat before 6.0.39, 7.x before 7.0.50, and 8.x before 8.0.0-RC10 allows attackers to obt…", "s": 3.29}, "30b": {"ok": false, "why": "missing 'Tomcat internals'", "pred": "CVE-2013-4590 (Vulnerability) has weakness Exposure of Sensitive Information to an Unauthorized Actor (Weakness). CVE-2013-4590 (Vulnerability) affects product apache tomcat 5 (Product).", "s": 0.43}, "cpu": {"ok": false, "why": "none of ['XXE', 'XML External Entity']", "pred": "allows attackers to obtain \"Tomcat internals\" information.", "s": 8.9}}}, {"id": 139, "entity": "CVE-2013-4590", "kind": "fact", "question": "Which Tomcat versions fix CVE-2013-4590?", "gold": "6.0.39, 7.0.50 and 8.0.0-RC10.", "expect": {"any": [["6.0.39"]], "min_distinct": {"pattern": "6\\.0\\.39|7\\.0\\.50|8\\.0\\.0-RC10", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Tomcat 6.0.39 and later, 7.0.50 and later, and 8.0.0‑RC10 and later fix CVE‑2013‑4590 (see “CVE-2013-4590 (Vulnerability) Apache Tomcat before 6.0.39, 7.x before 7.0.50, and 8.x before 8.0.0‑RC10 …”).", "s": 2.53}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2013-4590 (Vulnerability) affects product apache tomcat 6.0.39 (Product).\nCVE-2013-4590 (Vulnerability) affects product apache tomcat 7.0.50 (Product).\nCVE-2013-4590 (Vulnerability) affects product apache tomcat 8.0.0-RC10 (Product).", "s": 0.53}, "cpu": {"ok": true, "why": "ok", "pred": "Apache Tomcat before 6.0.39, 7.x before 7.0.50, and 8.x before 8.0.0-RC10.", "s": 9.56}}}, {"id": 140, "entity": "CVE-2013-4590", "kind": "fact", "question": "What EPSS score does the dossier give CVE-2013-4590?", "gold": "0.09487.", "expect": {"must": ["0.09487"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "0.09487 – epss_score=0.09487", "s": 0.8}, "30b": {"ok": true, "why": "ok", "pred": "The dossier gives CVE-2013-4590 an EPSS score of 0.09487.", "s": 0.24}, "cpu": {"ok": true, "why": "ok", "pred": "EPSS 0.09487.", "s": 7.78}}}, {"id": 141, "entity": "CVE-2013-4590", "kind": "reasoning", "question": "Besides Apache Tomcat, which vendor's product does CVE-2013-4590 affect?", "gold": "Oracle Solaris 11.2.", "expect": {"must": ["solaris"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "oracle solaris 11.2 – CVE-2013-4590 (Vulnerability) affects product oracle solaris 11.2 (Product).", "s": 1.72}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2013-4590 (Vulnerability) affects product oracle solaris 11.2 (Product).", "s": 0.32}, "cpu": {"ok": true, "why": "ok", "pred": "product oracle solaris 11.2.", "s": 8.25}}}, {"id": 142, "entity": "CVE-2026-25985", "kind": "fact", "question": "How much memory does the crafted SVG make ImageMagick try to allocate?", "gold": "About 674 GB.", "expect": {"must": ["674"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "~674 GB of memory — “CVE-2026-25985: A crafted SVG file containing an malicious element causes ImageMagick to attempt to allocate ~674 GB of memory, leading to an out-of-memory abort.” (first line)", "s": 0.99}, "30b": {"ok": true, "why": "ok", "pred": "~674 GB of memory", "s": 0.25}, "cpu": {"ok": true, "why": "ok", "pred": "causes ImageMagick to attempt to allocate ~674 GB of memory.", "s": 7.69}}}, {"id": 143, "entity": "CVE-2026-25985", "kind": "fact", "question": "How was CVE-2026-25985 discovered?", "gold": "AFL++ fuzzing with afl-clang-lto instrumentation and AddressSanitizer.", "expect": {"any": [["AFL++", "fuzzing"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Found via AFL++ fuzzing with afl-clang-lto instrumentation and AddressSanitizer. (“Found via AFL++ fuzzing with afl-clang-lto instrumentation and AddressSanitizer.”)", "s": 1.35}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2026-25985 was discovered via AFL++ fuzzing with afl-clang-lto instrumentation and AddressSanitizer.", "s": 0.32}, "cpu": {"ok": true, "why": "ok", "pred": "Found via AFL++ fuzzing with afl-clang-lto instrumentation and AddressSanitizer.", "s": 7.81}}}, {"id": 144, "entity": "CVE-2026-25985", "kind": "fact", "question": "Which two weaknesses are recorded for CVE-2026-25985?", "gold": "Allocation of Resources Without Limits or Throttling; Memory Allocation with Excessive Size Value.", "expect": {"any": [["Allocation of Resources Without Limits"]], "min_distinct": {"pattern": "Allocation of Resources Without Limits|Memory Allocation with Excessive Size", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Allocation of Resources Without Limits or Throttling; Memory Allocation with Excessive Size Value — used lines: “On 2026-02-24, CVE-2026-25985 (Vulnerability) has weakness Allocation of Resources Without Limits or Throttling (Weakness).” and “On 2026-02-24, CVE-2026-25985 (Vulnerability) has weakness Memory Allocation with Excessive Size Value (Weakness).”", "s": 2.18}, "30b": {"ok": true, "why": "ok", "pred": "Allocation of Resources Without Limits or Throttling (Weakness), Memory Allocation with Excessive Size Value (Weakness)", "s": 0.17}, "cpu": {"ok": true, "why": "ok", "pred": "weakness Allocation of Resources Without Limits or Throttling, weakness Memory Allocation with Excessive Size Value.", "s": 1.64}}}, {"id": 145, "entity": "CVE-2026-25985", "kind": "date", "question": "When did the velocity spike for CVE-2026-25985 occur?", "gold": "2026-03-29.", "expect": {"any": [["2026-03-29", "29 March 2026", "March 29, 2026"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2026-03-29 — On 2026-03-29, Velocity spike: CVE-2026-25985 (+19 edges, z=7.8) (EmergentSignal) involves CVE-2026-25985 (Vulnerability).", "s": 1.81}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-03-29, Velocity spike: CVE-2026-25985 (+19 edges, z=7.8) (EmergentSignal) involves CVE-2026-25985 (Vulnerability).", "s": 0.41}, "cpu": {"ok": true, "why": "ok", "pred": "Velocity spike: CVE-2026-25985 (+19 edges, z=7.8) on 2026-03-29.", "s": 8.52}}}, {"id": 146, "entity": "CVE-2021-26872", "kind": "fact", "question": "Which Windows component does CVE-2021-26872 concern?", "gold": "Windows Event Tracing.", "expect": {"must": ["Event Tracing"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Windows Event Tracing (CVE-2021-26872: Windows Event Tracing Elevation of Privilege Vulnerability)", "s": 1.22}, "30b": {"ok": true, "why": "ok", "pred": "CVE-2021-26872: Windows Event Tracing Elevation of Privilege Vulnerability", "s": 0.26}, "cpu": {"ok": true, "why": "ok", "pred": "Windows Event Tracing Elevation of Privilege Vulnerability.", "s": 4.96}}}, {"id": 147, "entity": "CVE-2021-26872", "kind": "fact", "question": "What CVSS score and attack vector does CVE-2021-26872 carry?", "gold": "7.8, local (AV:L).", "expect": {"must": ["7.8"], "any": [["AV:L", "local"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVSS score=7.8; attack vector=AV:L (cve_id=CVE-2021-26872; cvss_score=7.8; cvss_vector=CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H; severity=HIGH; epss_score=0.00747; epss_percentile=0.54416; published_at=2021-03-11T16:15:14.863)", "s": 2.5}, "30b": {"ok": true, "why": "ok", "pred": "CVSS score=7.8; cvss_vector=CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H", "s": 0.2}, "cpu": {"ok": true, "why": "ok", "pred": "cvss_score=7.8; cvss_vector=CVSS:3.1/AV:L/AC:L/PR:L/UI:N/S:U/C:H/I:H/A:H.", "s": 6.09}}}, {"id": 148, "entity": "CVE-2021-26872", "kind": "negative", "question": "Does the dossier list a KEV entry for CVE-2021-26872?", "gold": "No. The dossier does not mention a KEV entry.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.39}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.11}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention KEV.", "s": 4.34}}}, {"id": 149, "entity": "CVE-2021-26872", "kind": "yesno", "question": "Does CVE-2021-26872 affect Windows 7?", "gold": "Yes.", "expect": {"must": ["windows 7"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. \"CVE-2021-26872 (Vulnerability) affects product microsoft windows 7 (Product).\"", "s": 0.74}, "30b": {"ok": true, "why": "ok", "pred": "Yes. CVE-2021-26872 (Vulnerability) affects product microsoft windows 7 (Product).", "s": 0.24}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. CVE-2021-26872 (Vulnerability) affects product microsoft windows 7 (Product).", "s": 5.35}}}, {"id": 150, "entity": "Pikabot", "kind": "fact", "question": "Since when has Pikabot been active, and what is it used for?", "gold": "Early 2023; initial access and follow-on tool deployment.", "expect": {"must": ["2023"], "any": [["initial access", "backdoor"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Since early 2023, Pikabot is a backdoor used for initial access and follow‑on tool deployment, frequently deploying follow‑on tools such as Cobalt Strike or ransomware variants. 【“Pikabot is a backdoor used for initial access and follow-on tool deployment active since early 2023.”】【“Pikabot is frequently used to deploy follow on tools such as Cobalt Strike o…", "s": 2.02}, "30b": {"ok": true, "why": "ok", "pred": "Pikabot has been active since early 2023 and is used for initial access and follow-on tool deployment.", "s": 0.33}, "cpu": {"ok": false, "why": "none of ['initial access', 'backdoor']", "pred": "active since early 2023.", "s": 9.45}}}, {"id": 151, "entity": "Pikabot", "kind": "fact", "question": "Which malware family does Pikabot overlap with?", "gold": "QakBot.", "expect": {"must": ["QakBot"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "QakBot – “Pikabot has some overlaps with QakBot, but insufficient evidence exists to definitively link these two malware families.” (first line)", "s": 1.24}, "30b": {"ok": true, "why": "ok", "pred": "Pikabot has some overlaps with QakBot, but insufficient evidence exists to definitively link these two malware families.", "s": 0.34}, "cpu": {"ok": false, "why": "missing 'QakBot'", "pred": "insufficient evidence exists to definitively link these two malware families.", "s": 9.54}}}, {"id": 152, "entity": "Pikabot", "kind": "fact", "question": "Which threat actor uses Pikabot?", "gold": "TA577.", "expect": {"must": ["TA577"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "TA577 (ThreatActor), TA577 (EcoActor), TA577 (DwActor) – “On 2024-09-17, TA577 (ThreatActor) uses Pikabot (Software, malware).” “On 2026-10-24, TA577 (EcoActor) uses Pikabot (Malware).” “TA577 (DwActor) dw uses Pikabot (DwMalware).”", "s": 2.44}, "30b": {"ok": true, "why": "ok", "pred": "TA577 (ThreatActor) uses Pikabot (Software, malware).\nTA577 (DwActor) dw uses Pikabot (DwMalware).", "s": 0.39}, "cpu": {"ok": true, "why": "ok", "pred": "TA577.", "s": 9.26}}}, {"id": 153, "entity": "Pikabot", "kind": "fact", "question": "Name two campaigns that distributed Pikabot.", "gold": "Pikabot Distribution February 2024 and Water Curupira Pikabot Distribution.", "expect": {"any": [["Water Curupira"]], "min_distinct": {"pattern": "Water Curupira|February 2024", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Pikabot Distribution February 2024 (Campaign) and Water Curupira Pikabot Distribution (Campaign) – lines “On 2024‑07‑17, Pikabot Distribution February 2024 (Campaign) uses Pikabot (Software, malware).” and “On 2024‑07‑17, Water Curupira Pikabot Distribution (Campaign) uses Pikabot (Software, malware).”", "s": 2.02}, "30b": {"ok": true, "why": "ok", "pred": "On 2024-07-17, Pikabot Distribution February 2024 (Campaign) uses Pikabot (Software, malware).  \nOn 2024-07-17, Water Curupira Pikabot Distribution (Campaign) uses Pikabot (Software, malware).", "s": 0.49}, "cpu": {"ok": true, "why": "ok", "pred": "Pikabot Distribution February 2024, Water Curupira Pikabot Distribution.", "s": 10.05}}}, {"id": 154, "entity": "Pikabot", "kind": "negative", "question": "Does the dossier say Pikabot implements Keylogging?", "gold": "No. The dossier does not mention Keylogging.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "No. The dossier does not say.", "s": 2.01}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.3}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention Keylogging.", "s": 9.27}}}, {"id": 155, "entity": "Mispadu", "kind": "fact", "question": "What language is Mispadu written in, and who operates it?", "gold": "Delphi; the Malteiro group.", "expect": {"must": ["Delphi", "Malteiro"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Delphi; operated by the Malteiro cybercriminal group. [Mispadu: Mispadu is a banking trojan written in Delphi that was first observed in 2019 and uses a Malware-as-a-Service (MaaS) business model. This malware is operated, managed, and sold by the Malteiro cybercriminal group.]", "s": 1.59}, "30b": {"ok": true, "why": "ok", "pred": "Mispadu is written in Delphi and is operated by the Malteiro cybercriminal group.", "s": 0.34}, "cpu": {"ok": false, "why": "missing 'Malteiro'", "pred": "a banking trojan written in Delphi that was first observed in 2019.", "s": 9.79}}}, {"id": 156, "entity": "Mispadu", "kind": "fact", "question": "Which countries has Mispadu mainly targeted?", "gold": "Brazil and Mexico.", "expect": {"must": ["Brazil", "Mexico"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Brazil and Mexico — “Mispadu has mainly been used to target victims in Brazil and Mexico…” (first description line).", "s": 1.75}, "30b": {"ok": true, "why": "ok", "pred": "Mispadu has mainly been used to target victims in Brazil and Mexico, and has also had confirmed operations throughout Latin America and Europe.", "s": 0.33}, "cpu": {"ok": true, "why": "ok", "pred": "Brazil and Mexico.", "s": 8.18}}}, {"id": 157, "entity": "Mispadu", "kind": "date", "question": "When was Process Injection recorded for Mispadu?", "gold": "2024-04-18.", "expect": {"any": [["2024-04-18", "18 April 2024", "April 18, 2024"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2024-04-18 – “On 2024-04-18, Mispadu (Software, malware) implements Process Injection (Technique).”", "s": 1.5}, "30b": {"ok": true, "why": "ok", "pred": "On 2024-04-18, Mispadu (Software, malware) implements Process Injection (Technique).", "s": 0.34}, "cpu": {"ok": true, "why": "ok", "pred": "2024-04-18.", "s": 9.26}}}, {"id": 158, "entity": "Mispadu", "kind": "yesno", "question": "Does Mispadu implement Keylogging?", "gold": "Yes.", "expect": {"must": ["Keylogging"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. On 2024-03-15, Mispadu (Software, malware) implements Keylogging (Technique).", "s": 0.92}, "30b": {"ok": true, "why": "ok", "pred": "Yes. On 2024-03-15, Mispadu (Software, malware) implements Keylogging (Technique).", "s": 0.32}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. On 2024-03-15, Mispadu (Software, malware) implements Keylogging (Technique).", "s": 9.92}}}, {"id": 159, "entity": "BOLDMOVE", "kind": "fact", "question": "Which zero-day is BOLDMOVE linked to?", "gold": "CVE-2022-42475 in FortiOS SSL-VPN.", "expect": {"must": ["CVE-2022-42475"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "CVE‑2022‑42475 – “BOLDMOVE is linked to zero‑day exploitation of CVE‑2022‑42475 in FortiOSS SSL‑VPNs.” (line 1)", "s": 1.51}, "30b": {"ok": true, "why": "ok", "pred": "BOLDMOVE is linked to zero-day exploitation of CVE-2022-42475 in FortiOSS SSL-VPNs.", "s": 0.37}, "cpu": {"ok": true, "why": "ok", "pred": "zero-day exploitation of CVE-2022-42475 in FortiOSS SSL-VPNs.", "s": 11.0}}}, {"id": 160, "entity": "BOLDMOVE", "kind": "fact", "question": "Which devices are BOLDMOVE's Linux variants designed for?", "gold": "FortiGate firewall devices.", "expect": {"must": ["FortiGate"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "FortiGate Firewall devices — “BOLDMOVE includes both Windows and Linux variants, with some Linux variants specifically designed for FortiGate Firewall devices.”【BOLDMOVE: BOLDMOVE is a type of backdoor malware written in C linked to People’s Republic of China operations from 2022 through 2023. BOLDMOVE includes both Windows and Linux variants, with some Linu…", "s": 1.04}, "30b": {"ok": true, "why": "ok", "pred": "BOLDMOVE's Linux variants are specifically designed for FortiGate Firewall devices.", "s": 0.14}, "cpu": {"ok": true, "why": "ok", "pred": "specifically designed for FortiGate Firewall devices.", "s": 0.9}}}, {"id": 161, "entity": "BOLDMOVE", "kind": "fact", "question": "Which country's operations is BOLDMOVE linked to?", "gold": "The People's Republic of China.", "expect": {"must": ["China"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "People’s Republic of China — “BOLDMOVE is a type of backdoor malware written in C linked to People’s Republic of China operations from 2022 through 2023.”", "s": 0.93}, "30b": {"ok": true, "why": "ok", "pred": "BOLDMOVE is linked to People’s Republic of China operations from 2022 through 2023.", "s": 0.17}, "cpu": {"ok": false, "why": "missing 'China'", "pred": "zero-day exploitation of CVE-2022-42475 in FortiOSS SSL-VPNs.", "s": 2.0}}}, {"id": 162, "entity": "NBTscan", "kind": "fact", "question": "What is NBTscan used for?", "gold": "Internal reconnaissance within a compromised network.", "expect": {"any": [["internal reconnaissance", "reconnaissance"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "NBTscan: NBTscan is an open source tool that has been used by state groups to conduct internal reconnaissance within a compromised network. 【NBTscan (Software)】", "s": 1.34}, "30b": {"ok": false, "why": "none of ['internal reconnaissance', 'reconnaissance']", "pred": "NBTscan is used for System Owner/User Discovery, System Network Configuration Discovery, Network Sniffing, Network Service Discovery, and Remote System Discovery.", "s": 0.39}, "cpu": {"ok": true, "why": "ok", "pred": "an open source tool that has been used by state groups to conduct internal reconnaissance within a compromised network.", "s": 10.33}}}, {"id": 163, "entity": "NBTscan", "kind": "reasoning", "question": "Name three threat actors that use NBTscan.", "gold": "APT39, Agrius, BackdoorDiplomacy, Lotus Blossom, Tonto Team, GALLIUM, Mustang Panda, Earth Lusca, Threat Group-3390, Turla.", "expect": {"min_distinct": {"pattern": "APT39|Agrius|BackdoorDiplomacy|Lotus Blossom|Tonto Team|GALLIUM|Mustang Panda|Earth Lusca|Threat Group-3390|Turla", "n": 3}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "APT39, Agrius, BackdoorDiplomacy; On 2021-03-17, APT39 (ThreatActor) uses NBTscan (Software, tool). On 2024-05-22, Agrius (ThreatActor) uses NBTscan (Software, tool). On 2021-09-21, BackdoorDiplomacy (ThreatActor) uses NBTscan (Software, tool).", "s": 2.38}, "30b": {"ok": true, "why": "ok", "pred": "APT39 (ThreatActor), Agrius (ThreatActor), BackdoorDiplomacy (ThreatActor).", "s": 0.38}, "cpu": {"ok": true, "why": "ok", "pred": "APT39, Agrius, BackdoorDiplomacy.", "s": 10.18}}}, {"id": 164, "entity": "NBTscan", "kind": "date", "question": "When was Lotus Blossom recorded using NBTscan?", "gold": "2025-03-15.", "expect": {"any": [["2025-03-15", "15 March 2025", "March 15, 2025"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2025‑03‑15 (“On 2025-03-15, Lotus Blossom (ThreatActor) uses NBTscan (Software, tool).”) and 2026‑10‑24 (“On 2026-10-24, Lotus Blossom (EcoActor) uses NBTscan (Malware).”)", "s": 1.8}, "30b": {"ok": true, "why": "ok", "pred": "On 2025-03-15, Lotus Blossom (ThreatActor) uses NBTscan (Software, tool).", "s": 0.4}, "cpu": {"ok": true, "why": "ok", "pred": "2025-03-15: On 2025-03-15, Lotus Blossom (ThreatActor) uses NBTscan (Software, tool).", "s": 11.71}}}, {"id": 165, "entity": "Rclone", "kind": "fact", "question": "Which ransomware operations has Rclone been associated with?", "gold": "Conti and DarkSide.", "expect": {"must": ["Conti", "DarkSide"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Conti and DarkSide (Rclone has been used in a number of ransomware campaigns, including those associated with the Conti and DarkSide Ransomware-as-a-Service operations.)【Rclone has been used in a number of ransomware campaigns, including those associated with the Conti and DarkSide Ransomware-as-a-Service operations.】", "s": 1.18}, "30b": {"ok": true, "why": "ok", "pred": "Rclone has been used in a number of ransomware campaigns, including those associated with the Conti and DarkSide Ransomware-as-a-Service operations.", "s": 0.31}, "cpu": {"ok": true, "why": "ok", "pred": "those associated with the Conti and DarkSide Ransomware-as-a-Service operations.", "s": 6.12}}}, {"id": 166, "entity": "Rclone", "kind": "reasoning", "question": "Which exfiltration techniques does Rclone implement?", "gold": "Exfiltration Over Unencrypted Non-C2 Protocol, Exfiltration Over Asymmetric Encrypted Non-C2 Protocol, Exfiltration to Cloud Storage.", "expect": {"min_distinct": {"pattern": "Exfiltration Over Unencrypted Non-C2 Protocol|Exfiltration Over Asymmetric Encrypted Non-C2 Protocol|Exfiltration to Cloud Storage", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Exfiltration Over Unencrypted Non‑C2 Protocol; Exfiltration Over Asymmetric Encrypted Non‑C2 Protocol; Exfiltration to Cloud Storage — On 2022‑09‑30, Rclone (Software, tool) implements Exfiltration Over Unencrypted Non‑C2 Protocol (Technique). On 2022‑08‑30, Rclone (Software, tool) implements Exfiltration Over Asymmetric Encrypted Non‑C2 Protocol (Technique)…", "s": 2.81}, "30b": {"ok": true, "why": "ok", "pred": "Rclone (Software, tool) implements Exfiltration Over Unencrypted Non-C2 Protocol (Technique).  \nRclone (Software, tool) implements Exfiltration Over Asymmetric Encrypted Non-C2 Protocol (Technique).  \nRclone (Software, tool) implements Exfiltration to Cloud Storage (Technique).", "s": 0.4}, "cpu": {"ok": true, "why": "ok", "pred": "Exfiltration Over Unencrypted Non-C2 Protocol, Exfiltration Over Asymmetric Encrypted Non-C2 Protocol, Exfiltration to Cloud Storage, Archive via Utility, Data Transfer Size Limits, File and Directory Discovery.", "s": 7.71}}}, {"id": 167, "entity": "Rclone", "kind": "date", "question": "When was Scattered Spider recorded using Rclone?", "gold": "2025-10-14.", "expect": {"any": [["2025-10-14", "14 October 2025", "October 14, 2025"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2025-10-14 (On 2025-10-14, Scattered Spider (ThreatActor) uses Rclone (Software, tool).); 2026-10-24 (On 2026-10-24, Scattered Spider (EcoActor) uses tool Rclone (Tool).)", "s": 1.74}, "30b": {"ok": true, "why": "ok", "pred": "On 2025-10-14, Scattered Spider (ThreatActor) uses Rclone (Software, tool).", "s": 0.26}, "cpu": {"ok": true, "why": "ok", "pred": "2025-10-14: On 2025-10-14, Scattered Spider (ThreatActor) uses Rclone (Software, tool).", "s": 7.12}}}, {"id": 168, "entity": "Rclone", "kind": "negative", "question": "Does the dossier say LockBit uses Rclone?", "gold": "No. The dossier does not mention LockBit.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.78}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.41}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention LockBit.", "s": 5.22}}}, {"id": 169, "entity": "Kirkland & Ellis LLP", "kind": "fact", "question": "Which group claimed Kirkland & Ellis LLP, and in which sector is it recorded?", "gold": "SilentRansomGroup; Professional Services.", "expect": {"must": ["SilentRansomGroup", "Professional Services"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "SilentRansomGroup claimed Kirkland & Ellis LLP (RansomwareVictim) — recorded in sector Professional Services. 【Kirkland & Ellis LLP (RansomwareVictim) claimed by SilentRansomGroup (RansomwareGroup).】【Kirkland & Ellis LLP (RansomwareVictim) in sector Professional Services (Sector).】", "s": 1.78}, "30b": {"ok": true, "why": "ok", "pred": "SilentRansomGroup claimed Kirkland & Ellis LLP, and it is recorded in the sector Professional Services.", "s": 0.25}, "cpu": {"ok": false, "why": "missing 'SilentRansomGroup'", "pred": "RansomwareVictim.", "s": 2.16}}}, {"id": 170, "entity": "Kirkland & Ellis LLP", "kind": "fact", "question": "Where is Kirkland & Ellis headquartered?", "gold": "Chicago, Illinois.", "expect": {"must": ["Chicago"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Chicago, Illinois, United States — “Kirkland & Ellis LLP: [AI generated] Kirkland & Ellis LLP is a global law firm headquartered in Chicago, Illinois, United States.”", "s": 1.29}, "30b": {"ok": true, "why": "ok", "pred": "Kirkland & Ellis LLP is a global law firm headquartered in Chicago, Illinois, United States.", "s": 0.21}, "cpu": {"ok": true, "why": "ok", "pred": "a global law firm headquartered in Chicago, Illinois, United States.", "s": 2.45}}}, {"id": 171, "entity": "Kirkland & Ellis LLP", "kind": "date", "question": "When was the Kirkland & Ellis claim first seen?", "gold": "2026-10-22.", "expect": {"any": [["2026-10-22", "22 October 2026", "October 22, 2026"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2026-10-22T16:58:16+00:00; first_seen=2026-10-22T16:58:16+00:00.", "s": 1.19}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-22 Kirkland & Ellis LLP (RansomwareVictim) was seen.", "s": 0.24}, "cpu": {"ok": false, "why": "none of ['2026-10-22', '22 October 2026', 'October 22, 2026']", "pred": "2026-10-05: On 2026-10-05, leakeddata (Crew) claimed Kirkland & Ellis LLP (DarkVictim).", "s": 4.46}}}, {"id": 172, "entity": "All Smiles Dental of Falls Church", "kind": "fact", "question": "Which group claimed All Smiles Dental of Falls Church?", "gold": "thegentlemen.", "expect": {"any": [["thegentlemen", "the gentlemen"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "thegentlemen — All Smiles Dental of Falls Church (RansomwareVictim) claimed by thegentlemen (RansomwareGroup).【All Smiles Dental of Falls Church (RansomwareVictim) claimed by thegentlemen (RansomwareGroup).】", "s": 1.16}, "30b": {"ok": true, "why": "ok", "pred": "The gentlemen (RansomwareGroup).", "s": 0.19}, "cpu": {"ok": true, "why": "ok", "pred": "the gentlemen.", "s": 2.4}}}, {"id": 173, "entity": "All Smiles Dental of Falls Church", "kind": "fact", "question": "Which two dentists founded All Smiles Dental?", "gold": "Dr. Thang Pham and Dr. Nu Dang.", "expect": {"must": ["Thang Pham", "Nu Dang"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Dr. Thang Pham and Dr. Nu Dang — from the line “founded in the early 1990s by Vietnamese‑immigrant dentists Dr. Thang Pham and Dr. Nu Dang, who met at Saigon Dental School and retrained cum laude at MCV/VCU.”", "s": 1.95}, "30b": {"ok": true, "why": "ok", "pred": "Dr. Thang Pham and Dr. Nu Dang", "s": 0.21}, "cpu": {"ok": true, "why": "ok", "pred": "Vietnamese-immigrant dentists Dr. Thang Pham and Dr. Nu Dang.", "s": 3.23}}}, {"id": 174, "entity": "All Smiles Dental of Falls Church", "kind": "fact", "question": "What is the practice's flagship clinical focus?", "gold": "Biomimetic dentistry.", "expect": {"must": ["biomimetic"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "biomimetic dentistry — “The practice offers full-service dentistry with a flagship focus on biomimetic dentistry — Dr. Long Pham is one of the few US dentists with mastership from the Alleman Center for Biomimetic Dentistry — plus CEREC same-day crowns, Invisalign/SureSmile aligners”. (Dossier line)", "s": 1.69}, "30b": {"ok": true, "why": "ok", "pred": "The practice's flagship clinical focus is biomimetic dentistry.", "s": 0.18}, "cpu": {"ok": true, "why": "ok", "pred": "biomimetic dentistry.", "s": 1.34}}}, {"id": 175, "entity": "All Smiles Dental of Falls Church", "kind": "fact", "question": "Under which sector and country is All Smiles Dental recorded?", "gold": "Healthcare, US.", "expect": {"must": ["Healthcare", "US"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Healthcare sector, US country — using “All Smiles Dental of Falls Church (RansomwareVictim) in sector Healthcare (Sector).” and “All Smiles Dental of Falls Church (RansomwareVictim) in country US (Region).”", "s": 1.4}, "30b": {"ok": true, "why": "ok", "pred": "All Smiles Dental of Falls Church (RansomwareVictim) in country US (Region).  \nAll Smiles Dental of Falls Church (RansomwareVictim) in sector Healthcare (Sector).", "s": 0.29}, "cpu": {"ok": true, "why": "ok", "pred": "sector=Healthcare; country=US.", "s": 2.77}}}, {"id": 176, "entity": "Husky Ltee", "kind": "fact", "question": "What does Husky Ltee sell, and where is it based?", "gold": "Workwear and safety footwear; Quebec (Joliette / Saint-Paul).", "expect": {"any": [["footwear", "work boots", "workwear"]], "min_distinct": {"pattern": "Quebec|Joliette|Saint-Paul", "n": 1}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "It sells work boots, safety apparel, PPE and custom embroideries, prints and promotional products; it is a Quebec‑based company founded in Joliette with its Saint‑Paul headquarters – “Husky Ltee: … is a Quebec family‑owned workwear and safety footwear retailer founded in 1969 in Joliette … Core business: retail and B2B sales of work boots, safety apparel, an…", "s": 3.22}, "30b": {"ok": true, "why": "ok", "pred": "Husky Ltee sells work boots, safety apparel, and PPE (Carhartt, Ariat, Timberland, Helly Hansen, Big Bill), plus an in-house atelier producing 50,000+ custom embroideries, prints, and promotional products per year for corporate clients. It is based in Saint-Paul headquarters.", "s": 0.38}, "cpu": {"ok": true, "why": "ok", "pred": "a Quebec family-owned workwear and safety footwear retailer founded in 1969 in Joliette.", "s": 3.43}}}, {"id": 177, "entity": "Husky Ltee", "kind": "fact", "question": "What is the HUSKY Mobile Store, and since when has it run?", "gold": "A fully equipped truck that travels to job sites for on-site fittings, since 1985.", "expect": {"must": ["truck", "1985"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The HUSKY Mobile Store is a fully equipped truck that travels to job sites and factories for on‑site employee fittings (since 1985) — “Its signature service is the HUSKY Mobile Store (since 1985) — a fully equipped truck that travels to job sites and factories for on‑site employee fittings.”", "s": 1.73}, "30b": {"ok": true, "why": "ok", "pred": "The HUSKY Mobile Store is a fully equipped truck that travels to job sites and factories for on-site employee fittings. It has run since 1985.", "s": 0.32}, "cpu": {"ok": false, "why": "missing '1985'", "pred": "a fully equipped truck that travels to job sites and factories for on-site employee fittings.", "s": 3.07}}}, {"id": 178, "entity": "Husky Ltee", "kind": "fact", "question": "Which group claimed Husky Ltee, and what country code is recorded?", "gold": "thegentlemen; CA.", "expect": {"must": ["CA"], "any": [["thegentlemen", "the gentlemen"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "thegentlemen, CA — “Husky Ltee (RansomwareVictim) claimed by thegentlemen (RansomwareGroup).”; “Country run (velocity-weighted): actor=thegentlemen, country=CA (Story) involves Husky Ltee (RansomwareVictim).”", "s": 1.6}, "30b": {"ok": true, "why": "ok", "pred": "Thegentlemen (RansomwareGroup) claimed Husky Ltee, and the country code recorded is CA.", "s": 0.25}, "cpu": {"ok": true, "why": "ok", "pred": "the gentlemen (Crew), country=CA.", "s": 2.74}}}, {"id": 179, "entity": "Cutting Edge", "kind": "fact", "question": "Which actor designations are attributed to the Cutting Edge campaign?", "gold": "UNC5221/UTA0178 and UNC5325.", "expect": {"any": [["UNC5221", "UTA0178", "UNC5325"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "UNC5221/UTA0178 and UNC5325 — “Cutting Edge was a campaign conducted by suspected China-nexus espionage actors, variously identified as UNC5221/UTA0178 and UNC5325” (first line).", "s": 1.3}, "30b": {"ok": true, "why": "ok", "pred": "UNC5221/UTA0178 and UNC5325", "s": 0.27}, "cpu": {"ok": true, "why": "ok", "pred": "UNC5221/UTA0178 and UNC5325.", "s": 7.68}}}, {"id": 180, "entity": "Cutting Edge", "kind": "fact", "question": "Which VPN appliance did Cutting Edge exploit?", "gold": "Ivanti Connect Secure (formerly Pulse Secure).", "expect": {"any": [["Ivanti Connect Secure", "Ivanti"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Ivanti Connect Secure (previously Pulse Secure) VPN appliances【Cutting Edge was a campaign … with the exploitation of zero‑day vulnerabilities in Ivanti Connect Secure (previously Pulse Secure) VPN appliances.】", "s": 1.12}, "30b": {"ok": true, "why": "ok", "pred": "Cutting Edge exploited zero-day vulnerabilities in Ivanti Connect Secure (previously Pulse Secure) VPN appliances.", "s": 0.29}, "cpu": {"ok": true, "why": "ok", "pred": "zero-day vulnerabilities in Ivanti Connect Secure (previously Pulse Secure) VPN appliances.", "s": 9.06}}}, {"id": 181, "entity": "Cutting Edge", "kind": "reasoning", "question": "Name three custom malware families used in Cutting Edge.", "gold": "LIGHTWIRE, FRAMESTING, ZIPLINE, GLASSTOKEN, PITSTOP, WARPWIRE, WIREFIRE, LITTLELAMB.WOOLTEA.", "expect": {"min_distinct": {"pattern": "LIGHTWIRE|FRAMESTING|ZIPLINE|GLASSTOKEN|PITSTOP|WARPWIRE|WIREFIRE|LITTLELAMB\\.WOOLTEA", "n": 3}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "FRAMESTING, LIGHTWIRE, PITSTOP — On 2024-03-11, Cutting Edge (Campaign, campaign) uses FRAMESTING (Software).  \nOn 2024-03-07, Cutting Edge (Campaign, campaign) uses LIGHTWIRE (Software).  \nOn 2024-03-13, Cutting Edge (Campaign, campaign) uses PITSTOP (Software).", "s": 1.73}, "30b": {"ok": false, "why": "fewer than 3 distinct LIGHTWIRE|FRAMESTING|ZIPLINE|GLASSTOKEN|PITSTOP|WARPWIRE|WIREFIRE|LITTLELAMB\\.WOOLTEA", "pred": "The dossier does not say.", "s": 0.24}, "cpu": {"ok": false, "why": "fewer than 3 distinct LIGHTWIRE|FRAMESTING|ZIPLINE|GLASSTOKEN|PITSTOP|WARPWIRE|WIREFIRE|LITTLELAMB\\.WOOLTEA", "pred": "Impacket, web shells and other custom malware.", "s": 7.99}}}, {"id": 182, "entity": "Cutting Edge", "kind": "date", "question": "When did the Cutting Edge campaign begin?", "gold": "As early as December 2023.", "expect": {"any": [["December 2023", "2023-12"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "December 2023 – “Cutting Edge was a campaign … that began as early as December 2023 with the exploitation of zero‑day vulnerabilities…”【Cutting Edge (Campaign)】", "s": 0.88}, "30b": {"ok": true, "why": "ok", "pred": "The Cutting Edge campaign began as early as December 2023.", "s": 0.16}, "cpu": {"ok": true, "why": "ok", "pred": "began as early as December 2023.", "s": 1.17}}}, {"id": 183, "entity": "Cutting Edge", "kind": "negative", "question": "Does the dossier say Cutting Edge used Cobalt Strike?", "gold": "No. The dossier does not mention Cobalt Strike.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.64}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.1}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention Cobalt Strike.", "s": 1.15}}}, {"id": 184, "entity": "CrashStealer: C++ macOS Infostealer Posing as Crash Reporter", "kind": "fact", "question": "What does CrashStealer impersonate?", "gold": "Apple's crash-reporting framework.", "expect": {"any": [["crash-reporting", "crash reporting", "crash reporter"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Apple's crash‑reporting framework — “impersonates Apple's crash‑reporting framework”【CrashStealer: C++ macOS Infostealer Posing as Crash Reporter: A newly discovered macOS infostealer, implemented in native C++, impersonates Apple's crash‑reporting framework to harvest sensitive data.】", "s": 1.38}, "30b": {"ok": true, "why": "ok", "pred": "CrashStealer impersonates Apple's crash-reporting framework.", "s": 0.36}, "cpu": {"ok": true, "why": "ok", "pred": "impersonates Apple's crash-reporting framework to harvest sensitive data.", "s": 12.19}}}, {"id": 185, "entity": "CrashStealer: C++ macOS Infostealer Posing as Crash Reporter", "kind": "fact", "question": "How does the CrashStealer dropper get past Gatekeeper?", "gold": "It is a signed and notarized dropper application.", "expect": {"any": [["signed and notarized", "notarized"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dropper is “signed and notarized … that bypasses Gatekeeper”【CrashStealer: C++ macOS Infostealer Posing as Crash Reporter: A newly discovered macOS infostealer, implemented in native C++, impersonates Apple's crash-reporting framework to harvest sensitive data. The malware is distributed through a signed and notarized dropper application that bypasses Ga…", "s": 1.4}, "30b": {"ok": true, "why": "ok", "pred": "The malware is distributed through a signed and notarized dropper application that bypasses Gatekeeper.", "s": 0.41}, "cpu": {"ok": false, "why": "none of ['signed and notarized', 'notarized']", "pred": "bypasses Gatekeeper.", "s": 9.22}}}, {"id": 186, "entity": "CrashStealer: C++ macOS Infostealer Posing as Crash Reporter", "kind": "fact", "question": "How does CrashStealer encrypt stolen data before exfiltration?", "gold": "AES-GCM.", "expect": {"must": ["AES-GCM"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Collected data is encrypted using AES‑GCM before being packaged into hidden ZIP archives and exfiltrated. 【Collected data is encrypted using AES‑GCM before being packaged into hidden ZIP archives and exfiltrated】", "s": 1.23}, "30b": {"ok": true, "why": "ok", "pred": "CrashStealer encrypts collected data using AES-GCM before being packaged into hidden ZIP archives and exfiltrated to a command-and-control server.", "s": 0.18}, "cpu": {"ok": true, "why": "ok", "pred": "Collected data is encrypted using AES-GCM before being packaged into hidden ZIP archives and exfiltrated to a command-and-control server.", "s": 2.03}}}, {"id": 187, "entity": "CrashStealer: C++ macOS Infostealer Posing as Crash Reporter", "kind": "fact", "question": "How does CrashStealer establish persistence?", "gold": "It copies itself to a hidden directory and installs a LaunchAgent.", "expect": {"must": ["LaunchAgent"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The malware establishes persistence by copying itself to a hidden directory and installing a LaunchAgent. [“The malware establishes persistence by copying itself to a hidden directory and installing a LaunchAgent.”]", "s": 0.84}, "30b": {"ok": true, "why": "ok", "pred": "The malware establishes persistence by copying itself to a hidden directory and installing a LaunchAgent.", "s": 0.14}, "cpu": {"ok": true, "why": "ok", "pred": "copying itself to a hidden directory and installing a LaunchAgent.", "s": 1.17}}}, {"id": 188, "entity": "Popular DAEMON Tools software compromised", "kind": "fact", "question": "Which DAEMON Tools versions contain the trojaned binaries?", "gold": "12.5.0.2421 to 12.5.0.2434.", "expect": {"must": ["12.5.0.2421", "12.5.0.2434"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Versions 12.5.0.2421 to 12.5.0.2434 contain trojaned binaries (DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp.exe)【Popular DAEMON Tools software compromised: Since April 8, 2026, installers of DAEMON Tools software have been compromised … Versions 12.5.0.2421 to 12.5.0.2434 contain trojaned binaries (DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp…", "s": 1.67}, "30b": {"ok": true, "why": "ok", "pred": "Versions 12.5.0.2421 to 12.5.0.2434 contain trojaned binaries (DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp.exe) signed with legitimate developer certificates.", "s": 0.46}, "cpu": {"ok": true, "why": "ok", "pred": "Versions 12.5.0.2421 to 12.5.0.2434 contain trojaned binaries.", "s": 10.45}}}, {"id": 189, "entity": "Popular DAEMON Tools software compromised", "kind": "date", "question": "Since when have the DAEMON Tools installers been compromised?", "gold": "Since April 8, 2026.", "expect": {"any": [["April 8, 2026", "8 April 2026", "2026-04-08"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Since April 8, 2026, installers of DAEMON Tools software have been compromised【Popular DAEMON Tools software compromised: Since April 8, 2026, installers of DAEMON Tools software have been compromised…】", "s": 0.86}, "30b": {"ok": true, "why": "ok", "pred": "Since April 8, 2026, installers of DAEMON Tools software have been compromised with malicious payloads distributed through the legitimate website.", "s": 0.18}, "cpu": {"ok": true, "why": "ok", "pred": "Since April 8, 2026.", "s": 1.01}}}, {"id": 190, "entity": "Popular DAEMON Tools software compromised", "kind": "fact", "question": "Which trojaned binaries does the report name?", "gold": "DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp.exe.", "expect": {"min_distinct": {"pattern": "DTHelper\\.exe|DiscSoftBusServiceLite\\.exe|DTShellHlp\\.exe", "n": 2}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp.exe – “Versions 12.5.0.2421 to 12.5.0.2434 contain trojaned binaries (DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp.exe) signed with legitimate developer certificates.” (first paragraph).", "s": 2.03}, "30b": {"ok": true, "why": "ok", "pred": "The report names the following trojaned binaries: DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp.exe.", "s": 0.33}, "cpu": {"ok": true, "why": "ok", "pred": "DTHelper.exe, DiscSoftBusServiceLite.exe, DTShellHlp.exe.", "s": 8.43}}}, {"id": 191, "entity": "Popular DAEMON Tools software compromised", "kind": "fact", "question": "What evidence points to the actor's language in the DAEMON Tools compromise?", "gold": "Chinese-language strings in the malicious components.", "expect": {"must": ["Chinese"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Chinese-language strings found in malicious components suggest a Chinese‑speaking threat actor. 【Popular DAEMON Tools software compromised: ... Chinese-language strings found in malicious components suggest a Chinese‑speaking threat actor.】", "s": 0.85}, "30b": {"ok": true, "why": "ok", "pred": "Chinese-language strings found in malicious components suggest a Chinese-speaking threat actor.", "s": 0.13}, "cpu": {"ok": true, "why": "ok", "pred": "Chinese-language strings found in malicious components suggest a Chinese-speaking threat actor.", "s": 1.28}}}, {"id": 192, "entity": "APT29", "kind": "fact", "question": "Which intelligence service is APT29 attributed to?", "gold": "Russia's Foreign Intelligence Service (SVR).", "expect": {"any": [["SVR", "Foreign Intelligence Service"]]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Russia's Foreign Intelligence Service (SVR) – **APT29 is threat group that has been attributed to Russia's Foreign Intelligence Service (SVR).**", "s": 1.55}, "30b": {"ok": true, "why": "ok", "pred": "APT29 is threat group that has been attributed to Russia's Foreign Intelligence Service (SVR).", "s": 0.32}, "cpu": {"ok": true, "why": "ok", "pred": "Russia's Foreign Intelligence Service (SVR).", "s": 8.64}}}, {"id": 193, "entity": "APT29", "kind": "fact", "question": "Name three aliases of APT29.", "gold": "Cozy Bear, The Dukes, NOBELIUM, UNC2452, Midnight Blizzard, Dark Halo, YTTRIUM, IRON RITUAL, IRON HEMLOCK, NobleBaron, CozyDuke, SolarStorm, Blue Kitsune, UNC3524.", "expect": {"min_distinct": {"pattern": "Cozy Bear|The Dukes|NOBELIUM|UNC2452|Midnight Blizzard|Dark Halo|YTTRIUM|IRON RITUAL|IRON HEMLOCK|NobleBaron|CozyDuke|SolarStorm|Blue Kitsune|UNC3524", "n": 3}}, "results": {"120b": {"ok": true, "why": "ok", "pred": "APT29, IRON RITUAL, IRON HEMLOCK — from the line  \n`aliases=APT29, IRON RITUAL, IRON HEMLOCK, NobleBaron, Dark Halo, NOBELIUM, UNC2452, YTTRIUM, The Dukes, Cozy Bear, CozyDuke, SolarStorm, Blue Kitsune, UNC3524, Midnight Blizzard; type=intrusion-set`", "s": 1.58}, "30b": {"ok": false, "why": "fewer than 3 distinct Cozy Bear|The Dukes|NOBELIUM|UNC2452|Midnight Blizzard|Dark Halo|YTTRIUM|IRON RITUAL|IRON HEMLOCK|NobleBaron|CozyDuke|SolarStorm|Blue Kitsune|UNC3524", "pred": "APT29, Cozy Bear, CozyDuke", "s": 0.29}, "cpu": {"ok": false, "why": "fewer than 3 distinct Cozy Bear|The Dukes|NOBELIUM|UNC2452|Midnight Blizzard|Dark Halo|YTTRIUM|IRON RITUAL|IRON HEMLOCK|NobleBaron|CozyDuke|SolarStorm|Blue Kitsune|UNC3524", "pred": "APT29, IRON RITUAL, IRON HEMLOCK.", "s": 9.14}}}, {"id": 194, "entity": "APT29", "kind": "date", "question": "In which year did APT29 reportedly begin compromising the Democratic National Committee?", "gold": "2015.", "expect": {"must": ["2015"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "2015 — “APT29 reportedly compromised the Democratic National Committee starting in the summer of 2015.” (first line).", "s": 0.81}, "30b": {"ok": true, "why": "ok", "pred": "In the summer of 2015.", "s": 0.14}, "cpu": {"ok": true, "why": "ok", "pred": "starting in the summer of 2015.", "s": 1.12}}}, {"id": 195, "entity": "APT29", "kind": "yesno", "question": "Does APT29 use SUNBURST?", "gold": "Yes.", "expect": {"must": ["SUNBURST"], "expect_yes": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Yes. \"On 2021-04-16, APT29 (ThreatActor, intrusion-set) uses SUNBURST (Software).\"", "s": 1.0}, "30b": {"ok": true, "why": "ok", "pred": "Yes. On 2021-04-16, APT29 (ThreatActor, intrusion-set) uses SUNBURST (Software).", "s": 0.51}, "cpu": {"ok": true, "why": "ok", "pred": "Yes. On 2021-04-16, APT29 (ThreatActor, intrusion-set) uses SUNBURST (Software).", "s": 10.24}}}, {"id": 196, "entity": "APT29", "kind": "negative", "question": "Does the dossier say APT29 uses NBTscan?", "gold": "No. The dossier does not mention NBTscan.", "expect": {"expect_no": true}, "results": {"120b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.83}, "30b": {"ok": true, "why": "ok", "pred": "The dossier does not say.", "s": 0.16}, "cpu": {"ok": true, "why": "ok", "pred": "No. The dossier does not mention NBTscan.", "s": 1.05}}}, {"id": 197, "entity": "Orangeworm", "kind": "fact", "question": "Which sector does Orangeworm target, and which malware is directly associated with it?", "gold": "Healthcare; Kwampirs.", "expect": {"must": ["healthcare", "Kwampirs"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Healthcare sector — “Orangeworm is a group that has targeted organizations in the healthcare sector in the United States, Europe, and Asia since at least 2015…” and Kwampirs — “Reverse engineering of Kwampirs, directly associated with Orangeworm activity…” (lines 1‑2).", "s": 1.8}, "30b": {"ok": true, "why": "ok", "pred": "Orangeworm targets organizations in the healthcare sector, and Kwampirs is directly associated with Orangeworm activity.", "s": 0.38}, "cpu": {"ok": false, "why": "missing 'healthcare'", "pred": "Reverse engineering of Kwampirs, directly associated with Orangeworm activity.", "s": 10.02}}}, {"id": 198, "entity": "Orangeworm", "kind": "fact", "question": "Kwampirs shows development overlaps with which other malware?", "gold": "Shamoon.", "expect": {"must": ["Shamoon"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Shamoon – Reverse engineering of Kwampirs, directly associated with Orangeworm activity, indicates significant functional and development overlaps with Shamoon. (Line: “Reverse engineering of Kwampirs, directly associated with Orangeworm activity, indicates significant functional and development overlaps with Shamoon.”)", "s": 1.92}, "30b": {"ok": true, "why": "ok", "pred": "Kwampirs shows development overlaps with Shamoon.", "s": 0.36}, "cpu": {"ok": true, "why": "ok", "pred": "directly associated with Orangeworm activity, indicates significant functional and development overlaps with Shamoon.", "s": 9.66}}}, {"id": 199, "entity": "the gentlemen", "kind": "date", "question": "Which school district did the gentlemen claim on 21 October 2026?", "gold": "Weld Re 8 School District.", "expect": {"must": ["Weld Re 8"]}, "results": {"120b": {"ok": true, "why": "ok", "pred": "Weld Re 8 School District【On 2026-10-21, the gentlemen (Crew) claimed Weld Re 8 School District (DarkVictim).】【On 2026-10-21, the gentlemen (DwCrew) dw claimed Weld Re 8 School District (DwVictim).】", "s": 2.21}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-21, the gentlemen (Crew) claimed Weld Re 8 School District (DarkVictim).", "s": 0.72}, "cpu": {"ok": true, "why": "ok", "pred": "Weld Re 8 School District.", "s": 10.04}}}, {"id": 200, "entity": "qilin", "kind": "date", "question": "Which victims did qilin claim on 23 October 2026?", "gold": "Any two of that day's claims: SEPIVER, United Restaurant Supply (dark-web feed) or von Schaewen, Tamponnt, Miljaus Producciones, Labaid Cancer Hospital, MaxiSeguridad, Kale Balata, Revi Group, CDG Chartered Accounting (crew feed).", "expect": {"min_distinct": {"pattern": "SEPIVER|United Restaurant Supply|von Schaewen|Tamponnt|Miljaus|Labaid|MaxiSeguridad|Kale Balata|Revi Group|CDG Chartered", "n": 2}}, "results": {"120b": {"ok": false, "why": "fewer than 2 distinct SEPIVER|United Restaurant Supply|von Schaewen|Tamponnt|Miljaus|Labaid|MaxiSeguridad|Kale Balata|Revi Group|CDG Chartered", "pred": "", "s": 6.48}, "30b": {"ok": true, "why": "ok", "pred": "On 2026-10-23, qilin (Crew) claimed von Schaewen (DarkVictim).  \nOn 2026-10-23, qilin (Crew) claimed Tamponnt (DarkVictim).  \nOn 2026-10-23, qilin (Crew) claimed Miljaus Producciones S.L (www.miljaus.com) (DarkVictim).  \nOn 2026-10-23, qilin (Crew) claimed Labaid Cancer Hospital And Super Speciality Center (DarkVictim).  \nOn 2026-10-23, qilin (Crew) claimed", "s": 1.45}, "cpu": {"ok": true, "why": "ok", "pred": "von Schaewen, Tamponnt, Miljaus Producciones S.L, Labaid Cancer Hospital And Super Speciality Center, MaxiSeguridad, Kale Balata A, Revi Group, CDG Chartered Accounting, Aerotop, Aryza, Coop Italia.", "s": 13.5}}}]}