Reading the benchmark tables

Reading the benchmark tables A data-flow diagram generated by Archify. 01 / Each vendor's card 02 / Comparison target 03 / Evaluation setup 04 / What you can read Meta model card · Muse-Glimmer 30B · 01 / Each vendor's card · 2026-08-10 Meta model card Muse-Glimmer 30B 2026-08-10 Qwen model card · Qwen3.8-27B · 01 / Each vendor's card · 2026-08-14 Qwen model card Qwen3.8-27B 2026-08-14 Qwen3.6-27B · previous generation · 02 / Comparison target · 2026-04 Qwen3.6-27B previous generation 2026-04 Muse-Glimmer 30B · current release · 02 / Comparison target · 2026-08 Muse-Glimmer 30B current release 2026-08 Meta's harness · SWE-bench Verified · 03 / Evaluation setup · own setup Meta's harness SWE-bench Verified own setup Qwen's harness · Claude Code, 256K · 03 / Evaluation setup · own setup Qwen's harness Claude Code, 256K own setup SWE-bench Pro 51.2 · both cards, same harness · 04 / What you can read · aligned SWE-bench Pro 51.2 both cards, same harness aligned Reading rule · gaps within a table · 04 / What you can read · across tables: discount Reading rule gaps within a table across tables: discount compares against Meta's choice compares against Qwen's choice evaluated under vendor setup evaluated under vendor setup SWE-bench Pro public harness same 51.2 public harness only aligned row caveat Legend primary data policy / PII async batch data store data flow

The mirror

  • • Qwen's card quotes Muse 51.2 on SWE-bench Pro, 65.9 on OSWorld
  • • Meta's card quotes Qwen3.6 77.2 SWE-bench Verified, 75.6 OSWorld
  • • Each side benchmarks against the other's generation of its choosing

One aligned number

  • • SWE-bench Pro under Claude Code is the only row both tables share
  • • Muse trails Qwen3.8's 61.7 by 10.5 and Qwen3.6's 53.5 by 2.3
  • • No current-vs-current, same-setup table exists yet

The rule

  • • Relative gaps within one table are readable
  • • Absolute values read across tables: discount
  • • Vendor headline numbers are claims, not measurements