StepFun's Step 5 Preview Open Weights Landed On Schedule. The Benchmarks Still Don't Agree.
When StepFun unveiled Step 5 Preview on September 20, the open-weights promise that came with it looked like the usual vaporware pattern: a Hugging Face repository, stepfun-ai/Step-5-Preview-BF16, that held nothing but a placeholder .gitattributes file and a date, October 15, that was still weeks away. On October 15, StepFun actually delivered, publishing BF16 and FP8 checkpoints to both Hugging Face and ModelScope alongside vLLM and SGLang serving support, according to a same-day report from Chinese financial outlet 10jqka. That alone is notable: in a year when several Chinese labs have announced open weights that slipped by months, StepFun hit its date.
A 600-billion-parameter MoE built for long agent runs
Step 5 Preview is a sparse mixture-of-experts model with 600 billion total parameters and roughly 27 billion active per token, a 92-layer 'narrow and deep' design the company says is tuned for long-horizon agentic work rather than single-turn chat. It claims a 1-million-token context window, though a third-party configuration file reviewed by model-routing analysts at OrcaRouter listed a conflicting 350,000-token context with a 64,000-token output cap that StepFun has not publicly resolved. Pricing landed at $1.00 per million input tokens and $2.70 per million output tokens, positioning it well below Claude Opus 5's $5/$25 but above DeepSeek's V4.1 Flash.
Two scorecards, two different models
StepFun's own benchmark table put Step 5 ahead of Kimi K3 and GLM-5.3 on several agentic coding tests, including a StepCodeBench score of 49.0% against Kimi K3's 43.9%. Artificial Analysis, running its own Intelligence Index independently, scored the model at 44, good for 24th of roughly 200 tracked models, tied with Kimi K3 and Claude Opus 5 at medium reasoning effort but a point behind GLM-5.3 — a far less flattering picture than the vendor table's framing. Epoch AI's cross-lab comparison, which still listed Step 5 as closed-weight as of its latest snapshot, is less kind still: 42% on ProofBench against GPT-6 Astra's 99% and Claude Fable 5.1's 100%, and a Text Arena coding Elo of 1570 versus GPT-6 Astra's 1820. The one place Step 5 doesn't trail badly, SciCode, has it essentially tied with Kimi K3 and ahead of GPT-6 Astra.
The gap is partly explained by how the model spends tokens. Artificial Analysis found Step 5 generated about 160 million output tokens running its benchmark suite, against a median of 92 million for comparable models — verbosity that pushed the all-in cost of that run past $920, with output tokens alone accounting for roughly $432 of it. A model priced cheaply per token can still be an expensive model to actually run.
The model is callable today and downloadable in about three and a half weeks, and those are two different things.
That line, from an independent technical breakdown published shortly after the September announcement, captured the core tension before the October 15 release resolved it: an API being live says nothing about whether weights, a license, or reproducible numbers will follow. StepFun's on-time delivery answers the first question. It does not answer the second — the company has still not published a license for the checkpoints, and the benchmark rows beyond the Artificial Analysis index remain entirely self-reported.
What it means for the open-weight pile-up
Step 5 Preview's real significance isn't its ranking — by any independent measure it sits well behind GPT-6 Astra, Claude Fable 5.1, and even some open Chinese peers on hard reasoning and coding tasks. It's that StepFun has now joined DeepSeek, Alibaba's Qwen team, Zhipu's GLM line, and Moonshot's Kimi in treating a prompt, on-time open-weights drop as a competitive requirement rather than a courtesy. Chinese financial press noted Step 5 had already climbed to second place by call volume on OpenRouter's weekly leaderboard before the weights even shipped — a usage signal, not a capability one, but one that shows practitioners were routing real traffic to an unverifiable model simply because it was cheap and available. With the weights now public, that gap between adoption and independent verification is closing, but the mismatch between StepFun's internal benchmark table and what Artificial Analysis and Epoch AI are measuring suggests the verification problem in this tier of model isn't going away — it's just moving from 'can we download it' to 'can we trust the number on the box.'