Reward-Weighted On-Policy Distillation with an Open Property-Equivalence Verifier for NL-to-SVA Generation
Qingyun Zou, Yingze Li, Tianen Liu, Bingsheng He, Weng-Fai Wong · 2026
Read Reward-Weighted On-Policy Distillation with an Open Property-Equivalence Verifier for NL-to-SVA Generation by Qingyun Zou, Yingze Li, Tianen Liu, Bingsheng He, Weng-Fai Wong (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2605-13501.txt.
Introduction
Qingyun Zou and four colleagues introduce reward-weighted on-policy distillation for SystemVerilog assertions. The strongest model reaches about 76 percent on NL2SVA-Human, yet collapses to a few templates on delay and liveness cases. Fine-tuning on language-assertion pairs optimises token mimicry rather than property equivalence. Their method samples student outputs, scores them with an open SymbiYosys-Z3 checker, and applies a verifier-weighted gradient. Distilling CodeV-SVA-14B into a Qwen2.5-Coder-7B student sets a new record on NL2SVA-Human and NL2SVA-Machine.
Catalog record
| CREATOR | Qingyun Zou, Yingze Li, Tianen Liu, Bingsheng He, Weng-Fai Wong |
|---|---|
| YEAR | 2026 |
| COLLECTION | PAPERS |
| FORMAT | TXT |
| FILE SIZE | 92581 |
| LICENSE | http://creativecommons.org/licenses/by/4.0/ |
| RIGHTS | CC BY |
| SOURCE | arXiv |
Source: https://arxiv.org/abs/2605.13501
Browse the full library · About the Tower of Babel
Full text
The opening of the work, from the library's hosted text:
Reward-Weighted On-Policy Distillation with an Open Property-Equivalence Verifier for NL-to-SVA Generation Qingyun Zou, Yingze Li, Tianen Liu, Bingsheng He, Weng-Fai Wong Abstract LLM-based generation of SystemVerilog Assertions (SVA) is often reported as nearing saturation, with the strongest specialized model reaching ∼ 76 % {\sim}76\% accuracy on NL2SVA-Human. We show that this aggregate hides a temporal gap: models that appear strong overall still collapse to a few implication templates on bounded-delay and liveness specifications. The core issue is that the dominant recipe, supervised fine-tuning on NL/SVA pairs, optimizes token-level mimicry rather than the property equivalence that defines SVA correctness. We introduce Reward-Weighted On-Policy Distillation (RWOPD), an on-policy distillation method that samples student rollouts, scores them with an open SymbiYosys+Z3 Property-Equivalence Checker (PEC), and applies a verifier-reward-weighted forward-KL gradient from a frozen 14B teacher on verifier-passable rollouts. This keeps the supervision dense at every response token while grounding both selection and loss weight in property-equivalent behavior. RWOPD distills CodeV-SVA-14B into a Qwen2.5-Coder-7B-Instruct student that sets a new state of the art on NL2SVA-Human and NL2SVA-Machine across pass@1, pass@5, and pass@10, surpassing both specialized prior SOTA models and 671B general-purpose baselines. 1 Introduction SystemVerilog Assertions (SVA) specify safety, liveness, and bounded-time obligations for register-transfer-level designs, making them a common entry point for industrial formal verification 17 ; 6 . Because hand-writing SVAs from natural-language intent is slow and error-prone, recent work has trained LLMs to translate natural-language specifications into assertions 22 ; 21 ; 13 . The strongest specialized model, CodeV-SVA-14B 21 , reports roughly 76% functional accuracy on NL2SVA-Human, suggesting that the benchmark may be close to saturated. The aggregate score hides a temporal failure mode. On bounded-delay, ranged-delay, and liveness specifications, both a 14B state-of-the-art model and 7B baselines often fall back to a small set of implication templates (§ 3 ). We call this template collapse : the generated SVA is syntactically plausible, but its temporal class or property meaning does not match the specification. The failure matters because the non-trivial part of NL-to-SVA is not producing assertion-shaped text; it is producing an assertion that is property-equivalent to the intended behavior. The training objective used by existing NL-to-SVA systems does not directly optimize that target. Supervised fine-tuning over NL/SVA pairs rewards token-level agreement with one reference assertion, even though SVA correctness is semantic: two assertions can be syntactically distant and equivalent, while two assertions that differ by one temporal operator can be opposed. A more natural supervision signal should evaluate the property emitted by the model and then update the model on its own rollouts. Reinforcement learning with verifier feedback is the direct way to implement that idea, following the broader use of verifiers and verifiable rewards in language-model training 7 ; 15 ; 9 . In our setting, however, it is not the most effective use of a small LoRA update budget 11 . The complete oracle used by prior NL-to-SVA evaluation, Cadence JasperGold’s prop_eq_checker , is closed-source and too costly to call inside every gradient step. We therefore build an open SymbiYosys+Z3 Property-Equivalence Checker (PEC) for the bounded-model-checking fragment, but a single verifier verdict per rollout remains a sparse learning signal. Across seven GRPO and two IPO configurations on the same SFT seed, the verifier reward does not exceed the strict open-PEC SFT baseline; a rejection-sampling SFT control that fine-tunes on the seed’s own PEC-accepted rollouts (filter alone, without the teacher) lifts pass@1 only to roughly GRPO’s level and stays well below RWOPD, isolating the dense teacher KL as the dominant value-add (§ 5.3.2 ). Our main method uses the verifier differently: as a reward-shaped filter for dense on-policy distillation. Reward-Weighted OPD (RWOPD, § 4.1 ) samples rollouts from the current student, keeps only rollouts that the open PEC marks as verifier-passable, weights each kept rollout by its verifier-equivalence reward, and applies the forward-KL gradient from a frozen CodeV-SVA-14B teacher on those response tokens. A stratified-curriculum SFT seed with operator-token-weighted cross-entropy (§ 4.2 ) gives the student non-degenerate coverage across temporal classes, so verifier-passing rollouts appear often enough for OPD to train. JasperGold remains the complete evaluation oracle, while the open PEC supplies a reproducible, sound filter on the C1/C2 fragment (§ 4.3 ). The contributions of this paper are: (1) Reward-Weighted On-Policy Distillation (RWOPD) : an OPD enhancement that restricts and reward-weights the teacher’s dense forward-KL supervision using an open property-equivalence checker (§ 4.1 ). (2) An open SymbiYosys+Z3 Property-Equivalence Checker that supplies a license-free, second-scale verifier signal for training-time filtering and reward experiments. The checker is sound on the bounded C1/C2 fragment and abstains on unsupported liveness cases; JasperGold is retained for complete final evaluation (§ 4.3 ). (3) Empirical results. In our 7B+LoRA setting, RWOPD sets the new state of the art on NL2SVA-Human and NL2SVA-Machine across pass@1, pass@5, and pass@10, surpassing both the specialized prior SOTA (CodeV-SVA-14B) and 671B general-purpose baselines (Table 1 ); the per-class breakdown shows the gain is concentrated on the temporally non-trivial slice (C2 +7.5 pp, C3 +11.5 pp on NL2SVA-Machine) while the C1 majority is already saturated (Figure 3 ). 2 Related Work LLM-based NL \to SVA generation. Recent NL-to-SVA systems fine-tune code LLMs 5 ; 18 ; 14 on synthesized assertion corpora: AssertLLM 22 , CodeV-SVA 21 , and the FVEval benchmark family 13 establish the task and the JasperGold evaluation protocol. Their aggregate accuracies do not separate combinational, bounded-temporal, and liveness behavior; our temporal analysis shows that this aggregation hides template collapse. Verifier feedback for assertion synthesis. Prior verifier-assisted methods use formal tools outside the training loss — AssertFix 16 for counterexample-driven inference repair for prompt-loop refinement. We instead call the verifier inside post-training and compare sparse reward optimization with reward-weighted distillation. On-policy distillation and verifier rewards. On-policy distillation 1 trains a student on its own rollouts via a teacher distribution; RL and preference optimization 9 ; 2 train on rollout-level rewards or pairwise preferences. We combine the two: the verifier weights each rollout, and the teacher supplies the dense token-level gradient on the weighted rollouts. 3 The Hidden Temporal Gap The aggregate accuracy of prior NL-to-SVA generators hides a temporally stratified failure. We expose it by classifying each SVA by its Temporal Complexity Level (TCL) into three classes by maximum-complexity temporal operator: C1 combinational, C2 bounded temporal, and C3 liveness. The operator-level definitions, a regex-AST classifier with 100% accuracy on a 90-SVA hand-labeled corpus, and a 32-test edge-case suite are in Appendix D . Decomposed by reference temporal class on NL2SVA-Human (62 C1, 6 C2, 11 C3), the 14B state of the art CodeV-SVA-14B 21 produces a same-class output on only 18 of 62 C1, 3 of 6 C2, and 4 of 11 C3 references; an unfine-tuned 7B base (Qwen2.5-Coder-7B-Instruct) produces 19, 0, and 4. The 75.8% functional accuracy reported by prior work is dominated by the C1 majority and obscures these gaps on the temporally non-trivial slice. We call the underlying failure mode template collapse . Against the reference 62/6/11 class distribution, the 14B teacher emits 19/33/9 and the 7B base emits 31/41/6 across C1/C2/C3 — both substantially overproduce C2 (the implication-template class) and underproduce C1. When the dispatch decision is hard — a bounded delay or a liveness obligation — the model defaults to