Evaluating Nova 2.0 Lite model under Amazon’s Frontier Model Safety Framework

Satyapriya Krishna, Matteo Memelli, Tong Wang, Abhinav Mohanty, Claire O’Brien Rajkumar, Payal Motwani, Rahul Gupta, and Spyros Matsoukas Amazon Nova Responsible AI · 2026

Read Evaluating Nova 2.0 Lite model under Amazon’s Frontier Model Safety Framework by Satyapriya Krishna, Matteo Memelli, Tong Wang, Abhinav Mohanty, Claire O’Brien Rajkumar, Payal Motwani, Rahul Gupta, and Spyros Matsoukas Amazon Nova Responsible AI (2026) in the Tower of Babel, a free online library of public-domain works preserved in clean, searchable text. No sign-up. The full text is served as a plain-text file at a predictable URL, where text is present: https://antarctic-labs.com/tower-of-babel/arxiv-2601-19134.txt.

Introduction

Satyapriya Krishna and six co-authors grade a new Amazon model against the firm's own safety frame. Amazon set out its Frontier Model Safety Framework at the Paris AI summit and has since scored its top models against it. Here the subject is Nova 2.0 Lite, a lighter model now open to all users.

The report runs the model through the frame's checks for risky capabilities and behavior. Public scoring of a shipped model lets outsiders see how a lab's safety claims are tested, not just stated. Such reports are fast becoming the norm for frontier model releases.

Catalog record

CREATORSatyapriya Krishna, Matteo Memelli, Tong Wang, Abhinav Mohanty, Claire O’Brien Rajkumar, Payal Motwani, Rahul Gupta, and Spyros Matsoukas Amazon Nova Responsible AI
YEAR2026
COLLECTIONPAPERS
FORMATTXT
FILE SIZE30 KB
VERSION1.0
LICENSEhttp://creativecommons.org/licenses/by/4.0/
RIGHTSCC BY
SOURCEarXiv

Source: https://arxiv.org/abs/2601-19134

Browse the full library · About the Tower of Babel

Full text

The opening of the work, from the library's hosted text:

Evaluating Nova 2.0 Lite model under Amazon’s Frontier Model Safety Framework Satyapriya Krishna, Matteo Memelli, Tong Wang, Abhinav Mohanty, Claire O’Brien Rajkumar, Payal Motwani, Rahul Gupta, and Spyros Matsoukas Amazon Nova Responsible AI Abstract Amazon published its Frontier Model Safety Framework (FMSF) as part of the Paris AI summit, following which we presented a report on Amazon’s Premier model [ 7 ] . In this report, we present an evaluation of Nova 2.0 Lite. Nova 2.0 Lite was made generally available from amongst the Nova 2.0 series and is one of its most capable reasoning models. The model processes text, images, and video with a context length of up to 1M tokens, enabling analysis of large codebases, documents, and videos in a single prompt. We present a comprehensive evaluation of Nova 2.0 Lite’s critical risk profile under the FMSF [ 2 ] . Evaluations target three high-risk domains–Chemical, Biological, Radiological and Nuclear (CBRN), Offensive Cyber Operations, and Automated AI R&D–and combine automated benchmarks, expert red-teaming, and uplift studies to determine whether the model exceeds release thresholds. We summarize our methodology and report core findings. We will continue to enhance our safety evaluation and mitigation pipelines as new risks and capabilities associated with frontier models are identified. 1 Introduction Amazon Nova 2.0 Lite [ 5 ] is a frontier-scale, multimodal foundation model that can reason over extended contexts of up to 1M tokens, spanning source-code repositories and lengthy documents. The breadth of competence in this model raises commensurate safety obligations. In line with the commitments announced at the 2025 Paris AI Safety Summit, Amazon’s Frontier Model Safety Framework (FMSF) requires that every release undergo rigorous, domain-specific risk assessments before deployment. The FMSF concentrates on three high-consequence domains: Chemical, Biological, Radiological and Nuclear (CBRN) weapons proliferation, Offensive Cyber Operations, and Automated AI R&D, each with a defined critical threshold. We will be publishing the safety reports of other follow-up models in the Nova family as they are made generally available. We also note that since Nova Act is powered by a custom version of Nova 2.0 Lite [ 1 ] , these evaluations are also indicative of this model powering our Nova Act offering. This report is a system-card-style disclosure of Nova 2.0 Lite’s performance against those thresholds. We integrate two complementary methodologies: (i) reproducible automated benchmarks that quantify the model’s knowledge of the high risk domains, and (ii) human-centric risk evaluations such as expert red-teaming, uplift studies, and multi-agent stress tests that probe for emergent, real-world failure modes. Independent auditors (Nemesys Insights for CBRN and Model Evaluation and Threat Research (METR) for Automated AI R&D) reviewed results on test sets, scoring rubrics, and safety-judge notes to verify our internal findings. Across all three domains, Nova 2.0 Lite shows measurable capability gains over its predecessors (Nova 1.0), however within safety thresholds. The model demonstrates higher factual accuracy on CBRN and cybersecurity knowledge tests, yet automated and human-in-the-loop evaluations indicate that it remains within safety thresholds when performing tasks such as executing end-to-end weaponisation workflows, solving real-world capture-the-flag exploits, or autonomously conducting machine-learning research relevant to CBRN or cyber offensive applications. External reviewers concur that the model is safe for public release under the current mitigation stack, which layers policy-tuned refusal behaviour, dynamic content filters, and continuous safeguard monitoring. By formalising the evaluation protocol and disclosing empirical risk signals, this paper aims to provide a transparent template for future cross-organisational safety audits of frontier models. Figure 1 : CBRN automated benchmark results comparing models across four datasets. For Nova 2.0 Lite, we present the best result among all the reasoning modes (low’,’medium’, and ‘high’). 2 Chemical, Biological, Radiological, and Nuclear (CBRN) Weapons Proliferation LLMs present a dual mandate in the CBRN domain: they can streamline benign laboratory workflows while simultaneously lowering the barrier to the misuse of sensitive know-how. Guided by Amazon’s Frontier Model Safety Framework, we combine automated stress tests with independent, human-in-the-loop "uplift" studies to determine whether Nova 2.0 Lite can be considered safe with respect to the threshold for CBRN weapons proliferation. As defined in the framework, this threshold is crossed if a model is capable of providing expert-level, interactive instruction that delivers material uplift beyond what is available through public tools or research, in a manner that enables a non-expert to reliably produce and deploy a CBRN weapon. 2.1 Automated Benchmarks Our automated suite covers complementary abilities that span the spectrum of CBRN-relevant safety aspects, from basic knowledge recall to complex procedural understanding and execution planning. Hazardous-knowledge recall is measured with the Weapons of Mass Destruction Proxy (WMDP) [ 9 ] benchmark: WMDP-Bio (1,273 items) and WMDP-Chem (408 items), each formatted as multiple-choice questions on topics like cloning, pathogen handling, and toxin synthesis. Protocol troubleshooting is evaluated with ProtocolQA [ 8 ] , a 108-question benchmark where each item embeds a single error into a wet-lab procedure and asks for the correct fix. Long-answer plan generation is tested with BioLP-Bench [ 6 ] , an 800-sample benchmark covering 11 real-world protocols where responses are graded by an LLM-as-a-judge rubric. Finally, we include the Virology Capabilities Test ( VCT ) [ 4 ] , a benchmark that measures the capability to troubleshoot complex virology laboratory protocols. VCT is designed to be difficult, with expert virologists scoring an average of only 22.1% even with internet access. The evaluation results (Figure 1 ) show the performance of Nova 2.0 Lite relative to Nova 1.0 Pro and Nova Premier across all four datasets. On knowledge-oriented multiple-choice tasks, Nova 2.0 Lite reaches 0.71 on WMDP-Chem , exceeding Nova Premier (0.66) and Nova 1.0 Pro (0.63). On WMDP-Bio , Nova 2.0 Lite scores approximately 0.82, performing on par with Nova 1.0 Pro (0.82) and slightly below Nova Premier (0.84). These scores place Nova 2.0 Lite within the range typical of frontier language models on hazardous knowledge benchmarks. On procedure-heavy benchmarks, Nova 2.0 Lite shows consistently higher scores relative to the baseline. In ProtocolQA , Nova 2.0 Lite attains 0.49 versus Nova 1.0 Pro’s 0.34 and Nova Premier’s 0.48. A similar pattern emerges on BioLP-Bench , where the mean solve rate is 0.24 for Nova 2.0 Lite compared to 0.11 for Nova 1.0 Pro and 0.23 for Nova Premier. Finally, on the expert-level VCT benchmark, Nova 2.0 Lite achieves a score of 0.29, slightly below Nova Premier (0.30) and exceeding Nova 1.0 Pro (0.15). To further evaluate multimodal risk, we utilized a chemical image understanding dataset collected by subject matter experts. The dataset comprises 105 chemicals (categorized as dangerous, safe, or contested) paired with diverse structural images, including noisy samples. Experts developed prompts for each chemical to test safety boundaries, creating accurate responses for safe queries and refusal criteria for dangerous ones. These challenging questions require recognizing chemicals from structure images, assessing their hazards, judging response appropriateness, and answering accurately using advanced chemical knowledge, thereby serving as an early warning system that measures the trade-off between safety and utility. Results, shown as Chem-Vision in Figure 1 , indicate that Nova 2.0 Lite balances these capabilities effectively. We observe that Nova 2.0 Lite obtains an accuracy between Nova Premier and Nova 1.0 Pro. Additionally, our assessment also indicated that Nova 2.0 Lite demonstrates a stronger safety posture

Read the complete text