Safety in the Loop: Advancing Autonomous Vehicle Safety Validation Through Simulation

How can simulation be leveraged to accelerate AV safety validation? In this livestream, we will present a general, theoretically grounded framework for AV safety validation whereby real-world tests are paired with simulated tests on corresponding reconstructed scenarios. We will discuss how this approach can dramatically reduce the number of real-world tests required to estimate safety KPIs with a specified level of confidence. Attendees will learn: · Main considerations to validate high-end autonomy stacks. · How simulation can be used to significantly accelerate the task of AV safety validation. · More broadly, opportunities offered by generative AI to instantiate a safety data flywheel. Key Moments: 00:19 — Welcome & agenda: AV safety validation via simulation 04:03 — Why end-to-end AV stacks need new safety validation (foundation models) 06:37 — Three pillars of safety: Diversity, Monitoring, Evidence 15:12 — NVIDIA approach: end-to-end policy + modular guardrails for fail-safe behavior 18:21 — Car2Sim: neural scene reconstruction (Cosmos) to re-simulate real logs 20:28 — SIM-to-VAL method: control-variates pairing of real + sim with sim-only boosts 32:39 — Case study (intersections): ~6× fewer real-world miles for same confidence 40:41 — Demo results: 34.5% variance reduction vs. vanilla Monte Carlo 42:57 — Metric Correlator Function (MCF): learning better sim→real correlation 44:06 — Key takeaways: sim as variance-reduction, not sim-only validation Q&A Highlights: 47:41 — Can importance sampling complement control-variates? A: Yes, both can be combined for rare or long-tail validation cases. 49:01 — How many paired samples are typically needed? A: A few hundred paired samples with 1–5× additional sim-only samples give strong results. 50:35 — How does this help with rare corner cases? A: Simulation fills in gaps; each real-world rare sample becomes more valuable. 54:32 — How do you prove redundancy and independence between AV subsystems? A: Architectural and training diversity help reduce common-mode failures; ongoing monitoring confirms it. 56:35 — Is SIM-to-VAL related to Prediction-Powered Inference? A: Yes — PPI is a special case where β = 1 in the same control-variates framework. 58:19 — Are simulators deterministic? A: Determinism improves correlation; one rollout per scenario is used, with multi-rollouts planned for future versions. 1:00:00 — Why is real-world testing so expensive? A: Time, driver labor, vehicle wear, and insurance costs add up rapidly. 1:01:01 — Can we move from statistical confidence to provable safety? A: Full formal proofs are difficult; best practice is hybrid — combine data-driven validation with formal methods. Resources: · Autonomous Vehicle Simulation webpage: https://www.nvidia.com/en-us/use-case... · Sim2Val project website: https://nvlabs.github.io/sim2val/ · Technical paper: https://www.arxiv.org/abs/2506.20553 · Github repo: https://github.com/NVLabs/sim2val Got questions, post them on our Discord Thread:   / discord   Discord Invite:   / discord   📆 Check out the full calendar for all of our upcoming events → https://nvda.ws/3JqaWnA ---------------------------------------------------------------------------- ⬇️Get Started → https://nvda.ws/4cZAZO1 ​ 👀Explore OpenUSD → https://nvda.ws/3CeozBQ ​ 👥Join the Community → https://nvda.ws/3ZMfc6e