The Bug Humans Can't Find

Property-based testing generates millions of test cases automatically. Nikhil Barthwal shows how it catches bugs humans can't imagine. "When the system gets too complicated it is not possible for humans to write every test cases." - Nikhil Barthwal In this episode, I chat with Nikhil Barthwal about property-based testing. We go into how property-based testing can uncover the hidden bugs that often slip past human testers. With its capacity to automatically generate a multitude of test cases, this method helps us see beyond typical limitations. Nikhil also shares when property-based testing may not be ideal, like when it incurs high resource costs. He emphasizes that this approach serves as an assistant to testers rather than a replacement, enhancing productivity and reliability. 00:00 Exploring Property-Based Testing 05:16 Testing Overflow with Property-Based Methods 07:37 Identifying Bugs in Complex Steps 11:53 Testing Intermittent System Failures 14:02 Testing E-commerce System Properties 16:47 Costly Testing Challenges 19:25 Convincing Skeptical Colleagues 23:04 System Enhances Tester Efficiency 📘 Free e-book: The 7 success factors of software testing. 25 years of project experience in one 33-page workbook, now also in English 👉 https://tul.fm/ebook 🎯 Highlights: Property-based testing generates thousands of test combinations automatically from a formal description of system behavior, reaching failure modes no human-authored test suite can anticipate. In distributed systems with many services, the number of possible failure combinations grows exponentially, making manual test case authorship impractical at scale. The shrinker component is what makes property-based testing usable: it reduces a long generated failure sequence to the precise step where the fault occurred, so engineers can act on it. Property-based testing becomes costly where each test execution consumes billable resources, such as cloud storage writes or live network calls, because automated test volume can spiral expenses out of control. Introducing property-based testing in a large organization is primarily a cultural challenge: winning one willing team first, letting them advocate to peers, is more effective than a top-down rollout. 🔗 Links Blog Post for Episode: https://www.richard-seidl.com/en/podc... 🎙️ More from Richard Seidl Website: https://www.richard-seidl.com Linkedin:   / richardseidl   Podcast Software Testing: https://www.testing-unleashed.fm #softwaretesting #QA #propertybasedtesting