What does HackerNews think of hypothesis?

Hypothesis is a powerful, flexible, and easy to use library for property-based testing.

Language: Python

#102 in Python
#8 in Testing
Author note: The original title is "Property based testing: tale of two shrinkers", which is retroactively a bit bad title choice on my part; the title I submitted it under sums the content up better I believe.

This is a talk I gave internally at my company (I'm also planning to give an updated version at [Haskell Exchange 2022](https://events.skillsmatter.com/haskellx2022) in a month or so), about approaches to shrinking in PBT libraries and their issues, and how the [Hypothesis](https://github.com/HypothesisWorks/hypothesis) approach ("internal shrinking") solves those issues yet remains mostly unknown. There's also a bit of a call to action to PBT library maintainers to adopt this approach at the end :)

I'm a co-maintainer of [elm-test](https://github.com/elm-explorations/test), the Elm PBT library, which has historically used the rose tree approach (second one mentioned in the video) and recently switched to the internal shrinking approach in v2.0.0. We like it so far!

We've had a lot of success combining that approach (cffi instead of ctypes) with property-based testing (https://github.com/HypothesisWorks/hypothesis) for the query engine at backtrace: https://engineering.backtrace.io/2020-03-11-how-hard-is-it-t... .
The inputs shouldn’t be “random”, but the idea is that you should provide an input distribution that’s roughly representative of your expected distribution of real-world values (+ some perturbation to find edge cases).

As for frameworks in the languages you listed...

Ruby and Python: https://github.com/HypothesisWorks/hypothesis

Elixir and Erlang: https://github.com/proper-testing/proper

Node and JS: https://jsverify.github.io/

As for real world use-cases, imagine you’re writing a program that accepts timestamps as input and has to implement branching, requirements-defined business logic based off of them. When you’re writing your unit tests you can use the requirements to select timestamps that are “known good” and “known bad”, but it’s hard to explore this state space on your own.

Same thing goes for handling unexpected inputs to certain functions. You probably don’t want to check _every_ type of input for _every_ dynamic function, but it might make sense to make sure that certain “entry points” to your program fail in the expected manner when they get poorly typed input.

> lack of typing information isn't "free solo" hard but it certainly makes life a lot less pleasant.

One of the projects I work on has switched to full type hinting along with heavy mypy¹ usage, and it has become an absolute pleasure to work with. Along with hypothesis², I can't recommend mypy enough. It must be said that retrofitting either to an existing project is a lot of work though.

1. http://www.mypy-lang.org/ 2. https://github.com/HypothesisWorks/hypothesis

"While Python is usually great for programming puzzles, this step in particular is crying out for something like QuickCheck: without it, we'll have to roll our own."

Instead of rolling his own, the author could have just used Python's Hypothesis package.[1][2]

[1] - https://github.com/HypothesisWorks/hypothesis

[2] - https://pypi.org/project/hypothesis/