# `ExUnitProperties`
[🔗](https://github.com/whatyouhide/stream_data/blob/v1.4.0/lib/ex_unit_properties.ex#L1)

Provides macros for property-based testing.

This module provides a few macros that can be used for property-based testing. The core is `check/3`,
which allows executing arbitrary tests on many pieces of generated data. Another one is
`property/3`, which is meant as a utility to replace the `ExUnit.Case.test/3` macro when writing
properties. The last one is `gen/3`, which can be used as syntactic sugar to build generators
(see `StreamData` for other ways of building generators and for core generators).

## Overview of property-based testing

One of the most common ways of writing tests (in Elixir and many other
languages) is to write tests by hand. For example, say that we want to write a
`starts_with?/2` function that takes two binaries and returns `true` if the
first starts with the second and `false` otherwise. We would likely test such
function with something like this:

    test "starts_with?/2" do
      assert starts_with?("foo", "f")
      refute starts_with?("foo", "b")
      assert starts_with?("foo", "")
      assert starts_with?("", "")
      refute starts_with?("", "something")
    end

This test highlights the method used to write such kind of tests: they're
written by hand. The process usually consists of testing an expected output on
a set of expected inputs. This works especially well for edge cases, but the
robustness of this test could be improved. This is what property-based testing aims
to solve. Property testing is based on two ideas:

  * specify a set of **properties** that a piece of code should satisfy
  * test those properties on a very large number of randomly generated data

The point of specifying **properties** instead of testing manual scenarios is
that properties should hold for all the data that the piece of code should be
able to deal with, and in turn, this plays well with generating data at
random. Writing properties has the added benefit of forcing the programmer to
think about their code differently: they have to think about which are
invariant properties that their code satisfies.

To go back to the `starts_with?/2` example above, let's come up with a
property that this function should hold. Since we know that the `Kernel.<>/2`
operator concatenates two binaries, we can say that a property of
`starts_with?/2` is that the concatenation of binaries `a` and `b` always
starts with `a`. This is easy to model as a property using the `check/3` macro
from this module and generators taken from the `StreamData` module:

    test "starts_with?/2" do
      check all a <- StreamData.binary(),
                b <- StreamData.binary() do
        assert starts_with?(a <> b, a)
      end
    end

When run, this piece of code will generate a random binary and assign it to
`a`, do the same for `b`, and then run the assertion. This step will be
repeated for a large number of times (`100` by default, but it's
configurable), hence generating many combinations of random `a` and `b`. If
the body passes for all the generated data, then we consider the property to
hold. If a combination of randomly generated terms fails the body of the
property, then `ExUnitProperties` tries to find the smallest set of random
generated terms that still fails the property and reports that; this step is
called shrinking.

### Shrinking

Say that our `starts_with?/2` function blindly returns false when the second
argument is the empty binary (such as `starts_with?("foo", "")`). It's likely
that in 100 runs an empty binary will be generated and bound to `b`. When that
happens, the body of the property fails but `a` is a randomly generated binary
and this might be inconvenient: for example, `a` could be `<<0, 74, 192, 99,
24, 26>>`. In this case, the `check/3` macro tries to **shrink** `a` to the
smallest term that still fails the property (`b` is not shrunk because `""` is
the smallest binary possible). Doing so will lead to `a = ""` and `b = ""`
which is the "minimal" failing case for our function.

The example above is a contrived example but shrinking is a very powerful tool
that aims at taking the noise out of the failing data.

For detailed information on shrinking, see also the "Shrinking" section in the
documentation for `StreamData`.

## Building structs

We can use the built-in generators to generate other kinds of structs. For
example, imagine we wanted to test the following function.

    def noon?(~T[12:00:00]), do: true
    def noon?(_), do: false

We could generate `%Time{}` structs as follows:

    defp non_noon_generator do
      gen all time <- valid_time_generator(), time != ~T[12:00:00] do
        time
      end
    end

    defp valid_time_generator do
      gen all hour <- StreamData.integer(0..23),
              minute <- StreamData.integer(0..59),
              second <- StreamData.integer(0..59) do
        Time.new!(hour, minute, second)
      end
    end

and use them in properties:

    describe "noon?/1" do
      test "returns true for noon" do
        assert noon?(~T[12:00:00]) == true
      end

      property "returns false for other times" do
        check all time <- non_noon_generator() do
          assert noon?(time) == false
        end
      end
    end

## Resources on property-based testing

There are many resources available online on property-based testing. An interesting
read is the original paper that introduced QuickCheck, ["QuickCheck: A
Lightweight Tool for Random Testing of Haskell
Programs"](http://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf), a
property-testing tool for the Haskell programming language. Another very
useful resource especially geared towards Erlang and the BEAM is
[propertesting.com](http://propertesting.com), a website created by Fred
Hebert: it's a great explanation of property-based testing that includes many
examples. Fred's website uses an Erlang property-based testing tool called
[PropEr](https://github.com/manopapad/proper) but many of the things he talks
about apply to `ExUnitProperties` as well.

## Options

When an error occurs, StreamData will shrink the generated values to find the smallest set of values that still reproduces the error.
It will then print out the generated values using `inspect/2`.

You can customize the `inspect/2` options used by setting the `:inspect_opts` option in your test config.

    # config/test.exs
    import Config

    config :stream_data,
      inspect_opts: [limit: :infinity]

# `__using__`
*macro* 

Sets up an `ExUnit.Case` module for property-based testing.

# `check`
*macro* 

Runs tests for a property.

This macro provides ad hoc syntax to write properties. Let's see a quick
example to get a feel of how it works:

    check all int1 <- integer(),
              int2 <- integer(),
              int1 > 0 and int2 > 0,
              sum = int1 + int2 do
      assert sum > int1
      assert sum > int2
    end

Everything between `check all` and `do` is referred to as **clauses**. Clauses
are used to specify the values to generate in order to test the properties.
The actual tests that the properties hold live in the `do` block.

Clauses work exactly like they work in the `gen/1` macro.

The body passed in the `do` block is where you test that the property holds
for the generated values. The body is just like the body of a test: use
`ExUnit.Assertions.assert/2` (and friends) to assert whatever you want.

## Options

  * `:initial_size` - (non-negative integer) the initial generation size used
    to start generating values. The generation size is then incremented by `1`
    on each iteration. See the "Generation size" section of the `StreamData`
    documentation for more information on generation size. Defaults to `1`.

  * `:max_runs` - (non-negative integer) the total number of generations to
    run. Defaults to `100`.

  * `:max_run_time` - (non-negative integer) the total number of time (in milliseconds)
    to run a given check for. This is not used by default, so unless a value
    is given then the length of the test will be determined by `:max_runs`.
    If both `:max_runs` and `:max_run_time` are given, then the check will finish at
    whichever comes first, `:max_runs` or `:max_run_time`.

  * `:max_shrinking_steps` - (non-negative integer) the maximum numbers of
    shrinking steps to perform in case a failing case is found. Defaults to
    `100`.

  * `:max_generation_size` - (non-negative integer) the maximum generation
    size to reach. Note that the size is increased by one on each run. By
    default, the generation size is unbounded.

  * `:initial_seed` - (integer) the initial seed used to drive the random generation.
    When `check all` is run with the same initial seed more than once, then every time
    the terms generated by the generators will be the same as all other runs. This is useful
    when you want to deterministically reproduce a result. However, it's usually better
    to leave `:initial_seed` to its default value, which is taken from ExUnit's seed: this
    way, the random generation will follow options like `--seed` used in ExUnit to
    deterministically reproduce tests.

It is also possible to set the values for `:initial_size`, `:max_runs`, `:max_run_time`, and
`:max_shrinking_steps` through your project's config files. This is especially helpful
in combination with `:max_runs` when you want to run more iterations on your continuous
integration platform, but keep your local tests fast:

    # config/test.exs
    import Config

    config :stream_data,
      max_runs: if System.get_env("CI"), do: 1_000, else: 50

## Examples

Check that all values generated by the `StreamData.integer/0` generator are
integers:

    check all int <- integer() do
      assert is_integer(int)
    end

Check that `String.starts_with?/2` and `String.ends_with?/2` always hold for
concatenated strings:

    check all start <- binary(),
              finish <- binary(),
              concat = start <> finish do
      assert String.starts_with?(concat, start)
      assert String.ends_with?(concat, finish)
    end

Check that `Kernel.in/2` returns `true` when checking if an element taken out
of a list is in that same list (changing the number of runs):

    check all list <- list_of(integer()),
              member <- member_of(list),
              max_runs: 50 do
      assert member in list
    end

### Using `check all` in doctests

`check all` can be used in doctests. Make sure that the module where you call
`doctest(MyModule)` calls `use ExUnitProperties`. Then, you can call `check all`
in your doctests:

    @doc """
    Tells if a term is an integer.

        iex> check all i <- integer() do
        ...>   assert int?(i)
        ...> end
        :ok

    """
    def int?(i), do: is_integer(i)

`check all` always returns `:ok`, so you can use that as the return value of
the whole expression.

# `gen`
*macro* 

Syntactic sugar to create generators.

This macro provides ad-hoc syntax to write complex generators. Let's see a
quick example to get a feel of how it works. Say we have a `User` struct:

    defmodule User do
      defstruct [:name, :email]
    end

We can create a generator of users like this:

    email_generator = map({binary(), binary()}, fn {left, right} -> left <> "@" <> right end)

    user_generator =
      gen all name <- binary(),
              email <- email_generator do
        %User{name: name, email: email}
      end

Everything between `gen all` and `do` is referred to as **clauses**. You can write
clauses to specify the values to generate. You can then use those values in the `do` body.
The newly-created generator will generate values that are the return value of the
`do` body using the generated values in the clauses.

### Clauses

As seen in the example above, clauses can be of the following types:

  * **value generation** - they have the form `pattern <- generator` where `generator` must be a
    generator. These clauses take a value out of `generator` on each run and match it against
    `pattern`. Variables bound in `pattern` can be then used throughout subsequent clauses and
    in the `do` body. If `pattern` doesn't match a generated value, it's treated like a filter
    (see the "filtering" clauses described below).

  * **filtering and binding** - they have the form `expression`. If a filtering clause returns
    a truthy value, then the set of generated values that appear before the
    filtering clause is considered valid and generation continues. If the
    filtering clause returns a falsey value, then the current value is
    considered invalid and a new value is generated. Note that filtering
    clauses should not filter out too many times; in case they do, a
    `StreamData.FilterTooNarrowError` error is raised (same as `StreamData.filter/3`).
    Filtering clauses can be used also to assign variables: for example, `a = :foo` is a valid
    clause.

The behaviour of the clauses above is similar to the behaviour of clauses in
`Kernel.SpecialForms.for/1`.

### Body

The return value of the body passed in the `do` block is what is ultimately
generated by the generator return by this macro.

## Shrinking

See the module documentation for more information on shrinking. Clauses affect
shrinking in the following way:

  * filtering clauses affect shrinking like `StreamData.filter/3`
  * value generation clauses affect shrinking similarly to `StreamData.bind/2`

# `pick`

```elixir
@spec pick(StreamData.t(a)) :: a when a: term()
```

Picks a random element generated by the `StreamData` generator `data`.

This function uses the current ExUnit seed to generate a random term from `data`. The generation
size (see [*Generation size*](StreamData.html#module-generation-size)) is chosen at random between in `1..100`. If you want finer
control over the generation size, you can use functions like `StreamData.resize/2` to resize
`data` or `StreamData.scale/2` to scale the generation size.

## Examples

    ExUnitProperties.pick(StreamData.integer())
    #=> -21

# `property`
*macro* 

Defines a not-implemented property test with a string.

Provides a convenient macro that allows a property test to be defined with a
string, but not yet implemented. The resulting property test will always
fail and print a "Not implemented" error message. The resulting test case is
also tagged with `:not_implemented`.

This behavior is similar to `ExUnit.Case.test/1`.

## Examples

    property "this will be a property test in the future"

# `property`
*macro* 

Defines a property and imports property-testing facilities in the body.

This macro is similar to `ExUnit.Case.test/3`, except that it denotes a
**property**. In the given body, all the functions exposed by `StreamData` are
imported, as well as `check/2`.

When defining a test whose body only consists of one or more `check/2` calls,
it's advised to use `property/3` so as to clearly denote and scope properties.
Doing so will also improve reporting.

## Examples

    use ExUnitProperties

    property "reversing a list doesn't change its length" do
      check all list <- list_of(integer()) do
        assert length(list) == length(:lists.reverse(list))
      end
    end

---

*Consult [api-reference.md](api-reference.md) for complete listing*