Advantages:
- Pure test evaluations
- The test framework can now be used by flakes that extend nix-bitcoin
- Most features of `run-tests.sh` are now accessible via `nix build`/`nix run`.
We keep `run-tests.sh` for advanced features like `scenarioOverridesFile` and adhoc scenarios.
Other changes:
- `run-tests.sh` now builds aggregate VM tests like `basic` or
`buildable` by creating all VMs in a single evaluation.
This speeds up the tests and eases debugging by separating the eval and build steps.
- Use the new `nix` CLI which has improved build output logging
by prefixing output lines with the origin drv name.
Avoid adding flake resource paths to the store (via string
interpolation).
This reduces performance and can lead to modules getting imported
twice, once through a local path and once through a store path.
This might not be needed in a future Nix release, in which case we can
revert this.
The result of `import tests.nix {}` is now an attrset of tests.
This makes it easier and more efficient to evaluate or build multiple
tests in one call to `nix build`.
Simplify tests.nix by removing the large module args scope in favor of
self-contained scenario module definitions.
Add CPU core and memory size defaults to the test configuration to
enable building tests without `run-tests.sh`.
Add the following top-level args to tests.nix:
- `extraScenarios` to provide a nix-level way to define extra scenarios.
- `pkgs` to allow building tests with custom pkgs or systems.
1. fixedTest: Remove some unneeded layers of function calls.
2. Don't add a modified version of `black` to the global pkgs set.
Tests should not affect the pkgs of the tested system modules.
Fix the driver build script instead by adding an extra arg to the
call to `black`.
This improves debugging and experimenting by making it easy to compose fine-grained
scenarios that have specific tests and features enabled.
The VM test output now includes the subtest name and duration.
Remove the 'raise Exception()' hack for interactive mode.
Run 'banlist-and-restart' test before 'backups'. This speeds up the test
by avoiding an extra shutdown of all bitcoin-related services.
Erik Arvstedt
nixbitcoin
Jonas Nick