Advantages:
- Pure test evaluations
- The test framework can now be used by flakes that extend nix-bitcoin
- Most features of `run-tests.sh` are now accessible via `nix build`/`nix run`.
We keep `run-tests.sh` for advanced features like `scenarioOverridesFile` and adhoc scenarios.
Other changes:
- `run-tests.sh` now builds aggregate VM tests like `basic` or
`buildable` by creating all VMs in a single evaluation.
This speeds up the tests and eases debugging by separating the eval and build steps.
- Use the new `nix` CLI which has improved build output logging
by prefixing output lines with the origin drv name.
Don't auto-switch to root when executing make-container.sh, because
auto root switching is also implemented in extra-container.
Besides simplifying the code, this is useful for a later commit that
introduces flakes-based container building.
With this change, the container is built under the regular user
instead of root, thereby utilizing the user's regular fetcher and
evaluation caches.
Avoid adding flake resource paths to the store (via string
interpolation).
This reduces performance and can lead to modules getting imported
twice, once through a local path and once through a store path.
This might not be needed in a future Nix release, in which case we can
revert this.
In this case, `git commit --amend` fails.
The state of a .git repo existing without a branch being checked out
can be reached through the following sequence of events (which I just
seem to have triggered):
1. The script calls `git init`
2. The user interrupts the script before `git commit` is finished
The result of `import tests.nix {}` is now an attrset of tests.
This makes it easier and more efficient to evaluate or build multiple
tests in one call to `nix build`.
Simplify tests.nix by removing the large module args scope in favor of
self-contained scenario module definitions.
Add CPU core and memory size defaults to the test configuration to
enable building tests without `run-tests.sh`.
Add the following top-level args to tests.nix:
- `extraScenarios` to provide a nix-level way to define extra scenarios.
- `pkgs` to allow building tests with custom pkgs or systems.
1. fixedTest: Remove some unneeded layers of function calls.
2. Don't add a modified version of `black` to the global pkgs set.
Tests should not affect the pkgs of the tested system modules.
Fix the driver build script instead by adding an extra arg to the
call to `black`.
Since nixpks 20.09, the test output is just an empty directory.
Restore saving the log output and linking to the driver.
Without linking to the driver, the driver is eligible for
garbage collection after running a test via `run-tests.sh --out-link-prefix`,
which implies lengthy driver rebuilds.
This improves debugging and experimenting by making it easy to compose fine-grained
scenarios that have specific tests and features enabled.
The VM test output now includes the subtest name and duration.
Remove the 'raise Exception()' hack for interactive mode.
Run 'banlist-and-restart' test before 'backups'. This speeds up the test
by avoiding an extra shutdown of all bitcoin-related services.
Erik Arvstedt
Otto Sabart
nixbitcoin
Jonas Nick