Commit Graph

57 Commits

Author SHA1 Message Date
Erik Arvstedt af115d746b
internal scripts: use pinned, cached pkgs
Instead of setting up the script PATH via nix-shell, use
`nix shell` with inputs from the nix-bitcoin flake.
Advantages:
- Uses the nixpkgs version from the nix-bitcoin flake instead of
  `<nixpkgs>` from the user env (NIX_PATH), so the script runtime
  env is reproducible.
- The pkg derivations for the runtime env are cached, which greatly
  increases script startup speed.

This commit was generated by running the following script inside the
repo root dir:

def transform(path, src)
  if src =~ /#! *nix-shell +-i +bash +-p +(.*)/
    pkgs = $1
    if src =~ /^.*?(set -e.*?pipefail)\n/
      set_statement = $1
      src.sub!($&, '')
    end
    src.sub!(/\A.*?#! *nix-shell.*?\n/m, '')

    parents = ([ '..' ] * (path.split('/').count - 1)).join('/')

    [
      '#!/usr/bin/env bash',
      *set_statement,
      %(. "${BASH_SOURCE[0]%/*}/#{parents}/helper/run-in-nix-env" "#{pkgs}" "$@"),
      nil,
      src
    ].join("\n")
  end
end

Dir['**/*.sh'].each do |f|
  src = File.read(f)
  if new_src = transform(f, src)
    puts "Changed file #{f}"
    File.write(f, new_src)
  end
end
2022-08-28 23:49:12 +02:00
Otto Sabart f184bb34e6
shellcheck: fix lint of scripts in tests 2022-08-28 18:25:37 +02:00
Otto Sabart 9a92d29111
tests: add shellcheck 2022-08-28 18:25:37 +02:00
Erik Arvstedt 952cd3773f
run-tests: support tests with multiple VMs in `run` 2022-08-05 13:48:36 +00:00
nixbitcoin 17507835fc
clightning: native database replication
Don't put `clightning.replication` options in
`examples/configuration.nix` until it is more "battle-tested."
2022-08-05 13:48:24 +00:00
Erik Arvstedt 2ae2eea54e
run-tests: fix command `debug` on NixOS 22.05 2022-07-05 10:13:55 +02:00
Erik Arvstedt 98bc0917d7
run-tests: fix `vm` cmd
Previously, qemu failed with error
`restrict=on : Parameter 'restrict' expects 'on' or 'off'`
due to the trailing space after `on`.
2022-01-04 22:30:35 +01:00
Erik Arvstedt f13ffe0146
run-tests: make `nixosSearch` compatible with `--copy-src` 2021-12-15 10:58:04 +01:00
Erik Arvstedt 0fd3be5343
move ci/ to test/ 2021-12-12 16:21:11 +01:00
Erik Arvstedt eaf98f92db
test: add test `nixosSearch`
This checks that creating module docs for search.nixos.org succceeds.
Errors can happen when complex `default` values can't be evaluated
or on malformed Docbook XML in descriptions.
2021-12-12 16:21:02 +01:00
Erik Arvstedt 77d8c00c53
run-tests.sh: extract `checkFlakeSupport`
Required by the following commit.
2021-12-12 16:20:40 +01:00
Erik Arvstedt 96df81a4d0
run-tests: move `flake` next to other tests 2021-12-12 16:20:40 +01:00
Jonas Nick daeea4012e
tests: remove pkgsUnstable
The latest nixpkgs-unstable update would require node-based packages to
distinguish between stable and unstable when building. Instead of dealing with
that complexity, we will only guarantee compatability of our packages with
stable.
2021-11-28 19:16:21 +00:00
Erik Arvstedt 32ce9d0ff4
run-tests: fix 'eval' command for newer versions of nix
There's no common `nix` command argument syntax for eval'ing a nix
expression that supports both older and newer (flake support) versions of nix.
So fall back to nix-instantiate.
2021-09-04 08:17:38 +02:00
Erik Arvstedt f7c2133250
add flake support
This change is fully backwards compatible.

We continue to use the standard non-flake evaluation mode in our
examples and internal tooling until the flakes design has stabilized.

'clightning-plugins = pkgs.recurseIntoAttrs' in pkgs/default.nix is
needed by flake-utils.lib.flattenTree in flake.nix.
It transforms the packages in `clightning-plugins` to top-level packages
named like `clightning-plugins/summary`. (The flake attr `packages`
must be a non-nested attrset of derivations.)
2021-08-26 12:45:10 +02:00
Erik Arvstedt b0c66c41e1
tests: add container-minimal example 2021-08-15 22:42:03 +02:00
Erik Arvstedt 6de9aba854
run-tests: quote scriptDir
scriptDir may contain spaces.
2021-08-15 11:29:35 +02:00
Erik Arvstedt 3cce822c64
run-tests: add 'vm' command 2021-03-26 23:23:22 +01:00
Erik Arvstedt 44439e2a81
tests: optimize building multiple tests at once
The result of `import tests.nix {}` is now an attrset of tests.
This makes it easier and more efficient to evaluate or build multiple
tests in one call to `nix build`.

Simplify tests.nix by removing the large module args scope in favor of
self-contained scenario module definitions.

Add CPU core and memory size defaults to the test configuration to
enable building tests without `run-tests.sh`.

Add the following top-level args to tests.nix:
- `extraScenarios` to provide a nix-level way to define extra scenarios.
- `pkgs` to allow building tests with custom pkgs or systems.
2021-03-22 14:35:29 +01:00
Erik Arvstedt 9ca52af523
tests: improve make-test-vm.nix
1. fixedTest: Remove some unneeded layers of function calls.

2. Don't add a modified version of `black` to the global pkgs set.
   Tests should not affect the pkgs of the tested system modules.
   Fix the driver build script instead by adding an extra arg to the
   call to `black`.
2021-03-22 14:35:29 +01:00
Erik Arvstedt e8b47f099c
examples: add deploy-krops.sh 2021-03-15 19:02:58 +01:00
Erik Arvstedt 0a2c8e4864
run-tests: add option --copy-src 2021-02-08 12:20:20 +01:00
Erik Arvstedt 7533f12ef1
bitcoind, clightning, run-tests: minor refactoring
bitcoind: use builtins.toFile
clightning: use boolToString
run-tests: remove leftover var
2021-02-07 22:39:05 +01:00
Erik Arvstedt 44546561fc
run-tests: allow defining scenarios via cmdline args
This simplifies running self-contained scenarios for testing and debugging.
2021-01-30 11:38:47 +01:00
nixbitcoin bcedf69549
readme: update and split into various parts 2020-12-30 15:59:22 +00:00
Erik Arvstedt 3403795c86
tests: add example scripts 2020-12-18 19:56:56 +01:00
Erik Arvstedt ff94985b8b
tests: add test 'hardened' 2020-12-18 19:56:56 +01:00
Erik Arvstedt 9977fa69af
ci: use run-tests.sh 2020-12-11 13:27:06 +01:00
Erik Arvstedt a82f0f5f48
add test 'pkgsUnstable'
Included in 'basic' tests.
Function 'doBuild' is needed by the following commit.
2020-12-11 13:27:05 +01:00
Erik Arvstedt 95bc1237e2
run-tests: rename testDir -> scriptDir 2020-12-11 13:27:05 +01:00
Erik Arvstedt 7265742655
run-tests: add 'instantiate' command
Useful for diffing test derivations.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 8cbdef8bf6
run-tests: fix CLI
Restore the original behavior that was accidentally changed:
When no args are given, run the basic test suite.
Otherwise, run the given command with default scenario 'default'.
Previously, `run-tests.sh build` ran the basic test suite instead of
building the default scenario.
2020-12-11 13:27:04 +01:00
Erik Arvstedt 1c0233c0a8
use Cirrus CI
- Make more economic use of the free CI resources by removing redundant build tasks:
  - Build unstable pkgs in a single separate task ("pkgs_unstable").
  - All stable pkgs are implicitly built by the modules tests.
- The build script (ci/build.sh) can now be executed locally for easier
  debugging.
- Use an explicit 'cachix push' command instead of helper/wait-for-network-idle.rb.
  This is simpler and more reliable.
2020-12-06 19:07:54 +01:00
Erik Arvstedt 0e2ff948d3
test: add scenario 'netnsRegtest'
The 'basic' test command now cover regtest mode and using nix-bitcoin without
the secure-node preset.
2020-10-29 21:21:27 +01:00
Erik Arvstedt 9951f10e74
test: add scenario 'regtest' 2020-10-16 23:55:13 +02:00
Erik Arvstedt bfed10b2fa
run-tests: add command 'all'
'all' includes test 'full', which now succeeds.
2020-10-16 16:46:56 +02:00
Erik Arvstedt 0a6b9beda5
run-tests: simplify setting default scenario 2020-10-16 16:46:56 +02:00
Erik Arvstedt 04075b108c
test: use QEMU from stable nixpkgs 2020-10-16 15:53:34 +02:00
Erik Arvstedt 03f8dbba47
test: add non-secure-node eval test
This tests that the modules work without the secure-node template.

The test currently fails at runtime, but evaluating already helps
catching module-related errors.
2020-10-16 15:53:34 +02:00
Erik Arvstedt bb763d6a26
run-tests: add 'eval' command 2020-10-16 15:53:34 +02:00
Erik Arvstedt c9251e72a1
README: add run-tests.sh to examples 2020-10-16 15:53:33 +02:00
Erik Arvstedt fcc67da9f4
test: add container support 2020-10-16 15:53:33 +02:00
Erik Arvstedt e99b7edb8e
run-tests: pass script args verbatim to command without word splitting
Needed for the upcoming 'container' command
2020-10-16 15:53:33 +02:00
Erik Arvstedt e7c397a485
run-tests: rename scriptDir -> testDir
Needed for container support.
2020-10-16 15:53:32 +02:00
Erik Arvstedt b552d17d55
run-tests: fix arg error messages
$1 was not substituted due to single quotes.
2020-10-16 15:53:32 +02:00
Erik Arvstedt 1e18d3ea3b
test: improve modularization
This improves debugging and experimenting by making it easy to compose fine-grained
scenarios that have specific tests and features enabled.

The VM test output now includes the subtest name and duration.

Remove the 'raise Exception()' hack for interactive mode.

Run 'banlist-and-restart' test before 'backups'. This speeds up the test
by avoiding an extra shutdown of all bitcoin-related services.
2020-10-11 19:40:08 +02:00
Erik Arvstedt 14d2d97ba6
test: rename scenario withnetns -> netns
This makes the naming consistent with scenarios added in later
commits.
2020-09-30 11:26:41 +02:00
Erik Arvstedt 45bcbf683d
test: rename test.nix -> tests.nix
The plural is consistent with tests.py and run-tests.sh
2020-09-30 11:26:40 +02:00
Erik Arvstedt e5fb3f6a7f
run-tests: document how to pass extra build args 2020-08-25 14:58:04 +02:00
Erik Arvstedt df790f6766
run-tests: allow linking test build results for all scenarios 2020-08-25 14:58:04 +02:00