Commit Graph

1465 Commits

Author SHA1 Message Date
Erik Arvstedt
52aaa8388e
fetch-release: write error messages to stderr
Previously, when used to update `nix-bitcoin-release`, the error
wasn't displayed but instead written to `nix-bitcoin-release`.

Also, show curl error messages.
2021-09-14 19:56:23 +02:00
Erik Arvstedt
c8774375d3
modules: use consistent service variables
Benefits of adding top-level variables for used services:
- Makes it obvious which other services are referenced by a service
- Less code

We already do this in many other places.
2021-09-13 13:41:47 +02:00
Erik Arvstedt
ad97c268c6
modules: move user/group options to bottom
These are insignificant, generic options; place them above readonly options.
We already do this in other services.

Also move user/group config to bottom in spark-wallet.
2021-09-13 13:41:47 +02:00
Erik Arvstedt
27c45b82cc
modules: move options to the top
This greatly improves readability and makes it easier to discover options.

This commit was genereated by running the following script inside the
repo root dir:

#!/usr/bin/env ruby

def transform(src)
  return false if src.include?('inherit options;')

  success = false

  options = nil
  src.sub!(/^  options.*?^  }.*?;/m) do |match|
    options = match
    "  inherit options;"
  end
  return false if !options

  src.sub!(/^with lib;\s*let\n+/m) do |match|
    success = true
    <<~EOF
      with lib;
      let
      #{options}

    EOF
  end

  success
end

Dir['modules/**/*.nix'].each do |f|
  src = File.read(f)
  if transform(src)
    puts "Changed file #{f}"
    File.write(f, src)
  end
end
2021-09-13 13:41:47 +02:00
Erik Arvstedt
731cf647ff
modules: remove unneeded use of options module arg
Needed by the following commit.
2021-09-13 13:41:47 +02:00
Jonas Nick
cf70d05be0
Merge fort-nix/nix-bitcoin#385: Misc. improvments
fdcb68e96e examples/shell.nix: add new commands (Erik Arvstedt)
a2466b1127 secrets: allow extending generate-secrets (Erik Arvstedt)
24fd1e9bdc improve examples/shell.nix (Erik Arvstedt)
8a757e0486 push-release.sh: improve --dry-run mode (Erik Arvstedt)
82a2b148d8 secrets: minor fixes (Erik Arvstedt)
e1e3d8a92b secrets: simplify cert generation (Erik Arvstedt)
2c8e29b35b lnd: extract option `certPath` (Erik Arvstedt)
be12a49933 lightning-pool/loop: extract lnd variable (Erik Arvstedt)
955b44404c delete helper/fetch-channel (Erik Arvstedt)
5087ce245f minor cleanups (Erik Arvstedt)
0d2db4e79f backups: add option `postgresqlDatabases` (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK fdcb68e96e
  jonasnick:
    ACK  fdcb68e96e

Tree-SHA512: a0fef5b6f8704a445b0e381a1713c14d1447e16798e7035bb005d2b61c4cde208f96fc6f152238b6ea2e9080c04fffe7f841073fa41a5c1e0597204e9ed805c2
2021-09-12 13:22:10 +00:00
Erik Arvstedt
fdcb68e96e
examples/shell.nix: add new commands 2021-09-12 11:29:54 +02:00
Erik Arvstedt
a2466b1127
secrets: allow extending generate-secrets
`generate-secrets` is no longer a monolithic script. Instead, it's
composed of the values of option `nix-bitcoin.generateSecretsCmds`.

This has the following advantages:
- generate-secrets is now extensible by users
- Only secrets of enabled services are generated
- RPC IPs in the `lnd` and `loop` certs are no longer hardcoded.

Secrets are no longer automatically generated when entering nix-shell.
Instead, they are generated before deployment (via `krops-deploy`)
because secrets generation is now dependant on the node configuration.
2021-09-12 11:29:54 +02:00
Erik Arvstedt
24fd1e9bdc
improve examples/shell.nix
The user's local node configuration directory usually contains a copy of
examples/shell.nix.

1. Move the shell implementation from shell.nix to nix-bitcoin/helper/makeShell.nix
   Because the shell is no longer defined locally in the user's config
   directory, we can now ship new shell features via nix-bitcoin updates.

2. Simplify examples/nix-bitcoin-release.nix
   nix-bitcoin-release.nix, as generated via `fetch-release`, now
   contains a simple fetchTarball statement which can be directly imported.
   This allows us to get rid of the extra `nix-bitcoin-unpacked` derivation
   which adds a dependency on the user's local nixpkgs.

   To keep `fetch-release` as simple as possible for easy auditing, we just
   fetch and verify a `nar-hash.txt` file that is now uploaded
   via `push-release.sh`.

A migration guide for updating the user's local `shell.nix` is
automatically printed when the user starts a new shell after updating
nix-bitcoin.
This is achieved by throwing an error in `generate-secrets`, which is called
on shell startup.

This commit is required to deploy the new extensible `generate-secrets`
mechanism introduced in the next commit.
2021-09-12 11:29:54 +02:00
Erik Arvstedt
8a757e0486
push-release.sh: improve --dry-run mode
- Don't require OAUTH_TOKEN
- Skip interactive query
2021-09-11 15:07:24 +02:00
Erik Arvstedt
82a2b148d8
secrets: minor fixes
- Improve comment.
- `secretsSetupMethod` is not internal because it can be set to "manual"
  by the user.
2021-09-11 15:07:24 +02:00
Erik Arvstedt
e1e3d8a92b
secrets: simplify cert generation
- Remove openssl.cnf which includes many unused settings.
- Generate the key and cert files with a single call to openssl.
  - Option `-nodes` ("no DES") disables encryption of the key file.
  - Option `-addext` is used to specify `subjectAltName` settings
    that were previously defined by openssl.cnf.

The key type is unchanged.
Certificate changes:
- Certificate duration is now 10 years
- Organization (subj 'O') is now 'loop' instead of 'loopd' for
  lightning-loop to simplify the code.
  For reference, the org. name in auto-generated loop certs is
  "loop autogenerated cert".
- The certificate now includes all default x509v3 extensions.
  These were previously restricted to just `subjectAltName` by openssl.cnf.
  We now use the openssl defaults for simplicity.
2021-09-11 15:07:24 +02:00
Erik Arvstedt
2c8e29b35b
lnd: extract option certPath
Improves service encapsulation.
2021-09-11 15:07:24 +02:00
Erik Arvstedt
be12a49933
lightning-pool/loop: extract lnd variable 2021-09-11 15:07:24 +02:00
Erik Arvstedt
955b44404c
delete helper/fetch-channel
This script was obsoleted by switching to flakes.
2021-09-11 15:07:23 +02:00
Erik Arvstedt
5087ce245f
minor cleanups
- btcpayserver: remove unneeded trailing semicolons

- krops/get-sha256:
  `tail` is unneeded because `nix-prefetch-url` just outputs a single
  line containing the hash.
2021-09-11 15:07:23 +02:00
Erik Arvstedt
0d2db4e79f
backups: add option postgresqlDatabases
This simplifies defining postgresql backups.
This change is covered by tests.py.
2021-09-11 15:07:23 +02:00
Jonas Nick
faa7831708
Merge fort-nix/nix-bitcoin#384: joinmarket: Update patch hash
c35e96a553 joinmarket: update patch hash (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK c35e96a553

Tree-SHA512: 40f1bbe6990fa940c0153e00719d2e56e20ce9dc01a5975c48e0da070544b873cafa6cb9aeb860498aad8c104c379f3e368496c96cc569966963a707f478178c
2021-09-06 11:39:40 +00:00
Erik Arvstedt
c35e96a553
joinmarket: update patch hash
The patch hash has changed due to an update of the PR branch.
The PR has now been merged.
2021-09-05 22:33:17 +02:00
Jonas Nick
fb2fec852b
Merge fort-nix/nix-bitcoin#381: run-tests: Fix 'eval' command for newer versions of nix
32ce9d0ff4 run-tests: fix 'eval' command for newer versions of nix (Erik Arvstedt)
926f1febb7 make-container: update extra-container version (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK 32ce9d0ff4

Tree-SHA512: 16665901eea84d8493c44a5334b02aa470820dfb76a186caa9e7e90b2c5688aea12348674e30c4b98b32b113e7613d956b36a40b23a7a4e94c8f57461ca92e0a
2021-09-05 16:10:48 +00:00
Erik Arvstedt
32ce9d0ff4
run-tests: fix 'eval' command for newer versions of nix
There's no common `nix` command argument syntax for eval'ing a nix
expression that supports both older and newer (flake support) versions of nix.
So fall back to nix-instantiate.
2021-09-04 08:17:38 +02:00
Erik Arvstedt
926f1febb7
make-container: update extra-container version
Keep this file in sync with the latest extra-container update.
2021-09-04 08:17:38 +02:00
Jonas Nick
1c5154cfcf
Merge fort-nix/nix-bitcoin#380: joinmarket: 0.8.3 -> 0.9.1
9730be9282 joinmarket-yieldgenerator: simplify start script (Erik Arvstedt)
179b86d19c joinmarket: allow recreating wallet from seed (Erik Arvstedt)
7c5ef32b50 versioning: move list of changes to the top (Erik Arvstedt)
b15d71605e joinmarket: fix leaking passwords (Erik Arvstedt)
5c14453389 joinmarket-ob-watcher: don't assert running, assert rpc failure (nixbitcoin)
00a0759884 joinmarket-ob-watcher: extra permissions & functionality for fidelity bonds (nixbitcoin)
d7f9e33e1c joinmarket-ob-watcher: move resource files to extra dir (Erik Arvstedt)
32d0f08d77 docs: fix usage steps numbering (nixbitcoin)
e95abf6c7e joinmarket: 0.8.3 -> 0.9.1 (nixbitcoin)

Pull request description:

ACKs for top commit:
  erikarvstedt:
    ACK 9730be9282

Tree-SHA512: b6e693d3e293ad3d590479eefdb5d1e144a5d7b16c4160fc7cf4ba890a78b6e94b170c43f61a541363a17dddc3cf4441917270e23ece643b7cff4c0cb4581337
2021-08-30 18:38:41 +00:00
Erik Arvstedt
9730be9282
joinmarket-yieldgenerator: simplify start script 2021-08-30 13:37:05 +02:00
Erik Arvstedt
179b86d19c
joinmarket: allow recreating wallet from seed
This allows users to easily upgrade their wallets to use Fidelity Bonds.
2021-08-30 13:37:05 +02:00
Erik Arvstedt
7c5ef32b50
versioning: move list of changes to the top
Improves readability.
2021-08-30 13:37:05 +02:00
Erik Arvstedt
b15d71605e
joinmarket: fix leaking passwords
Previously, `bitcoin-rpcpassword-privileged` and `jm-wallet-password` were
passed as world readable arguments to sed and jm-genwallet subprocesses.
2021-08-30 13:37:04 +02:00
nixbitcoin
5c14453389
joinmarket-ob-watcher: don't assert running, assert rpc failure
joinmarket-ob-watcher now makes extensive use of bitcoind because of
JoinMarket's new fidelity bond functionality. Therefore it fails on
non-synced nodes, as those in the test suite. We now test that the
service fails with the correct error, rather than asserting that it is
running.
2021-08-30 13:37:04 +02:00
nixbitcoin
00a0759884
joinmarket-ob-watcher: extra permissions & functionality for fidelity bonds 2021-08-30 13:37:04 +02:00
Erik Arvstedt
d7f9e33e1c
joinmarket-ob-watcher: move resource files to extra dir
Don't clutter joinmarket/bin with ob-watcher resource files.
2021-08-30 13:37:04 +02:00
nixbitcoin
32d0f08d77
docs: fix usage steps numbering 2021-08-30 09:02:30 +00:00
nixbitcoin
e95abf6c7e
joinmarket: 0.8.3 -> 0.9.1 2021-08-30 09:02:26 +00:00
Jonas Nick
4040e4fd32
Merge fort-nix/nix-bitcoin#379: Add flake support
dde04f8cbe update nixpkgs-unstable (Erik Arvstedt)
87df809a88 add helper/update-flake.sh (Erik Arvstedt)
f7c2133250 add flake support (Erik Arvstedt)
de77281cba pkgs: import pinned nixpkgs in default.nix (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK dde04f8cbe
  jonasnick:
    ACK dde04f8cbe

Tree-SHA512: 251bbaf748bc429e1f094bf37f1bbf00eed91108f165e378f3ac4de817d42f3e1e4e8c1bbf79ea81447f24d0c172e81fd4eec98164379ea27095d6ba5420af6c
2021-08-28 06:51:25 +00:00
Erik Arvstedt
dde04f8cbe
update nixpkgs-unstable
Includes:
btcpayserver: 1.1.2 -> 1.2.0
lightning-loop: 0.14.2-beta -> 0.15.0-beta
nbxplorer: 2.1.52 -> 2.1.58
2021-08-26 12:45:10 +02:00
Erik Arvstedt
87df809a88
add helper/update-flake.sh
This greatly simplifies updating nixpkgs.
See the comment at the top of update-flake.sh for a description.
2021-08-26 12:45:10 +02:00
Erik Arvstedt
f7c2133250
add flake support
This change is fully backwards compatible.

We continue to use the standard non-flake evaluation mode in our
examples and internal tooling until the flakes design has stabilized.

'clightning-plugins = pkgs.recurseIntoAttrs' in pkgs/default.nix is
needed by flake-utils.lib.flattenTree in flake.nix.
It transforms the packages in `clightning-plugins` to top-level packages
named like `clightning-plugins/summary`. (The flake attr `packages`
must be a non-nested attrset of derivations.)
2021-08-26 12:45:10 +02:00
Erik Arvstedt
de77281cba
pkgs: import pinned nixpkgs in default.nix
pkgs/default.nix now explicitly specifies all its dependencies as arguments.
This is required for flake support.

Also simplify pinned.nix and python-packages by removing unused attrs.
2021-08-16 10:43:07 +02:00
Jonas Nick
9b24a74b23
Merge fort-nix/nix-bitcoin#378: Misc. improvements
b0c66c41e1 tests: add container-minimal example (Erik Arvstedt)
a8a8b9ce4d backups: backup NixOS uid, gid mappings (Erik Arvstedt)
ee8b83681b modules: document module dependencies (Erik Arvstedt)
9f7d048769 modules: move assertion to lnd.nix (Erik Arvstedt)
cce9a3f6b2 modules: move nix-bitcoin options to file 'nix-bitcoin.nix' (Erik Arvstedt)
fdc278a0b8 lib: fix comment (Erik Arvstedt)
13b4650e84 versioning: add usage comment (Erik Arvstedt)
ca3c7a281b secrets: mark option 'secretsSetupMethod' as internal (Erik Arvstedt)
f9a0fd7a17 nodeinfo: fix indentation (Erik Arvstedt)
4ece606e8b examples/minimal-configuration: improve comment (Erik Arvstedt)
6de9aba854 run-tests: quote scriptDir (Erik Arvstedt)
1ef8cbb384 joinmarket: fix allowRunAsUsers setting (Erik Arvstedt)
fb36f2abe5 joinmarket-ob-watcher: use consistent mode formatting (Erik Arvstedt)
f14af1fc48 treewide: use consistent echo message formatting (Erik Arvstedt)
b8043d3db5 treewide: use consistent bash script indentation (Erik Arvstedt)
c758d68ea4 lib: rename privileged -> rootScript (Erik Arvstedt)
1c3735b600 examples/README: add nixbitcoin.org server repo (Erik Arvstedt)
c041079ae1 configuration.nix: reorder sections (Erik Arvstedt)
3734ab38a6 configuration.nix: improve wording and formatting (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK b0c66c41e1

Tree-SHA512: 11411e13de4ed8a6c8c942b2843b9ad45f3340a0682fe673a5cee18db93bb75f995c717eb9624f6a754615c508d089c03bf1790848c5112a7d5e9546d583fb24
2021-08-15 21:28:01 +00:00
Erik Arvstedt
b0c66c41e1
tests: add container-minimal example 2021-08-15 22:42:03 +02:00
Erik Arvstedt
a8a8b9ce4d
backups: backup NixOS uid, gid mappings
Now that service uid, gid mappings are included in the backups, along
with the service data dirs, we can remove 'chown -R' for
clightning and liquidd data dirs.

Note that we used 'chown -R' only for these two services, while this
approach would have been relevant for all services with data dirs.
2021-08-15 22:40:35 +02:00
Erik Arvstedt
ee8b83681b
modules: document module dependencies 2021-08-15 22:40:35 +02:00
Erik Arvstedt
9f7d048769
modules: move assertion to lnd.nix
nix-bitcoin.nix is now no longer dependent on clightning.nix and lnd.nix.
Due to condition '!(config.services ? clightning)' lnd.nix still
doesn't depend on clightning.nix.

Also fix the assertion message by renaming clightning.bindPort to clightning.port.
2021-08-15 22:40:35 +02:00
Erik Arvstedt
cce9a3f6b2
modules: move nix-bitcoin options to file 'nix-bitcoin.nix'
This allows modules.nix to consist only of a list of modules.
2021-08-15 22:40:35 +02:00
Erik Arvstedt
fdc278a0b8
lib: fix comment 2021-08-15 11:29:36 +02:00
Erik Arvstedt
13b4650e84
versioning: add usage comment 2021-08-15 11:29:36 +02:00
Erik Arvstedt
ca3c7a281b
secrets: mark option 'secretsSetupMethod' as internal 2021-08-15 11:29:36 +02:00
Erik Arvstedt
f9a0fd7a17
nodeinfo: fix indentation 2021-08-15 11:29:36 +02:00
Erik Arvstedt
4ece606e8b
examples/minimal-configuration: improve comment 2021-08-15 11:29:35 +02:00
Erik Arvstedt
6de9aba854
run-tests: quote scriptDir
scriptDir may contain spaces.
2021-08-15 11:29:35 +02:00
Erik Arvstedt
1ef8cbb384
joinmarket: fix allowRunAsUsers setting
This option requires user names instead of groups.
2021-08-15 11:29:35 +02:00