Commit Graph

1351 Commits

Author SHA1 Message Date
Erik Arvstedt
4d5bc810eb
secrets: fix setup-secrets in case of no secrets
Previously, the glob (*) returned '*' when no files existed in the
secrets dir, leading to error `chown: cannot access '*'`.

Now `unprocessedFiles` is empty when there are no secrets.

Also remove the unneeded sorting of `unprocessedFiles` and
remove redundant leading zero in the default mode.
2021-10-04 00:33:27 +02:00
Erik Arvstedt
e61c743644
test: add option extraTestScript
This allows extending tests.py.
2021-10-04 00:33:27 +02:00
Erik Arvstedt
2cf12d8765
README: minor fixes
- `packages and dependencies are pinned`
  Reworded and moved to `Integrity`
- `most packages are built from the NixOS stable channel...`
  Removed because this is false
- `builds happen in a sandboxed environment`
  Removed because this depends on the user's local nix configuration
  and cannot be enforced by nix-bitcoin
2021-10-04 00:33:27 +02:00
Erik Arvstedt
e57ab83a51
docs/hardware: update
- Update pruning info
- Add Raspberry Pi 4
  Mentioned here: https://github.com/fort-nix/nix-bitcoin/issues/39#issuecomment-917418830
2021-10-04 00:33:27 +02:00
Erik Arvstedt
1b597f92a6
docs/hardware: add line breaks 2021-10-04 00:33:27 +02:00
Erik Arvstedt
a92d6a8e80
netns: expose bridgeIp as an option
Previously, this variable was not accessible to other modules.
2021-10-04 00:33:26 +02:00
Erik Arvstedt
f36df8f563
secure-node: remove redundant bitcoind settings
- `discover` is automatically disabled by bitcoind because we're
   setting `externalip` via the `nix-bitcoin.onionServices` mechanism
- `bech32` is bitcoind's default addresstype
2021-10-04 00:33:26 +02:00
Erik Arvstedt
09169365d8
liquid: remove unused features
- `hexStr` is unused
- Simplify ExecStart options
- Quote `dataDir`
- Remove unneeded `pidFile` setting
2021-10-04 00:33:26 +02:00
Erik Arvstedt
82d910e937
nbxplorer: fix bitcoind, liquidd settings
- Add nbxplorer to whitelists.
  This is recommended by the nbxplorer docs and guarantees that nbxplorer
  can always p2p-connect to bitcoind/liquidd.

- Enable bitcoind/liquidd p2p servers via `listen`.
2021-10-04 00:33:26 +02:00
Erik Arvstedt
f61e928139
services: support 0.0.0.0/:: in address options
Previously, client services didn't decode these special INADDR_ANY
addresses and failed to connect.
2021-10-04 00:33:26 +02:00
Erik Arvstedt
1848c3dd98
btcpayserver: minor improvements
- Quote datadir
- Extract liquidd service variable
- Move btcpayserver below liquid in modules list because it depends
  on liquid
2021-10-01 11:52:57 +02:00
Erik Arvstedt
e561637600
minor fixes
- bitcoind: Remove obsolete defaultText
- clightning: Fix description
  Option `address` can't be used to specify a socket path because it's
  used explicitly as an IP address in many places.
- lnd: Break up overlong line
  This is required by commit `services: support 0.0.0.0/:: in `address` options`
- nix-bitcoin.nix: Formatting
- secrets: Improve descriptions
2021-10-01 11:52:56 +02:00
Jonas Nick
e86d1ff0ff
Merge fort-nix/nix-bitcoin#399: examples: Fix shell on non-Linux systems
46af0b1a8d examples: fix shell on non-Linux systems (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK 46af0b1a8d

Tree-SHA512: bc3c24c75b2d26a30f9f4f88700da8a78d8276802b47ddaf3d52097ba6ed16a7b7ca662085e2775a2cd1654b07c1759c36918ee0ab05eb90fbfeac3265576510
2021-09-27 09:14:44 +00:00
Jonas Nick
a4ac735cd3
Merge fort-nix/nix-bitcoin#396: examples: add importable-configuration.nix
2a16db6919 readme: add 'Get started' section (Erik Arvstedt)
d713e7b15c examples: add importable-configuration.nix (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK 2a16db6919

Tree-SHA512: 76338cbd852503df2fa316d4fd6833ad423f166aed4ac556c6917bdf6b39610f8a62323e6bd7c9d191238bb6f6dce9e918b0b303dc80a6534497eb89cb7ec344
2021-09-27 09:11:21 +00:00
Erik Arvstedt
46af0b1a8d
examples: fix shell on non-Linux systems
extra-container is not supported on these systems.

Also mention that VM examples require Linux.
2021-09-26 23:58:46 +02:00
Erik Arvstedt
2a16db6919
readme: add 'Get started' section
Also, shorten the overview a bit by removing some redundancy:
- Remove sentence 'A Bitcoin node verifies ... network.' because
  its content is implied by the use cases (wallet, public infrastructure) mentioned in the
  following sentence.

- 'second layer public infrastructure' -> 'public infrastructure',
  because we're also serving the first layer.
2021-09-26 22:34:39 +02:00
Erik Arvstedt
d713e7b15c
examples: add importable-configuration.nix
This replaces minimal-configuration.nix.
importable-configuration.nix can be directly added to an existing
NixOS configuration. This makes it easy for users to get started quickly.
2021-09-26 22:34:39 +02:00
Jonas Nick
5626558222
Merge fort-nix/nix-bitcoin#397: backups: make extraFiles list of strings
c483f1694d examples: correct localBackups scp command (nixbitcoin)
cb54891484 backups: make extraFiles list of strings (nixbitcoin)

Pull request description:

ACKs for top commit:
  erikarvstedt:
    ACK c483f1694d

Tree-SHA512: cb178382655e70aaf85f5930ec7a2c631c28e4c447d0b5d02d973eb73e3bd902ab091cc496d79efbfba5355b9574cb5ba6dd5a4d395b0cda4e3da715fd88d9c5
2021-09-25 10:34:51 +00:00
nixbitcoin
c483f1694d
examples: correct localBackups scp command 2021-09-24 11:51:33 +00:00
nixbitcoin
cb54891484
backups: make extraFiles list of strings 2021-09-24 11:38:47 +00:00
Jonas Nick
c72f6b0a31
Merge fort-nix/nix-bitcoin#393: Update nixpkgs
e94e071ad6 update nixpkgs (Jonas Nick)
0c31130ac8 secure-node: remove default bitcoin addnode (Jonas Nick)

Pull request description:

ACKs for top commit:
  erikarvstedt:
    ACK e94e071ad6
  nixbitcoin:
    ACK e94e071ad6

Tree-SHA512: 26ece3c295db0086198aedb50ccef9689c87b7d9860ca5a09e2c8247dd259a9be6eb8e22bed17798cb901abc7de361f0d660ddebed4c2fb69a6b1f4b999d78e0
2021-09-20 15:48:56 +00:00
Jonas Nick
e94e071ad6
update nixpkgs
bitcoin: 0.21.1 -> 22.0
bitcoind: 0.21.1 -> 22.0
electrs: 0.8.10 -> 0.8.11
2021-09-19 20:07:55 +00:00
Jonas Nick
0c31130ac8
secure-node: remove default bitcoin addnode
Onion v2 addresses are incompatible with the upcoming bitcoind version.
2021-09-19 20:03:40 +00:00
Jonas Nick
020d9486dd
Merge fort-nix/nix-bitcoin#388: Modules formatting
9114ec669a lnd: improve options formatting (Erik Arvstedt)
c8774375d3 modules: use consistent service variables (Erik Arvstedt)
ad97c268c6 modules: move user/group options to bottom (Erik Arvstedt)
27c45b82cc modules: move options to the top (Erik Arvstedt)
731cf647ff modules: remove unneeded use of `options` module arg (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK 9114ec669a

Tree-SHA512: 7817b33d00459b31cacedc365ec067c8162200d46d13a3c2a405eba61e014373d6be8de4cb3c270886da8b1555f86bc7909023ce20d33d3f7ec5429f5eae9d8b
2021-09-19 11:28:17 +00:00
Erik Arvstedt
9114ec669a
lnd: improve options formatting 2021-09-16 12:51:00 +02:00
Jonas Nick
6b4b74bbac
Merge fort-nix/nix-bitcoin#391: examples/shell.nix: Add shell version variable
1f8f2b2139 examples/shell.nix: Add shell version variable (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK 1f8f2b2139

Tree-SHA512: 30eea3294d7e308d0e3271241e114c65ce9cab75ead91642a5c00b4724d8808bd0221675591856a1d2188570707afeef814ccdce093bea4896031c0ec533403e
2021-09-15 10:27:18 +00:00
Erik Arvstedt
1f8f2b2139
examples/shell.nix: Add shell version variable
This simplifies future shell upgrades.
2021-09-15 12:22:10 +02:00
Jonas Nick
6bf6c225e2
Merge fort-nix/nix-bitcoin#389: examples/shell.nix: Add upgrade note for NixOps users
0186b2a764 examples/shell.nix: Add upgrade note for NixOps users (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK 0186b2a764

Tree-SHA512: 656ea58e730f12cbb9c9856b79978a50adcdeaae8a693b32f4aaf6a0c6118a1728946ba4ec6a09b0dc15b6c0379b367b7c102300c13c3df252300716068ab30c
2021-09-15 10:08:03 +00:00
Erik Arvstedt
0186b2a764
examples/shell.nix: Add upgrade note for NixOps users 2021-09-15 12:01:37 +02:00
Jonas Nick
ed8792c72b
Merge fort-nix/nix-bitcoin#387: Shell improvements
b49c74545f fetch-release: make GPG key searchable (Erik Arvstedt)
7356a34d88 docs/install.md: update (Erik Arvstedt)
77af2e4538 makeShell: improve `update-nix-bitcoin` (Erik Arvstedt)
52aaa8388e fetch-release: write error messages to stderr (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK b49c74545f

Tree-SHA512: 4117cbe5839a7cf6ec1855687d75d53569582263064ec04207c2b8ea5de9638a6fca2f6367fad1427dfb0855b2c6656d51492b603baf44115db564b1a2b7be46
2021-09-15 09:00:49 +00:00
Jonas Nick
5c88be3403
Merge fort-nix/nix-bitcoin#383: update nixpkgs
3e146512d7 joinmarket: add copy of twisted 20.3.0 (nixbitcoin)
7008a5118a update nixpkgs (Jonas Nick)

Pull request description:

ACKs for top commit:
  erikarvstedt:
    ACK 3e146512d7

Tree-SHA512: 512fde9641c4309ff98e4c53d6392511fa099d01192a5f970b9d523d21635abda853c1a55148aac75b05286cc252036cd6041cd7dc60bf48e2f2bcc0f9d3fbac
2021-09-15 08:25:38 +00:00
nixbitcoin
3e146512d7
joinmarket: add copy of twisted 20.3.0 2021-09-14 20:06:35 +00:00
Jonas Nick
7008a5118a
update nixpkgs
btcpayserver: 1.2.0 -> 1.2.3
charge-lnd: 0.2.3 -> 0.2.4
nbxplorer: 2.1.58 -> 2.2.5
2021-09-14 20:06:27 +00:00
Erik Arvstedt
b49c74545f
fetch-release: make GPG key searchable
The non-spaced version gives no matches in major search engines.
This is useful for auditing this script.
2021-09-14 19:56:35 +02:00
Erik Arvstedt
7356a34d88
docs/install.md: update 2021-09-14 19:56:35 +02:00
Erik Arvstedt
77af2e4538
makeShell: improve update-nix-bitcoin
- Don't overwrite `nix-bitcoin-release.nix` on errors
- Show a message to indicate whether `nix-bitcoin-release.nix` was
  updated
- Don't start a shell when called noninteractively

Also, update `usage.md` and reformat `shell.nix`.
2021-09-14 19:56:35 +02:00
Erik Arvstedt
52aaa8388e
fetch-release: write error messages to stderr
Previously, when used to update `nix-bitcoin-release`, the error
wasn't displayed but instead written to `nix-bitcoin-release`.

Also, show curl error messages.
2021-09-14 19:56:23 +02:00
Erik Arvstedt
c8774375d3
modules: use consistent service variables
Benefits of adding top-level variables for used services:
- Makes it obvious which other services are referenced by a service
- Less code

We already do this in many other places.
2021-09-13 13:41:47 +02:00
Erik Arvstedt
ad97c268c6
modules: move user/group options to bottom
These are insignificant, generic options; place them above readonly options.
We already do this in other services.

Also move user/group config to bottom in spark-wallet.
2021-09-13 13:41:47 +02:00
Erik Arvstedt
27c45b82cc
modules: move options to the top
This greatly improves readability and makes it easier to discover options.

This commit was genereated by running the following script inside the
repo root dir:

#!/usr/bin/env ruby

def transform(src)
  return false if src.include?('inherit options;')

  success = false

  options = nil
  src.sub!(/^  options.*?^  }.*?;/m) do |match|
    options = match
    "  inherit options;"
  end
  return false if !options

  src.sub!(/^with lib;\s*let\n+/m) do |match|
    success = true
    <<~EOF
      with lib;
      let
      #{options}

    EOF
  end

  success
end

Dir['modules/**/*.nix'].each do |f|
  src = File.read(f)
  if transform(src)
    puts "Changed file #{f}"
    File.write(f, src)
  end
end
2021-09-13 13:41:47 +02:00
Erik Arvstedt
731cf647ff
modules: remove unneeded use of options module arg
Needed by the following commit.
2021-09-13 13:41:47 +02:00
Jonas Nick
cf70d05be0
Merge fort-nix/nix-bitcoin#385: Misc. improvments
fdcb68e96e examples/shell.nix: add new commands (Erik Arvstedt)
a2466b1127 secrets: allow extending generate-secrets (Erik Arvstedt)
24fd1e9bdc improve examples/shell.nix (Erik Arvstedt)
8a757e0486 push-release.sh: improve --dry-run mode (Erik Arvstedt)
82a2b148d8 secrets: minor fixes (Erik Arvstedt)
e1e3d8a92b secrets: simplify cert generation (Erik Arvstedt)
2c8e29b35b lnd: extract option `certPath` (Erik Arvstedt)
be12a49933 lightning-pool/loop: extract lnd variable (Erik Arvstedt)
955b44404c delete helper/fetch-channel (Erik Arvstedt)
5087ce245f minor cleanups (Erik Arvstedt)
0d2db4e79f backups: add option `postgresqlDatabases` (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK fdcb68e96e
  jonasnick:
    ACK  fdcb68e96e

Tree-SHA512: a0fef5b6f8704a445b0e381a1713c14d1447e16798e7035bb005d2b61c4cde208f96fc6f152238b6ea2e9080c04fffe7f841073fa41a5c1e0597204e9ed805c2
2021-09-12 13:22:10 +00:00
Erik Arvstedt
fdcb68e96e
examples/shell.nix: add new commands 2021-09-12 11:29:54 +02:00
Erik Arvstedt
a2466b1127
secrets: allow extending generate-secrets
`generate-secrets` is no longer a monolithic script. Instead, it's
composed of the values of option `nix-bitcoin.generateSecretsCmds`.

This has the following advantages:
- generate-secrets is now extensible by users
- Only secrets of enabled services are generated
- RPC IPs in the `lnd` and `loop` certs are no longer hardcoded.

Secrets are no longer automatically generated when entering nix-shell.
Instead, they are generated before deployment (via `krops-deploy`)
because secrets generation is now dependant on the node configuration.
2021-09-12 11:29:54 +02:00
Erik Arvstedt
24fd1e9bdc
improve examples/shell.nix
The user's local node configuration directory usually contains a copy of
examples/shell.nix.

1. Move the shell implementation from shell.nix to nix-bitcoin/helper/makeShell.nix
   Because the shell is no longer defined locally in the user's config
   directory, we can now ship new shell features via nix-bitcoin updates.

2. Simplify examples/nix-bitcoin-release.nix
   nix-bitcoin-release.nix, as generated via `fetch-release`, now
   contains a simple fetchTarball statement which can be directly imported.
   This allows us to get rid of the extra `nix-bitcoin-unpacked` derivation
   which adds a dependency on the user's local nixpkgs.

   To keep `fetch-release` as simple as possible for easy auditing, we just
   fetch and verify a `nar-hash.txt` file that is now uploaded
   via `push-release.sh`.

A migration guide for updating the user's local `shell.nix` is
automatically printed when the user starts a new shell after updating
nix-bitcoin.
This is achieved by throwing an error in `generate-secrets`, which is called
on shell startup.

This commit is required to deploy the new extensible `generate-secrets`
mechanism introduced in the next commit.
2021-09-12 11:29:54 +02:00
Erik Arvstedt
8a757e0486
push-release.sh: improve --dry-run mode
- Don't require OAUTH_TOKEN
- Skip interactive query
2021-09-11 15:07:24 +02:00
Erik Arvstedt
82a2b148d8
secrets: minor fixes
- Improve comment.
- `secretsSetupMethod` is not internal because it can be set to "manual"
  by the user.
2021-09-11 15:07:24 +02:00
Erik Arvstedt
e1e3d8a92b
secrets: simplify cert generation
- Remove openssl.cnf which includes many unused settings.
- Generate the key and cert files with a single call to openssl.
  - Option `-nodes` ("no DES") disables encryption of the key file.
  - Option `-addext` is used to specify `subjectAltName` settings
    that were previously defined by openssl.cnf.

The key type is unchanged.
Certificate changes:
- Certificate duration is now 10 years
- Organization (subj 'O') is now 'loop' instead of 'loopd' for
  lightning-loop to simplify the code.
  For reference, the org. name in auto-generated loop certs is
  "loop autogenerated cert".
- The certificate now includes all default x509v3 extensions.
  These were previously restricted to just `subjectAltName` by openssl.cnf.
  We now use the openssl defaults for simplicity.
2021-09-11 15:07:24 +02:00
Erik Arvstedt
2c8e29b35b
lnd: extract option certPath
Improves service encapsulation.
2021-09-11 15:07:24 +02:00
Erik Arvstedt
be12a49933
lightning-pool/loop: extract lnd variable 2021-09-11 15:07:24 +02:00