Commit Graph

142 Commits

Author SHA1 Message Date
Erik Arvstedt
77af2e4538
makeShell: improve update-nix-bitcoin
- Don't overwrite `nix-bitcoin-release.nix` on errors
- Show a message to indicate whether `nix-bitcoin-release.nix` was
  updated
- Don't start a shell when called noninteractively

Also, update `usage.md` and reformat `shell.nix`.
2021-09-14 19:56:35 +02:00
Erik Arvstedt
24fd1e9bdc
improve examples/shell.nix
The user's local node configuration directory usually contains a copy of
examples/shell.nix.

1. Move the shell implementation from shell.nix to nix-bitcoin/helper/makeShell.nix
   Because the shell is no longer defined locally in the user's config
   directory, we can now ship new shell features via nix-bitcoin updates.

2. Simplify examples/nix-bitcoin-release.nix
   nix-bitcoin-release.nix, as generated via `fetch-release`, now
   contains a simple fetchTarball statement which can be directly imported.
   This allows us to get rid of the extra `nix-bitcoin-unpacked` derivation
   which adds a dependency on the user's local nixpkgs.

   To keep `fetch-release` as simple as possible for easy auditing, we just
   fetch and verify a `nar-hash.txt` file that is now uploaded
   via `push-release.sh`.

A migration guide for updating the user's local `shell.nix` is
automatically printed when the user starts a new shell after updating
nix-bitcoin.
This is achieved by throwing an error in `generate-secrets`, which is called
on shell startup.

This commit is required to deploy the new extensible `generate-secrets`
mechanism introduced in the next commit.
2021-09-12 11:29:54 +02:00
Erik Arvstedt
179b86d19c
joinmarket: allow recreating wallet from seed
This allows users to easily upgrade their wallets to use Fidelity Bonds.
2021-08-30 13:37:05 +02:00
Erik Arvstedt
f7c2133250
add flake support
This change is fully backwards compatible.

We continue to use the standard non-flake evaluation mode in our
examples and internal tooling until the flakes design has stabilized.

'clightning-plugins = pkgs.recurseIntoAttrs' in pkgs/default.nix is
needed by flake-utils.lib.flattenTree in flake.nix.
It transforms the packages in `clightning-plugins` to top-level packages
named like `clightning-plugins/summary`. (The flake attr `packages`
must be a non-nested attrset of derivations.)
2021-08-26 12:45:10 +02:00
Erik Arvstedt
cce9a3f6b2
modules: move nix-bitcoin options to file 'nix-bitcoin.nix'
This allows modules.nix to consist only of a list of modules.
2021-08-15 22:40:35 +02:00
Erik Arvstedt
4ece606e8b
examples/minimal-configuration: improve comment 2021-08-15 11:29:35 +02:00
Erik Arvstedt
1c3735b600
examples/README: add nixbitcoin.org server repo 2021-08-15 11:29:33 +02:00
Erik Arvstedt
c041079ae1
configuration.nix: reorder sections
Move backups and netns-isolation to the end.
2021-08-15 11:29:33 +02:00
Erik Arvstedt
3734ab38a6
configuration.nix: improve wording and formatting 2021-08-15 11:29:33 +02:00
Erik Arvstedt
0ef66c920b
treewide: use services.getty option
services.mingetty is equivalent but deprecated.
2021-08-14 10:46:40 +02:00
Erik Arvstedt
a25ceecca5
update to NixOS 21.05 2021-08-12 11:18:26 +02:00
nixbitcoin
e2c6eb6681
joinmarket: create bitcoind wallet
Starting with 0.21.0, bitcoin no longer automatically creates and loads
a default wallet.
This was being ignored because of a JoinMarket issue [1] in CI builds prior
to this version. Now a watch-only Bitcoin Core wallet is created in ExecStartPost.

[1] https://github.com/JoinMarket-Org/joinmarket-clientserver/issues/812
2021-08-10 10:12:37 +00:00
Martin Milata
b666bb2903 charge-lnd: add module 2021-07-12 17:36:31 +02:00
Jonas Nick
9e7d9ee475
Merge #348: run-tests: add 'vm' command
3cce822c64 run-tests: add 'vm' command (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK 3cce822c64

Tree-SHA512: 1b7cdc0bc3b73634fdc6abd6817417fd49853be076cf71cc54487e5db9e97e67832eef5ecbb8e899e30a91a60ff333e022e51cc664df7364417240f05096eb16
2021-04-02 12:57:44 +00:00
nixbitcoin
1ff5f8f01b
hardening: use generic hostname by default
Minimize identifying information
2021-04-02 10:59:21 +00:00
nixbitcoin
3b938a909f
add hardened-extended preset 2021-04-02 10:59:09 +00:00
Erik Arvstedt
3cce822c64
run-tests: add 'vm' command 2021-03-26 23:23:22 +01:00
Erik Arvstedt
45d0964e27
examples/shell.nix: minor improvements
- Use idiomatic var name `pkgs` for the imported nixpkgs.
- Don't add `figlet` to PATH because it's only used internally.
- Only print figlet in interactive shells to avoid interfering with
  stdout when running `nix-shell --run <cmd>`.
- Define `fetch-release` as a function to enable running it via
  `nix-shell --run fetch-release`
2021-03-16 12:46:18 +01:00
Erik Arvstedt
cc7149eb78
examples: improve robustness of deploy scripts
- The scripts now work within arbitrary nix-shells.
  Previously, they failed when run from nix shells other than
  `examples/shell.nix`.

- The scripts now work from arbitrary working dirs.
2021-03-16 12:46:18 +01:00
Jonas Nick
eb6dff5478
docs: various improvements to installation tutorial
- Update nixos iso
- Make sure to be root before parting and give instructions for checking
  UEFI vs. legacy
- Instead of making user replace automatically generated nixos config,
  explain how to add options.
- Add boot.loader options directly to hardware-configuration.nix
- Copy .gitignore when setting up a deployment dir
- Stress that the first config that is deployed remotely must include ssh key.
- Debian: 9 Strech -> Debian 10 Buster
- nix: 2.3.3 -> 2.3.10
2021-03-15 19:02:58 +01:00
Erik Arvstedt
e8b47f099c
examples: add deploy-krops.sh 2021-03-15 19:02:58 +01:00
Jonas Nick
fe118b28ff
examples: add krops deployment method 2021-03-15 18:53:07 +01:00
Erik Arvstedt
03db1a61b1
lnd, joinmarket: don't write to secrets dir
Keeping the secrets dir read-only is more simple and robust.

- lnd seed mnemonic creation and joinmarket wallet creation can be
  run as the regular service user instead of root.

- It is easier to switch to a third-party secrets deployment
  method in the future.

Don't create a seed mnemonic for lnd when a wallet exists.
This avoids creating unused mnemonics and helps simplifying
the migration command in `versioning.nix`.
2021-03-15 18:50:15 +01:00
Erik Arvstedt
b701cb5603
secrets: add option 'generateSecrets'
Move this feature from a module preset to a regular option, so that it's
easily discoverable and accessible.

Simplify the implementation of `generateSecrets` by adding it to the
existing `setup-secrets` service script.

Also rename option setup-secrets -> setupSecrets.
2021-03-15 12:42:52 +00:00
Erik Arvstedt
03515a8da6
examples/qemu: disable hardened preset
- Hardening is irrelevant in examples
- The decreased performance is especially noticeable in VMs
2021-03-15 12:42:52 +00:00
Erik Arvstedt
908af3bfb8
examples/deploy-qemu-vm: show progress when waiting
This improves the user experience on VM startup, which can take a few
seconds.
2021-03-15 12:42:51 +00:00
Erik Arvstedt
ccba86a0f0
examples: extract common features for running QEMU VMs
Needed by commit: "examples: add deploy-krops.sh"
2021-03-15 12:42:51 +00:00
Erik Arvstedt
cb3d5446c8
examples: use relative paths to example files
This ensures that the examples files from the current example dir are used.
Previously, the default examples from the nix-bitcoin source were used.
2021-03-15 12:42:51 +00:00
Jonas Nick
a4dabc7390
Remove nixops examples and documentation 2021-03-15 12:42:47 +00:00
kon
eb21012745 pool: add pkg, module & tests 2021-03-01 10:59:35 +01:00
Jonas Nick
f9683889d9
Merge #312: Refactorings, cleanups
0a2c8e4864 run-tests: add option --copy-src (Erik Arvstedt)
803584a288 backups: don't use hardcoded secrets dir (Erik Arvstedt)
c29d44b49a ci: use 'cachix watch-exec' (Erik Arvstedt)
6a32812412 services: add names for systemd helper scripts (Erik Arvstedt)
6982699613 services: use consistent layout (Erik Arvstedt)
a43534dda0 services: improve config file setup (Erik Arvstedt)
18f2002cf0 joinmarket-yieldgenerator: improve systemd journal output (Erik Arvstedt)
9d0b8c8f6f joinmarket-ob-watcher: use DynamicUser (Erik Arvstedt)
e9c98f415c joinmarket: explain need for tor control socket (Erik Arvstedt)
d9c87b6a8f joinmarket: fix wallet creation (Erik Arvstedt)
7458350108 treewide: remove deprecated types.loaOf (Erik Arvstedt)
9cf038939c treewide: use mkEnableOption (Erik Arvstedt)
7a97304f13 treewide: remove unit descriptions (Erik Arvstedt)
a942177ecf treewide: remove user descriptions (Erik Arvstedt)
4f6ff408ef treewide: remove unneeded string literals (Erik Arvstedt)
e6a6c721c1 treewide: streamline 'extraConfig' descriptions (Erik Arvstedt)
e774c045de treewide: fix formatting (Erik Arvstedt)
0b5b29a2a3 netns-isolation: simplify permission definition for netns-exec (Erik Arvstedt)
a587a2b02a defaultHardening: explain where @system-service is defined (Erik Arvstedt)
bb3a69797e README: minor improvements (Erik Arvstedt)
13fc9dfabf examples: improve introductory comments (Erik Arvstedt)
af2040f4c4 netns-isolation: use 'true' for systemd option (Erik Arvstedt)
c246bbb36e bitcoind, clightning, lnd: improve descriptions (Erik Arvstedt)
7533f12ef1 bitcoind, clightning, run-tests: minor refactoring (Erik Arvstedt)
41fe9b0c1d elementsd: minor refactoring (Erik Arvstedt)
f0850d3f23 btcpayserver: reorder config settings (Erik Arvstedt)
d1c0ea9f85 btcpayserver: add missing systemd postgresql dependency (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  jonasnick:
    ACK 0a2c8e4864

Tree-SHA512: 5c81b36042fbb2f016c8e58ba9e05ef3389d5376b8df713d3258d2cd0b6a9239904531171aca8e49bea7039341d5fa91aa9474c6d98de849c25ede52deccc5a3
2021-02-08 20:32:03 +00:00
Jonas Nick
2ebd1129a5
Merge #317: Pkg updates
a0f48c9de9 examples: fix deploy-container interactive flag (nixbitcoin)
a2f265cd35 secp256k1: move to top-level packages (Erik Arvstedt)
d41a843167 jmbitcoin: remove secp256k1 from propagatedBuildInputs (Erik Arvstedt)
c22adb03af extra-container: 0.5 -> 0.6 (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK a0f48c9de9
  jonasnick:
    ACK a0f48c9de9

Tree-SHA512: 29fa58a960673df407831dd41594c66b26dad1de1e792f4fcc8e35641f39dd873d77b725651be5e01c875bf42284fa78903bab0ea677ec5a0e7eccf98816845d
2021-02-07 21:44:10 +00:00
Erik Arvstedt
13fc9dfabf
examples: improve introductory comments 2021-02-07 22:39:05 +01:00
nixbitcoin
a0f48c9de9
examples: fix deploy-container interactive flag 2021-02-06 11:43:36 +01:00
nixbitcoin
ebd478fd0d
lnd: add option 'restOnionService' 2021-02-05 09:17:14 +01:00
Erik Arvstedt
332d0e70c8
bitcoind: support onion address announcing 2021-01-31 22:26:49 +01:00
Erik Arvstedt
44546561fc
run-tests: allow defining scenarios via cmdline args
This simplifies running self-contained scenarios for testing and debugging.
2021-01-30 11:38:47 +01:00
nixbitcoin
8c125ec48c
joinmarket-obwatcher: add pkg & module 2021-01-17 17:40:12 +00:00
Erik Arvstedt
f6b883a9ac
remove webindex
This module is outdated and incomplete. We can readd an improved version in
the future.

Move nanopos nginx proxy tests to the nanopos test.
2021-01-14 13:25:10 +01:00
Erik Arvstedt
2a240d6f4a
enable-tor: disable default onion services for clightning, lnd, btcpayserver
In case of btcpayserver the default onion service is a security risk
because any visitor can register an admin account on a freshly setup node.
2021-01-14 13:25:09 +01:00
Erik Arvstedt
bed00fe937
lnd: use onionServices for address announcing 2021-01-14 13:25:09 +01:00
Erik Arvstedt
3980cd5a41
clightning: use onionServices for address announcing 2021-01-14 13:25:08 +01:00
Erik Arvstedt
e78a609687
clightning: add consistent address options
Also remove option 'autolisten'. This option has no effect because
option 'bind-addr' is always set.
2021-01-14 13:25:04 +01:00
Erik Arvstedt
b41a720c28
lnd: add consistent address options
Also fix btcpayserver by connecting to the lnd restAddress instead of the p2p address.
2021-01-14 13:25:03 +01:00
Jonas Nick
79f4723cda
lightning-charge: remove package and module 2021-01-01 19:16:46 +00:00
Jonas Nick
58de79d401
nanopos: remove package and module 2021-01-01 17:37:30 +00:00
nixbitcoin
bcedf69549
readme: update and split into various parts 2020-12-30 15:59:22 +00:00
Jonas Nick
7b32a78de2
Merge #284: Fix containers
2bfb4efbd8 make-container: fix usage comment (Erik Arvstedt)
3403795c86 tests: add example scripts (Erik Arvstedt)
ff94985b8b tests: add test 'hardened' (Erik Arvstedt)
c8e73c959e fix 'hardened' profile for NixOS 20.09 (Erik Arvstedt)
44b06aea5a extra-container: 0.5-pre -> 0.5 (Erik Arvstedt)
a359cdfb66 generate-secrets: use pwgen (Erik Arvstedt)
a5a2fc7274 make-container: fix renamed variable (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK 2bfb4efbd8
  jonasnick:
    utACK 2bfb4efbd8

Tree-SHA512: 421b1fc5bf695d6815f060d129855ae0fecc06f7946ed8ac2bfe53895d7dc9529aad40099fc16844547791010232252f74b1ce32cbc9c6458e6d77f327450e94
2020-12-21 12:24:14 +00:00
Erik Arvstedt
3403795c86
tests: add example scripts 2020-12-18 19:56:56 +01:00
Erik Arvstedt
c8e73c959e
fix 'hardened' profile for NixOS 20.09
The 'scudo' memory allocator set by the 'hardened' profile breaks some
services on 20.09.
The fix for NixOS unstable (https://github.com/NixOS/nixpkgs/pull/104052)
is ineffective on 20.09.

As a workaround, add a custom 'hardened' preset that uses the default allocator.
2020-12-18 19:56:56 +01:00
nixbitcoin
fdfafb2f40
joinmarket: 0.7.4 -> 0.8.0-bcfa7eb
Update to 0.8.0 with hotfix for genwallet script
Includes module versioning error with migration instructions
2020-12-17 11:12:58 +00:00
Ian Shipman
1d44b99340 add curated clightning plugins 2020-11-18 20:21:34 -06:00
Jonas Nick
5ec9ea1f6d
Merge #250: Improve examples
b574cb097f examples: add deploy-container-minimal.sh (Erik Arvstedt)
e6340426c1 deploy-container.sh: fix sudo (Erik Arvstedt)
c19f7ebb01 examples: add option --interactive|-i (Erik Arvstedt)
33ff8d82be examples: fix running outside of examples dir (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK b574cb097f

Tree-SHA512: 1cf73206af950b157c87476a205e035311416ebb801f90f573868a8c0b8437f900e8b5b85e45e3bacbd3235972b4d3d311de394c63dd93fe4c222919d3364851
2020-10-20 11:18:43 +00:00
Jonas Nick
6933b0ef47
Merge #251: Services: Auto-enable dependencies
67e49fe415 services: auto-enable dependencies (Erik Arvstedt)

Pull request description:

ACKs for top commit:
  nixbitcoin:
    ACK 67e49fe415

Tree-SHA512: 6e0f2e2ca4acdb7c5edd41eb3b56a9e95fc6d2ea9cfd08c1142429f88455c9d771f2f2be6339336448a289632f9768c4ae8f6c307038c5aa69c48b303043dda0
2020-10-20 11:17:10 +00:00
Erik Arvstedt
b574cb097f
examples: add deploy-container-minimal.sh 2020-10-20 12:35:23 +02:00
Erik Arvstedt
67e49fe415
services: auto-enable dependencies 2020-10-19 14:55:59 +02:00
Erik Arvstedt
d2dbad256f
shell.nix: fix failing generate-secrets
generate-secrets failed due to the `PYTHONPATH` env var that was set
through the nixops19_09 buildInput.

Fixes:
- Don't pull in the build environments of binaries that should be
  available in PATH. Only extend PATH instead.
- Run generate-secrets in an empty environment
2020-10-19 11:35:32 +02:00
Erik Arvstedt
e6340426c1
deploy-container.sh: fix sudo
The 'echo sudo' approach used previously failed when PATH or
NIX_PATH contains spaces. Exec the script with sudo instead.
2020-10-18 13:42:58 +02:00
Erik Arvstedt
c19f7ebb01
examples: add option --interactive|-i 2020-10-18 13:42:58 +02:00
Erik Arvstedt
33ff8d82be
examples: fix running outside of examples dir 2020-10-18 13:42:54 +02:00
Erik Arvstedt
d3ece59919
add module 'versioning' 2020-10-16 23:23:00 +02:00
Erik Arvstedt
1cc432a136
examples/deploy-container: use new extra-container features 2020-10-16 15:53:34 +02:00
Erik Arvstedt
16b2783ae7
examples: enable running outside of working dir 2020-10-16 15:53:34 +02:00
Erik Arvstedt
ac6cee5c12
pkgs: add extra-container 2020-10-11 19:40:26 +02:00
nixbitcoin
173891fa5b
joinmarket: add module 2020-09-22 13:50:37 +00:00
nixbitcoin
15b574faa7
nbxplorer/btcpayserver: add module 2020-09-15 12:09:12 +00:00
nixbitcoin
e4fb7a52de
backups: add module 2020-08-04 15:25:37 +00:00
nixbitcoin
ac96fd59db
assertions: make lnd.enable depend on !clightning.enable or port != 9735 2020-08-04 14:07:10 +00:00
nixbitcoin
491d83a658
lightning-loop: add module 2020-07-28 15:55:52 +00:00
nixbitcoin
f3d2aaa5d4
lnd: prepare for netns and bring in line with clightning
- Adds bitcoind-host, and tor-socks options to allow using with
  network namespaces.
- Adds listen, rpclisten, and restlisten option to specify host on which
  to listen on for peer, rpc and rest connections respectively
- Adds announce-tor option and generates Tor Hidden Service with nix
  instead of lnd to bring in line with clightning.

WARNING: Breaking changes for Tor Hidden Service. Manual migration
necessary.
2020-07-21 09:38:32 +00:00
nixbitcoin
65b5dab3d4
clightning: add announce-tor
From the clightning manpage:

autolisten=BOOL By default, we bind (and maybe announce) on IPv4 and
IPv6 interfaces if no addr, bind-addr or  announce-addr options  are
specified. Setting this to false disables that.

We already set bind-addr by default, so autolisten had no effect.
Therefore, this commit replaces autolisten with the more granular
announce-addr option.

For now we are Tor-only, so we only need to announce our hidden service
to accept incoming connections. In the future, we can add clearnet
connectivity with `addr` and route connections into our netns with NAT.
2020-07-21 09:38:26 +00:00
nixbitcoin
e5e07b91f7
netns-isolation: netns architecture
- Adds network namespace instantiation and routing architecture.
- netns-isolation disabled by default. Can be enabled with
  configuration.nix FIXME.
- Uses mkMerge to toggle certain options for non netns and netns
  systems.
- Adds security wrapper for netns-exec which allows operator to exec
  with cap_sys_admin
- User can select the 169.254.N.0/24 addressblock netns's are created in.
- nix-bitcoin-services IpAddressAllow is amended with link-local
  addresses
2020-07-21 09:38:20 +00:00
nixbitcoin
23cd323ad1
assertions: add lnd, clightning exclusivity 2020-06-15 13:02:58 +00:00
nixbitcoin
0ad524ca2d
example config: clarify nix-bitcoin will auto-detect invalid settings 2020-06-15 10:56:01 +00:00
nixbitcoin
c16924b850
example config: change hwi excluding dependency to high-memory
HWI can be enabled if electrs is enabled as long as electrs.high-memory
is disabled.
2020-06-15 10:55:59 +00:00
nixbitcoin
0fd99c4cc0
bitcoind: simplify pruning
Remove the possible null value for bitcoind.prune and set prune = 0 in
bitcoind as a default. Remove prune = 0 in secure-node.nix and the
mkForce in configuration.nix (bitcoind.prune = lib.mkForce ).
2020-06-15 10:55:57 +00:00
nixbitcoin
b9a7a71873
example config: document enabling pruning 2020-06-15 10:55:55 +00:00
Erik Arvstedt
0f1ee5f533
examples: improve shell session usability
- Add usage prompt when starting shell sessions

- Give all examples an uniform interface ("c") for running commands
  or starting a shell on the node.
2020-05-17 23:53:32 +02:00
Erik Arvstedt
719dcd77bb
examples: execute bash sessions in script environment
Previously, the sessions contained only explicitly exported variables
and functions.
This was fragile and in part buggy due to lacking exports.

Interactive features like user-defined aliases and functions are still
working as before.
2020-05-17 14:30:39 +02:00
nixbitcoin
742aef1e0f
Only set dataDirReadableByGroup if cfg.high-memory is enabled 2020-04-24 16:21:12 +02:00
Erik Arvstedt
ade3f84317
examples: add shell session snippets 2020-04-15 18:55:02 +02:00
Erik Arvstedt
1a63f0ca6a
remove option 'services.nix-bitcoin.enable'
Users can enable the node config just by importing secure-node.nix
2020-04-08 17:35:12 +02:00
Erik Arvstedt
28792f79dc
rename nix-bitcoin.nix -> presets/secure-node.nix 2020-04-08 17:35:12 +02:00
Jonas Nick
5ed0284db9
Add fetch-release script
This allows getting the hash of the latest (or some other) release
using github releases and gpg verification.
2020-04-08 07:01:35 +00:00
Erik Arvstedt
705d187a35
examples/shell.nix: don't run shellHook on subsequent nix-shells
This avoids an extra delay and the unexpected creation of secrets when
run in another dir.

Needed for the 'fetch-release' script introduced in a later commit.
2020-03-30 11:00:31 +02:00
Erik Arvstedt
9df22a2764
add deploy-qemu-vm.sh example 2020-03-30 10:56:57 +02:00
Erik Arvstedt
abcee651d3
add deploy-container.sh 2020-03-30 10:49:15 +02:00
Erik Arvstedt
5dadea310c
add deploy-nixops.sh 2020-03-24 21:43:22 +00:00
Erik Arvstedt
0c74c365de
mention performance loss with hardened kernel profile 2020-03-24 21:43:22 +00:00
Erik Arvstedt
f3121892ef
move main module import to configuration.nix 2020-03-24 21:43:21 +00:00
Erik Arvstedt
0c0978c007
extract module 'deployment/nixops.nix', add option 'deployment.secretsDir' 2020-03-24 21:43:21 +00:00
Jonas Nick
87d0286498
Change the nix-bitcoin deployment from forking this repo to importing the module
Instead of forking this repo, it is now recommended that users simply import the
nix-bitcoin module. This commit adds an example directory that contains the
network/ examples and a shell.nix for deployment with nixops.
2020-03-24 21:43:17 +00:00