From a4dabc73900cef226bac8a33d567a0dc6f415b85 Mon Sep 17 00:00:00 2001 From: Jonas Nick Date: Mon, 8 Mar 2021 15:11:13 +0100 Subject: [PATCH 01/15] Remove nixops examples and documentation --- docs/faq.md | 30 --- docs/install.md | 295 +++++++----------------------- docs/usage.md | 30 +-- examples/README.md | 4 - examples/configuration.nix | 7 +- examples/deploy-nixops.sh | 50 ----- examples/nixops/node-libvirtd.nix | 30 --- examples/nixops/node-vbox.nix | 13 -- examples/nixops/node.nix | 9 +- examples/shell.nix | 8 +- helper/vbox-resize-disk1.sh | 81 -------- modules/presets/secure-node.nix | 7 - 12 files changed, 82 insertions(+), 482 deletions(-) delete mode 100755 examples/deploy-nixops.sh delete mode 100644 examples/nixops/node-libvirtd.nix delete mode 100644 examples/nixops/node-vbox.nix delete mode 100755 helper/vbox-resize-disk1.sh diff --git a/docs/faq.md b/docs/faq.md index 62b8986..7fa972e 100644 --- a/docs/faq.md +++ b/docs/faq.md @@ -1,33 +1,3 @@ -* **Q:** When deploying or trying to SSH into the machine I see - ``` - bitcoin-node> waiting for SSH... - Received disconnect from 10.1.1.200 port 22:2: Too many authentication failures - ``` - * **A:** Somehow ssh-agent and nixops don't play well together. Try killing the ssh-agent. -* **Q:** When deploying or trying to SSH into the machine I see - ``` - root@xxx.xxx.xxx.xxx: Permission denied (publickey,password,keyboard-interactive). - ``` - Make sure you don't have something like - ``` - Host * - PubkeyAuthentication no - ``` - in your ssh config. -* **Q:** When deploying to virtualbox for the first time I see - ``` - bitcoin-node> Mar 19 09:22:27 bitcoin-node systemd[1]: Started Get NixOps SSH Key. - bitcoin-node> Mar 19 09:22:27 bitcoin-node get-vbox-nixops-client-key-start[2226]: VBoxControl: error: Failed to connect to the guest property service, error VERR_INTERNAL_ERROR - bitcoin-node> Mar 19 09:22:27 bitcoin-node systemd[1]: get-vbox-nixops-client-key.service: Main process exited, code=exited, status=1/FAILURE - bitcoin-node> Mar 19 09:22:27 bitcoin-node systemd[1]: get-vbox-nixops-client-key.service: Failed with result 'exit-code'. - bitcoin-node> error: Traceback (most recent call last): - File "/nix/store/6zyvpi0q6mvprycadz2dpdqag4742y18-python2.7-nixops-1.6pre0_abcdef/lib/python2.7/site-packages/nixops/deployment.py", line 731, in worker - raise Exception("unable to activate new configuration") - Exception: unable to activate new configuration - ``` - * **A:** This is issue https://github.com/NixOS/nixops/issues/908. The machine needs to be rebooted. You can do that by running `nixops deploy` with the `--force-reboot` flag once. -* **Q:** I'm deploying to virtualbox it's not able to connect anymore. - * **A:** Maybe the IP address of the box changed. Run `nixops deploy --check` to update nixops with the new IP address. Sometimes you need to run `nixops modify -d network/network.nix network/network-vbox.nix`. Sometimes you also need to remove the old IP address from `~/.ssh/known_hosts`. * **Q:** The clightning service is running but when I try to use it (f.e. by running `lightning-cli getinfo` as user operator) all I get is `lightning-cli: Connecting to 'lightning-rpc': Connection refused`. * **A:** Check your clightning logs with `journalctl -eu clightning`. Do you see something like `bitcoin-cli getblock ... false` failed? Are you using pruned mode? That means that clightning hasn't seen all the blocks it needs to and it can't get that block because your node is pruned. If you're just setting up a new node you can `systemctl stop clightning` and wipe your `/var/lib/clightning` directory. Otherwise you need to reindex the Bitcoin node. * **Q:** My disk space is getting low due to nix. diff --git a/docs/install.md b/docs/install.md index a7881e5..1826e0a 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,204 +1,10 @@ Preliminary steps --- Get a machine to deploy nix-bitcoin on. -This could be a VirtualBox, a machine that is already running [NixOS](https://nixos.org/nixos/manual/index.html) or a cloud provider. -Have a look at the options in the [NixOps manual](https://nixos.org/nixops/manual/). # Tutorials -1. [Install and configure NixOS for nix-bitcoin on VirtualBox](#tutorial-install-and-configure-nixos-for-nix-bitcoin-on-virtualbox) -2. [Install and configure NixOS for nix-bitcoin on VirtualBox (macOS host)](#tutorial-install-and-configure-nixos-for-nix-bitcoin-on-virtualbox-macOS-host) -3. [Install and configure NixOS for nix-bitcoin on your own hardware](#tutorial-install-and-configure-nixos-for-nix-bitcoin-on-your-own-hardware) ----- - -Tutorial: install and configure NixOS for nix-bitcoin on VirtualBox ---- -## 1. VirtualBox installation - -1. Add virtualbox.list to /etc/apt/sources.list.d (Debian 9 stretch) - - ``` - echo "deb http://download.virtualbox.org/virtualbox/debian stretch contrib" | sudo tee /etc/apt/sources.list.d/virtualbox.list - ``` - -2. Add Oracle VirtualBox public key - - ``` - wget https://www.virtualbox.org/download/oracle_vbox_2016.asc - gpg2 oracle_vbox_2016.asc - ``` - - Proceed _only_ if fingerprint reads B9F8 D658 297A F3EF C18D 5CDF A2F6 83C5 2980 AECF - - ``` - sudo apt-key add oracle_vbox_2016.asc - ``` - -3. Install virtualbox-5.2 - - ``` - sudo apt-get update - sudo apt-get install virtualbox-5.2 - ``` - -3. Create Host Adapter in VirtualBox - - ``` - vboxmanage hostonlyif create - ``` - -## 2. Nix installation -The following steps are meant to be run on the machine you deploy from, not the machine you deploy to. -You can also build Nix from source by following the instructions at https://nixos.org/nix/manual/#ch-installing-source. - -1. Install Dependencies (Debian 9 stretch) - - ``` - sudo apt-get install curl git gnupg2 dirmngr - ``` - -2. Install latest Nix in "multi-user mode" with GPG Verification according to https://nixos.org/nix/download.html - - ``` - curl -o install-nix-2.3.3 https://releases.nixos.org/nix/nix-2.3.3/install - curl -o install-nix-2.3.3.asc https://releases.nixos.org/nix/nix-2.3.3/install.asc - gpg2 --recv-keys B541D55301270E0BCF15CA5D8170B4726D7198DE - gpg2 --verify ./install-nix-2.3.3.asc - sh ./install-nix-2.3.3 --daemon - ``` - - Then follow the instructions. Open a new terminal window when you're done. - - If you get an error similar to - ``` - error: cloning builder process: Operation not permitted - error: unable to start build process - /tmp/nix-binary-tarball-unpack.hqawN4uSPr/unpack/nix-2.2.1-x86_64-linux/install: unable to install Nix into your default profile - ``` - you're likely not installing as multi-user because you forgot to pass the `--daemon` flag to the install script. - -3. Optional: Disallow substitutes - - You can put `substitute = false` to your `nix.conf` usually found in `/etc/nix/` to build the packages from source. - This eliminates an attack vector where nix's build server or binary cache is compromised. - - -## 3. Setup deployment directory - -1. Clone this project - - ``` - cd - git clone https://github.com/fort-nix/nix-bitcoin - ``` - -2. Obtain the hash of the latest nix-bitcoin release - - ``` - cd nix-bitcoin/examples - nix-shell - ``` - - This will download the nix-bitcoin dependencies and might take a while without giving an output. - Now in the nix-shell run - - ``` - fetch-release > nix-bitcoin-release.nix - ``` - -3. Create a new directory for your nix-bitcoin deployment and copy initial files from nix-bitcoin - - ``` - cd ../../ - mkdir nix-bitcoin-node - cd nix-bitcoin-node - cp -r ../nix-bitcoin/examples/{configuration.nix,nixops,shell.nix,nix-bitcoin-release.nix} . - ``` - -## 4. Deploy with NixOps - -1. Enter `nix-shell`, which drops you into an environment with all the project dependencies loaded ([docs](https://nixos.wiki/wiki/Development_environment_with_nix-shell)). - - ``` - nix-shell - ``` - - Note that a new directory `secrets/` appeared which contains the secrets for your node. - -2. Create a [nixops deployment](https://nixos.org/nixops/manual/#sec-deploying-to-physical-nixos) in nix-shell. - - ``` - nixops create nixops/node.nix nixops/node-vbox.nix -d bitcoin-node - ``` - -3. Adjust your configuration by opening the `configuration.nix` file and enable/disable the modules you want by editing this file. Pay particular attention to lines that are preceded by `FIXME` comments. - -4. Deploy with nixops in nix-shell - - ``` - nixops deploy -d bitcoin-node - ``` - - This will now create a nix-bitcoin node on the target machine. - -5. Nixops automatically creates an ssh key for use with `nixops ssh`. Access `bitcoin-node` through ssh in nix-shell with - - ``` - nixops ssh operator@bitcoin-node - ``` - -For security reasons, all normal system management tasks can and should be performed with the `operator` user. Logging in as `root` should be done as rarely as possible. - -See [usage.md](usage.md) for usage instructions, such as how to update. - -To resize the VM disk image, you can use this helper script from within nix-shell: -``` -../nix-bitcoin/helper/vbox-resize-disk1.sh --help -``` ----- - -Tutorial: install and configure NixOS for nix-bitcoin on VirtualBox (macOS host) ---- -## 1. VirtualBox installation (macOS) - -1. Download and install VirtualBox - Use the official Downloads page: [https://www.virtualbox.org/wiki/Downloads](https://www.virtualbox.org/wiki/Downloads) - - Make sure that the installer you download shows a lock icon in the top right corner to make sure it is signed by Oracle. - -2. Create Host Adapter in VirtualBox - - ``` - vboxmanage hostonlyif create - ``` - -## 2. Nix installation (macOS) - -Follow the instructions from [Nix installation on debian](#2-nix-installation). You will may need to replace `gpg2` with `gpg`. - -## 3. LinuxKit Nix installation -In order to build binaries for your linux (NixOS) virtual machine on a macOS host machine, you need to use [linuxkit-nix](https://github.com/nix-community/linuxkit-nix). It uses hyperkit to spin up a separate VM on which it builds binaries. An alternative solution is [nix-docker-build-slave](https://github.com/LnL7/nix-docker/blob/master/start-docker-nix-build-slave). - -1. Installation - - ``` - nix-env -i /nix/store/jgq3savsyyrpsxvjlrz41nx09z7r0lch-linuxkit-builder - nix-linuxkit-configure - ``` - - You may want to use `nix-linuxkit-configure -c 4` to give the builder 4 CPUs. - -2. Confirm that nix-linuxkit works - - ``` - nix-build ~/.cache/nix-linuxkit-builder/example.nix - ``` - - As the installer says, run a `nix-build` to make sure that you are able to build linux binaries. The `example.nix` is specifically configured to force a x86_64-linux build. Remove the generated `result` folder afterwards. - -## 4. Nixops deployment (macOS) - -Follow the instructions from [Nixops deployment on debian](#3-nixops-deployment). Add `--option system x86_64-linux` to the `nixops deploy` command in step 5 to force your system to use linuxkit-nix. +1. [Install and configure NixOS for nix-bitcoin on your own hardware](#tutorial-install-and-configure-nixos-for-nix-bitcoin-on-your-own-hardware) ---- @@ -344,33 +150,78 @@ This is borrowed from the [NixOS manual](https://nixos.org/nixos/manual/index.ht reboot ``` - ## 2. Nix installation -Follow the [Nix installation](#2-nix-installation) instructions from the tutorial above (on the machine you are going to deploy from). +The following steps are meant to be run on the machine you deploy from, not the machine you deploy to. +You can also build Nix from source by following the instructions at https://nixos.org/nix/manual/#ch-installing-source. +1. Install Dependencies (Debian 9 stretch) + + ``` + sudo apt-get install curl git gnupg2 dirmngr + ``` + +2. Install latest Nix in "multi-user mode" with GPG Verification according to https://nixos.org/nix/download.html + + ``` + curl -o install-nix-2.3.3 https://releases.nixos.org/nix/nix-2.3.3/install + curl -o install-nix-2.3.3.asc https://releases.nixos.org/nix/nix-2.3.3/install.asc + gpg2 --recv-keys B541D55301270E0BCF15CA5D8170B4726D7198DE + gpg2 --verify ./install-nix-2.3.3.asc + sh ./install-nix-2.3.3 --daemon + ``` + + Then follow the instructions. Open a new terminal window when you're done. + + If you get an error similar to + + ``` + error: cloning builder process: Operation not permitted + error: unable to start build process + /tmp/nix-binary-tarball-unpack.hqawN4uSPr/unpack/nix-2.2.1-x86_64-linux/install: unable to install Nix into your default profile + ``` + + you're likely not installing as multi-user because you forgot to pass the `--daemon` flag to the install script. + +3. Optional: Disallow substitutes + + You can put `substitute = false` to your `nix.conf` usually found in `/etc/nix/` to build the packages from source. + This eliminates an attack vector where nix's build server or binary cache is compromised. ## 3. Setup deployment directory -Follow the [Setup deployment directory](#3-setup-deployment-directory) instructions from the tutorial above (on the machine you are going to deploy from). - -## 4. Deploy with NixOps -1. Make sure you are in the deployment directory and create a nixops network file as follows +1. Clone this project ``` - nano nixops/node-nixos.nix + cd + git clone https://github.com/fort-nix/nix-bitcoin ``` +2. Obtain the hash of the latest nix-bitcoin release + ``` - { - bitcoin-node = - { config, pkgs, ... }: - { deployment.targetHost = "1.2.3.4"; - }; - } + cd nix-bitcoin/examples + nix-shell ``` - Replace 1.2.3.4 with NixOS machine's IP address. + This will download the nix-bitcoin dependencies and might take a while without giving an output. + Now in the nix-shell run + ``` + fetch-release > nix-bitcoin-release.nix + ``` + +3. Create a new directory for your nix-bitcoin deployment and copy initial files from nix-bitcoin + + ``` + cd ../../ + mkdir nix-bitcoin-node + cd nix-bitcoin-node + # TODO + cp -r ../nix-bitcoin/examples/{configuration.nix,shell.nix,nix-bitcoin-release.nix} . + ``` + +## 4. Deploy with TODO +1. TODO 2. Edit `configuration.nix` ``` @@ -390,10 +241,13 @@ Follow the [Setup deployment directory](#3-setup-deployment-directory) instructi 4. Add boot option to `hardware-configuration.nix` Option 1: Enable systemd boot for UEFI + ``` boot.loader.systemd-boot.enable = true; ``` + Option 2: Set grub device for Legacy Boot (MBR) + ``` boot.loader.grub.device = "/dev/sda"; ``` @@ -404,29 +258,12 @@ Follow the [Setup deployment directory](#3-setup-deployment-directory) instructi nix-shell ``` - Note that a new directory `secrets/` appeared which contains the secrets for your node. - -6. Create nixops [deployment](https://nixos.org/nixops/manual/#sec-deploying-to-physical-nixos) in nix-shell. - - ``` - nixops create nixops/node.nix nixops/node-nixos.nix -d bitcoin-node - ``` + NOTE that a new directory `secrets/` appeared which contains the secrets for your node. +6. TODO 7. Adjust configuration by opening the `configuration.nix` file and enable/disable the modules you want by editing this file. Pay particular attention to lines that are preceded by `FIXME` comments. -8. Deploy with nixops in nix-shell - - ``` - nixops deploy -d bitcoin-node - ``` - - This will now create a nix-bitcoin node on the target machine. - -9. Nixops automatically creates an ssh key for use with `nixops ssh`. Access `bitcoin-node` through ssh in nix-shell with - - ``` - nixops ssh operator@bitcoin-node - ``` +8. TODO For security reasons, all normal system management tasks can and should be performed with the `operator` user. Logging in as `root` should be done as rarely as possible. diff --git a/docs/usage.md b/docs/usage.md index 0525cb9..3f3ef87 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -30,10 +30,6 @@ Connect to spark-wallet 2. Deploy new `configuration.nix` - ``` - nixops deploy -d bitcoin-node - ``` - 3. Enable Orbot VPN for spark-wallet ``` @@ -123,10 +119,6 @@ Connect to electrs 2. Deploy new `configuration.nix` - ``` - nixops deploy -d bitcoin-node - ``` - 3. Get electrs onion address with format `:` ``` @@ -156,7 +148,7 @@ Connect to nix-bitcoin node through the SSH onion service 1. Get the SSH onion address (excluding the port suffix) ``` - nixops ssh operator@bitcoin-node + ssh operator@bitcoin-node nodeinfo | jq -r .sshd.onion_address | sed 's/:.*//' ``` @@ -182,19 +174,9 @@ Connect to nix-bitcoin node through the SSH onion service ssh -i ~/.ssh/id_ed25519 -L :localhost:22 root@ ``` -5. Edit your `network-nixos.nix` to look like this +5. Edit your network to change the node's address to `localhost` and the ssh port to `[random port of your choosing]` - ``` - { - bitcoin-node = - { config, pkgs, ... }: - { deployment.targetHost = "127.0.0.1"; - deployment.targetPort = ; - }; - } - ``` - -6. Now you can run `nixops deploy -d bitcoin-node` and it will connect through the SSH tunnel you established in step iv. This also allows you to do more complex SSH setups that `nixops ssh` doesn't support. An example would be authenticating with [Trezor's SSH agent](https://github.com/romanz/trezor-agent), which provides extra security. +6. After deploying the new configuration, it will connect through the SSH tunnel you established in step iv. This also allows you to do more complex SSH setups that some deployment tools don't support. An example would be authenticating with [Trezor's SSH agent](https://github.com/romanz/trezor-agent), which provides extra security. Initialize a Trezor for Bitcoin Core's Hardware Wallet Interface --- @@ -212,14 +194,10 @@ Initialize a Trezor for Bitcoin Core's Hardware Wallet Interface 2. Deploy new `configuration.nix` - ``` - nixops deploy -d bitcoin-node - ``` - 3. Check that your nix-bitcoin node recognizes your Trezor ``` - nixops ssh operator@bitcoin-node + ssh operator@bitcoin-node lsusb ``` Should show something relating to your Trezor diff --git a/examples/README.md b/examples/README.md index dba4e53..c154bfe 100644 --- a/examples/README.md +++ b/examples/README.md @@ -20,10 +20,6 @@ By default, [`configuration.nix`](configuration.nix) enables `bitcoind` and `cli - [`./deploy-qemu-vm.sh`](deploy-qemu-vm.sh) creates a QEMU VM.\ Requires: [Nix](https://nixos.org/nix/) -- [`./deploy-nixops.sh`](deploy-nixops.sh) creates a VirtualBox VM via [NixOps](https://github.com/NixOS/nixops).\ - NixOps can be used to deploy to various other backends like cloud providers.\ - Requires: [Nix](https://nixos.org/nix/), [VirtualBox](https://www.virtualbox.org) - - [`./deploy-container-minimal.sh`](deploy-container-minimal.sh) creates a container defined by [minimal-configuration.nix](minimal-configuration.nix) that doesn't use the [secure-node.nix](../modules/presets/secure-node.nix) preset. diff --git a/examples/configuration.nix b/examples/configuration.nix index 5b99217..85c44eb 100644 --- a/examples/configuration.nix +++ b/examples/configuration.nix @@ -13,7 +13,6 @@ # FIXME: Uncomment next line to import your hardware configuration. If so, # add the hardware configuration file to the same directory as this file. - # This is not needed when deploying to a virtual box. #./hardware-configuration.nix ]; # FIXME: Enable modules by uncommenting their respective line. Disable @@ -73,10 +72,10 @@ # If you use lnd, you should manually backup your wallet mnemonic # seed. This will allow you to recover on-chain funds. You can run the # following command after the lnd service starts: - # nixops scp --from bitcoin-node /secrets/lnd-seed-mnemonic ./secrets/lnd-seed-mnemonic + # scp bitcoin-node:/secrets/lnd-seed-mnemonic ./secrets/lnd-seed-mnemonic # You should also backup your channel state after opening new channels. # This will allow you to recover off-chain funds, by force-closing channels. - # nixops scp --from bitcoin-node /var/lib/lnd/chain/bitcoin/mainnet/channel.backup /my-backup-path/channel.backup + # scp bitcoin-node:/var/lib/lnd/chain/bitcoin/mainnet/channel.backup /my-backup-path/channel.backup ### SPARK WALLET # Enable this module to use spark-wallet, a minimalistic wallet GUI for @@ -176,7 +175,7 @@ # /var/lib/localBackups once a day. # services.backups.enable = true; # You can pull the localBackups folder with - # `nixops scp --from bitcoin-node /var/lib/localBackups /my-backup-path/` + # `scp bitcoin-node:/var/lib/localBackups /my-backup-path/` # Alternatively, you can also set a remote target url, for example # services.backups.destination = "sftp://user@host[:port]/[relative|/absolute]_path"; # Supply the sftp password by appending the FTP_PASSWORD environment variable diff --git a/examples/deploy-nixops.sh b/examples/deploy-nixops.sh deleted file mode 100755 index 8e8307b..0000000 --- a/examples/deploy-nixops.sh +++ /dev/null @@ -1,50 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -# This script demonstrates how to setup a VirtualBox nix-bitcoin node with nixops. -# Running this script leaves no traces on your host system. - -# This demo is a template for your own experiments. -# Run with option `--interactive` or `-i` to start a shell for interacting with -# the node. - -if [[ ! -v IN_NIX_SHELL ]]; then - echo "Running script in nix shell env..." - cd "${BASH_SOURCE[0]%/*}" - exec nix-shell --run "./${BASH_SOURCE[0]##*/} $*" -fi - -# Cleanup on exit -cleanup() { - set +e - if nixops list | grep -q bitcoin-node; then - nixops destroy --confirm -d bitcoin-node - fi - rm -rf $tmpDir -} -trap "cleanup" EXIT - -tmpDir=/tmp/nix-bitcoin-nixops -mkdir -p $tmpDir - -# Don't write nixops and VirtualBox data to the $USER's home -export HOME=$tmpDir - -# Disable interactive queries and don't write to the $USER's known_hosts file -export NIXOPS_SSH_OPTS="-o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null" - -nixops create nixops/node.nix nixops/node-vbox.nix -d bitcoin-node -nixops deploy -d bitcoin-node - -# Connect to node -nixops ssh bitcoin-node systemctl status bitcoind - -c() { nixops ssh bitcoin-node "$@"; } - -case ${1:-} in - -i|--interactive) - . start-bash-session.sh - ;; -esac - -# Cleanup happens at exit (see above) diff --git a/examples/nixops/node-libvirtd.nix b/examples/nixops/node-libvirtd.nix deleted file mode 100644 index b6fe7dd..0000000 --- a/examples/nixops/node-libvirtd.nix +++ /dev/null @@ -1,30 +0,0 @@ -{ - bitcoin-node = - { config, pkgs, ... }: - { - deployment.targetEnv = "libvirtd"; - deployment.libvirtd.memorySize = 8192; # megabytes - deployment.libvirtd.vcpu = 4; # number of cpus - deployment.libvirtd.headless = true; - deployment.libvirtd.baseImageSize = 400; - boot.kernelParams = [ "console=ttyS0,115200" ]; - deployment.libvirtd.extraDevicesXML = '' - - - - - - - ''; - # Remove when fixed: https://github.com/NixOS/nixops/issues/931 - system.activationScripts.nixops-vm-fix-931 = { - text = '' - if ls -l /nix/store | grep sudo | grep -q nogroup; then - mount -o remount,rw /nix/store - chown -R root:nixbld /nix/store - fi - ''; - deps = []; - }; - }; -} diff --git a/examples/nixops/node-vbox.nix b/examples/nixops/node-vbox.nix deleted file mode 100644 index 2f75481..0000000 --- a/examples/nixops/node-vbox.nix +++ /dev/null @@ -1,13 +0,0 @@ -{ - bitcoin-node = - { config, pkgs, ... }: - { - deployment.targetEnv = "virtualbox"; - deployment.virtualbox = { - memorySize = 4096; # megabytes - vcpu = 4; # number of cpus - disks.disk1.size = 358400; # 350 GiB - headless = true; - }; - }; -} diff --git a/examples/nixops/node.nix b/examples/nixops/node.nix index d8eac3b..8a3978c 100644 --- a/examples/nixops/node.nix +++ b/examples/nixops/node.nix @@ -1,5 +1,8 @@ +# This is an example network definition for deploying a nix-bitcoin node via NixOps. +# NixOps deployment is currently untested. + { - network.description = "Bitcoin Core node"; + network.description = "Bitcoin node"; bitcoin-node = { config, pkgs, lib, ... }: { imports = [ @@ -8,5 +11,9 @@ ]; nix-bitcoin.deployment.secretsDir = toString ../secrets; + + #FIXME: + # Set `deployment.*` options like + # deployment.targetHost = ""; }; } diff --git a/examples/shell.nix b/examples/shell.nix index a7787e3..8f57cd3 100644 --- a/examples/shell.nix +++ b/examples/shell.nix @@ -20,7 +20,7 @@ with nixpkgs; stdenv.mkDerivation rec { name = "nix-bitcoin-environment"; - path = lib.makeBinPath [ nix-bitcoin.nixops19_09 nix-bitcoin.extra-container figlet ]; + path = lib.makeBinPath [ nix-bitcoin.extra-container figlet ]; shellHook = '' export NIX_PATH="nixpkgs=${nixpkgs-path}:nix-bitcoin=${toString nix-bitcoin-path}:." @@ -28,12 +28,6 @@ stdenv.mkDerivation rec { alias fetch-release="${toString nix-bitcoin-path}/helper/fetch-release" - # ssh-agent and nixops don't play well together (see - # https://github.com/NixOS/nixops/issues/256). I'm getting `Received disconnect - # from 10.1.1.200 port 22:2: Too many authentication failures` if I have a few - # keys already added to my ssh-agent. - export SSH_AUTH_SOCK="" - figlet "nix-bitcoin" (mkdir -p secrets; cd secrets; env -i ${nix-bitcoin.generate-secrets}) diff --git a/helper/vbox-resize-disk1.sh b/helper/vbox-resize-disk1.sh deleted file mode 100755 index b537764..0000000 --- a/helper/vbox-resize-disk1.sh +++ /dev/null @@ -1,81 +0,0 @@ -#!/usr/bin/env nix-shell -#! nix-shell -i bash -p jq - -while getopts ":d:m:s:f:yh" opt; do - case $opt in - d) - DEPLOYMENT="$OPTARG" - ;; - m) - MACHINE="$OPTARG" - ;; - s) - NEW_SIZE="$OPTARG" - ;; - f) - DISK_FILE="$OPTARG" - ;; - y) - YES="yes" - ;; - h) - echo "Usage: $0 [-d ] [-m ] [-s ] [-f ] [-y]" - echo "" - echo "Options:" - echo " -d NixOps deployment name. Default: bitcoin-node." - echo " -m NixOps machine name. Default: bitcoin-node." - echo " -s New disk size in megabytes. Default: 358400 (350gb)." - echo " -f Path to vbox disk file/VDI. Default: read from nixops export." - echo " -y Don't ask for confirmation." - exit 0 - ;; - \?) - echo "Invalid option: -$OPTARG" >&2 - exit 1 - ;; - :) - echo "Option -$OPTARG requires an argument." >&2 - exit 1 - ;; - esac -done - -DEPLOYMENT=${DEPLOYMENT:-"bitcoin-node"} -MACHINE=${MACHINE:-"bitcoin-node"} -NEW_SIZE=${NEW_SIZE:-358400} -DISK_FILE=${DISK_FILE:-$(nixops export -d $DEPLOYMENT | jq -r '..|."virtualbox.disks"?|select(.!=null)' | jq -r .disk1.path)} - -echo "Resizing virtualbox disk for use with nixops and nix-bitcoin." -echo "Using deployment: $DEPLOYMENT" -echo "Using machine: $MACHINE" -echo "Using size: $NEW_SIZE" -echo "Using disk file: $DISK_FILE" - -if [ "$YES" != "yes" ]; then - read -p "Continue? [Y/n] " -n 1 -r - echo - if [[ ! "$REPLY" =~ ^[Yy]$ ]]; then - exit 1 - fi -fi - -set -ex - -nixops stop -d $DEPLOYMENT -VBoxManage modifyhd --resize $NEW_SIZE "$DISK_FILE" -nixops start -d $DEPLOYMENT - -# ( -# echo d # [d]elete 50gb partition -# echo n # [n]ew partition -# echo p # [p]rimary partition -# echo # partition number (Accept default: 1) -# echo # first sector (Accept default: 1) -# echo # last sector (Accept default: determined by $NEW_SIZE) -# echo w # [w]rite changes -# ) | fdisk -nixops ssh -d $DEPLOYMENT $MACHINE -- '(echo d; echo n; echo p; echo; echo; echo; echo w; ) | fdisk /dev/sda' - -nixops reboot -d $DEPLOYMENT -nixops ssh -d $DEPLOYMENT $MACHINE -- resize2fs /dev/sda1 -nixops ssh -d $DEPLOYMENT $MACHINE -- df -h diff --git a/modules/presets/secure-node.nix b/modules/presets/secure-node.nix index 9bea4d5..70a850e 100644 --- a/modules/presets/secure-node.nix +++ b/modules/presets/secure-node.nix @@ -57,12 +57,5 @@ in { users.users.${operatorName} = { openssh.authorizedKeys.keys = config.users.users.root.openssh.authorizedKeys.keys; }; - # Enable nixops ssh for operator (`nixops ssh operator@mynode`) on nixops-vbox deployments - systemd.services.get-vbox-nixops-client-key = - mkIf (builtins.elem ".vbox-nixops-client-key" config.services.openssh.authorizedKeysFiles) { - postStart = '' - cp "${config.users.users.root.home}/.vbox-nixops-client-key" "${config.users.users.${operatorName}.home}" - ''; - }; }; } From cb3d5446c8f16e5008e18adc5ab7e5fbb4de0d3b Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Mon, 8 Mar 2021 15:11:14 +0100 Subject: [PATCH 02/15] examples: use relative paths to example files This ensures that the examples files from the current example dir are used. Previously, the default examples from the nix-bitcoin source were used. --- examples/deploy-container.sh | 2 +- examples/deploy-qemu-vm.sh | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/examples/deploy-container.sh b/examples/deploy-container.sh index 53b3843..408c590 100755 --- a/examples/deploy-container.sh +++ b/examples/deploy-container.sh @@ -82,7 +82,7 @@ read -d '' src < + <${configuration}> ]; }; diff --git a/examples/deploy-qemu-vm.sh b/examples/deploy-qemu-vm.sh index c83a96e..0fa009e 100755 --- a/examples/deploy-qemu-vm.sh +++ b/examples/deploy-qemu-vm.sh @@ -38,7 +38,7 @@ nix-build --out-link $tmpDir/vm - < { configuration = { imports = [ - + ]; virtualisation.graphics = false; From ccba86a0f0d07f0c82fba363c5314304ad076a5c Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Mon, 8 Mar 2021 15:11:15 +0100 Subject: [PATCH 03/15] examples: extract common features for running QEMU VMs Needed by commit: "examples: add deploy-krops.sh" --- examples/deploy-qemu-vm.sh | 48 +++++----------------------------- examples/qemu-vm/run-vm.sh | 48 ++++++++++++++++++++++++++++++++++ examples/qemu-vm/vm-config.nix | 7 +++++ 3 files changed, 62 insertions(+), 41 deletions(-) create mode 100644 examples/qemu-vm/run-vm.sh create mode 100644 examples/qemu-vm/vm-config.nix diff --git a/examples/deploy-qemu-vm.sh b/examples/deploy-qemu-vm.sh index 0fa009e..7e68c2f 100755 --- a/examples/deploy-qemu-vm.sh +++ b/examples/deploy-qemu-vm.sh @@ -17,61 +17,27 @@ if [[ ! -v IN_NIX_SHELL ]]; then exec nix-shell --run "./${BASH_SOURCE[0]##*/} $*" fi -cd "${BASH_SOURCE[0]%/*}" - -tmpDir=/tmp/nix-bitcoin-qemu-vm -mkdir -p $tmpDir - -# Cleanup on exit -cleanup() { - set +eu - kill -9 $qemuPID - rm -rf $tmpDir -} -trap "cleanup" EXIT - -identityFile=qemu-vm/id-vm -chmod 0600 $identityFile +source qemu-vm/run-vm.sh echo "Building VM" -nix-build --out-link $tmpDir/vm - < { configuration = { imports = [ + ]; - virtualisation.graphics = false; - services.mingetty.autologinUser = "root"; - users.users.root = { - openssh.authorizedKeys.keys = [ "$(cat $identityFile.pub)" ]; - }; }; }).vm EOF -vmMemoryMiB=2048 vmNumCPUs=4 +vmMemoryMiB=2048 sshPort=60734 +runVM $tmpDir/vm $vmNumCPUs $vmMemoryMiB $sshPort -export NIX_DISK_IMAGE=$tmpDir/img -export QEMU_NET_OPTS=hostfwd=tcp::$sshPort-:22 -/dev/null & -qemuPID=$! - -# Run command in VM -c() { - ssh -p $sshPort -i $identityFile -o ConnectTimeout=1 \ - -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \ - -o ControlMaster=auto -o ControlPath=$tmpDir/ssh-connection -o ControlPersist=60 \ - root@127.0.0.1 "$@" -} - -echo -echo "Waiting for SSH connection..." -while ! c : 2>/dev/null; do :; done - -echo +vmWaitForSSH echo "Waiting until services are ready..." c ' attempts=300 @@ -99,4 +65,4 @@ case ${1:-} in ;; esac -# Cleanup happens at exit (see above) +# Cleanup happens at exit (defined in qemu-vm/run-vm.sh) diff --git a/examples/qemu-vm/run-vm.sh b/examples/qemu-vm/run-vm.sh new file mode 100644 index 0000000..b195635 --- /dev/null +++ b/examples/qemu-vm/run-vm.sh @@ -0,0 +1,48 @@ +qemuDir=$(cd "${BASH_SOURCE[0]%/*}" && pwd) + +tmpDir=/tmp/nix-bitcoin-qemu-vm +mkdir -p $tmpDir + +# Cleanup on exit +cleanup() { + set +eu + if [[ $qemuPID ]]; then + kill -9 $qemuPID + fi + rm -rf $tmpDir +} +trap "cleanup" EXIT + +identityFile=$qemuDir/id-vm +chmod 0600 $identityFile + +runVM() { + vm=$1 + vmNumCPUs=$2 + vmMemoryMiB=$3 + sshPort=$4 + + export NIX_DISK_IMAGE=$tmpDir/img + export QEMU_NET_OPTS=hostfwd=tcp::$sshPort-:22 + /dev/null & + qemuPID=$! +} + +vmWaitForSSH() { + echo + printf "Waiting for SSH connection..." + while ! c : 2>/dev/null; do :; done + echo +} + +# Run command in VM +c() { + ssh -p $sshPort -i $identityFile -o ConnectTimeout=1 \ + -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \ + -o ControlMaster=auto -o ControlPath=$tmpDir/ssh-connection -o ControlPersist=60 \ + root@127.0.0.1 "$@" +} +export identityFile +export sshPort +export tmpDir +export -f c diff --git a/examples/qemu-vm/vm-config.nix b/examples/qemu-vm/vm-config.nix new file mode 100644 index 0000000..a1f8b70 --- /dev/null +++ b/examples/qemu-vm/vm-config.nix @@ -0,0 +1,7 @@ +{ + virtualisation.graphics = false; + services.mingetty.autologinUser = "root"; + users.users.root = { + openssh.authorizedKeys.keyFiles = [ ./id-vm.pub ]; + }; +} From 908af3bfb8f4dc90bc2168345c60d8a0e15bfbf3 Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Mon, 8 Mar 2021 15:11:16 +0100 Subject: [PATCH 04/15] examples/deploy-qemu-vm: show progress when waiting This improves the user experience on VM startup, which can take a few seconds. --- examples/deploy-qemu-vm.sh | 15 +++++++-------- examples/qemu-vm/run-vm.sh | 4 +++- examples/qemu-vm/wait-until.sh | 21 +++++++++++++++++++++ 3 files changed, 31 insertions(+), 9 deletions(-) create mode 100644 examples/qemu-vm/wait-until.sh diff --git a/examples/deploy-qemu-vm.sh b/examples/deploy-qemu-vm.sh index 7e68c2f..c0a005e 100755 --- a/examples/deploy-qemu-vm.sh +++ b/examples/deploy-qemu-vm.sh @@ -38,14 +38,13 @@ sshPort=60734 runVM $tmpDir/vm $vmNumCPUs $vmMemoryMiB $sshPort vmWaitForSSH -echo "Waiting until services are ready..." -c ' -attempts=300 -while ! systemctl is-active clightning &> /dev/null; do - ((attempts-- == 0)) && { echo "timeout"; exit 1; } - sleep 0.2 -done -' +printf "Waiting until services are ready" +c " +$(cat qemu-vm/wait-until.sh) +waitUntil 'systemctl is-active clightning &> /dev/null' 100 +" +echo + echo echo "Bitcoind service:" c systemctl status bitcoind diff --git a/examples/qemu-vm/run-vm.sh b/examples/qemu-vm/run-vm.sh index b195635..0181796 100644 --- a/examples/qemu-vm/run-vm.sh +++ b/examples/qemu-vm/run-vm.sh @@ -1,5 +1,7 @@ qemuDir=$(cd "${BASH_SOURCE[0]%/*}" && pwd) +source "$qemuDir/wait-until.sh" + tmpDir=/tmp/nix-bitcoin-qemu-vm mkdir -p $tmpDir @@ -31,7 +33,7 @@ runVM() { vmWaitForSSH() { echo printf "Waiting for SSH connection..." - while ! c : 2>/dev/null; do :; done + waitUntil "c : 2>/dev/null" 500 echo } diff --git a/examples/qemu-vm/wait-until.sh b/examples/qemu-vm/wait-until.sh new file mode 100644 index 0000000..b419349 --- /dev/null +++ b/examples/qemu-vm/wait-until.sh @@ -0,0 +1,21 @@ +# Wait until $condition is true, retrying every $intervalMs milliseconds. +# Print a '.' character every second as a progress indicator. +waitUntil() { + condition=$1 + intervalMs=$2 + + lastDotTime=$(getTimeMs) + while ! { t0=$(getTimeMs); eval "$condition"; }; do + now=$(getTimeMs) + if ((now - lastDotTime >= 1000)); then + printf . + lastDotTime=$now + fi + toSleep=$((t0 + intervalMs - now)) + if ((toSleep > 0)); then + sleep $((toSleep / 1000)).$((toSleep % 1000)); + fi + done +} + +getTimeMs() { date +%s%3N; } From 03515a8da600cbe56ba9d4c3c9b33a2cca45ef08 Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Mon, 8 Mar 2021 15:11:17 +0100 Subject: [PATCH 05/15] examples/qemu: disable hardened preset - Hardening is irrelevant in examples - The decreased performance is especially noticeable in VMs --- examples/qemu-vm/vm-config.nix | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-) diff --git a/examples/qemu-vm/vm-config.nix b/examples/qemu-vm/vm-config.nix index a1f8b70..ca91e55 100644 --- a/examples/qemu-vm/vm-config.nix +++ b/examples/qemu-vm/vm-config.nix @@ -1,7 +1,12 @@ { - virtualisation.graphics = false; - services.mingetty.autologinUser = "root"; - users.users.root = { - openssh.authorizedKeys.keyFiles = [ ./id-vm.pub ]; + # Disable the hardened preset to improve VM performance + disabledModules = [ ]; + + config = { + virtualisation.graphics = false; + services.mingetty.autologinUser = "root"; + users.users.root = { + openssh.authorizedKeys.keyFiles = [ ./id-vm.pub ]; + }; }; } From b701cb5603c5f996c8c202c679820180ad7f28e9 Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Wed, 10 Mar 2021 14:08:34 +0100 Subject: [PATCH 06/15] secrets: add option 'generateSecrets' Move this feature from a module preset to a regular option, so that it's easily discoverable and accessible. Simplify the implementation of `generateSecrets` by adding it to the existing `setup-secrets` service script. Also rename option setup-secrets -> setupSecrets. --- examples/deploy-container.sh | 2 +- examples/deploy-qemu-vm.sh | 2 +- examples/minimal-configuration.nix | 3 +- modules/obsolete-options.nix | 2 ++ modules/secrets/generate-secrets.nix | 30 +++-------------- modules/secrets/secrets.nix | 50 +++++++++++++++++++++------- test/tests.nix | 6 ++-- 7 files changed, 51 insertions(+), 44 deletions(-) diff --git a/examples/deploy-container.sh b/examples/deploy-container.sh index 408c590..8cc424b 100755 --- a/examples/deploy-container.sh +++ b/examples/deploy-container.sh @@ -83,8 +83,8 @@ read -d '' src < - ]; + nix-bitcoin.generateSecrets = true; }; }; } diff --git a/examples/deploy-qemu-vm.sh b/examples/deploy-qemu-vm.sh index c0a005e..bff14db 100755 --- a/examples/deploy-qemu-vm.sh +++ b/examples/deploy-qemu-vm.sh @@ -26,8 +26,8 @@ nix-build --out-link $tmpDir/vm - <<'EOF' imports = [ - ]; + nix-bitcoin.generateSecrets = true; }; }).vm EOF diff --git a/examples/minimal-configuration.nix b/examples/minimal-configuration.nix index db6cc66..e7c2757 100644 --- a/examples/minimal-configuration.nix +++ b/examples/minimal-configuration.nix @@ -1,9 +1,10 @@ { config, pkgs, lib, ... }: { imports = [ - ]; + nix-bitcoin.generateSecrets = true; + services.bitcoind.enable = true; services.clightning.enable = true; diff --git a/modules/obsolete-options.nix b/modules/obsolete-options.nix index 4c39aff..fc10181 100644 --- a/modules/obsolete-options.nix +++ b/modules/obsolete-options.nix @@ -22,6 +22,8 @@ in { (mkRenamedOptionModule [ "services" "liquidd" "bind" ] [ "services" "liquidd" "address" ]) (mkRenamedOptionModule [ "services" "liquidd" "rpcbind" ] [ "services" "liquidd" "rpc" "address" ]) + (mkRenamedOptionModule [ "nix-bitcoin" "setup-secrets" ] [ "nix-bitcoin" "setupSecrets" ]) + (mkRenamedAnnounceTorOption "clightning") (mkRenamedAnnounceTorOption "lnd") ]; diff --git a/modules/secrets/generate-secrets.nix b/modules/secrets/generate-secrets.nix index 83ef423..9061a62 100644 --- a/modules/secrets/generate-secrets.nix +++ b/modules/secrets/generate-secrets.nix @@ -1,26 +1,4 @@ -{ config, pkgs, lib, ... }: - -# This is mainly for testing. -# When using this for regular deployments, make sure to create a backup of the -# generated secrets. - -with lib; -{ - nix-bitcoin.setup-secrets = true; - - systemd.services.generate-secrets = { - requiredBy = [ "setup-secrets.service" ]; - before = [ "setup-secrets.service" ]; - serviceConfig = { - Type = "oneshot"; - RemainAfterExit = true; - }; - script = '' - mkdir -p "${config.nix-bitcoin.secretsDir}" - cd "${config.nix-bitcoin.secretsDir}" - chown root: . - chmod 0700 . - ${config.nix-bitcoin.pkgs.generate-secrets} - ''; - }; -} +throw '' + The module `generate-secrets.nix` has been removed. + Set option `nix-bitcoin.generateSecrets = true;` instead. +'' diff --git a/modules/secrets/secrets.nix b/modules/secrets/secrets.nix index 5358bae..83f84ea 100644 --- a/modules/secrets/secrets.nix +++ b/modules/secrets/secrets.nix @@ -3,9 +3,6 @@ with lib; let cfg = config.nix-bitcoin; - setupSecrets = concatStrings (mapAttrsToList (n: v: '' - setupSecret ${n} ${v.user} ${v.group} ${v.permissions} } - '') cfg.secrets); in { options.nix-bitcoin = { @@ -15,6 +12,24 @@ in description = "Directory to store secrets"; }; + setupSecrets = mkOption { + type = types.bool; + default = false; + description = '' + Set permissions for existing secrets in `nix-bitcoin.secretsDir`. + ''; + }; + + generateSecrets = mkOption { + type = types.bool; + default = false; + description = '' + Automatically generate all required secrets. + Make sure to create a backup of the generated secrets. + ''; + }; + + # Currently, this is used only by ../deployment/nixops.nix deployment.secretsDir = mkOption { type = types.path; description = '' @@ -43,27 +58,34 @@ in } )); }; - - setup-secrets = mkEnableOption "Set permissions for secrets generated by 'generate-secrets.sh'"; }; - config = mkIf cfg.setup-secrets { - systemd.targets.nix-bitcoin-secrets = { - requires = [ "setup-secrets.service" ]; - after = [ "setup-secrets.service" ]; - }; + config = { + systemd.targets.nix-bitcoin-secrets = {}; + + nix-bitcoin.setupSecrets = mkIf cfg.generateSecrets true; # Operation of this service: # - Set owner and permissions for all used secrets # - Make all other secrets accessible to root only # For all steps make sure that no secrets are copied to the nix store. # - systemd.services.setup-secrets = { + systemd.services.setup-secrets = mkIf cfg.setupSecrets { + requiredBy = [ "nix-bitcoin-secrets.target" ]; + before = [ "nix-bitcoin-secrets.target" ]; serviceConfig = { Type = "oneshot"; RemainAfterExit = true; }; script = '' + ${optionalString cfg.generateSecrets '' + mkdir -p "${cfg.secretsDir}" + cd "${cfg.secretsDir}" + chown root: . + chmod 0700 . + ${cfg.pkgs.generate-secrets} + ''} + setupSecret() { file="$1" user="$2" @@ -87,7 +109,11 @@ in cd "$dir" processedFiles=() - ${setupSecrets} + ${ + concatStrings (mapAttrsToList (n: v: '' + setupSecret ${n} ${v.user} ${v.group} ${v.permissions} } + '') cfg.secrets) + } # Make all other files accessible to root only unprocessedFiles=$(comm -23 <(printf '%s\n' *) <(printf '%s\n' "''${processedFiles[@]}" | sort)) diff --git a/test/tests.nix b/test/tests.nix index c7d11a2..f9db2e0 100644 --- a/test/tests.nix +++ b/test/tests.nix @@ -12,10 +12,10 @@ let testEnv = rec { imports = [ ./lib/test-lib.nix ../modules/modules.nix - ../modules/secrets/generate-secrets.nix { # Features required by the Python test suite nix-bitcoin.secretsDir = "/secrets"; + nix-bitcoin.generateSecrets = true; nix-bitcoin.operator.enable = true; environment.systemPackages = with pkgs; [ jq ]; } @@ -80,8 +80,8 @@ let testEnv = rec { tests.backups = cfg.backups.enable; # To test that unused secrets are made inaccessible by 'setup-secrets' - systemd.services.generate-secrets.postStart = mkIfTest "security" '' - install -o nobody -g nogroup -m777 <(:) /secrets/dummy + systemd.services.setup-secrets.preStart = mkIfTest "security" '' + install -D -o nobody -g nogroup -m777 <(:) /secrets/dummy ''; } (mkIf config.test.features.clightningPlugins { From b0c223e71676c26a76aaf6704f347e8089a8d68d Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Wed, 10 Mar 2021 14:08:35 +0100 Subject: [PATCH 07/15] secrets: don't stop services when the secrets target stops With krops or nixops the secrets target is always restarted during the deployment process. This previously caused unnecessary restarts of all nix-bitcoin services. --- modules/bitcoind.nix | 7 ++++++- modules/secrets/secrets.nix | 11 +++++++++-- 2 files changed, 15 insertions(+), 3 deletions(-) diff --git a/modules/bitcoind.nix b/modules/bitcoind.nix index f5fce4d..ad804c3 100644 --- a/modules/bitcoind.nix +++ b/modules/bitcoind.nix @@ -312,9 +312,14 @@ in { ]; systemd.services.bitcoind = { - requires = [ "nix-bitcoin-secrets.target" ]; + # Use `wants` instead of `requires` so that bitcoind and all dependent services + # are not restarted when the secrets target restarts. + # The secrets target always restarts when deploying with one of the methods + # in ./deployment. + wants = [ "nix-bitcoin-secrets.target" ]; after = [ "network.target" "nix-bitcoin-secrets.target" ]; wantedBy = [ "multi-user.target" ]; + preStart = let extraRpcauth = concatMapStrings (name: let user = cfg.rpc.users.${name}; diff --git a/modules/secrets/secrets.nix b/modules/secrets/secrets.nix index 83f84ea..ccb495f 100644 --- a/modules/secrets/secrets.nix +++ b/modules/secrets/secrets.nix @@ -60,8 +60,15 @@ in }; }; - config = { - systemd.targets.nix-bitcoin-secrets = {}; + config = { + # This target is active when secrets have been setup successfully. + systemd.targets.nix-bitcoin-secrets = { + # This ensures that the secrets target is always activated when switching + # configurations. + # In this way `switch-to-configuration` is guaranteed to show an error + # when activating the secrets target fails on deployment. + wantedBy = [ "multi-user.target" ]; + }; nix-bitcoin.setupSecrets = mkIf cfg.generateSecrets true; From 55d87490ec330b2fe78b4cc9275db287990f59fc Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Wed, 10 Mar 2021 14:08:36 +0100 Subject: [PATCH 08/15] secrets: make configuration more robust - Fail at evaluation when secrets setup is not configured. Previously, bitcoind failed at runtime due to the missing secrets target. - Fail at evaluation when conflicting secrets setup methods are used. This happens when `secretsSetupMethod` has more than one definition. --- modules/deployment/nixops.nix | 2 ++ modules/secrets/secrets.nix | 18 +++++++++++++++++- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/modules/deployment/nixops.nix b/modules/deployment/nixops.nix index 50f9468..980d91c 100644 --- a/modules/deployment/nixops.nix +++ b/modules/deployment/nixops.nix @@ -1,5 +1,7 @@ { config, ... }: { + nix-bitcoin.secretsSetupMethod = "nixops"; + deployment.keys = builtins.mapAttrs (n: v: { keyFile = "${config.nix-bitcoin.deployment.secretsDir}/${n}"; destDir = config.nix-bitcoin.secretsDir; diff --git a/modules/secrets/secrets.nix b/modules/secrets/secrets.nix index ccb495f..30618e8 100644 --- a/modules/secrets/secrets.nix +++ b/modules/secrets/secrets.nix @@ -58,11 +58,25 @@ in } )); }; + + secretsSetupMethod = mkOption { + type = types.str; + default = throw '' + Error: No secrets setup method has been defined. + To fix this, choose one of the following: + + - Use one of the deployment methods in ${toString ./../deployment} + + - Set `nix-bitcoin.generateSecrets = true` to automatically generate secrets + + - Set `nix-bitcoin.secretsSetupMethod = "manual"` if you want to manually setup secrets + ''; + }; }; config = { # This target is active when secrets have been setup successfully. - systemd.targets.nix-bitcoin-secrets = { + systemd.targets.nix-bitcoin-secrets = mkIf (cfg.secretsSetupMethod != "manual") { # This ensures that the secrets target is always activated when switching # configurations. # In this way `switch-to-configuration` is guaranteed to show an error @@ -72,6 +86,8 @@ in nix-bitcoin.setupSecrets = mkIf cfg.generateSecrets true; + nix-bitcoin.secretsSetupMethod = mkIf cfg.setupSecrets "setup-secrets"; + # Operation of this service: # - Set owner and permissions for all used secrets # - Make all other secrets accessible to root only From 03db1a61b1121954460300aeb3651c09b052169b Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Wed, 10 Mar 2021 14:08:37 +0100 Subject: [PATCH 09/15] lnd, joinmarket: don't write to secrets dir Keeping the secrets dir read-only is more simple and robust. - lnd seed mnemonic creation and joinmarket wallet creation can be run as the regular service user instead of root. - It is easier to switch to a third-party secrets deployment method in the future. Don't create a seed mnemonic for lnd when a wallet exists. This avoids creating unused mnemonics and helps simplifying the migration command in `versioning.nix`. --- examples/configuration.nix | 11 +++++++---- modules/backups.nix | 4 +--- modules/joinmarket.nix | 9 ++++----- modules/lnd.nix | 29 ++++++++++++----------------- modules/versioning.nix | 22 ++++++++++++++++++++++ test/tests.py | 14 ++++++++++---- 6 files changed, 56 insertions(+), 33 deletions(-) diff --git a/examples/configuration.nix b/examples/configuration.nix index 85c44eb..0b61906 100644 --- a/examples/configuration.nix +++ b/examples/configuration.nix @@ -71,11 +71,13 @@ ## WARNING # If you use lnd, you should manually backup your wallet mnemonic # seed. This will allow you to recover on-chain funds. You can run the - # following command after the lnd service starts: - # scp bitcoin-node:/secrets/lnd-seed-mnemonic ./secrets/lnd-seed-mnemonic + # following commands after the lnd service starts: + # mkdir -p ./backups/lnd/ + # scp bitcoin-node:/var/lib/lnd/lnd-seed-mnemonic ./backups/lnd/ + # # You should also backup your channel state after opening new channels. # This will allow you to recover off-chain funds, by force-closing channels. - # scp bitcoin-node:/var/lib/lnd/chain/bitcoin/mainnet/channel.backup /my-backup-path/channel.backup + # scp bitcoin-node:/var/lib/lnd/chain/bitcoin/mainnet/channel.backup ./backups/lnd/ ### SPARK WALLET # Enable this module to use spark-wallet, a minimalistic wallet GUI for @@ -229,5 +231,6 @@ # The nix-bitcoin release version that your config is compatible with. # When upgrading to a backwards-incompatible release, nix-bitcoin will display an # an error and provide hints for migrating your config to the new release. - nix-bitcoin.configVersion = "0.0.30"; + nix-bitcoin.configVersion = "0.0.41"; + } diff --git a/modules/backups.nix b/modules/backups.nix index 2669620..6209d64 100644 --- a/modules/backups.nix +++ b/modules/backups.nix @@ -4,7 +4,6 @@ with lib; let cfg = config.services.backups; - secretsDir = config.nix-bitcoin.secretsDir; filelist = pkgs.writeText "filelist.txt" '' ${optionalString (!cfg.with-bulk-data) "- ${config.services.bitcoind.dataDir}/blocks"} @@ -12,7 +11,6 @@ let ${config.services.bitcoind.dataDir} ${config.services.clightning.dataDir} ${config.services.lnd.dataDir} - ${secretsDir}/lnd-seed-mnemonic ${optionalString (!cfg.with-bulk-data) "- ${config.services.liquidd.dataDir}/*/blocks"} ${optionalString (!cfg.with-bulk-data) "- ${config.services.liquidd.dataDir}/*/chainstate"} ${config.services.liquidd.dataDir} @@ -20,8 +18,8 @@ let ${config.services.nbxplorer.dataDir} ${config.services.btcpayserver.dataDir} ${config.services.joinmarket.dataDir} - ${secretsDir}/jm-wallet-seed ${config.services.postgresqlBackup.location}/btcpaydb.sql.gz + ${optionalString config.nix-bitcoin.generateSecrets "${config.nix-bitcoin.secretsDir}"} /var/lib/tor # Extra files ${cfg.extraFiles} diff --git a/modules/joinmarket.nix b/modules/joinmarket.nix index 1dcb83a..8f46652 100644 --- a/modules/joinmarket.nix +++ b/modules/joinmarket.nix @@ -240,20 +240,19 @@ in { ''; # Generating wallets (jmclient/wallet.py) is only supported for mainnet or testnet ExecStartPost = mkIf (bitcoind.network == "mainnet") - (nbLib.privileged "joinmarket-create-wallet" '' + (nbLib.script "joinmarket-create-wallet" '' walletname=wallet.jmdat wallet=${cfg.dataDir}/wallets/$walletname if [[ ! -f $wallet ]]; then echo "Create wallet" pw=$(cat "${secretsDir}"/jm-wallet-password) cd ${cfg.dataDir} - if ! ${pkgs.utillinux}/bin/runuser -u ${cfg.user} -- \ - ${nbPkgs.joinmarket}/bin/jm-genwallet --datadir=${cfg.dataDir} $walletname $pw \ + if ! ${nbPkgs.joinmarket}/bin/jm-genwallet --datadir=${cfg.dataDir} $walletname $pw \ | grep 'recovery_seed' \ | cut -d ':' -f2 \ - | (umask u=r,go=; cat > "${secretsDir}/jm-wallet-seed"); then + | (umask u=r,go=; cat > jm-wallet-seed); then echo "wallet creation failed" - rm -f "$wallet" "${secretsDir}/jm-wallet-seed" + rm -f "$wallet" jm-wallet-seed exit 1 fi fi diff --git a/modules/lnd.nix b/modules/lnd.nix index aa6aada..00493a5 100644 --- a/modules/lnd.nix +++ b/modules/lnd.nix @@ -200,32 +200,28 @@ in { ExecStartPost = let restUrl = "https://${cfg.restAddress}:${toString cfg.restPort}/v1"; in [ - # Run fully privileged for secrets dir write access - (nbLib.privileged "lnd-create-mnemonic" '' + (nbLib.script "lnd-create-wallet" '' attempts=250 while ! { exec 3>/dev/tcp/${cfg.restAddress}/${toString cfg.restPort} && exec 3>&-; } &>/dev/null; do ((attempts-- == 0)) && { echo "lnd REST service unreachable"; exit 1; } sleep 0.1 done - mnemonic=${secretsDir}/lnd-seed-mnemonic - if [[ ! -f $mnemonic ]]; then - echo Create lnd seed - umask u=r,go= - ${pkgs.curl}/bin/curl -s \ - --cacert ${secretsDir}/lnd-cert \ - -X GET ${restUrl}/genseed | ${pkgs.jq}/bin/jq -c '.cipher_seed_mnemonic' > "$mnemonic" - fi - chown ${cfg.user}: "$mnemonic" - '') - (nbLib.script "lnd-create-wallet" '' if [[ ! -f ${networkDir}/wallet.db ]]; then - echo Create lnd wallet + mnemonic="${cfg.dataDir}/lnd-seed-mnemonic" + if [[ ! -f "$mnemonic" ]]; then + echo Create lnd seed + umask u=r,go= + ${pkgs.curl}/bin/curl -s \ + --cacert ${secretsDir}/lnd-cert \ + -X GET ${restUrl}/genseed | ${pkgs.jq}/bin/jq -c '.cipher_seed_mnemonic' > "$mnemonic" + fi + echo Create lnd wallet ${pkgs.curl}/bin/curl -s --output /dev/null --show-error \ --cacert ${secretsDir}/lnd-cert \ -X POST -d "{\"wallet_password\": \"$(cat ${secretsDir}/lnd-wallet-password | tr -d '\n' | base64 -w0)\", \ - \"cipher_seed_mnemonic\": $(cat ${secretsDir}/lnd-seed-mnemonic | tr -d '\n')}" \ + \"cipher_seed_mnemonic\": $(cat "$mnemonic" | tr -d '\n')}" \ ${restUrl}/initwallet # Guarantees that RPC calls with cfg.cli succeed after the service is started @@ -248,9 +244,8 @@ in { while ! { exec 3>/dev/tcp/${cfg.rpcAddress}/${toString cfg.rpcPort}; } &>/dev/null; do sleep 0.1 done - '') - # Run fully privileged for chown + # Setting macaroon permission for other users needs root permissions (nbLib.privileged "lnd-create-macaroons" '' umask ug=r,o= ${lib.concatMapStrings (macaroon: '' diff --git a/modules/versioning.nix b/modules/versioning.nix index f06a593..f34eaac 100644 --- a/modules/versioning.nix +++ b/modules/versioning.nix @@ -69,6 +69,28 @@ let (mkOnionServiceChange "clightning") (mkOnionServiceChange "lnd") (mkOnionServiceChange "btcpayserver") + { + version = "0.0.41"; + condition = config.services.lnd.enable || config.services.joinmarket.enable; + message = let + secretsDir = config.nix-bitcoin.secretsDir; + lnd = config.services.lnd; + jm = config.services.joinmarket; + in '' + Secret files generated by services at runtime are now stored in the service + data dirs instead of the global secrets dir. + + To migrate, run the following Bash script as root on your nix-bitcoin node: + + if [[ -e ${secretsDir}/lnd-seed-mnemonic ]]; then + install -o ${lnd.user} -g ${lnd.group} -m400 "${secretsDir}/lnd-seed-mnemonic" "${lnd.dataDir}" + fi + if [[ -e ${secretsDir}/jm-wallet-seed ]]; then + install -o ${jm.user} -g ${jm.group} -m400 "${secretsDir}/jm-wallet-seed" "${jm.dataDir}" + fi + rm -f "${secretsDir}"/{lnd-seed-mnemonic,jm-wallet-seed} + ''; + } ]; incompatibleChanges = optionals diff --git a/test/tests.py b/test/tests.py index dd37785..48cc4ca 100644 --- a/test/tests.py +++ b/test/tests.py @@ -326,16 +326,22 @@ def _(): files = { "bitcoind": "var/lib/bitcoind/test/wallet.dat", "clightning": "var/lib/clightning/bitcoin/hsm_secret", - "lnd": "secrets/lnd-seed-mnemonic", - "joinmarket": "secrets/jm-wallet-seed", + "lnd": "var/lib/lnd/lnd-seed-mnemonic", + "joinmarket": "var/lib/joinmarket/jm-wallet-seed", "btcpayserver": "var/backup/postgresql/btcpaydb.sql.gz", } actual_files = succeed(f"{run_duplicity} list-current-files file:///var/lib/localBackups") - for test, file in files.items(): - if test in enabled_tests and file not in actual_files: + def assert_file_exists(file): + if file not in actual_files: raise Exception(f"Backup file '{file}' is missing.") + for test, file in files.items(): + if test in enabled_tests: + assert_file_exists(file) + + assert_file_exists("secrets/lnd-wallet-password") + # Impure: restarts services @test("banlist-and-restart") From e017675d5e961d622314663e711ae752bac222a2 Mon Sep 17 00:00:00 2001 From: Jonas Nick Date: Wed, 10 Mar 2021 14:08:38 +0100 Subject: [PATCH 10/15] krops: add package --- pkgs/default.nix | 1 + pkgs/krops/default.nix | 12 ++++++++++++ pkgs/krops/get-sha256.sh | 14 ++++++++++++++ 3 files changed, 27 insertions(+) create mode 100644 pkgs/krops/default.nix create mode 100755 pkgs/krops/get-sha256.sh diff --git a/pkgs/default.nix b/pkgs/default.nix index 9a2e9e8..4275100 100644 --- a/pkgs/default.nix +++ b/pkgs/default.nix @@ -8,6 +8,7 @@ let self = { joinmarket = pkgs.callPackage ./joinmarket { inherit (self) nbPython3Packages; }; generate-secrets = pkgs.callPackage ./generate-secrets { }; nixops19_09 = pkgs.callPackage ./nixops { }; + krops = import ./krops { }; netns-exec = pkgs.callPackage ./netns-exec { }; lightning-loop = pkgs.callPackage ./lightning-loop { }; lightning-pool = pkgs.callPackage ./lightning-pool { }; diff --git a/pkgs/krops/default.nix b/pkgs/krops/default.nix new file mode 100644 index 0000000..a80ceba --- /dev/null +++ b/pkgs/krops/default.nix @@ -0,0 +1,12 @@ +{ pkgs ? import {} }: + +let + src = pkgs.fetchgit { + url = "https://cgit.krebsco.de/krops"; + rev = "804c79a14dc8f81a602d31d5a1eed5f82b3f2457"; + sha256 = "1k20l7zqprsrm9s38xslr7190vssf4sjdprd9gh146hxlvln2qrf"; + }; +in { + lib = import "${src}/lib"; + pkgs = import "${src}/pkgs" {}; +} diff --git a/pkgs/krops/get-sha256.sh b/pkgs/krops/get-sha256.sh new file mode 100755 index 0000000..91b4049 --- /dev/null +++ b/pkgs/krops/get-sha256.sh @@ -0,0 +1,14 @@ +#! /usr/bin/env nix-shell +#! nix-shell -i bash -p git +set -euo pipefail + +archive_hash () { + repo=$1 + rev=$2 + nix-prefetch-url --unpack "https://github.com/${repo}/archive/${rev}.tar.gz" 2> /dev/null | tail -n 1 +} + +echo "Fetching latest krops commit" +latest=$(git ls-remote https://github.com/krebs/krops master | cut -f 1) +echo "rev: ${latest}" +echo "sha256: $(archive_hash krebs/krops $latest)" From fe118b28ffa3c65b45a8cbbe6f637b8f465fe65d Mon Sep 17 00:00:00 2001 From: Jonas Nick Date: Wed, 10 Mar 2021 14:08:39 +0100 Subject: [PATCH 11/15] examples: add krops deployment method --- examples/krops/deploy.nix | 20 ++++++++++++++++ examples/krops/krops-configuration.nix | 7 ++++++ examples/krops/sources.nix | 33 ++++++++++++++++++++++++++ examples/shell.nix | 7 ++++++ modules/deployment/krops.nix | 24 +++++++++++++++++++ 5 files changed, 91 insertions(+) create mode 100644 examples/krops/deploy.nix create mode 100644 examples/krops/krops-configuration.nix create mode 100644 examples/krops/sources.nix create mode 100644 modules/deployment/krops.nix diff --git a/examples/krops/deploy.nix b/examples/krops/deploy.nix new file mode 100644 index 0000000..7ed634f --- /dev/null +++ b/examples/krops/deploy.nix @@ -0,0 +1,20 @@ +let + # FIXME: + target = "root@HOSTNAME_OR_IP_ADDRESS"; + + extraSources = { + "hardware-configuration.nix".file = toString ../hardware-configuration.nix; + }; + + krops = (import {}).krops; +in +krops.pkgs.krops.writeDeploy "deploy" { + inherit target; + + source = import ./sources.nix { inherit extraSources krops; }; + + # Avoid having to create a sentinel file. + # Otherwise /var/src/.populate must be created on the target node to signal krops + # that it is allowed to deploy. + force = true; +} diff --git a/examples/krops/krops-configuration.nix b/examples/krops/krops-configuration.nix new file mode 100644 index 0000000..0182bca --- /dev/null +++ b/examples/krops/krops-configuration.nix @@ -0,0 +1,7 @@ +# This file allows you to build your krops configuration locally +{ + imports = [ + ../configuration.nix + + ]; +} diff --git a/examples/krops/sources.nix b/examples/krops/sources.nix new file mode 100644 index 0000000..64ae641 --- /dev/null +++ b/examples/krops/sources.nix @@ -0,0 +1,33 @@ +{ extraSources, krops }: + +krops.lib.evalSource [({ + nixos-config.file = builtins.toFile "nixos-config" '' + { + imports = [ + ./configuration.nix + + ]; + } + ''; + + "configuration.nix".file = toString ../configuration.nix; + + # Enable `useChecksum` for sources which might be located in the nix store + # and which therefore might have static timestamps. + + nixpkgs.file = { + path = toString ; + useChecksum = true; + }; + + nix-bitcoin.file = { + path = toString ; + useChecksum = true; + filters = [{ + type = "exclude"; + pattern = ".git"; + }]; + }; + + secrets.file = toString ../secrets; +} // extraSources)] diff --git a/examples/shell.nix b/examples/shell.nix index 8f57cd3..7f30899 100644 --- a/examples/shell.nix +++ b/examples/shell.nix @@ -28,6 +28,13 @@ stdenv.mkDerivation rec { alias fetch-release="${toString nix-bitcoin-path}/helper/fetch-release" + krops-deploy() { + # Ensure strict permissions on secrets/ directory before rsyncing it to + # the target machine + chmod 700 ${toString ./secrets} + $(nix-build --no-out-link ${toString ./krops/deploy.nix}) + } + figlet "nix-bitcoin" (mkdir -p secrets; cd secrets; env -i ${nix-bitcoin.generate-secrets}) diff --git a/modules/deployment/krops.nix b/modules/deployment/krops.nix new file mode 100644 index 0000000..a31a01b --- /dev/null +++ b/modules/deployment/krops.nix @@ -0,0 +1,24 @@ +{ lib, ... }: +{ + nix-bitcoin = { + secretsDir = "/var/src/secrets"; + setupSecrets = true; + }; + environment.variables.NIX_PATH = lib.mkForce "/var/src"; + + # The file syncing step in krops resets the secrets file permissions. + # So force `setup-secrets.service` to restart on deployment. + # Stop it at activation start so that it gets restarted at the end. + system.activationScripts.nixBitcoinStopSetupSecrets = '' + ${/* Skip this step if systemd is not running, i.e. when booting or in nixos-install */ ""} + if [[ -e /run/systemd/system ]]; then + if ! output=$(/run/current-system/systemd/bin/systemctl stop setup-secrets.service --no-block 2>&1); then + # Ignore if the unit is not loaded, which can happen on the first deployment + if [[ $output != *setup-secrets.service\ not\ loaded* ]]; then + echo "$output" + false + fi + fi + fi + ''; +} From e8b47f099c9a149b85300f30d41455c41fc1b4e8 Mon Sep 17 00:00:00 2001 From: Erik Arvstedt Date: Wed, 10 Mar 2021 14:08:40 +0100 Subject: [PATCH 12/15] examples: add deploy-krops.sh --- examples/README.md | 4 + examples/deploy-krops.sh | 122 ++++++++++++++++++++++++++++ examples/krops-vm-configuration.nix | 8 ++ test/run-tests.sh | 1 + 4 files changed, 135 insertions(+) create mode 100755 examples/deploy-krops.sh create mode 100644 examples/krops-vm-configuration.nix diff --git a/examples/README.md b/examples/README.md index c154bfe..aca0366 100644 --- a/examples/README.md +++ b/examples/README.md @@ -20,6 +20,10 @@ By default, [`configuration.nix`](configuration.nix) enables `bitcoind` and `cli - [`./deploy-qemu-vm.sh`](deploy-qemu-vm.sh) creates a QEMU VM.\ Requires: [Nix](https://nixos.org/nix/) +- [`./deploy-krops.sh`](deploy-krops.sh) creates a QEMU VM and deploys a + nix-bitcoin configuration to it using [krops](https://github.com/krebs/krops).\ + Requires: [Nix](https://nixos.org/nix/) + - [`./deploy-container-minimal.sh`](deploy-container-minimal.sh) creates a container defined by [minimal-configuration.nix](minimal-configuration.nix) that doesn't use the [secure-node.nix](../modules/presets/secure-node.nix) preset. diff --git a/examples/deploy-krops.sh b/examples/deploy-krops.sh new file mode 100755 index 0000000..ca6f42e --- /dev/null +++ b/examples/deploy-krops.sh @@ -0,0 +1,122 @@ +#!/usr/bin/env bash +set -euo pipefail + +# This script demonstrates how to setup a nix-bitcoin node with krops. +# The node is deployed to a minimal NixOS QEMU VM. +# Running this script leaves no traces on your host system. + +# This demo is a template for your own experiments. +# Run with option `--interactive` or `-i` to start a shell for interacting with +# the node. + +# MAKE SURE TO REPLACE the SSH identity file if you use this script for +# anything serious. + +if [[ ! -v IN_NIX_SHELL ]]; then + echo "Running script in nix shell env..." + cd "${BASH_SOURCE[0]%/*}" + exec nix-shell --run "./${BASH_SOURCE[0]##*/} $*" +fi + +source qemu-vm/run-vm.sh + +echo "Building the target VM" +# Build the initial VM to which the nix-bitcoin node is deployed via krops +nix-build --out-link $tmpDir/vm - <<'EOF' +(import { + configuration = { lib, ... }: { + imports = [ ]; + services.openssh.enable = true; + + # Silence the following warning that appears when deploying via krops: + # warning: Nix search path entry '/nix/var/nix/profiles/per-user/root/channels' does not exist, ignoring + nix.nixPath = lib.mkForce []; + }; +}).vm +EOF + +vmNumCPUs=4 +vmMemoryMiB=2048 +sshPort=60734 +# Start the VM in the background +runVM $tmpDir/vm $vmNumCPUs $vmMemoryMiB $sshPort + +# Build the krops deploy script +export sshPort +nix-build --out-link $tmpDir/krops-deploy - <<'EOF' +let + krops = (import {}).krops; + + extraSources = { + # Skip uploading nixpkgs to the target node. + # This works because /nix/store is shared with the target VM. + nixpkgs.symlink = toString ; + + nixos-config.file = toString ; + + qemu-vm.file = toString ; + }; +in +krops.pkgs.krops.writeCommand "krops-deploy" { + source = import { inherit extraSources krops; }; + force = true; + target = { + user = "root"; + host = "127.0.0.1"; + port = builtins.getEnv "sshPort"; + extraOptions = [ + "-i" (toString ) "-oConnectTimeout=1" + "-oStrictHostKeyChecking=no" "-oUserKnownHostsFile=/dev/null" "-oLogLevel=ERROR" + "-oControlMaster=auto" "-oControlPath=${builtins.getEnv "tmpDir"}/ssh-connection" "-oControlPersist=60" + ]; + }; + + # "test" instead of "switch" to avoid installing a bootloader which + # is not possible in this VM + command = targetPath: '' + nixos-rebuild test -I /var/src + ''; +} +EOF + +echo "Building the nix-bitcoin node" +# Pre-build the nix-bitcoin node outside of the VM to save some time +nix-build --out-link $tmpDir/store-paths -E ' +let + system = (import { configuration = ; }).system; + pkgsUnstable = (import ).nixpkgs-unstable; + pkgs = import {}; +in + pkgs.closureInfo { rootPaths = [ system pkgsUnstable ]; } +' > /dev/null + +vmWaitForSSH + +# Add the store paths that include the nix-bitcoin node +# to the nix store db in the VM +c "nix-store --load-db < $(realpath $tmpDir/store-paths)/registration" + +echo +echo "Deploy with krops" +$tmpDir/krops-deploy + +echo +echo "Bitcoind service:" +c systemctl status bitcoind +echo +echo "Bitcoind network:" +c bitcoin-cli getnetworkinfo +echo +echo "lightning-cli state:" +c lightning-cli getinfo +echo +echo "Node info:" +c nodeinfo + +case ${1:-} in + -i|--interactive) + . start-bash-session.sh + ;; +esac + +# Cleanup happens at exit (defined in qemu-vm/run-vm.sh) diff --git a/examples/krops-vm-configuration.nix b/examples/krops-vm-configuration.nix new file mode 100644 index 0000000..7815347 --- /dev/null +++ b/examples/krops-vm-configuration.nix @@ -0,0 +1,8 @@ +{ lib, ... }: { + imports = [ + ./configuration.nix + + + + ]; +} diff --git a/test/run-tests.sh b/test/run-tests.sh index 3f30408..7bfceb7 100755 --- a/test/run-tests.sh +++ b/test/run-tests.sh @@ -249,6 +249,7 @@ examples() { set -e ./deploy-container.sh ./deploy-qemu-vm.sh + ./deploy-krops.sh " (cd $scriptDir/../examples && nix-shell --run "$script") } From eb6dff54788b7fc77998f124c2ba3e8cc4012c9f Mon Sep 17 00:00:00 2001 From: Jonas Nick Date: Wed, 10 Mar 2021 14:08:41 +0100 Subject: [PATCH 13/15] docs: various improvements to installation tutorial - Update nixos iso - Make sure to be root before parting and give instructions for checking UEFI vs. legacy - Instead of making user replace automatically generated nixos config, explain how to add options. - Add boot.loader options directly to hardware-configuration.nix - Copy .gitignore when setting up a deployment dir - Stress that the first config that is deployed remotely must include ssh key. - Debian: 9 Strech -> Debian 10 Buster - nix: 2.3.3 -> 2.3.10 --- .gitignore | 1 - docs/install.md | 139 ++++++++++++++++++++++++-------------------- examples/.gitignore | 1 + 3 files changed, 76 insertions(+), 65 deletions(-) delete mode 100644 .gitignore create mode 100644 examples/.gitignore diff --git a/.gitignore b/.gitignore deleted file mode 100644 index dc05ada..0000000 --- a/.gitignore +++ /dev/null @@ -1 +0,0 @@ -/examples/secrets/ diff --git a/docs/install.md b/docs/install.md index 1826e0a..5d5a9c0 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,6 +1,6 @@ Preliminary steps --- -Get a machine to deploy nix-bitcoin on. +Get a machine to deploy nix-bitcoin on (see [hardware.md](hardware.md)). # Tutorials @@ -26,23 +26,34 @@ This is borrowed from the [NixOS manual](https://nixos.org/nixos/manual/index.ht 1. Obtain latest [NixOS](https://nixos.org/nixos/download.html). For example: ``` - wget https://releases.nixos.org/nixos/19.09/nixos-19.09.2284.bf7c0f0461e/nixos-minimal-19.09.2284.bf7c0f0461e-x86_64-linux.iso - sha256sum nixos-minimal-19.09.2284.bf7c0f0461e-x86_64-linux.iso - # output: 9768eb945bef410fccfb82cb3d2e7ce7c02c3430aed0f2f1527273cb080fff3e + wget https://releases.nixos.org/nixos/20.09/nixos-20.09.2405.e065200fc90/nixos-minimal-20.09.2405.e065200fc90-i686-linux.iso + sha256sum nixos-minimal-20.09.2405.e065200fc90-x86_64-linux.iso + # output: 5fc182e27a71a297b041b5c287558b21bdabde7068d4fc049752dad3025df867 ``` Alternatively you can build NixOS from source by following the instructions at https://nixos.org/nixos/manual/index.html#sec-building-cd. 2. Write NixOS iso to install media (USB/CD). For example: ``` - cp nixos-minimal-19.09.2284.bf7c0f0461e-x86_64-linux.iso /dev/sdX + cp nixos-minimal-20.09.2405.e065200fc90-x86_64-linux.iso /dev/sdX ``` Replace /dev/sdX with the correct device name. You can find this using `sudo fdisk -l` -3. Boot the system +3. Boot the system and become root + + ``` + sudo -i + ``` + + You will have to find out if your hardware uses UEFI or Legacy Boot for the next step. You can do that, for example, by executing + + ``` + ls /sys/firmware/efi + ``` + + If the file exists exists, you should continue the installation for UEFI otherwise for Legacy Boot. - You will have to find out if your hardware uses UEFI or Legacy Boot for the next step. 4. Option 1: Partition and format for UEFI @@ -84,52 +95,64 @@ This is borrowed from the [NixOS manual](https://nixos.org/nixos/manual/index.ht nano /mnt/etc/nixos/configuration.nix ``` - Option 1: Edit NixOS configuration for UEFI + We now need to adjust the configuration to make sure that we can ssh into the system and that it boots correctly. We add some lines to set `services.openssh` such that the configuration looks as follows: ``` - { config, pkgs, ... }: { + { config, pkgs, ... }: + + { imports = [ - # Include the results of the hardware scan. - ./hardware-configuration.nix + ... ]; + # Enable the OpenSSH server. + services.openssh = { + enable = true; + permitRootLogin = "yes"; + }; + + # The rest of the file are default options and hints. + } + ``` + + Now we open `hardware-configuration.nix` + + ``` + nano /mnt/etc/nixos/hardware-configuration.nix + ``` + + which will look similar to + + ``` + { config, pkgs, ... }: + + { + imports = [ ]; + + # Add line here as explained below + + # The rest of the file are generated options. + } + ``` + + Now add one of the following lines to the location mentioned in above example hardware config. + + **Option 1**: UEFI + + ``` boot.loader.systemd-boot.enable = true; - - # Note: setting fileSystems is generally not - # necessary, since nixos-generate-config figures them out - # automatically in hardware-configuration.nix. - #fileSystems."/".device = "/dev/disk/by-label/nixos"; - - # Enable the OpenSSH server. - services.openssh = { - enable = true; - permitRootLogin = "yes"; - }; - } ``` - Option 2: Edit NixOS configuration for Legacy Boot (MBR) + **Option 2**: Legacy Boot (MBR) ``` - { config, pkgs, ... }: { - imports = [ - # Include the results of the hardware scan. - ./hardware-configuration.nix - ]; - boot.loader.grub.device = "/dev/sda"; + ``` - # Note: setting fileSystems is generally not - # necessary, since nixos-generate-config figures them out - # automatically in hardware-configuration.nix. - #fileSystems."/".device = "/dev/disk/by-label/nixos"; + Lastly, in rare circumstances the hardware configuration does not have a `fileSystems` option. In that case you need to add it with the folllowing line: - # Enable the OpenSSH server. - services.openssh = { - enable = true; - permitRootLogin = "yes"; - }; - } + ``` + fileSystems."/".device = "/dev/disk/by-label/nixos"; ``` 6. Do the installation @@ -137,7 +160,9 @@ This is borrowed from the [NixOS manual](https://nixos.org/nixos/manual/index.ht ``` nixos-install ``` + Set root password + ``` setting root password... Enter new UNIX password: @@ -154,7 +179,7 @@ This is borrowed from the [NixOS manual](https://nixos.org/nixos/manual/index.ht The following steps are meant to be run on the machine you deploy from, not the machine you deploy to. You can also build Nix from source by following the instructions at https://nixos.org/nix/manual/#ch-installing-source. -1. Install Dependencies (Debian 9 stretch) +1. Install Dependencies (Debian 10 Buster) ``` sudo apt-get install curl git gnupg2 dirmngr @@ -163,11 +188,11 @@ You can also build Nix from source by following the instructions at https://nixo 2. Install latest Nix in "multi-user mode" with GPG Verification according to https://nixos.org/nix/download.html ``` - curl -o install-nix-2.3.3 https://releases.nixos.org/nix/nix-2.3.3/install - curl -o install-nix-2.3.3.asc https://releases.nixos.org/nix/nix-2.3.3/install.asc + curl -o install-nix-2.3.10 https://releases.nixos.org/nix/nix-2.3.10/install + curl -o install-nix-2.3.10.asc https://releases.nixos.org/nix/nix-2.3.10/install.asc gpg2 --recv-keys B541D55301270E0BCF15CA5D8170B4726D7198DE - gpg2 --verify ./install-nix-2.3.3.asc - sh ./install-nix-2.3.3 --daemon + gpg2 --verify ./install-nix-2.3.10.asc + sh ./install-nix-2.3.10 --daemon ``` Then follow the instructions. Open a new terminal window when you're done. @@ -217,7 +242,7 @@ You can also build Nix from source by following the instructions at https://nixo mkdir nix-bitcoin-node cd nix-bitcoin-node # TODO - cp -r ../nix-bitcoin/examples/{configuration.nix,shell.nix,nix-bitcoin-release.nix} . + cp -r ../nix-bitcoin/examples/{configuration.nix,shell.nix,nix-bitcoin-release.nix,.gitignore} . ``` ## 4. Deploy with TODO @@ -238,21 +263,7 @@ You can also build Nix from source by following the instructions at https://nixo Copy contents of your NixOS machine's `/etc/nixos/hardware-configuration.nix` to this file. -4. Add boot option to `hardware-configuration.nix` - - Option 1: Enable systemd boot for UEFI - - ``` - boot.loader.systemd-boot.enable = true; - ``` - - Option 2: Set grub device for Legacy Boot (MBR) - - ``` - boot.loader.grub.device = "/dev/sda"; - ``` - -5. Enter environment +4. Enter environment ``` nix-shell @@ -260,10 +271,10 @@ You can also build Nix from source by following the instructions at https://nixo NOTE that a new directory `secrets/` appeared which contains the secrets for your node. -6. TODO -7. Adjust configuration by opening the `configuration.nix` file and enable/disable the modules you want by editing this file. Pay particular attention to lines that are preceded by `FIXME` comments. +5. TODO +6. Adjust configuration by opening the `configuration.nix` file and enable/disable the modules you want by editing this file. Pay particular attention to lines that are preceded by `FIXME` comments. Make sure to set your SSH pubkey. Otherwise, you loose remote access because the config does not enable `permitRootLogin` (unless you add that manually). -8. TODO +7. TODO For security reasons, all normal system management tasks can and should be performed with the `operator` user. Logging in as `root` should be done as rarely as possible. diff --git a/examples/.gitignore b/examples/.gitignore new file mode 100644 index 0000000..4bd922a --- /dev/null +++ b/examples/.gitignore @@ -0,0 +1 @@ +secrets/ From 7c0b521785377e2e68fca3dc9ab1f7c84af93ded Mon Sep 17 00:00:00 2001 From: Jonas Nick Date: Wed, 10 Mar 2021 14:08:42 +0100 Subject: [PATCH 14/15] docs: describe deploying with krops in installation tutorial --- docs/install.md | 115 +++++++++++++++++++++++++++++++++++------------- docs/usage.md | 4 +- 2 files changed, 87 insertions(+), 32 deletions(-) diff --git a/docs/install.md b/docs/install.md index 5d5a9c0..4af65cf 100644 --- a/docs/install.md +++ b/docs/install.md @@ -1,21 +1,15 @@ -Preliminary steps ---- -Get a machine to deploy nix-bitcoin on (see [hardware.md](hardware.md)). +# Tutorial: Install and configure NixOS for nix-bitcoin on a dedicated machine -# Tutorials - -1. [Install and configure NixOS for nix-bitcoin on your own hardware](#tutorial-install-and-configure-nixos-for-nix-bitcoin-on-your-own-hardware) - ----- - -Tutorial: install and configure NixOS for nix-bitcoin on your own hardware ---- +This tutorial describes how to manage your Bitcoin node comfortably from your personal computer with the deployment tool [krops](https://github.com/krebs/krops). +However, nix-bitcoin is agnostic to the deployment method and can be used with different or without such tools (see [examples](../examples/README.md)). ## 0. Preparation -1. Optional: Make sure you have the latest firmware for your system (BIOS, microcode updates). +1. Find a machine to deploy nix-bitcoin on (see [hardware.md](hardware.md)). -2. Optional: Disable Simultaneous Multi-Threading (SMT) in the BIOS +2. Optional: Make sure you have the latest firmware for your system (BIOS, microcode updates). + +3. Optional: Disable Simultaneous Multi-Threading (SMT) in the BIOS Researchers recommend disabling (SMT), also known as Hyper-Threading Technology in the Intel® world to significantly reduce the impact of speculative execution-based attacks (https://mdsattacks.com/). @@ -241,29 +235,77 @@ You can also build Nix from source by following the instructions at https://nixo cd ../../ mkdir nix-bitcoin-node cd nix-bitcoin-node - # TODO - cp -r ../nix-bitcoin/examples/{configuration.nix,shell.nix,nix-bitcoin-release.nix,.gitignore} . + cp -r ../nix-bitcoin/examples/{nix-bitcoin-release.nix,configuration.nix,shell.nix,krops,.gitignore} . ``` -## 4. Deploy with TODO -1. TODO -2. Edit `configuration.nix` +## 4. Deploy with krops + +1. Edit your ssh config + + ``` + nano ~/.ssh/config + ``` + + and add the node with an entry similar to the following (make sure to fix `Hostname` and `IdentityFile`): + + ``` + Host bitcoin-node + # FIXME + Hostname NODE_IP_ADDRESS_OR_HOST_NAME_HERE + User root + PubkeyAuthentication yes + # FIXME + IdentityFile ~/.ssh/id_... + AddKeysToAgent yes + ``` + +2. Make sure you are in the deployment directory and edit `krops/deploy.nix` + + ``` + nano krops/deploy.nix + ``` + + Locate the `FIXME` and set the target to the name of the ssh config entry created earlier, i.e. `bitcoin-node`. + + Note that any file imported by your `configuration.nix` must be copied to the target machine by krops. + For example, if there is an import of `networking.nix` you must add it to `extraSources` in `krops/deploy.nix` like this: + ``` + extraSources = { + "hardware-configuration.nix".file = toString ../hardware-configuration.nix; + "networking.nix".file = toString ../networking.nix; + }; + ``` + +3. Optional: Disallow substitutes + + If you prefer to build the system from source instead of copying binaries from the Nix cache, add the following line to `configuration.nix`: + ``` + nix.extraOptions = "substitute = false"; + ``` + + If the build process fails for some reason when deploying with `krops-deploy` (see later step), it may be difficult to find the cause due to the missing output. + To see the build output, SSH into the target machine and run + ``` + nixos-rebuild -I /var/src switch + ``` + +4. Copy `hardware-configuration.nix` from your node to the deployment directory. + + ``` + scp root@bitcoin-node:/etc/nixos/hardware-configuration.nix . + ``` + +5. Adjust configuration by opening the `configuration.nix` file and enable/disable the modules you want by editing this file. ``` nano configuration.nix ``` - Uncomment `./hardware-configuration.nix` line by removing #. + Pay attention to lines that are preceded by `FIXME` comments. In particular: + 1. Make sure to set your SSH pubkey. Otherwise, you loose remote access because the config does not enable `permitRootLogin` (unless you add that manually). + 2. Uncomment the line `./hardware-configuration.nix` by removing `#`. -3. Create `hardware-configuration.nix`. - - ``` - nano hardware-configuration.nix - ``` - - Copy contents of your NixOS machine's `/etc/nixos/hardware-configuration.nix` to this file. - -4. Enter environment +6. Enter the deployment environment ``` nix-shell @@ -271,10 +313,21 @@ You can also build Nix from source by following the instructions at https://nixo NOTE that a new directory `secrets/` appeared which contains the secrets for your node. -5. TODO -6. Adjust configuration by opening the `configuration.nix` file and enable/disable the modules you want by editing this file. Pay particular attention to lines that are preceded by `FIXME` comments. Make sure to set your SSH pubkey. Otherwise, you loose remote access because the config does not enable `permitRootLogin` (unless you add that manually). +7. Deploy with krops in nix-shell -7. TODO + ``` + krops-deploy + ``` + + This will now create a nix-bitcoin node on the target machine. + +8. You can now access `bitcoin-node` via ssh + + ``` + ssh operator@bitcoin-node + ``` + + Note that you're able to log in as the unprivileged `operator` user because nix-bitcoin automatically authorizes the ssh key added to `root`. For security reasons, all normal system management tasks can and should be performed with the `operator` user. Logging in as `root` should be done as rarely as possible. diff --git a/docs/usage.md b/docs/usage.md index 3f3ef87..1216114 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -174,7 +174,9 @@ Connect to nix-bitcoin node through the SSH onion service ssh -i ~/.ssh/id_ed25519 -L :localhost:22 root@ ``` -5. Edit your network to change the node's address to `localhost` and the ssh port to `[random port of your choosing]` +5. Edit your deployment tool's configuration and change the node's address to `localhost` and the ssh port to ``. + If you use krops as described in the [installation tutorial](./install.md), set `target = "localhost:";` in `krops/deploy.nix`. + 6. After deploying the new configuration, it will connect through the SSH tunnel you established in step iv. This also allows you to do more complex SSH setups that some deployment tools don't support. An example would be authenticating with [Trezor's SSH agent](https://github.com/romanz/trezor-agent), which provides extra security. From 520a983862728320ca6fec3677e5fcbf57d05217 Mon Sep 17 00:00:00 2001 From: nixbitcoin Date: Wed, 10 Mar 2021 14:08:43 +0100 Subject: [PATCH 15/15] docs: nixops-krops migration guide --- docs/nixops-krops-migration.md | 130 +++++++++++++++++++++++++++++++++ 1 file changed, 130 insertions(+) create mode 100644 docs/nixops-krops-migration.md diff --git a/docs/nixops-krops-migration.md b/docs/nixops-krops-migration.md new file mode 100644 index 0000000..d402327 --- /dev/null +++ b/docs/nixops-krops-migration.md @@ -0,0 +1,130 @@ +# Tutorial: Moving from a NixOps to a Krops deployment + +1. Add a new ssh key to your nix-bitcoin node + + Krops doesn't automatically generate ssh keys like NixOps, instead you add your own. + + If you don't have a ssh key yet + + ``` + ssh-keygen -t ed25519 -f ~/.ssh/bitcoin-node + ``` + + Edit `configuration.nix` + + ``` + users.users.root = { + openssh.authorizedKeys.keys = [ + "" + ]; + }; + ``` + + Deploy new key + + ``` + nixops deploy -d bitcoin-node + ``` + +2. Update your nix-bitcoin, depending on your setup either with `fetch-release` or `git`. Make sure you are at least on `v0.0.41`. + +3. Pull the latest nix-bitcoin source + + ``` + cd ~/nix-bitcoin + git pull + ``` + +4. Copy new and updated files into your deployment folder + + ``` + cd + cp -r ~/nix-bitcoin/examples/{krops,shell.nix} . + ``` + +5. Edit your ssh config + + ``` + nano ~/.ssh/config + ``` + + and add the node with an entry similar to the following (make sure to fix `Hostname` and `IdentityFile`): + + ``` + Host bitcoin-node + # FIXME + Hostname NODE_IP_ADDRESS_OR_HOST_NAME_HERE + User root + PubkeyAuthentication yes + # FIXME + IdentityFile + AddKeysToAgent yes + ``` + +6. Make sure you are in the deployment directory and edit `krops/deploy.nix` + + ``` + nano krops/deploy.nix + ``` + + Locate the `FIXME` and set the target to the name of the ssh config entry created earlier, i.e. `bitcoin-node`. + + Note that any file imported by your `configuration.nix` must be copied to the target machine by krops. + For example, if there is an import of `networking.nix` you must add it to `extraSources` in `krops/deploy.nix` like this: + ``` + extraSources = { + "hardware-configuration.nix".file = toString ../hardware-configuration.nix; + "networking.nix".file = toString ../networking.nix; + }; + ``` + +7. If `lnd` or `joinmarket` is enabled on your node, run the commmand + ``` + nix-shell --run 'nix-instantiate --eval -E " + (import { + configuration = { lib, ... }: { + imports = [ ./configuration.nix ]; + nix-bitcoin.configVersion = lib.mkDefault \"0.0.31\"; + nix-bitcoin.secretsSetupMethod = lib.mkForce \"manual\"; + }; + }).vm.outPath + "' + ``` + and follow the migration instructions from the error message. + +8. Optional: Disallow substitutes + + You may have been building nix-bitcoin "without substitutes" to avoid pulling in binaries from the Nix cache. If you want to continue doing so, you have to add the following line to the `configuration.nix`: + ``` + nix.extraOptions = "substitute = false"; + ``` + + If the build process fails for some reason when deploying with `krops-deploy` (see later step), it may be difficult to find the cause due to the missing output. + In that case, it is possible to SSH into the target machine and run + ``` + nixos-rebuild -I /var/src switch + ``` + +9. Deploy with krops + + ``` + nix-shell --run krops-deploy + ``` + Remove the old secrets directory. For krops deployments, secrets are always + located at `/var/src/secrets`. + ``` + ssh bitcoin-node 'rm -rf /secrets' + ``` + +9. You can now access `bitcoin-node` via ssh + + ``` + ssh operator@bitcoin-node + ``` + +10. You can remove the remaining traces of nixops as follows: + ``` + nix-shell + nix run -f '' nixops19_09 -c nixops delete -d bitcoin-node --force + git rm -r nixops + ```