# `bitcoind` Quickstart
Links: [[bitcoind]]
## Prerequisites
### Clone
```bash
git clone https://github.com/bitcoin/bitcoin
cd bitcoin
```
## Build from source
- See [`doc/build-*.md` at bitcoin/bitcoin](https://github.com/bitcoin/bitcoin/tree/master/doc) for platform-specific instructions
### [(Ubuntu & Debian) Build Requirements](https://github.com/bitcoin/bitcoin/blob/master/doc/build-unix.md#ubuntu--debian)
Links: [[Ubuntu]] / [[Linux]]
```bash
sudo apt install build-essential libtool autotools-dev automake pkg-config bsdextrautils python3
sudo apt install libevent-dev libboost-dev
```
**Optional build requirements**
```bash
# SQLite is required for the descriptor wallet
sudo apt install libsqlite3-dev
# Optional port mapping libraries
sudo apt install libminiupnpc-dev libnatpmp-dev
# ZMQ dependencies (provides ZMQ API)
sudo apt-get install libzmq3-dev
# User-Space, Statically Defined Tracing (USDT) dependencies
sudo apt install systemtap-sdt-dev
```
### [(macOS) Build Requirements](https://github.com/bitcoin/bitcoin/blob/master/doc/build-osx.md)
Links: [[macOS]]
Build prerequisites that are usually already installed:
- Xcode Command Line Tools
- `xcode-select --install`
- [[Homebrew]]
**Required dependencies**
```bash
brew install automake libtool boost pkg-config libevent
```
**Optional dependencies**
```bash
# [GUI dependencies]
# If 'qt' is installed, the build process will fail.
brew uninstall qt
brew install qt@5
# QR Codes in the GUI
brew install qrencode
# [Port mapping dependencies]
brew install miniupnpc
brew install libnatpmp
# ZMQ
brew install zeromq
# Test suite
brew install python
```
### Build
```bash
./autogen.sh
./configure --without-wallet --with-gui=no
make # use "-j N" for N parallel jobs
sudo make install # sudo required to move into /usr/bin
make check # Run tests if Python 3 is available
```
The resulting binary will be at `./src/bitcoind` and `./src/bitcoin-cli`.
If setting up `bitcoind` on a VM, you may want to copy the binary to a more obvious location:
```bash
mkdir -p ~/bin
cp ./src/bitcoind ~/bin
cp ./src/bitcoin-cli ~/bin
```
## Configure `bitcoin.conf`
### (Option 1) Create `bitcoin.conf` in custom data dir (e.g. on VM)
```bash
sudo mkdir -p /data/bitcoin
cd bitcoin
touch bitcoin.conf
```
### (Option 2) Create `bitcoin.conf` at the default location
- [[Linux]]: `~/.bitcoin/bitcoin.conf`
- `mkdir ~/.bitcoin`
- `touch ~/.bitcoin/bitcoin.conf`
- [[macOS]]: `~/Library/Application Support/Bitcoin/bitcoin.conf`
- `mkdir -p "~/Library/Application Support/Bitcoin"`
- `touch "~/Library/Application Support/Bitcoin/bitcoin.conf"`
### Generate `rpcauth` credentials
Explanation of the `-rpcauth` option from `bitcoind -help`:
```text
-rpcauth=<userpw>
Username and HMAC-SHA-256 hashed password for JSON-RPC connections. The
field <userpw> comes in the format: <USERNAME>:<SALT>
lt;HASH>. A
canonical python script is included in share/rpcauth. The client
then connects normally using the
rpcuser=<USERNAME>/rpcpassword=<PASSWORD> pair of arguments. This
option can be specified multiple times
```
Generate the `rpcauth` string. `-` indicates that you will specify a password at the prompt. e.g. for username `kek` and password `sadge`:
```bash
$ python3 share/rpcauth/rpcauth.py kek -
Password:
String to be appended to bitcoin.conf:
rpcauth=kek:47cdd9eeef4445b980eb77b63956a6a2$41253a60903a25e55339636b40d31b5adc0fd7eebb3f98bc53855b9f56715a1c
Your password:
sadge
```
Can also view the help of the `rpcauth` script if needed:
```bash
python3 share/rpcauth/rpcauth.py -h
```
### Quickstart bitcoin.conf
`bitcoin.conf`
```conf
chain=test
datadir=/data/bitcoin # We also set this in bitcoind.service
server=1
disablewallet=1
prune=0
txindex=1
zmqpubrawblock=tcp://127.0.0.1:18500
zmqpubrawtx=tcp://127.0.0.1:18501
# Options only for mainnet
[main]
rpcport=8332
# Options only for testnet
[test]
bind=0.0.0.0 # Testnet P2P
port=18332 # Testnet P2P
rpcbind=0.0.0.0 # insecure, for dev only!
rpcport=6969
rpcallowip=0.0.0.0/0 # insecure, for dev only!
rpcauth=<the string you generated above>
```
- Default RPC port on `mainnet` is `8332`; default RPC port for `testnet` is `18332`. Redundantly specifying these in the configuration for clarity.
- Can generate a better RPC password by key-mashing into [SHA256 online](https://emn178.github.io/online-tools/sha256.html)
- If a separate data directory was specified, make sure to create it first, e.g.: `mkdir -p /data/bitcoin`
- `rpcport=6969` prevents a conflic with the default port for inbound testnet P2P connections, `18332`
- `rpcauth` is the string you generated above
- Ignore anything that uses `rpcconnect` - it is not a supported option anymore.
- If you are setting up `bitcoind` purely to support `electrs`, then `txindex` can be set to `0`.
## Start
```bash
./src/bitcoind -datadir=/data/bitcoin
./src/bitcoind -datadir=/data/bitcoin -help | vim
./src/bitcoin-cli
./src/bitcoin-cli -help | vim
```
- If the default `datadir` (e.g. `~/.bitcoin`) is used, then the `-datadir` option can be omitted.
- If using a custom `datadir` such as `/data/bitcoin`, the `-datadir` option must be specified in all invocations of `bitcoind`, otherwise `bitcoind` will look for (and not find) the `bitcoin.conf` in the default location.
- `bitcoin-cli` does not require specifying `-datadir` regardless how `bitcoind` is set up.
- If `make install` was used, the `bitcoind` and `bitcoin-cli` binaries were copied to `/usr/bin` which is in `$PATH`, so `./src/bitcoind` can be changed to simply `bitcoind`.
- If the binaries were copied to some other location like `~/bin`, obviously invoking with `~/bin/bitcoind` works too.
## RPC
Test that RPC is working:
```bash
./src/bitcoin-cli getblockhash 0
```
## Daemonizing `bitcoind` with [[systemd]]
### [[systemd]] unit file
The following is partially based on the [sample bitcoind.service configuration](https://github.com/bitcoin/bitcoin/blob/master/contrib/init/bitcoind.service) provided by bitcoin/bitcoin.
`bitcoind.service`
```ini
[Unit]
Description=Bitcoin Core Service
After=network.target
[Service]
User=max
Group=max
Type=simple
ExecStart=/home/max/bin/bitcoind -datadir=/data/bitcoin
WorkingDirectory=/home/max/bitcoin
Restart=always
RestartSec=15
KillSignal=SIGINT
TimeoutStopSec=90
FinalKillSignal=SIGTERM
[Install]
WantedBy=multi-user.target
```
- `datadir` needs to be specified because otherwise it will default to `/.bitcoin` (where `.bitcoin` is prefixed by the root `/` rather than `$HOME`)
- We set `User` and `Group` to a non-privileged user ostensibly for security reasons, but actually mostly so `electrs` can read from the `bitcoind` datadir without running into permissions issues.
### Set up / reconfigure `bitcoind` service
```bash
# Create / modify the unit file
vim bitcoind.service
# Copy it into the location
sudo cp bitcoind.service /etc/systemd/system
# Check everything looks OK
cat /etc/systemd/system/bitcoind.service
# Reload the service file since it was changed
sudo systemctl daemon-reload
# Enable the service to run at boot
sudo systemctl enable bitcoind.service
# Outputs "Created symlink /etc/systemd/system/multi-user.target.wants/bitcoind.service → /etc/systemd/system/bitcoind.service."
# Verify service has been detected
systemctl list-units --type=service --all
```
### Using the `bitcoind` service
```bash
# Start
sudo systemctl start bitcoind
# Stop
sudo systemctl stop bitcoind
# Restart
sudo systemctl restart bitcoind
# Status
sudo systemctl status bitcoind
# See bitcoind output in Ubuntu/Debian syslog
tail -F /var/log/syslog
```
### Connecting to RPC remotely
Local development that uses a remote `bitcoind` testnet node requires that:
- The appropriate ports are open in the [[Azure]] / [[AWS]] firewall (network security settings)
- The `6969` port is strictly necessary for RPC
- `18332` is necessary if you want to enable inbound testnet P2P connections
Check if the testnet RPC port is open:
```bash
nc -zvw10 20.123.24.44 6969
```
- Time out means that the request was blocked by [[Azure]] (haven't checked for [[AWS]])
- Connection refused means that something is incorrectly configured in `bitcoin.conf`
**Overview**
- There are four variables to note:
- `bind`: Incoming connections from Bitcoin P2P network
- `port`: Port for incoming connections from Bitcoin P2P network
- Default `8332` for mainnet, default `18332` for testnet
- `rpcbind`: Incoming RPC connections
- `rpcport`: Port for incoming RPC connections
- Default `8332` for mainnet, default `18332` for testnet.
- Setting both `bind` and `rpcbind` to `0.0.0.0` and `port` and `rpcport` to the same value will result in conflicting bindings between Bitcoin P2P messages and Bitcoin RPC. Therefore, the quickstart `bitcoin.conf` uses `6969` as the RPC port to avoid conflicts with the P2P port.
- `rpcbind` should be `0.0.0.0`; `rpcbind=127.0.0.1` will result in "Connection refused"
## Bonus: Set up [[electrs]]
Before we begin:
- Note that these instructions were written with the Blockstream fork in mind, and might not work as exactly written if you intend to run the original romanz version intended for smaller scale deployments.
- Note that Blockstream's `electrs` must run on the same machine that `bitcoind` is running on, since it reads `blk*.dat` files directly from `bitcoind`'s `datadir` to achieve maximum performance (as opposed to fetching data via RPC).
Some notes on testnet system requirements:
- While `electrs` was indexing, I saw total memory on my VM (which only runs `bitcoind` and `electrs`) go up to as high as 10 GiB, so having at least 16GiB total of memory + swap space during `electrs` indexing is a good idea.
- As of 2023-06-23, a 128GiB disk was not enough to store the `bitcoind` datadir + the `electrs` datadir and db + a 16GiB swap file. I had to upsize my volume to 256GiB for the `electrs` indexing process to run smoothly, but ~192GiB or so should be enough. After syncing, indexing, and compaction were all complete, the total disk size of the `bitcoind` + `electrs` datadirs was 91GiB. Note that this is with `txindex=0` set in `bitcoin.conf`.
Install [[Rust]]
```bash
curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
```
Install [build dependencies](https://github.com/romanz/electrs/blob/master/doc/install.md#build-dependencies)
```bash
sudo apt update
sudo apt install clang cmake build-essential # for building 'rust-rocksdb'
```
Clone the repo, either romanz or [[Blockstream]] version
```bash
git clone https://github.com/romanz/electrs # romanz version
git clone https://github.com/Blockstream/electrs # Blockstream version
cd electrs
```
Build
```bash
cargo build --release --locked
```
- The [build instructions](https://github.com/romanz/electrs/blob/master/doc/install.md#build) note that the build may fail if you do not have enough memory. On [[2023-06-22]] I was able to build the Blockstream fork on a 3.5 GiB Ubuntu VM with no swap space, so the actual amount of memory required can't be that crazy.
Copy over the binary to somewhere more noticeable, either in PATH or elsewhere
```bash
cp target/release/electrs ~/bin
cd bin
```
See electrs help
```bash
electrs --help
```
Run electrs. Here is an example testnet deployment that I set up on an Azure VM:
```bash
electrs -vvvv \
--network testnet \
--cookie kek:sadge \
--blocks-dir /data/bitcoin/testnet3/blocks \
--daemon-dir /data/bitcoin \
--db-dir /data/electrs \
--daemon-rpc-addr 127.0.0.1:18332 \
--electrum-rpc-addr 127.0.0.1:60001 \
--http-addr 0.0.0.0:3001
```
- `kek:sadge` should correspond with the `rpcauth` parameter set in `bitcoin.conf`.
- Some of these options are exactly the same as the defaults, but I included them anyway for discoverability and clarity. See `--help` for more info on these, as well as more options.
This command is pretty long, so it might be worthwhile to simplify the process for others (and yourself) by saving it in a `run_electrs.sh` or similar. Don't forget to `chmod +x` the resulting script.
`~/bin/run_electrs.sh`
```bash
#!/bin/bash
/home/max/bin/electrs -vvvv \
<...>
```
- Note the use of the `/home/<user>/bin` prefix so that putting the binary in `PATH` isn't strictly required, and also so the working directory doesn't impact the operation of the script.
- It is important to include the `#!/bin/bash` shebang if you intend to daemonize `electrs` later. Otherwise, `systemd` will "not know how to run" this `run-electrs.sh` script.
### Daemonize `electrs`
`electrs.service`
```systemd
[Unit]
Description=electrs Service
After=network.target
[Service]
User=max
Group=max
Type=simple
ExecStart=/home/max/bin/run-electrs.sh
WorkingDirectory=/home/max
Restart=always
RestartSec=15
KillSignal=SIGINT
TimeoutStopSec=90
FinalKillSignal=SIGKILL
[Install]
WantedBy=multi-user.target
```
- For the `FinalKillSignal` we use `SIGKILL` instead of the usual `SIGTERM` because in my experience (Blockstream) `electrs` tends to be unresponsive to both.
### Load and enable `electrs.service`
These steps are very similar to the bitcoind steps above
```bash
# Create / modify the unit file
vim electrs.service
# Copy it into the location
sudo cp electrs.service /etc/systemd/system
# Check everything looks OK
cat /etc/systemd/system/electrs.service
# Reload the service file since it was changed
sudo systemctl daemon-reload
# Enable the service to run at boot
sudo systemctl enable electrs.service
# Outputs "Created symlink /etc/systemd/system/multi-user.target.wants/bitcoind.service → /etc/systemd/system/bitcoind.service."
# Verify service has been detected
systemctl list-units --type=service --all
```
### Manage `electrs` via `systemctl`
```bash
sudo systemctl start electrs
sudo systemctl stop electrs
sudo systemctl restart electrs
sudo systemctl status electrs
# See output in Ubuntu/Debian syslog
# If you've daemonized bitcoind as well (likely), the outputs of both should be mixed here.
tail -F /var/log/syslog
```