# Transaction Sync Resources A hodge-podge of helpful resources I came across in the process of implementing BDK wallet sync + LDK transaction sync in [[Async Rust|async]] [[Rust]] and deciding between [[Electrum]] or [[Esplora]]. Compiled [[2023-01-09]]. ### [Esplora and Other Alternatives to ElectrumX](https://blog.blockstream.com/en-esplora-and-other-alternatives-to-electrumx/) By [[Lawrence Nahum]] for [[Blockstream]] blog on [[2020-06-16]] - Helpful high-level explanation of the various options - Written in 2020; a little bit outdated - [[Blockstream]]-biased Covers: - ElectrumX - Electrum Personal Server - Bitcoin Wallet Tracker - romanz/electrs - Blockstream/electrs - Esplora - Connect to the Blockstream Electrum Server Blockstream.info provides a [public Electrum server](https://blog.blockstream.com/en-esplora-and-other-alternatives-to-electrumx/) which can be used over TCP or TLS. ### [Electrum server performance report (2022)](https://blog.keys.casa/electrum-server-performance-report-2022/) By [[Jameson Lopp]] for [[Casa]] blog - ElectrumX - Esplora Electrs - Fulcrum ### [[mempool.space]] - API docs - [FAQ](https://mempool.space/docs/faq) - [REST](https://mempool.space/docs/api/rest) - [Websockets](https://mempool.space/docs/api/websocket) - [GitHub repo](https://github.com/mempool/mempool) ### Esplora - [HTTP API docs](https://github.com/Blockstream/esplora/blob/master/API.md) ### Electrum - [ElectrumX protocol method docs](https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html) ### ([[Rust]]) [romanz/electrs](https://github.com/romanz/electrs) - Much more active than the [Blockstream fork](https://github.com/blockstream/electrs) - [Integration tests PR](https://github.com/Blockstream/electrs/pull/46) (unmerged as of [[2023-01-09]]) - [Patch to fix build for M1 macs](https://github.com/romanz/electrs/pull/773) (unmerged as of [[2023-01-09]]) - Original [issue](https://github.com/romanz/electrs/issues/753) opened by [[Casey Rodarmor]] - [docs.rs](https://docs.rs/electrs/latest/electrs/) (only has a `run()` function, would need work to embed in a test binary) ### ([[Rust]]) `electrum-client` - [docs.rs](https://docs.rs/electrum-client/latest/electrum_client/) - [[GitHub]] repo: [bitcoindevkit/rust-electrum-client](https://github.com/bitcoindevkit/rust-electrum-client) - As of [[2023-01-09]] unfortunately only supports a blocking API ### ([[Rust]]) `esplora-client` - [docs.rs](https://docs.rs/esplora-client/latest/esplora_client/) - Supports both async and blocking - [[GitHub]] repo: [bitcoindedvkit/rust-esplora-client](https://github.com/bitcoindevkit/rust-esplora-client) ### BDK `Blockchain` API requirements See the list of [trait implementations](https://docs.rs/bdk/latest/bdk/blockchain/esplora/struct.EsploraBlockchain.html#trait-implementations) for one of their provided blockchain clients, e.g. the [`EsploraBlockchain`](https://docs.rs/bdk/latest/bdk/blockchain/struct.EsploraBlockchain.html). ### BDK-provided wallet sync clients - `bitcoind` [client](https://docs.rs/bdk/latest/bdk/blockchain/rpc/struct.RpcBlockchain.html) (unfortunately blocking only) - [[Neutrino]] [client](https://docs.rs/bdk/latest/bdk/blockchain/compact_filters/struct.CompactFiltersBlockchain.html) (unfortunately blocking only) - electrum [client](https://docs.rs/bdk/latest/bdk/blockchain/electrum/struct.ElectrumBlockchain.html) (unfortunately blocking only, and would take significant work to rewrite as async since the underlying [`electrum-client`](https://docs.rs/electrum-client/latest/electrum_client/) crate is blocking only) - esplora [client](https://docs.rs/bdk/latest/bdk/blockchain/struct.EsploraBlockchain.html) - Both async and blocking API supported - Use `use-esplora-async` or `use-esplora-blocking` [feature flags](https://docs.rs/crate/bdk/latest/features) ### LDK's transaction sync API requirements - [lightningdevkit.org high-level docs](https://lightningdevkit.org/blockchain_data/confirmed_transactions/): "Confirmed Transactions" - The main way that we let LDK know of updated transaction data is via the [`Confirm` interface](https://docs.rs/lightning/0.0.113/lightning/chain/trait.Confirm.html). - The main way that LDK lets us know which transactions to track is via the [`Filter` interface](https://docs.rs/lightning/0.0.113/lightning/chain/trait.Filter.html). The `Confirm` interface also offers a `get_relevant_txids` method which allows us to query for txids that LDK has already indicated as of interest. **`chain::Confirm`** - LDK needs to know: - when a transaction is confirmed or unconfirmed - when the best block is updated **`chain::Filter`** - LDK tells us: - script pubkeys it is interested in (`Script`) - outputs it is interested in (`WatchedOutput`, which is a block hash, outpoint, and script pubkey) ### [lipa wallet docs on LDK transaction sync](https://github.com/getlipa/lipa-lightning-lib/blob/947b284f73b9267b2d8733a6d62f0fe2f8f9520a/docs/Sync.org) - Thanks to Elias (@[[Elias Rohrer|tnull]]) for pointing me towards this resource Sections: - LDK Interface - Implementation Responsibilities - Transactions confirmed, unconfirmed, reconfirmed - Spent Outputs - Best Block - Implementation Considerations - Initialization - Spent Output - Reorgs - Stateless Implementation ### Meeting LDK's API requirements with electrum The [electrum protocol](https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html) contains: - a [`blockchain.transaction.get`](https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html#blockchain-transaction-get) method which gives us the number of confirmations whenever we set `verbose` to true. - a [`blockchain.scripthash.subscribe`](https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-methods.html#blockchain-scripthash-subscribe) method which allows us to return the [`status`](https://electrumx-spesmilo.readthedocs.io/en/latest/protocol-basics.html#status) of the script pubkey that we're interested in. ### ([[Rust]]) Electrum / Esplora integration testing environment [[Riccardo Casatta]] has a little known crate called `electrsd` which is perfect for this, created in the same spirit as his excellent `bitcoind` regtest crate **`electrsd`** - [[Github]] repo: [RCasatta/electrsd](https://github.com/RCasatta/electrsd) - [docs.rs](https://docs.rs/electrsd/latest/electrsd/) - [Example usage](https://github.com/tnull/ldk-node/blob/20c878c019f70d3f0d2d3aef37890b589e17b053/src/tests/functional_tests.rs#L35) in `ldk-node` - [Example configuration](https://github.com/tnull/ldk-node/blob/20c878c019f70d3f0d2d3aef37890b589e17b053/Cargo.toml#L52) to include esplora api in `ldk-node` `Cargo.toml` ### `lightning-transaction-sync` By [[Elias Rohrer]] aka [[Elias Rohrer|tnull]] - [Open PR rust-lightning#1870](https://github.com/lightningdevkit/rust-lightning/pull/1870): Add transaction sync crate - Not merged as of [[2023-01-09]] - The transaction-oriented counterpart to `lightning-block-sync` ### [`ldk-node`](https://github.com/lightningdevkit/ldk-node) - [[Elias Rohrer|tnull]]'s "ready-to-go" node implementation based on LDK which also uses BDK wallet sync and LDK's transaction-based sync API, in [[Async Rust]]. - Almost nothing has been merged, all work is in open PRs based on branches in [[Elias Rohrer|tnull]]'s fork. - [PR#11](https://github.com/lightningdevkit/ldk-node/pull/11): Initial implementation of LDK Node (+2648 -183) - Search for `EsploraBlockchain` to find BDK integration with wallet sync - Search for `EsploraSyncClient` to see how `lightning-transaction-sync` is used - See `Cargo.toml` to see required features for `bdk`, `lightning-transaction-sync`, and `electrsd` integration testing - See `event.rs` for an example of async event handling - See `wallet.rs` to see the implementation of required wallet methods used throughout the rest of the node - Wallet PR: [ldk-node#33](https://github.com/lightningdevkit/ldk-node/pull/33) - This PR includes the changes from [PR#26](https://github.com/lightningdevkit/ldk-node/pull/26) ("Make everything `async`"), so every example is already async-ified.