Commit graph

1233 commits

Author SHA1 Message Date
Olaoluwa Osuntokun
af97007ead
build: update to latest versions of btcd+neutrino 2019-11-12 19:00:05 -08:00
Olaoluwa Osuntokun
d14cee90e5
Merge pull request #660 from wpaulino/zmq-reuse-msg-buffers
build+chain: reuse buffers when reading ZMQ messages from bitcoind
2019-11-12 18:45:29 -08:00
Wilmer Paulino
d7d2f14d61
build+chain: reuse buffers when reading ZMQ messages from bitcoind
`bitcoind` notifies transactions once they're accepted into the mempool
and once they're confirmed in a block. Previously, reading a message
from ZMQ would allocate a buffer with the size of the message. This can
cause nodes to perform a large number of allocations within a small
amount periodically (3000 300B allocations every 10 mins on average),
which can cause a lot of GC pressure on lower resourced nodes. To remedy
this, we introduce two static buffers, one for blocks and another for
transactions, that will be reused for every message read. Each is
constrained by its maximum expected size.
2019-11-12 18:18:59 -08:00
Olaoluwa Osuntokun
c49e7ef3ec
Merge pull request #658 from wpaulino/lingering-unconfirmed-input
wtxmgr: remove unconfirmed input reference for confirmed transcation …
2019-11-08 19:18:58 -08:00
Olaoluwa Osuntokun
ef3abfafbc
Merge pull request #655 from breez/fix-re-broadcast-flow
Fix broadcast existing transaction
2019-11-08 19:14:04 -08:00
Olaoluwa Osuntokun
b08b47817c
Merge pull request #653 from wpaulino/improve-tx-record-err-msg
wtxmgr: improve error message for failed tx record retrieval
2019-11-08 18:05:26 -08:00
Wilmer Paulino
58ac163d48
chain: trace log timed out ZMQ connections 2019-11-08 12:39:00 -08:00
Wilmer Paulino
36ca842905
chain: extract ZMQ command strings into constants 2019-11-08 12:38:28 -08:00
Wilmer Paulino
66e09f252d
wtxmgr: remove unconfirmed input reference for confirmed transcation
Previously, inserting a transaction as unconfirmed into the store and
later confirming it would leave a lingering unconfirmed input record.
This was discovered as part of
https://github.com/btcsuite/btcwallet/pull/655. This issue would only
affect the wallet if it tracked spent transaction outputs, which it
doesn't. We aim to resolve it in any case for the sake of internal
consistency.
2019-10-30 16:44:55 -07:00
Wilmer Paulino
ac731b8e52
wtxmgr: add TestInsertMempoolTxAndConfirm
This test ensures that there aren't any lingering unconfirmed records
for a transaction that existed within the store as unconfirmed before
becoming confirmed. At the moment, this is currently failing due to a
gap when moving a transaction from unconfirmed to confirmed within the
store. This will be resolved in a subsequent commit.
2019-10-30 16:44:54 -07:00
Roei Erez
1c43ed9294 wtxmgr: fix broadcast existing transaction
This commit ensures the wallet won't enter an inconsitent state
by checking tx confirmation before adding credit.
Without this fix, In the case the existing transaction is already
confirmed on-chain the flow updates the bucketUnminedCredits but
without adding en entry also to the bucketUnmined resulting in
inconsistent state.
2019-10-31 00:28:21 +02:00
Wilmer Paulino
1e2f445cdd
wtxmgr: improve error message for failed tx record retrieval 2019-10-24 09:24:32 -04:00
Olaoluwa Osuntokun
7abdd4f8ad
Merge pull request #649 from halseth/neutrino-dep-update-cfilter-fix
mod: update neutrino dep
2019-10-14 18:12:23 +02:00
Johan T. Halseth
ed825b0fd9
mod: update neutrino+btcd dep 2019-10-14 17:53:31 +02:00
Olaoluwa Osuntokun
5f205d607b
Merge pull request #648 from wpaulino/bolt-no-freelist-sync-option
multi: extend walletdb bolt driver with NoFreelistSync option
2019-10-03 21:45:42 -07:00
Wilmer Paulino
81797fe29d
multi: extend walletdb bolt driver with NoFreelistSync option
This allows external callers to set the option instead. All tests
remained with the option enabled.
2019-10-03 13:28:04 -04:00
Olaoluwa Osuntokun
95d7aa0b49
Merge pull request #647 from halseth/rbf-reject-matching
Cleanup publication error matching, add ErrDoubleSpend/ErrReplacment
2019-09-24 17:50:52 -07:00
Johan T. Halseth
0149963628
wallet: define ErrReplacement
This error will be returned if the backend rejects a transaction since
it doesn't satisfy the requirements for replacement.
2019-09-24 09:00:22 +02:00
Johan T. Halseth
7533180fd3
wallet: define ErrDoubleSpend
This error is returned in cases where there are conflicting transactions
either in the mempool or chain.
2019-09-24 09:00:10 +02:00
Johan T. Halseth
33770f11b2
wallet: clean up error string matching
Also note where the errors are found.
2019-09-20 09:31:02 +02:00
Olaoluwa Osuntokun
d5cdeb4b91
Merge pull request #644 from wpaulino/notify-on-not-filtered
chain: notify block when not filtered for BitcoindClient
2019-09-11 09:57:39 +03:00
Wilmer Paulino
333ecbf262 chain: notify block when not filtered for BitcoindClient 2019-09-09 17:10:38 +03:00
Olaoluwa Osuntokun
ae43a2a200
Merge pull request #643 from wpaulino/go1.13
build: pin against latest neutrino version and update travis to go 1.13
2019-09-05 18:38:08 -07:00
Olaoluwa Osuntokun
277ce501a4
build: run travis against go 1.13 2019-09-05 18:31:15 -07:00
Olaoluwa Osuntokun
cfb9bb1e32
build: pin against latest neutrino version 2019-09-05 18:31:03 -07:00
Olaoluwa Osuntokun
5480f628bc
Merge pull request #642 from wpaulino/submodule-wtxmgr-txauthor
build: create new modules for wtxmgr and internal packages
2019-09-05 17:46:43 -07:00
Olaoluwa Osuntokun
3fc43caa86
build: update top-level module to pin to new sub-modules 2019-09-05 17:34:09 -07:00
Olaoluwa Osuntokun
8dddf4319f
wallet/txauthor: create new wallet/txauthor module 2019-09-05 17:30:56 -07:00
Olaoluwa Osuntokun
a608ee4e4f
wallet/txrules: create new wallet/txrules module 2019-09-05 17:24:09 -07:00
Olaoluwa Osuntokun
d510e4772d
wallet/txsizes: create new txsizes package+module, remove internal version
In this commit, we move the existing `internal/txsizes` package into its
own package and make it a module along the way. This allows projects
like `neutrino` to depend on a slimmer set of `btcwallet` related
dependencies.
2019-09-05 17:18:14 -07:00
Olaoluwa Osuntokun
35a7a1a0f6
wtxmgr: create new wtxmgr submodule 2019-09-05 16:57:26 -07:00
Olaoluwa Osuntokun
4a8fb7b716
Merge pull request #641 from wpaulino/walletdb-module
build+walletdb: create new module for walletdb
2019-09-05 16:32:33 -07:00
Olaoluwa Osuntokun
0594b75a43
build+walletdb: create new module for walletdb
In this commit, we create a new module for `walletdb` in order to allow
external projects to use the project without also having to depend on
the entirety of `btcwallet`. With this commit, projects that depend on
`walletdb` will benefit from a slimmer set of additional dependencies.
2019-09-05 16:23:31 -07:00
Olaoluwa Osuntokun
300b649394
Merge pull request #634 from Crypt-iQ/bbolt_options_btcwallet_0801
bdb: specify freelist options by default
2019-09-04 14:59:10 -07:00
Olaoluwa Osuntokun
a5162f9db4
Merge pull request #639 from wpaulino/remove-golangcrypto-dep
build: replace use of btcsuite/golangcrypto with golang.org/x/crypto
2019-09-04 12:58:14 -07:00
Wilmer Paulino
26a58718a3
build: replace use of btcsuite/golangcrypto with golang.org/x/crypto
btcsuite/golangcrypto was created to cope with the lack of package
management for Go back in the day. This is no longer needed as we now
use modules.
2019-09-04 12:51:58 -07:00
Olaoluwa Osuntokun
505acf5150
Merge pull request #636 from wpaulino/empty-filter-block
chain: prevent filtering blocks when BitcoindClient has an empty filter
2019-08-13 19:34:31 -07:00
Wilmer Paulino
5124978a9b
chain: prevent filtering blocks when BitcoindClient has an empty filter 2019-08-09 18:16:40 -07:00
nsa
e809748b1e bdb: specify freelist options by default
This commit specifies two bbolt options when opening the bbolt db.
This reduces heap pressure in case there are a lot of pages on
the freelist.
2019-08-01 22:15:53 -04:00
nsa
604661be3d build: update bbolt to v1.3.3
This commit updates the bbolt dependency to v1.3.3 so that we
can use the NoFreelistSync option without triggering a panic
in case a rollback occurs.
2019-08-01 19:30:07 -04:00
Olaoluwa Osuntokun
7a3a3e82cb
Merge pull request #633 from wpaulino/insert-unconfirmed-after-confimed
wtxmgr: prevent adding existing confirmed transactions as unconfirmed
2019-07-11 20:49:38 -07:00
Olaoluwa Osuntokun
79761f5121
Merge pull request #632 from wpaulino/bitcoind-block-zmq-read
chain: explicitly close connections in BitcoindConn's Stop method
2019-07-11 15:45:49 -07:00
Wilmer Paulino
02b0a7c18d
wallet: fix nil txid in publishTransaction
A nil txid could've been returned from publishTransaction even if it was
successful. This was due to the underlying SendRawTransaction call
"failing", e.g., when the transaction being broadcast has already
confirmed, but publishTranasction interpreting such failure as a
success.
2019-07-10 16:25:01 -07:00
Wilmer Paulino
44d818d813
wtxmgr: prevent adding existing confirmed transactions as unconfirmed
We do this as a sanity check for users of the wallet to ensure the
wallet maintains a correct representation of the chain.
2019-07-10 16:25:00 -07:00
Wilmer Paulino
6c4dab5f7c
chain: remove extraneous rpcclient.Disconnect calls from BitcoindConn
These are not needed since they have no effect when the RPC client is
making HTTP POST requests.
2019-07-10 16:15:51 -07:00
Wilmer Paulino
f02166e5de
chain: explicitly close connections in BitcoindConn's Stop method
Since it's now possible for gozmq.Conn to block when calling Receive,
BitcoindConn hangs upon being stopped because its goroutines are waiting
for a message to be delivered. To address this, we modify it to close
its ZMQ connections driving the goroutines once it's been stopped. This
allows the goroutines to unblock by detecting the EOF error and exiting.
2019-07-10 16:15:50 -07:00
Olaoluwa Osuntokun
cd66e82bc8
Merge pull request #631 from wpaulino/dropwtxmgr-max-reorg-depth
cmd/dropwtxmgr+waddrmgr: handle constraint when resetting synced state
2019-07-10 15:45:58 -07:00
Wilmer Paulino
f710cd4a92
cmd/dropwtxmgr+waddrmgr: handle constraint when resetting synced state
We update the dropwtxmgr utility tool to take into account that the
wallet only stores MaxReorgDepth blocks, which introduced an additional
constraint when updating the wallet's synced state. The constraint
ensures that the previous block exists when updating the wallet's synced
state, but this does not hold for the birthday block since it's the
first block we'll store.
2019-07-08 13:26:43 -07:00
Olaoluwa Osuntokun
4a9774585e
Merge pull request #628 from cfromknecht/faster-btcd-init
wallet: speed up bare wallet init when using the btcd backend
2019-06-28 15:53:30 -07:00
Olaoluwa Osuntokun
2698f8434e
wallet: speed up bare wallet init when using the btcd backend
In this commit, we speed up creating a fresh wallet when using the btcd
backend. Before this commit, we would need to rescan 10k or so blocks
when creating a new wallet with the btcd backend. btcd will actually
fully scan all the blocks even though we have zero addresses or UTXOs to
look for. As a result, this can take quite some time.

In this commit we modify the starting height of the initial rescan to
start at the birthday height, and only modify it if it's unset, or the
best height of the chain is before this birthday height. As a result, we
won't always have the full 10k block re org safety horizon on disk, but
will tend to this level after we begin to sync forward.
2019-06-28 15:01:09 -07:00