Strip out the witnesses when serializing the non-witness utxo. However
witness serializations are allowed, so make sure we always deserialize
as witness.
GitHub-Pull: #13960
Rebased-From: bd19cc7
0-input transactions can be ambiguously deserialized as being witness
transactions. Since the unsigned transaction is never serialized as
a witness transaction as it has no witnesses, we should always
deserialize it as a non-witness transaction and set the serialization
flags as such.
Also always serialize the unsigned transaction as a non-witness transaction.
GitHub-Pull: #13960
Rebased-From: 43811e6
869193f5a6 docs: fixed bitcoin-cli -help output for help2man (Hennadii Stepanov)
Pull request description:
Currently `bitcon-cli -help` output forces help2man to produce `.TP` and `.IP` commands instead of a single `.IP` command for `-stdinrpcpass` option.
Removing an extra space fixes this issue.
This pull request is rebased from #13879
Tree-SHA512: 1c5b25ed2ef7b7de42bc6210165bdbabe63f045699487f2db4790e0d3176f6493dfd3e8e19f4ddc38b551539465d7b41aea570f20dccbc0609f00fdfee1b5180
4b7091a842 Replace median fee rate with feerate percentiles (Marcin Jachymiak)
Pull request description:
Currently, the `medianfeerate` statistic is calculated from the feerate of the middle transaction of a list of transactions sorted by feerate.
This PR instead uses the value of the 50th percentile weight unit in the block, and also calculates the feerate at the 10th, 25th, 75th, and 90th percentiles. This more accurately corresponds with what is generally meant by median feerate.
Tree-SHA512: 59255e243df90d7afbe69839408c58c9723884b8ab82c66dc24a769e89c6d539db1905374a3f025ff28272fb25a0b90e92d8101103e39a6d9c0d60423a596714
e306be7429 Use 72 byte dummy signatures when watching only inputs may be used (Andrew Chow)
48b1473c89 Use 71 byte signature for DUMMY_SIGNATURE_CREATOR (Andrew Chow)
18dfea0dd0 Always create 70 byte signatures with low R values (Andrew Chow)
Pull request description:
When creating signatures for transactions, always make one which has a 32 byte or smaller R and 32 byte or smaller S value. This results in signatures that are always less than 71 bytes (32 byte R + 32 byte S + 6 bytes DER + 1 byte sighash) with low R values. In most cases, the signature will be 71 bytes.
Because R is not mutable in the same way that S is, a low R value can only be found by trying different nonces. RFC 6979 for deterministic nonce generation has the option to specify additional entropy, so we simply use that and add a uin32_t counter which we increment in order to try different nonces. Nonces are sill deterministically generated as the nonce used will the be the first one where the counter results in a nonce that results in a low R value. Because different nonces need to be tried, time to produce a signature does increase. On average, it takes twice as long to make a signature as two signatures need to be created, on average, to find one with a low R.
Having a fixed size signature makes size calculations easier and also saves half a byte of transaction size, on average.
DUMMY_SIGNATURE_CREATOR has been modified to produce 71 byte dummy signatures instead of 72 byte signatures.
Tree-SHA512: 3cd791505126ce92da7c631856a97ba0b59e87d9c132feff6e0eef1dc47768e81fbb38bfbe970371bedf9714b7f61a13a5fe9f30f962c81734092a4d19a4ef33
18f690ec2f wallet: shuffle coins before grouping, where warranted (Karl-Johan Alm)
Pull request description:
Coins are randomly shuffled in coin selection to avoid unintentional privacy leaks regarding the user's coin set. For the case where a user has a lot of coins with the same destination, these will be grouped into groups of 10 *before* the shuffling.
It is unclear whether this has any implications at all, but this PR plugs the potential issue, if there ever is one, by shuffling the coins before they are grouped.
Issue brought up in https://github.com/bitcoin/bitcoin/pull/12257#discussion_r204554549
Tree-SHA512: fb50ed4b5fc03ab4853d45b76e1c64476ad5bcd797497179bc37b9262885c974ed6811159fd8e581f1461b6cc6d0a66146f4b70a2777c0f5e818d1322e0edb89
The `help2man` parses a string containing two spaces between words with an issue:
it gives out `.TP` and `.IP` commands instead of a single `.IP` command.
Removing an extra space fixes this issue.
Currently the `-help` output for the `-stdin` option looks without any issue due to eliminating
of two spaces between words by a `FormatParagraph` call for this particular case.
For consistency and preventing from future regressions extra spaces have been removed from the both lines.
The redundant `strprintf` call has been removed aswell.
Make sure that translations are synchronized with transifex before the
branch-off point to minimize the difference and prevent duplicate work.
Tree-SHA512: 41e71eaf14094606fd90011d035c551a635d5a715f865a49841dbe2b54a76b7fbf59a7918f86e5fd80a717e2934a9613fe463391fd01848d0a01e5c4e7e7fef0
Removes medianfeerate result from getblockstats.
Adds feerate_percentiles which give the feerate of the 10th, 25th, 50th,
75th, and 90th percentile weight unit in the block.
bb5b1c0b2d [Docs] upgrade rescan time warning from minutes to >1 hour (Mason Simon)
Pull request description:
When I rescanned just now it took well over an hour. The time warning "may take minutes" didn't prepare me for that.
```
2018-08-08T03:10:17Z [wallet] Still rescanning. At block 174747. Progress=0.008341
2018-08-08T03:11:17Z [wallet] Still rescanning. At block 204233. Progress=0.024533
2018-08-08T03:12:17Z [wallet] Still rescanning. At block 221170. Progress=0.038340
...
2018-08-08T04:16:17Z [wallet] Still rescanning. At block 524815. Progress=0.957105
2018-08-08T04:17:17Z [wallet] Still rescanning. At block 528572. Progress=0.971323
2018-08-08T04:18:17Z [wallet] Still rescanning. At block 532458. Progress=0.986824
```
This is on a 4-core 4ghz system with a 7200rpm drive.
Tree-SHA512: 722ccf566bfd6a3381fa173e08849cb676fe4c1f1cb2c4b86b07df2a5dc1ca0d54797cbe8fd606cdc2c60fef2be7c98e052460decdac2132ba759cff822132e8
227d27e70c Use pushKV in some new PSBT RPCs. (Daniel Kraft)
Pull request description:
Most of the code uses `UniValue::pushKV` where appropriate, but some new RPC code related to PSBTs did not. This fixes those places - after this change, there are no remaining source files I could find that contain `push_back(Pair(`.
Tree-SHA512: d6567cf144d05d7e42276bd66ff4cd44413328f985772d11bb9d7339d32ab7c3438d4bb0040a37e75f8d193c610b08fa971073935885e0a178546aa045daf9fa
e254ff5d53 Introduce a maximum size for locators. (Gregory Maxwell)
Pull request description:
The largest sensible size for a locator is log in the number of blocks.
But, as noted by Coinr8d on BCT a maximum size message could encode a
hundred thousand locators. If height were used to limit the messages
that could open new attacks where peers on long low diff forks would
get disconnected and end up stuck.
Ideally, nodes first first learn to limit the size of locators they
send before limiting what would be processed, but common implementations
back off with an exponent of 2 and have an implicit limit of 2^32
blocks, so they already cannot produce locators over some size.
Locators are cheap to process so allowing a few more is harmless,
so this sets the maximum to 64-- which is enough for blockchains
with 2^64 blocks before the get overhead starts increasing.
Tree-SHA512: da28df9c46c988980da861046c62e6e7f93d0eaab3083d32e408d1062f45c00316d5e1754127e808c1feb424fa8e00e5a91aea2cc3b80326b71c148696f7cdb3
With watching only inputs, we do not know how large the signatures
for those inputs will be as their signers may not have implemented
71 byte signatures. Thus we estimate their fees using the 72 byte
dummy signature to ensure that we pay enough fees.
This only effects fundrawtransaction when includeWatching is true.
When extra entropy is not specified by the caller, CKey::Sign will
now always create a signature that has a low R value and is at most
70 bytes. The resulting signature on the stack will be 71 bytes when
the sighash byte is included.
Using low R signatures means that the resulting DER encoded signature
will never need to have additional padding to account for high R
values.
The largest sensible size for a locator is log in the number of blocks.
But, as noted by Coinr8d on BCT a maximum size message could encode a
hundred thousand locators. If height were used to limit the messages
that could open new attacks where peers on long low diff forks would
get disconnected and end up stuck.
Ideally, nodes first first learn to limit the size of locators they
send before limiting what would be processed, but common implementations
back off with an exponent of 2 and have an implicit limit of 2^32
blocks, so they already cannot produce locators over some size.
This sets the limit to an absurdly high amount of 101 in order to
maximize compatibility with existing software.
faab63111d shutdown: Stop threads before resetting ptrs (MarcoFalke)
Pull request description:
On shutdown some threads would continue to run after or during a pointer reset. This leads to occasional segfaults on shutdown.
Fix this by resetting the smart pointers after all threads that might read from them have been stopped.
This should fix:
* A segfault in the txindex thread, that occurs when the txindex destructor is done, but the thread was not yet stopped (as this is done in the base index destructor)
* A segfault in the scheduler thread, which dereferences conman. (e.g. CheckForStaleTipAndEvictPeers)
Tree-SHA512: abbcf67fadd088e10fe8c384fadfb90bb115d5317145ccb5363603583b320efc18131e46384f55a9bc574969013dfcbd08c49e0d42c004ed7212eca193858ab2
3fc20632a3 qt: Set BLOCK_CHAIN_SIZE = 220 (DrahtBot)
2b6a2f4a28 Regenerate manpages (DrahtBot)
eb7daf4d60 Update copyright headers to 2018 (DrahtBot)
Pull request description:
Some trivial maintenance to avoid having to do it again after the 0.17 branch off.
(The scripts to do this are in `./contrib/`)
Tree-SHA512: 16b2af45e0351b1c691c5311d48025dc6828079e98c2aa2e600dc5910ee8aa01858ca6c356538150dc46fe14c8819ed8ec8e4ec9a0f682b9950dd41bc50518fa
23fbbb100f wallet: sum ancestors rather than taking max in output groups (Karl-Johan Alm)
Pull request description:
This is pointed out in https://github.com/bitcoin/bitcoin/pull/12257#discussion_r204549758.
Basically, the ancestors gives an indication as to how many ancestors the resulting transaction will have, which is more precise when summing up the values, rather than taking the maximum, since all the coins in the group will become ancestors if selected.
Tree-SHA512: 0588c4b6059669650614817e041526a2ab89dda8c07fca8e077c7669dca1fed51cd164f7df56340840ab60285d48f3b140dcee64f64bf696b2dd4ab16d556a13
13bb5cae31 Docs: fix GetWarnings docs to reflect behavior (Ben Woosley)
Pull request description:
In "gui", it returns all warnings, joined by a separator
In "statusbar", it returns the last warning set which seems notionally to be the most important, though that is debatable
Tree-SHA512: 5fc0dc68d143a040b7b893b7176188e2b064c2cf1d559420906e4de636e16e9ab7451a1b87603020a7a8f66d6b94f4ee6c7da2697efad879f9e6de9c0e0c9ac1
faa24441ec policy: Remove promiscuousmempoolflags (MarcoFalke)
Pull request description:
It seems odd to clutter validation code with features that can only ever be used for testing (testnet or regtest). Removing that test-only code makes the mempool logic less painful to understand and easier to reason about when changed or refactored in the future.
Tree-SHA512: 3b897aa9604ac8d82ebe9573c6efd468c93ddaa08d378ebc902e247b7aa6c68fcde71e5b449c08f17a067146cdc66dc50a67ce06d07607c27e5189a49c3fba3f
93de2891fa wallet: assert to ensure accuracy of CMerkleTx::GetBlocksToMaturity (Ben Woosley)
Pull request description:
According to my understanding, it should not be possible for coinbase
transactions to be conflicting, thus it should not be possible for
GetDepthInMainChain to return a negative result. If it did, this would
also result in innacurate results for GetBlocksToMaturity due to the
math therein. asserting ensures accuracy.
Tree-SHA512: 8e71c26f09fe457cfb00c362ca27066f7f018ea2af1f395090fdc7fd9f5964b76f4317c23f7a4923776f00087558511da5c1c368095be39fb1bacc614a93c32f
a1a998cf24 wallet: Fix backupwallet for multiwallets (Daniel Kraft)
Pull request description:
`backupwallet` was broken for multiwallets in their own directories (i.e. something like `DATADIR/wallets/mywallet/wallet.dat`). In this case, the backup would use `DATADIR/wallets/wallet.dat` as source file and not take the specific wallet's directory into account.
This led to either an error during the backup (if the wrong source file was not present) or would silently back up the wrong wallet; especially the latter behaviour can be quite bad for users.
Tree-SHA512: 7efe2450ca047e40719fcc7cc211ed94699056020ac737cada7b59e8240298675960570c45079add424d0aab520437d5050d956acd695a9c2452dd4317b4d2c4
This commit slightly changes the format of the "Usage" strings in CLI
`-help` messages to meet the expection of the help2man tool, which we
use to generate man pages. On the way, we remove a few calls to
`strprintf()`, which became superficial after commit 32fbfda.
7bf22bf0c2 gui: Reject options dialog when key escape is pressed (João Barbosa)
4a43306a4f gui: Reject edit address dialog when key escape is pressed (João Barbosa)
f7a553177d gui: Add GUIUtil::ItemDelegate with keyEscapePressed signal (João Barbosa)
Pull request description:
Currently `EditAddressDialog` and `OptionsDialog` don't close when the escape key is pressed. The `QDataWidgetMapper` instances prevents closing the dialogs because the escape key is used to reset the widgets values. More details and workarounds in https://stackoverflow.com/a/51487847 and http://qtramblings.blogspot.com/2010/10/qdatawidgetmapper-annoyances.html.
The adopted solution is different from the above references. It turns out that `QDataWidgetMapper::setItemDelegate` sets the event filter for all mapped widgets. So in this PR the mapper's delegate are changed to a custom `GUIUtil::ItemDelegate` that offers the signal `keyEscapePressed`, which is connected to the `QDialog::reject` slot.
Note that the installed event filter lets all events pass, so the current behaviour isn't changed, meaning that widgets values are reset in addition to closing the dialog.
Tree-SHA512: 9c961d488480b4ccc3880a11a8f1824b65f77570ee8918c7302c62775a1a73e52ae988a31a55ffff87b4170ddbecf833c2f09b66095c00eb6854a4d43f030f1f
312ff01ee5 -prune option -help output aligned with code (Hennadii Stepanov)
Pull request description:
The -help output for -prune is aligned with the code.
In the code (.../src/init.cpp#L1063):
```
if (nPruneTarget < MIN_DISK_SPACE_FOR_BLOCK_FILES) {
return InitError(strprintf(_("Prune configured below the minimum of %d MiB. Please use a higher number."), MIN_DISK_SPACE_FOR_BLOCK_FILES / 1024 / 1024));
}
```
So correct value of nPruneTarget is **greater than or equal to** MIN_DISK_SPACE_FOR_BLOCK_FILES.
Tree-SHA512: 8e55aa99c8f5a9d020677b0f1b016215e2dbda5fa4ee7c8504b12a3abef226bc21beca118fa332c0bf206a4aff913a5a717b55bb5b2ecdba38423e9c0161209e
fa365021bb doc: Remove outdated net comment (MarcoFalke)
Pull request description:
`mapAddresses` and the corresponding "critsect" has been removed in 5fee401fe1 more than 6 years ago. Now is probably a good time to remove this confusing comment.
Tree-SHA512: 498a403d5703da395c18a7ebb776aa6e693e59fe43a839fefd261e0a5af58621763813979d4cfbd8d1728ce73b325b82002e393cde79bdbff33e0fbf68ab6747
fe7180c5b2 [trivial,doc] Fix memory consistency model in comment (Jesse Cohen)
Pull request description:
Updating a comment overlooked during review in #13247
Tree-SHA512: 0bd54ba1c265fdd77fd6e12ad0be46dd422348f7d926ce9abaca53fdb3a3c55c0d1cd90b4382321352076f4a81e2249c0014cd789f47a3637cb93bd983cb4657
f6b7fc349c Support h instead of ' in hardened descriptor paths (Pieter Wuille)
fddea672eb Add experimental warning to scantxoutset (Jonas Schnelli)
6495849bfd [QA] Extend tests to more combinations (Pieter Wuille)
1af237faef [QA] Add xpub range tests in scantxoutset tests (Jonas Schnelli)
151600bb49 Swap in descriptors support into scantxoutset (Pieter Wuille)
0652c3284f Descriptor tests (Pieter Wuille)
fe8a7dcd78 Output descriptors module (Pieter Wuille)
e54d76044b Add simple FlatSigningProvider (Pieter Wuille)
29943a904a Add more methods to Span class (Pieter Wuille)
Pull request description:
As promised, here is an implementation of my output descriptor concept (https://gist.github.com/sipa/e3d23d498c430bb601c5bca83523fa82) and integration within the `scantxoutset` RPC that was just added through #12196.
It changes the RPC to use descriptors for everything; I hope the interface is simple enough to encompass all use cases. It includes support for P2PK, P2PKH, P2WPKH, P2SH, P2WSH, multisig, xpubs, xprvs, and chains of keys - combined in every possible way.
Tree-SHA512: 63b54a96e7a72f5b04a8d645b8517d43ecd6a65a41f9f4e593931ce725a8845ab0baa1e9db6a7243190d8ac841f6e7e2f520d98c539312d78f7fd687d2c7b88f
a13647b8bd [qa] Add test for too-large wallet output groups (Suhas Daftuar)
57ec1c97b2 [wallet] correctly limit output group size (Suhas Daftuar)
Pull request description:
Also add a test to ensure that output groups are being limited, even if a wallet has many outputs corresponding to the same scriptPubKey (the test fails without the first commit).
Tree-SHA512: 2aaa82005b0910488f5cbf40690d4c5e2f46949e299ef70b4cb6e440713811443d411dcbc6d71b1701fd82423073125e21747787d70830cd021c841afb732d51
cbeaa91dbb Update ValidationInterface() documentation to explicitly specify threading and memory model (Jesse Cohen)
b296b425a7 Update documentation for SingleThreadedSchedulerClient() to specify the memory model (Jesse Cohen)
9994d01d8b Add Unit Test for SingleThreadedSchedulerClient (Jesse Cohen)
Pull request description:
As discussed in #13023 I've split this test out into a separate pr
This test (and documentation update) makes explicit the guarantee (previously undefined, but implied by the 'SingleThreaded' in `SingleThreadedSchedulerClient()`) - that callbacks pushed to the `SingleThreadedSchedulerClient()` obey the single threaded model for memory and execution - specifically, the callbacks are executed fully and in order, and even in cases where a subsequent callback is executed by a different thread, sequential consistency of memory for all threads executing these callbacks is maintained.
Maintaining memory consistency should make the api more developer friendly - especially for users of the validationinterface. To the extent that there are performance implications from this decision, these are not currently present in practice because all use of this scheduler happens on a single thread currently, furthermore the lock should guarantee consistency across callback executions even when callbacks are executed by multiple threads (as the test does).
Tree-SHA512: 5d95a7682c402e5ad76b05bc9dfbca99ca64105f62ab9e78f6fc0f6ea8c5277aa399fbb94298e35cc677b0c2181ff17259584bb7ae230e38aa68b85ecbc22856
494634a052 bench: Make CoinSelection output groups pass eligibility filter (Andrew Chow)
Pull request description:
Set the depth of the output groups used in the CoinSelection benchmark to be 6 in order to pass the eligibility filter for the benchmark.
Fixes#13813
Tree-SHA512: 55fc6aeda0127f5e155efb982aec211b70dfd3257808dce627886af6866ffa25de4df3c9b10f8c45b6c298a42542c54654f36e59efb208e9055885361f0e501c
247d5740d2 Ignore unknown config file options for now (Pieter Wuille)
04ce0d88ca Report when unknown config file options are ignored (Pieter Wuille)
Pull request description:
As reported by @satwo on IRC a few days ago, the current mechanism of treating unknown config file options as errors is problematic for options like `-rpcclienttimeout` which aren't defined for `bitcoind`.
A full solution would be to either make all binaries be aware of each other's options, or to permit config file options that only apply to specific binaries (`bitcoind`, `bitcoin-qt`, `bitcoin-cli`). Both of these seem too invasive to introduce for 0.17.
As a compromise, this PR makes it ignores those options, but still warn about it in the log file.
Tree-SHA512: dfddc771b91df3031a9c98d9f3292f8f4fcd1b97ebb7317b2f457e12d9f205dc63f42721302e7258dbb53f273d7cc041a65a0a9120972769555784e1f1cc9aef
fa8f2d826c doc: Fix chainTxData comment (MarcoFalke)
fa6094f152 chainparams: Update with data from assumed valid chain (MarcoFalke)
Pull request description:
Can be reviewed by using the `getblock` and `getchaintxstats` rpcs of a synced node. Reviewers get extra points when their full node has checkpoints and assumevalid disabled.
Tree-SHA512: cedd61fde129ae4c16fc12275b61e4c5659b6d72dd801c608efc294188561bc986d94652fe9bea71ada48654258e2a074d2d2da78036c69608ccff3a6cc1ccf5
fa5ed4f8d2 refactor: Avoid locking tx pool cs thrice (MarcoFalke)
Pull request description:
`addUnchecked` is (outside the tests) only called by ATMP, which already takes the tx pool read lock. So locking it twice more in both `addUnchecked` methods seems redundant.
Similarly `CalculateMemPoolAncestors` is (beside once in the wallet) only called in contexts, where the tx pool lock is already taken. So remove the lock there as well.
Tree-SHA512: fcf603b570da0fc529fe6db8add218663eae52845510732bee0d4611263d2429d3d3c9c8ae68493d67287d13504500ed51905ccbe711eb15a0af3b019edad543
620361fce8 Fix accidental use of the addition assignment operator ("+="). Remove newlines from error message. (practicalswift)
Pull request description:
Fix accidental use of the addition assignment operator (`+=`).
_Note to reviewers:_ Perhaps the `\n`:s should be removed too?
Tree-SHA512: 4e8c2dfd6025d78ef9d60522297994829dacc447e6b6782e15c0bdd5dd2daa17ca9a8948bfa9a15be57d9286092356381d7e6747980303852d273eb0df0dd76b
46340b3337 [bench] Add benchmark for unserialize prevector (Akio Nakamura)
Pull request description:
This PR adds benchmarks for the unserialization of the prevector.
Note: Separated from #12324.
Tree-SHA512: c055a283328cc2634c01eb60f26604a8665939bbf77d367b6ba6b4e01e77d4511fab69cc3ddb1e62969adb3c48752ed870f45ceba153eee192302601341e18a7
3fe836b78d [moveonly] Extract RescanWallet to handle a simple rescan (Ben Woosley)
Pull request description:
Where the outcome does not depend on the result, apart from a simple
success check.
Tree-SHA512: e0d29c6fc0c7f99a730289e5a80deb586b2848aead56b5198a71ef01f65374812468dfd57be0b8b076eb9be4090d5101d28d979a1d5c3d2f1caeca77b303e90e
fad231ad41 Fix merging of global unknown data in PSBTs (Andrew Chow)
41df035ee1 Check that PSBT keys are the correct length (Andrew Chow)
Pull request description:
This PR fixes a few bugs that were found and adds tests checking for these errors.
Specifically:
- Single byte keys are checked to actually be one byte.
- Unknown global data must be merged when combining two PSBTs.
Tree-SHA512: c0e7b4bc607d510cc005aaa7c0813ee58c5467ab7ce4adce485522dfeee92b1af3d29fe89df778b0ea812bb3827e085b30e04d4f4ebcefd8364d809573991332
12dd101345 scripted-diff: Remove trailing whitespaces (João Barbosa)
Pull request description:
The script test/lint/lint-whitespace.sh should prevent new cases.
This happens in some pulls where the code editor and the author 'git add's them, so this would fix it all.
Tree-SHA512: bcdd3472fcd01a2754e52212c7db1de2fdc422728b06785481954a27162fb72001cb73708329cc56e95bcc5e45c1348ebc4eacc2ccfa6aa12413c7ec450b6a33
