be8ab7d08 Create new wallet databases as directories rather than files (Russell Yanofsky)
26c06f24e Allow wallet files not in -walletdir directory (Russell Yanofsky)
d8a99f65e Allow wallet files in multiple directories (Russell Yanofsky)
Pull request description:
This change consists of three commits:
* The first commit is a pure refactoring that removes the restriction that two wallets can only be opened at the same time if they are contained in the same directory.
* The second commit removes the restriction that `-wallet` filenames can only refer to files in the `-walletdir` directory.
* The third commit makes second commit a little safer by changing bitcoin to create wallet databases as directories rather than files, so they can be safely backed up.
All three commits should be straightforward:
* The first commit adds around 20 lines of new code and then updates a bunch of function signatures (generally updating them to take plain fs::path parameters, instead of combinations of strings, fs::paths, and objects like CDBEnv and CWalletDBWrapper).
* The second commit removes two `-wallet` filename checks and adds some test cases to the multiwallet unit test.
* The third commit just changes the mapping from specified wallet paths to bdb environment & data paths.
---
**Note:** For anybody looking at this PR for the first time, I think you can skip the comments before _20 Nov_ and start reading at https://github.com/bitcoin/bitcoin/pull/11687#issuecomment-345625565. Comments before _20 Nov_ were about an earlier version of the PR that didn't include the third commit, and then confusion from not seeing the first commit.
Tree-SHA512: 00bbb120fe0df847cf57014f75f1f7f1f58b0b62fa0b3adab4560163ebdfe06ccdfff33b4231693f03c5dc23601cb41954a07bcea9a4919c8d42f7d62bcf6024
08e0855b9 Give hint about gitian not able to download (kallewoof)
Pull request description:
Gitian fails to perform downloads right now on my set up. This can be circumvented by first checking out the tag being built and then doing the depends download step before running `gbuild`.
This should of course be fixed in gitian, but having this note until it's fixed is definitely useful.
Tree-SHA512: ae9d0eb44ecfdae44d35aecc6e5fd6db7d9e95b8e0badc76a1d9aaf8fe70bc00a2914dfcb4f516d030560835af411515ca13736ebf8b49b7040b340457882779
Gitian fails to perform downloads right now on my set up. This can be circumvented by first checking out the tag being built and then doing the depends download step before running `gbuild`.
cfaac2a60 Add build support for 'gprof' profiling. (murrayn)
Pull request description:
Support for profiling build: `./configure --enable-profiling`
Tree-SHA512: ea983cfce385f1893bb4ab7f94ac141b7d620951dc430da3bbc92ae1357fb05521eac689216e66dc87040171a8a57e76dd7ad98036e12a2896cfe5ab544347f0
13a399a46 depends: patch pthread_set_name_np out of zeromq (Cory Fields)
8f7922636 depends: zeromq 4.2.3 (fanquake)
Pull request description:
This is a followup to #9254 and #11981. Zeromq 4.2.3 was released just after #9254 was merged, and contains a years worth of improvements/bug fixes. See the release notes [here](https://github.com/zeromq/libzmq/releases/tag/v4.2.3).
Todo:
- [ ] Add zeromq-4.2.3.tar.gz to /depends-sources on bitcoincore.org
- [ ] Verify gitian builds are still OK
- [ ] Check: https://github.com/zeromq/libzmq/pull/2787
Tree-SHA512: 85e06f47be3e1fdedcee50ce90e3391d69df2ea1c167472ffc3126d8970d418eb75141b970e422eb2fda9a8cad00e6ba5b36afa53565171a9ebaa152a9dc9b60
7eb665fc8 [Trivial] link mentioned scripted-diff-commit (Felix Wolfsteller)
Pull request description:
Make it easier for people who do not operate on a cloned repository to access the example mentioned.
Tree-SHA512: 1c06e551c68cad03e6bd541bf0e0076cdf0b48ef9b8b4e4a61435367c3435e2e4ccb934112e8dc29d3d70217d8834924704aaf839e25d1133312df86848ca1a1
4d14d06fc docs: clarified systemd installation instructions in init.md for Ubuntu users. (DaveFromBinary)
Pull request description:
Added a note to init.md to clarify the .service copy path for Ubuntu because it differs from the described copy path.
Also noted which version of Ubuntu switched to systemd for the default system init to clarify when the systemd installation steps should be used instead of the upstart installation steps for Ubuntu users.
Tree-SHA512: 1ac6143a177d0f3782ff641029d71eb1f3b3be0c1482e266154d3ca093251b58a10a5f037d1cc82dbfaeae058df2bb8e904833ccb88b032f1a59a151724f95e2
fa9461473 [doc] dev-notes: Members should be initialized (MarcoFalke)
Pull request description:
Also, remove mention of threads that were removed long ago.
Motivation:
Make it easier to spot bugs such as #11654 and #12426
Tree-SHA512: 8ca1cb54e830e9368803bd98a8b08c39bf2d46f079094ed7e070b32ae15a6e611ce98d7a614f897803309f4728575e6bc9357fab1157c53d2536417eb8271653
This change should make it easier for users to make complete backups of wallets
because they can now just back up the specified `-wallet=<path>` path directly,
instead of having to back up the specified path as well as the transaction log
directory (for incompletely flushed wallets).
Another advantage of this change is that if two wallets are located in the same
directory, they will now use their own BerkeleyDB environments instead using a
shared environment. Using a shared environment makes it difficult to manage and
back up wallets separately because transaction log files will contain a mix of
data from all wallets in the environment.
Added a note to init.md to clarify the .service copy path for newer versions of Ubuntu because it differs from the described copy path.
Also noted what version of Ubuntu switched to systemd for the default system init.
a777244e4 Build: Add a makefile target for Doxygen documentation (Andrea Comand)
Pull request description:
You can now build the doxygen documentation with `make docs` and clean it with `make clean-docs`.
Fixes: #11949
Tree-SHA512: f2361ec7f771227367dd04bba1a444b44e59f13901463a678a5f2f579a10a56d67db2e29552e754e312a1c472a31593b6af189cbaac5cd351a428c57baf5ace7
9b6454c Improve "Turn Windows Features On or Off" step (Ernest Hemingway)
Pull request description:
Originally, this readme suggests searching for 'turn' to open this dialog but this will not necessarily work on all windows 10 PCs. It's better to use the executable name instead, which is consistent across installations.
Tree-SHA512: e5b95dd69a9a186ea5cd9c7aac2283e77f1857ecf628f8ad6ac0411f362c8aeb52e3bcffb46b90e3bab52f45fa244f269b1777f83d3e0519ac8a95935f7fb5b4
c8edc2c [docs] initial QT documentation, move Qt Creator instructions (Sjors Provoost)
Pull request description:
I'll update this as I figure out how everything is tied together, but I think it's a useful enough start.
Tree-SHA512: d96e5c9ba8ccc3a1b92a0894a8a8449317100eebb14e5d390b51793534458f50eac296cf2945fccf81b85aff23fa32d91d6015a0a76ada4f7091a400d7508ae5
fabb72b contrib: Remove xpired 522739F6 key (MarcoFalke)
faeab66 contrib: Replace developer keys with list of pgp fingerprints (MarcoFalke)
Pull request description:
Having to host a copy of the keys in this repo was a common source of discussion and distraction, caused by problems such as:
* Outdated keys. Unclear whether and when to replace by fresh copies.
* Unclear when to add a key of a new developer or Gitian builder.
The problems are solved by
* Having no keys but only the fingerprints
* Adding a rule of thumb, when to add a new key
<strike>Moving the keys to a different repo solves none of these issues, but since the keys are not bound to releases or git branches of Bitcoin Core, they should live somewhere else.
Obviously, all keys are hosted and distributed on key servers, but were added to the repo solely for convenience and redundancy.
Moving the mirror of those keys to a different repo makes it less distracting to update them -- let's say -- prior to every major release.
I updated our `doc/release-process.md` to reflect the new location.
DEPENDS_ON https://github.com/bitcoin-core/gitian.sigs/pull/621
</strike>
Tree-SHA512: c00795a07603190e26dc4526f6ce11e492fb048dc7ef54b38f859b77dcde25f58ec4449f5cf3f85a5e9c2dd2743bde53f7ff03c8eccf0d75d51784a6b164e47d
eeeb416 Remove suggestion to make cloned repository world-writable for Windows build. (murrayn)
Pull request description:
Current documentation for Windows build on Ubuntu suggests cloning the repository into /usr/src, as root, and making the tree world-writable(!). I can see no problem this solves, and it introduces obvious security issues.
Tree-SHA512: 05429a64319c046f5506f7d27c64c94f94cfe6d14ec5f01dccf843fc417e954fe96e1abc43126b9204a1178f101e4a8da9eece32b5de4b348c7c9358615c7e0f
7444149 Document method for reviewers to verify chainTxData (John Newbery)
Pull request description:
This commit adds the final block hash of the window to getchaintxstats
and documents how reviewers can verify changes to chainTxData.
Tree-SHA512: d16abb5f47d058e52660f4d495f1e453205b1b83716d7c810ff62a70338db721386c1808ec1fc8468f514e4d80cc58e3c96eeb3184cbbcb1d07830fa5e53f342
Master was bumped to 0.16.99, so update the man pages too to avoid
confusion.
Tree-SHA512: 63622d6ebea2fb052ffe05fb80fe08bd627c34310a7ca22b2bc1af74003b20ab1a1fde51746ee69d401379d65232981b68541a9fc7f329e04b854507f836b19e
Also clean out release notes.
Tree-SHA512: c4d5b52c089e14438be37381e1b0dab3711cc72aa8d345d1024169fff0055f3d021c8ca9d46fb794110694ebcbf7cbca0a12619f650873c9d381530adea7100e
Also warn if bitcoind is configured to use a relative -datadir path.
Specifying paths relative to the current working directory in a daemon process
can be dangerous, because files can fail to be located even if the
configuration doesn't change, but the daemon is started up differently.
Specifying a relative -datadir now adds a warning to the debug log. It would
not be backwards-compatible to forbid relative -datadir paths entirely, and it
could also be also inconvenient for command line testing.
Specifying a relative -walletdir now results in a startup error. But since the
-walletdir option is new in 0.16.0, there should be no compatibility issues.
Another reason not to use working directory paths for -walletdir specifically
is that the default -walletdir is a "wallets" subdirectory inside the datadir,
so it could be surprising that setting -walletdir manually would choose a
directory rooted in a completely different location.
97c3cad [docs] Clarify -walletdir usage (John Newbery)
Pull request description:
After discussion with @ryanofsky around #11687 , I think this documentation is a bit clearer for how the new `-walletdir` argument works.
Tree-SHA512: f279cab82524dbc0d75e6f9891f0e228ec4c8d0df3e16f351057fa243ddd263ff786f05383fd00a09b89edcc07dab211be5b64387f77271edf8af0177bcf667d
2be2b5d Remove the ending slashes from RPC URI format. (Jacky C)
Pull request description:
This resolves#11861 (A confusion caused by incorrect information in the release notes).
More information can be found at #11861.
Tree-SHA512: 35f85854b01a84acd5358e0c9deff881205111120277fa7cdf270801933c2603c2ae04fa4d55d233675c7298c2d37cc60c919f89e7e6091f5c61884025775ab0
ebcee1de2 bips: add bip176 (Bits Denomination) (William Casarin)
275b2eeed [qt] change µBTC to bits (William Casarin)
Pull request description:
Now that we have bip176, change "µBTC" to the more colloquial "bits"
Tree-SHA512: eba5e5f89c392728a4f0a3bd81a9779a117b8d72a490390fd031d4e7cc56c2bfee0016aba7ef9535903e8cf2262ce46497283424e378906d0e3bf5b0d2d981c7
5ec3eae remove brew c++ flag (Pablo Fernandez)
Pull request description:
the c++ is not needed since the listed brew formulas no longer have this option. It also raises a warning that may generate confusion.
Tree-SHA512: 37c0ebee2901947a21abfcd646ae0c7e73293568f37db9b130d1c521aa4222b0e82e21614e6ac60bb5347c34ef15bbd24eb8066ebf576ea3da9f365be9a4d062
6915f93 doc: Update OpenBSD build instructions for 6.2 (Wladimir J. van der Laan)
Pull request description:
(this continues #11442)
There is no more need to install a new compiler. This simplifies instructions a lot.
From discussion with @fanquake on IRC I first wanted to add a new section for 6.2, but that made the document a complex mess. I think it's good enough (and more maintainable too) to only support the most recent release.
Includes #11976.
I moved the "resource limits" section to the end as I didn't seem to need it with clang, but this may vary based on source changes and the phase of the moon so it's good to keep it as optional extra information.
Tree-SHA512: 15794afec6d682323d0aa13c7616d009acb7fce8b0ef5d2106261f2ebd86b7b2fe66040c04860d9bf2f0c1934fbdc2b594b8c09a98accfaac04f3daf9a6cadf3
GUI wallet uses RBF by default, regardless of -walletrbf.
RPC and debug console in the GUI remain unchanged; they don't
use RBF by default, unless launched with -walletrbf=1.
656fde5 Add script birthtime metadata to dump and import wallet (MeshCollider)
1bab9b2 Add script dump note to RPC help text and release notes (MeshCollider)
68c1e00 Add test for importwallet (MeshCollider)
9e1184d Add dumpwallet scripts test (MeshCollider)
ef0c730 Add scripts to importwallet RPC (MeshCollider)
b702ae8 Add CScripts to dumpwallet RPC (MeshCollider)
cdc260a Add GetCScripts to CBasicKeyStore (MeshCollider)
Pull request description:
As discussed in https://github.com/bitcoin/bitcoin/pull/11289#issuecomment-334600457, adds the CScripts from the wallet to the `dumpwallet` RPC and then allows them to be imported with the `importwallet` RPC. Includes a basic test, and modifies the helptext of the dumpwallet RPC.
Notes:
- Reviewers: use `?w=1` to avoid the indentation-only change in commit `Add scripts to importwallet RPC `
- currently the scripts are followed with `# addr=` comments just as the other keys are, unsure if this might confuse users into thinking all the scripts are for valid P2SH addresses though, but I don't think that should be an issue.
- there are no birthtimes for scripts, so script imports don't affect rescans
- `importwallet` imports the CScripts but I'm not sure how to approach specifying whether scripts are for P2SH addresses, BIP173 addresses, etc. whether that matters or not. Otherwise the RPC helptext might just need modification.
Fixes#11715
Tree-SHA512: 36c55837b3a58b9d3499d4c0c2ae82153d62aa71919e751574651b63a1d2b8ecc83796db4553cc65dad9b5341c3a42ae2fcf4d62598c30af267f8e1461ba8272
2712742 doc: Update FreeBSD build instructions to use bdb4 (Wladimir J. van der Laan)
d95c83d contrib: FreeBSD compatibility in install_db4.sh (Wladimir J. van der Laan)
c0298b0 contrib: Make X=Y arguments work in install_db4 (Wladimir J. van der Laan)
b798f9b contrib: New clang patch for install_db4 (Wladimir J. van der Laan)
Pull request description:
This PR improves the BSD compatibility of the bdb4 installer script.
See #11921, #11868.
I've tested this on OpenBSD 6.2 (clang) and Ubuntu 16.04 (gcc).
This needs testing on OSX at least, ~~and on gcc/Linux to make sure that applying the patch unconditionally doesn't negatively affect gcc~~.
~~NB: this is not yet sufficient to make `install_db4.sh` work on FreeBSD, as we need to use yet another `sha256` tool there. But it's a step in the right direction.~~
### contrib: New clang patch for install_db4
Replace the clang patch with a new and improved version that also fixes the build issues with OpenBSD and FreeBSD's clang, and apply it unconditionally.
Thanks to @fanquake for finding the patch.
### contrib: Make X=Y arguments work in install_db4
Trailing X=Y arguments are supposed to be passed through unchanged to bdb's configure. This was not the case, at least with OpenBSD 6.2's shell.
Fix this by not storing the arguments in a temporary variable but passing "$@" through directly.
### contrib: FreeBSD compatibility in install_db4.sh
Unfortunately, FreeBSD uses yet another syntax for `sha256`.
Support FreeBSD's syntax too. Using `uname` is a bit of a hack but it works and I found no way to distinguish the two.
Tree-SHA512: 12461a58dfeb4834701891762efc747c8187d834f41d98c8451edee1402a3958c4842bbc02c61bacbc7b0d90cc6b020a2ca158b65304d9760c9f0d2052ff36d4
aac6b3f067 Update files.md for new wallets/ subdirectory (MeshCollider)
b67342906c Cleanups for walletdir PR (MeshCollider)
Pull request description:
This addresses the remaining nits from https://github.com/bitcoin/bitcoin/pull/11466
- Updates `doc/files.md` with respect to the new default wallet directory
- Fixes @promag and @laanwj's error message nit, and Jonas' release notes nit
- ~Addresses @laanwj's net-specific wallet subdirectory concern in the case that a walletdir is specified~
- Changes the #includes from "" to <> style after #11651
Tree-SHA512: b86bf5fdc4de54c1b0f65b60a83af3cf82b35d216ce9c0de724803bfba6934796238b6c412659dcc29ae2e3e856d4eb97ae777c80f36f4089d8acecfddefe9aa
3121d76 doc: Update release notes for share/rpcauth/rpcauth.py rename (Henrik Jonsson)
3fdb297 Rename rpcuser.py to rpcauth.py (Henrik Jonsson)
Pull request description:
This script creates `rpcauth` entries for bitcoin.conf, not the deprecated `rpcuser` entry, so this changes the name of the script to match.
As discussed in #11830.
Tree-SHA512: cd71c2a4043ef1381d3810b057cc83be3fac612df576b91b683ef91fdb7998c534b3b97a3313845eb867dc4bf7cc42a1250474d2261ab3f9ed2f884ca8ebd9f4
543ab40 Docs: Bump OSX version to 10.13 in build-osx.md (Varunram Ganesh)
Pull request description:
Core works fine on macOS 10.13
Tree-SHA512: 44807920e3d5518c98d68191ed614019934bb702df9695389f8178a00a7c5afccd90fad68a07568cdc3e3d04068bddf434942fb81dceb6cbde9bb4ff4e49e425
c1e5d40 Make debugging test crash easier (MeshCollider)
8263f6a Create walletdir if datadir doesn't exist and fix tests (MeshCollider)
9587a9c Default walletdir is wallets/ if it exists (MeshCollider)
d987889 Add release notes for -walletdir and wallets/ dir (MeshCollider)
80c5cbc Add test for -walletdir (MeshCollider)
0530ba0 Add -walletdir parameter to specify custom wallet dir (MeshCollider)
Pull request description:
Closes#11348
Adds a `-walletdir` parameter which specifies a directory to use for wallets, allowing them to be stored separately from the 'main' data directory. Creates a new `wallets/` directory in datadir if this is the first time running, and defaults to using it if it exists.
Includes tests and release notes. Things which might need to be considered more:
- there is no 'lock' on the wallets directory, which might be needed?
- because this uses a new wallets/ directory by default, downgrading to an earlier version won't see the wallets in that directory (not a big deal though, users can just copy them up to the main dir)
- jnewbery suggested putting each wallet in its own directory, which is a good idea, but out of scope for this PR IMO. EDIT: this is being done in https://github.com/bitcoin/bitcoin/pull/11687
- doc/files.md needs updating (will do soon)
I also considered including a cleanup by removing caching of data directory paths and instead just initialise them once on startup (c.f. #3073), but decided it wasn't super relevant here will just complicate review.
Tree-SHA512: c8ac04bfe9a810c32055f2c8b8fa0d535e56125ceb8d96f12447dd3538bf3e5ee992b60b1cd2173bf5f3fa023a9feab12c9963593bf27ed419df929bb413398d
- Added steps to specify required source location for Windows WSL
builds.
- Attempted to make the introduction and options for building on Windows
clearer.
6e4cdd6 [docs] Add reference to install_db4.sh in OS X build instructions (James O'Beirne)
af9103e [build] Add a script for installing db4 (James O'Beirne)
Pull request description:
Instead of maintaining rote instructions for building BerkeleyDB in `doc/build-{unix,openbsd}.md`, reference a script that does the same thing and can be called from unanticipated contexts, e.g. Docker builds.
The script was written with portability in mind, though I haven't tested it on openbsd.
I wasn't sure if we wanted to create a separate directory for this sort of thing (e.g. `contrib/install`) so I just stuck it in `contrib/`; happy to move it around if anyone has another preference.
Tree-SHA512: d2fc83c065d083458c448e6041e5e9ef67f8165974925560a83881d22d1e9448ea3dd4f7a38196800a8cd6dcf206208a2d6d12417bfe094902d4754e4ca67f18
Instead of maintaining not-easily-tested instructions for building BerkeleyDB
in doc/build-unix.md, package the installation as a script in contrib/. This
allows shared usage from a number of contexts, e.g. Docker.
Thanks to @jonasschnelli, @laanwj for feedback.
89f0312 Remove redundant pwallet nullptr check (Matt Corallo)
c4784b5 Add a dev notes document describing the new wallet RPC blocking (Matt Corallo)
3ea8b75 Give ZMQ consistent order with UpdatedBlockTip on scheduler thread (Matt Corallo)
cb06edf Fix wallet RPC race by waiting for callbacks in sendrawtransaction (Matt Corallo)
e545ded Also call other wallet notify callbacks in scheduler thread (Matt Corallo)
17220d6 Use callbacks to cache whether wallet transactions are in mempool (Matt Corallo)
5d67a78 Add calls to CWallet::BlockUntilSyncedToCurrentChain() in RPCs (Matt Corallo)
5ee3172 Add CWallet::BlockUntilSyncedToCurrentChain() (Matt Corallo)
0b2f42d Add CallFunctionInQueue to wait on validation interface queue drain (Matt Corallo)
2b4b345 Add ability to assert a lock is not held in DEBUG_LOCKORDER (Matt Corallo)
0343676 Call TransactionRemovedFromMempool in the CScheduler thread (Matt Corallo)
a7d3936 Add a CValidationInterface::TransactionRemovedFromMempool (Matt Corallo)
Pull request description:
Based on #10179, this effectively reverts #9583, regaining most of the original speedups of #7946.
This concludes the work of #9725, #10178, and #10179.
See individual commit messages for more information.
Tree-SHA512: eead4809b0a75d1fb33b0765174ff52c972e45040635e38cf3686cef310859c1e6b3c00e7186cbd17374c6ae547bfbd6c1718fe36f26c76ba8a8b052d6ed7bc9
5ff01c236 [docs] Add instructions for lcov coverage report generation (James O'Beirne)
Pull request description:
After rediscovering the `lcov` report generation recipe one too many times, it seemed prudent to write some doc.
Tree-SHA512: 20e1b5f51ecd39e14bd67986a2c1578fb7da03a50625366eaca35b201db66aef99cd4a5456df3aaca5d2d66b18ed7d2e8eb8f3bd9c7aaf9af48164d9bac38931
7383d77 Updated instructions for Windows 10 Fall Creators Update. (Aaron Clauson)
e0fc4a7 Updated Windows build doc for WSL/Xenial workarounds. (Aaron Clauson)
Pull request description:
An update to the Windows build document that provides workarounds for the broken 64 bit mingw32 cross compiler on WSL/Xenial.
This update is an alternative to pull request #11437. While that pull request takes a valid approach by stating building on WSL should be avoided I think it is more useful to give Windows developers a workaround option.
The instructions have been tested on:
- Ubuntu 14.04 and 64 bit mingw32 tool chain
- Ubuntu 16.04 and 64 bit mingw32 tool chain
- Ubuntu 17.04 and 64 bit mingw32 tool chain
- Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 32 bit mingw32 tool chain
- Windows Subsystem for Linux (Windows 10 OS Build 15063.608) and 64 bit mingw32 tool chain
Related items:
- Serious incompatibility problems w/ newer mingw-64 on Ubuntu #8653
- `-fstack-protector-all` triggers crashes in mingw-w64 5.3.1 #8732
- Windows build appears broken on WSL (buntu okay) #10269
- Compilation error for windows target #11437
Tree-SHA512: 7c937e37ed7120ae5dcf61aba50e5228a7ed6f729647c724b8f48e7cbbd81366c1a83a818618766a8fe0418425e05ba2eba2b14f2616621c58606585444f45fc
