LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity
The blockchain that provides the digital content namespace for the LBRY protocol
16972 commits 22 branches 42 tags 99 MiB
C++ 66.5%
Python 15.1%
C 12.7%
M4 2.1%
Makefile 1.2%
Other 2.3%
Find a file
Pieter Wuille a07e8caa5d
Merge #13033: Build txindex in parallel with validation 
9b2704777c [doc] Include txindex changes in the release notes. (Jim Posen)
ed77dd6b30 [test] Simple unit test for TxIndex. (Jim Posen)
6d772a3d44 [rpc] Public interfaces to GetTransaction block until synced. (Jim Posen)
a03f804f2a [index] Move disk IO logic from GetTransaction to TxIndex::FindTx. (Jim Posen)
e0a3b80033 [validation] Replace tx index code in validation code with TxIndex. (Jim Posen)
8181db88f6 [init] Initialize and start TxIndex in init code. (Jim Posen)
f90c3a62f5 [index] TxIndex method to wait until caught up. (Jim Posen)
70d510d93c [index] Allow TxIndex sync thread to be interrupted. (Jim Posen)
94b4f8bbb9 [index] TxIndex initial sync thread. (Jim Posen)
34d68bf3a3 [index] Create new TxIndex class. (Jim Posen)
c88bcec93f [db] Migration for txindex data to new, separate database. (Jim Posen)
0cb8303241 [db] Create separate database for txindex. (Jim Posen)

Pull request description:

  I'm re-opening #11857 as a new pull request because the last one stopped loading for people

  -------------------------------

  This refactors the tx index code to be in it's own class and get built concurrently with validation code. The main benefit is decoupling and moving the txindex into a separate DB. The primary motivation is to lay the groundwork for other indexers that might be desired (such as the [compact filters](https://github.com/bitcoin/bips/pull/636)). The basic idea is that the TxIndex spins up its own thread, which first syncs the txindex to the current block index, then once in sync the BlockConnected ValidationInterface hook writes new blocks.

  ### DB changes

  At the suggestion of some other developers, the txindex has been split out into a separate database. A data migration runs at startup on any nodes with a legacy txindex. Currently the migration blocks node initialization until complete.

  ### Open questions

  - Should the migration of txindex data from the old DB to the new DB block in init or should it happen in a background thread? The downside to backgrounding it is that `getrawtransaction` would return an error message saying the txindex is syncing while the migration is running.

  ### Impact

  In a sample size n=1 test where I synced nodes from scratch, the average time [Index writing](https://github.com/bitcoin/bitcoin/blob/master/src/validation.cpp#L1903) was 3.36ms in master and 1.72ms in this branch. The average time between `UpdateTip` log lines for sequential blocks between 400,000 and IBD end on mainnet was 0.297204s in master and 0.286134s in this branch. Most likely this is just variance in IBD times, but I can try with some more trials if people want.

Tree-SHA512: 451fd7d95df89dfafceaa723cdf0f7b137615b531cf5c5035cfb54e9ccc2026cec5ac85edbcf71b7f4e2f102e36e9202b8b3a667e1504a9e1a9976ab1f0079c4
2018-04-25 18:47:47 -07:00
.github Make default issue text all comments to make issues more readable 2017-11-16 11:50:56 -05:00
.tx tx: Update transifex slug for 0.16 2018-01-24 16:35:40 +01:00
build-aux/m4 ax_boost_{chrono,unit_test_framework}.m4: take changes from upstream 2018-03-15 19:59:11 +01:00
contrib Merge #12972: Add python3 script shebang lint 2018-04-16 16:03:22 -04:00
depends Merge #12715: depends: Add 'make clean' rule 2018-04-18 11:28:05 +02:00
doc [doc] Include txindex changes in the release notes. 2018-04-25 11:25:19 -07:00
share [tests] Make rpcauth.py testable and add unit tests 2018-04-24 11:41:20 -07:00
src [test] Simple unit test for TxIndex. 2018-04-25 11:25:18 -07:00
test Don't test against min relay fee information in mining_prioritisetransaction.py 2018-04-26 00:30:38 +03:00
.gitattributes Separate protocol versioning from clientversion 2014-10-29 00:24:40 -04:00
.gitignore [build] .gitignore: add QT Creator artifacts 2017-12-22 12:37:00 +01:00
.travis.yml Add shell script linting: Check for shellcheck warnings in shell scripts 2018-04-11 15:41:18 +02:00
autogen.sh Add MIT license to autogen.sh and share/genbuild.sh 2016-09-21 23:01:36 +00:00
configure.ac test: add rpcauth-test to AC_CONFIG_LINKS to fix out-of-tree make check 2018-04-25 16:37:10 +02:00
CONTRIBUTING.md Docs: Improve documentation on standard communication channels 2018-03-22 12:58:57 -07:00
COPYING [Trivial] Update license year range to 2018 2018-01-01 04:33:09 +09:00
INSTALL.md Update INSTALL landing redirection notice for build instructions. 2016-10-06 12:27:23 +13:00
libbitcoinconsensus.pc.in Unify package name to as few places as possible without major changes 2015-12-14 02:11:10 +00:00
Makefile.am [tests] Make rpcauth.py testable and add unit tests 2018-04-24 11:41:20 -07:00
README.md Docs: Improve documentation on standard communication channels 2018-03-22 12:58:57 -07:00

README.md

Bitcoin Core integration/staging tree

Build Status

https://bitcoincore.org

What is Bitcoin?

Bitcoin is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Bitcoin Core is the name of open source software which enables the use of this currency.

For more information, as well as an immediately useable, binary version of the Bitcoin Core software, see https://bitcoin.org/en/download, or read the original whitepaper.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built and tested, but is not guaranteed to be completely stable. Tags are created regularly to indicate new official, stable release versions of Bitcoin Core.

The contribution workflow is described in CONTRIBUTING.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python, that are run automatically on the build server. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The Travis CI system makes sure that every pull request is built for Windows, Linux, and OS X, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.

Translators should also subscribe to the mailing list.