LBRYCommunity/lbrycrd
1
0
Fork 0
Code Issues 29 Pull requests 1 Projects Releases 32 Packages Wiki Activity
The blockchain that provides the digital content namespace for the LBRY protocol
21433 commits 22 branches 42 tags 99 MiB
C++ 66.5%
Python 15.1%
C 12.7%
M4 2.1%
Makefile 1.2%
Other 2.3%
Find a file
Wladimir J. van der Laan ab765c2ec7
Merge #16577: util: CBufferedFile fixes and unit test 
efd2474d17 util: CBufferedFile fixes (Larry Ruane)

Pull request description:

  The `CBufferedFile` object guarantees its user is able to "rewind" the data stream (that's being read from a file) up to a certain number of bytes, as specified by the user in the constructor. This guarantee is not honored due to a bug in the `SetPos` method.

  Such rewinding is done in `LoadExternalBlockFile()` (currently the only user of this object), which deserializes a series of `CBlock` objects. If that function encounters something unexpected in the data stream, which is coming from a `blocks/blk00???.dat` file, it "rewinds" to an earlier position in the stream to try to get in sync again. The `CBufferedFile` object does not actually rewind its file offset; it simply repositions its internal offset, `nReadPos`, to an earlier position within the object's private buffer; this is why there's a limit to how far the user may rewind.

  If `LoadExternalBlockFile()` needs to rewind (call `blkdat.SetPos()`), the stream may not be positioned as it should be, causing errors in deserialization. This need to rewind is probably rare, which is likely why this bug hasn't been noticed already. But if this object is used elsewhere in the future, this could be a serious problem, especially as, due to the nature of the bug, the `SetPos()` _sometimes_ works.

  This PR adds a unit test for `CBufferedFile` that fails due to this bug. (Until now it has had no unit tests.) The unit test provides good documentation and examples for developers trying to understand `LoadExternalBlockFile()` and for future users of this object.

  This PR also adds code to throw an exception from the constructor if the rewind argument is not less than the buffer size (since that doesn't make any sense).

  Finally, I discovered that the object is too restrictive in one respect: When the deserialization methods call this object's `read` method, a check ensures that the number of bytes being requested is less than the size of the buffer (adjusting for the rewind size), else it throws an exception. This restriction is unnecessary; the object being deserialized can be larger than the buffer because multiple reads from disk can satisfy the request.

ACKs for top commit:
  laanwj:
    ACK ~after squash.~ efd2474d17
  mzumsande:
    I had intended to follow up earlier on my last comment, ACK efd2474d17. I reviewed the code, ran tests and did a successful reindex on testnet with this branch.

Tree-SHA512: 695529e0af38bae2af4e0cc2895dda56a71b9059c3de04d32e09c0165a50f6aacee499f2042156ab5eaa6f0349bab6bcca4ef9f6f9ded4e60d4483beab7e4554
2019-09-26 13:38:39 +02:00
.github doc: Add issue templates for bug and feature request 2019-09-08 15:39:54 +02:00
.tx gui: Update transifex slug for 0.19 2019-09-02 13:40:01 +02:00
build-aux/m4 Merge #16870: build: update boost macros to latest upstream for improved error reporting 2019-09-21 10:00:28 +08:00
build_msvc Added libbitcoin_qt and bitcoin-qt to the msbuild configuration. 2019-09-08 14:13:05 +02:00
ci Merge #16961: test: Remove python dead code linter 2019-09-25 13:14:22 -04:00
contrib doc: Fix whitespace errs in .md files, bitcoin.conf, Info.plist.in, and find_bdb48.m4 2019-09-17 03:21:22 -04:00
depends Merge #16837: depends: qt: Fix {C{,XX},LD}FLAGS pickup 2019-09-26 13:19:20 +02:00
doc doc: Improve doxygen readme navigation section 2019-09-23 19:22:06 -04:00
share doc: Fix whitespace errs in .md files, bitcoin.conf, Info.plist.in, and find_bdb48.m4 2019-09-17 03:21:22 -04:00
src Merge #16577: util: CBufferedFile fixes and unit test 2019-09-26 13:38:39 +02:00
test Merge #16961: test: Remove python dead code linter 2019-09-25 13:14:22 -04:00
.appveyor.yml Added libbitcoin_qt and bitcoin-qt to the msbuild configuration. 2019-09-08 14:13:05 +02:00
.cirrus.yml test: Make PORT_MIN in test runner configurable 2019-09-19 12:03:40 -04:00
.gitattributes Separate protocol versioning from clientversion 2014-10-29 00:24:40 -04:00
.gitignore Merge #16371: build: ignore macOS make deploy artefacts & add them to clean-local 2019-08-21 08:02:20 +08:00
.python-version .python-version: Specify full version 3.5.6 2019-03-02 12:06:26 -05:00
.style.yapf test: .style.yapf: Set column_limit=160 2019-03-04 18:28:13 -05:00
.travis.yml travis: Disable feature_block in tsan run 2019-09-23 11:36:03 -04:00
autogen.sh Enable ShellCheck rules 2019-07-04 19:35:25 +03:00
configure.ac Merge #15146: Solve SmartOS FD_ZERO build issue 2019-09-18 16:33:23 +02:00
CONTRIBUTING.md doc: Update labels in CONTRIBUTING.md 2019-08-26 11:48:58 +03:00
COPYING [Trivial] Update license year range to 2019 2018-12-31 04:27:59 +01:00
INSTALL.md Update INSTALL landing redirection notice for build instructions. 2016-10-06 12:27:23 +13:00
libbitcoinconsensus.pc.in Unify package name to as few places as possible without major changes 2015-12-14 02:11:10 +00:00
Makefile.am build: ignore macOS make deploy artefacts & add them to clean-local 2019-08-14 08:07:12 +08:00
README.md doc: Remove travis badge from readme 2019-06-19 11:39:27 -04:00
SECURITY.md doc: Remove explicit mention of version from SECURITY.md 2019-06-14 06:39:17 -04:00

README.md

Bitcoin Core integration/staging tree

https://bitcoincore.org

What is Bitcoin?

Bitcoin is an experimental digital currency that enables instant payments to anyone, anywhere in the world. Bitcoin uses peer-to-peer technology to operate with no central authority: managing transactions and issuing money are carried out collectively by the network. Bitcoin Core is the name of open source software which enables the use of this currency.

For more information, as well as an immediately useable, binary version of the Bitcoin Core software, see https://bitcoincore.org/en/download/, or read the original whitepaper.

License

Bitcoin Core is released under the terms of the MIT license. See COPYING for more information or see https://opensource.org/licenses/MIT.

Development Process

The master branch is regularly built and tested, but is not guaranteed to be completely stable. Tags are created regularly to indicate new official, stable release versions of Bitcoin Core.

The contribution workflow is described in CONTRIBUTING.md and useful hints for developers can be found in doc/developer-notes.md.

Testing

Testing and code review is the bottleneck for development; we get more pull requests than we can review and test on short notice. Please be patient and help out by testing other people's pull requests, and remember this is a security-critical project where any mistake might cost people lots of money.

Automated Testing

Developers are strongly encouraged to write unit tests for new code, and to submit new unit tests for old code. Unit tests can be compiled and run (assuming they weren't disabled in configure) with: make check. Further details on running and extending unit tests can be found in /src/test/README.md.

There are also regression and integration tests, written in Python, that are run automatically on the build server. These tests can be run (if the test dependencies are installed) with: test/functional/test_runner.py

The Travis CI system makes sure that every pull request is built for Windows, Linux, and macOS, and that unit/sanity tests are run automatically.

Manual Quality Assurance (QA) Testing

Changes should be tested by somebody other than the developer who wrote the code. This is especially important for large or high-risk changes. It is useful to add a test plan to the pull request description if testing the changes is not straightforward.

Translations

Changes to translations as well as new translations can be submitted to Bitcoin Core's Transifex page.

Translations are periodically pulled from Transifex and merged into the git repository. See the translation process for details on how this works.

Important: We do not accept translation changes as GitHub pull requests because the next pull from Transifex would automatically overwrite them again.

Translators should also subscribe to the mailing list.