2015-09-26 18:25:54 +02:00
Developer Notes
===============
2013-05-20 06:30:00 +02:00
2018-03-22 18:34:27 +01:00
<!-- markdown - toc start -->
**Table of Contents**
- [Developer Notes ](#developer-notes )
- [Coding Style ](#coding-style )
- [Doxygen comments ](#doxygen-comments )
- [Development tips and tricks ](#development-tips-and-tricks )
- [Compiling for debugging ](#compiling-for-debugging )
- [Compiling for gprof profiling ](#compiling-for-gprof-profiling )
- [debug.log ](#debuglog )
- [Testnet and Regtest modes ](#testnet-and-regtest-modes )
- [DEBUG_LOCKORDER ](#debug_lockorder )
- [Valgrind suppressions file ](#valgrind-suppressions-file )
- [Compiling for test coverage ](#compiling-for-test-coverage )
- [Locking/mutex usage notes ](#lockingmutex-usage-notes )
- [Threads ](#threads )
- [Ignoring IDE/editor files ](#ignoring-ideeditor-files )
- [Development guidelines ](#development-guidelines )
- [General Bitcoin Core ](#general-bitcoin-core )
- [Wallet ](#wallet )
- [General C++ ](#general-c )
- [C++ data structures ](#c-data-structures )
- [Strings and formatting ](#strings-and-formatting )
- [Variable names ](#variable-names )
- [Threads and synchronization ](#threads-and-synchronization )
- [Source code organization ](#source-code-organization )
- [GUI ](#gui )
- [Subtrees ](#subtrees )
- [Git and GitHub tips ](#git-and-github-tips )
- [Scripted diffs ](#scripted-diffs )
- [RPC interface guidelines ](#rpc-interface-guidelines )
<!-- markdown - toc end -->
Coding Style
---------------
2014-07-09 18:17:23 +02:00
Various coding styles have been used during the history of the codebase,
and the result is not very consistent. However, we're now trying to converge to
2017-05-27 00:52:36 +02:00
a single style, which is specified below. When writing patches, favor the new
2017-08-16 00:24:39 +02:00
style over attempting to mimic the surrounding style, except for move-only
2017-05-27 00:52:36 +02:00
commits.
Do not submit patches solely to modify the style of existing code.
2017-01-28 16:58:29 +01:00
2017-05-27 00:52:36 +02:00
- **Indentation and whitespace rules** as specified in
[src/.clang-format ](/src/.clang-format ). You can use the provided
[clang-format-diff script ](/contrib/devtools/README.md#clang-format-diffpy )
tool to clean up patches automatically before submission.
2014-07-09 18:17:23 +02:00
- Braces on new lines for namespaces, classes, functions, methods.
- Braces on the same line for everything else.
- 4 space indentation (no tabs) for every block except namespaces.
2017-01-10 20:58:27 +01:00
- No indentation for `public` /`protected`/`private` or for `namespace` .
2014-07-09 18:17:23 +02:00
- No extra spaces inside parenthesis; don't do ( this )
2017-01-10 20:58:27 +01:00
- No space after function names; one space after `if` , `for` and `while` .
2017-05-27 00:52:36 +02:00
- If an `if` only has a single-statement `then` -clause, it can appear
on the same line as the `if` , without braces. In every other case,
braces are required, and the `then` and `else` clauses must appear
2017-01-10 20:58:27 +01:00
correctly indented on a new line.
2017-05-27 00:52:36 +02:00
- **Symbol naming conventions**. These are preferred in new code, but are not
required when doing so would need changes to significant pieces of existing
code.
- Variable and namespace names are all lowercase, and may use `_` to
2017-07-24 08:45:58 +02:00
separate words (snake_case).
2017-05-27 00:52:36 +02:00
- Class member variables have a `m_` prefix.
- Global variables have a `g_` prefix.
- Constant names are all uppercase, and use `_` to separate words.
2017-07-24 08:45:58 +02:00
- Class names, function names and method names are UpperCamelCase
(PascalCase). Do not prefix class names with `C` .
2017-05-27 00:52:36 +02:00
- **Miscellaneous**
2016-08-26 13:40:24 +02:00
- `++i` is preferred over `i++` .
2017-08-16 17:26:07 +02:00
- `nullptr` is preferred over `NULL` or `(void*)0` .
2017-08-16 21:22:56 +02:00
- `static_assert` is preferred over `assert` where possible. Generally; compile-time checking is preferred over run-time checking.
2018-03-27 17:00:03 +02:00
- `enum class` is preferred over `enum` where possible. Scoped enumerations avoid two potential pitfalls/problems with traditional C++ enumerations: implicit conversions to int, and name clashes due to enumerators being exported to the surrounding scope.
2014-06-26 11:49:51 +02:00
2014-07-09 18:17:23 +02:00
Block style example:
```c++
2017-05-27 00:52:36 +02:00
int g_count = 0;
2014-07-09 18:17:23 +02:00
namespace foo
{
class Class
{
2017-05-27 00:52:36 +02:00
std::string m_name;
public:
2017-01-10 20:58:27 +01:00
bool Function(const std::string& s, int n)
2014-07-09 18:17:23 +02:00
{
// Comment summarising what this section of code does
2016-08-26 13:40:24 +02:00
for (int i = 0; i < n ; + + i ) {
2017-05-27 00:52:36 +02:00
int total_sum = 0;
2014-07-09 18:17:23 +02:00
// When something fails, return early
2017-01-10 20:58:27 +01:00
if (!Something()) return false;
2014-07-09 18:17:23 +02:00
...
2017-05-27 00:52:36 +02:00
if (SomethingElse(i)) {
total_sum += ComputeSomething(g_count);
2017-01-10 20:58:27 +01:00
} else {
2017-05-27 00:52:36 +02:00
DoSomething(m_name, total_sum);
2017-01-10 20:58:27 +01:00
}
2014-07-09 18:17:23 +02:00
}
2013-05-20 06:30:00 +02:00
2014-07-09 18:17:23 +02:00
// Success return is usually at the end
return true;
}
}
2017-05-31 22:21:25 +02:00
} // namespace foo
2014-06-26 11:49:51 +02:00
```
2013-05-20 06:30:00 +02:00
2014-04-07 08:39:31 +02:00
Doxygen comments
-----------------
To facilitate the generation of documentation, use doxygen-compatible comment blocks for functions, methods and fields.
For example, to describe a function use:
```c++
/**
* ... text ...
* @param [in] arg1 A description
* @param [in] arg2 Another argument description
* @pre Precondition for function...
*/
bool function(int arg1, const char *arg2)
```
A complete list of `@xxx` commands can be found at http://www.stack.nl/~dimitri/doxygen/manual/commands.html.
As Doxygen recognizes the comments by the delimiters (`/**` and `*/` in this case), you don't
2015-04-28 16:48:28 +02:00
*need* to provide any commands for a comment to be valid; just a description text is fine.
2014-04-07 08:39:31 +02:00
To describe a class use the same construct above the class definition:
```c++
2015-10-17 12:10:45 +02:00
/**
2014-04-07 08:39:31 +02:00
* Alerts are for notifying old versions if they become too obsolete and
* need to upgrade. The message is displayed in the status bar.
* @see GetWarnings()
*/
class CAlert
{
```
To describe a member or variable use:
```c++
int var; //!< Detailed description after the member
```
2016-04-03 11:49:36 +02:00
or
```cpp
//! Description before the member
int var;
```
2014-04-07 08:39:31 +02:00
Also OK:
```c++
///
/// ... text ...
///
bool function2(int arg1, const char *arg2)
```
Not OK (used plenty in the current source, but not picked up):
```c++
//
// ... text ...
//
```
A full list of comment syntaxes picked up by doxygen can be found at http://www.stack.nl/~dimitri/doxygen/manual/docblocks.html,
but if possible use one of the above styles.
2017-12-26 18:12:04 +01:00
Documentation can be generated with `make docs` and cleaned up with `make clean-docs` .
2014-12-13 05:35:39 +01:00
Development tips and tricks
---------------------------
2018-03-22 18:34:27 +01:00
### Compiling for debugging
2014-12-13 05:35:39 +01:00
2018-03-22 18:34:27 +01:00
Run configure with `--enable-debug` to add additional compiler flags that
produce better debugging builds.
2014-12-13 05:35:39 +01:00
2018-03-22 18:34:27 +01:00
### Compiling for gprof profiling
2018-02-07 05:58:41 +01:00
2018-03-22 18:34:27 +01:00
Run configure with the `--enable-gprof` option, then make.
2018-02-07 05:58:41 +01:00
2018-03-22 18:34:27 +01:00
### debug.log
2014-12-13 05:35:39 +01:00
If the code is behaving strangely, take a look in the debug.log file in the data directory;
error and debugging messages are written there.
2018-03-22 18:34:27 +01:00
The `-debug=...` command-line option controls debugging; running with just `-debug` or `-debug=1` will turn
2014-12-13 05:35:39 +01:00
on all categories (and give you a very large debug.log file).
2018-03-22 18:34:27 +01:00
The Qt code routes `qDebug()` output to debug.log under category "qt": run with `-debug=qt`
2014-12-13 05:35:39 +01:00
to see it.
2018-03-22 18:34:27 +01:00
### Testnet and Regtest modes
2014-12-13 05:35:39 +01:00
2018-03-22 18:34:27 +01:00
Run with the `-testnet` option to run with "play bitcoins" on the test network, if you
2014-12-13 05:35:39 +01:00
are testing multi-machine code that needs to operate across the internet.
2018-03-22 18:34:27 +01:00
If you are testing something that can run on one machine, run with the `-regtest` option.
In regression test mode, blocks can be created on-demand; see [test/functional/ ](/test/functional ) for tests
that run in `-regtest` mode.
2014-12-13 05:35:39 +01:00
2018-03-22 18:34:27 +01:00
### DEBUG_LOCKORDER
2014-12-13 05:35:39 +01:00
2018-03-22 18:34:27 +01:00
Bitcoin Core is a multi-threaded application, and deadlocks or other
multi-threading bugs can be very difficult to track down. The `--enable-debug`
configure option adds `-DDEBUG_LOCKORDER` to the compiler flags. This inserts
run-time checks to keep track of which locks are held, and adds warnings to the
debug.log file if inconsistencies are detected.
2014-12-13 05:35:39 +01:00
2018-03-22 18:34:27 +01:00
### Valgrind suppressions file
2017-11-12 17:07:35 +01:00
Valgrind is a programming tool for memory debugging, memory leak detection, and
profiling. The repo contains a Valgrind suppressions file
([`valgrind.supp`](https://github.com/bitcoin/bitcoin/blob/master/contrib/valgrind.supp))
which includes known Valgrind warnings in our dependencies that cannot be fixed
in-tree. Example use:
```shell
$ valgrind --suppressions=contrib/valgrind.supp src/test/test_bitcoin
$ valgrind --suppressions=contrib/valgrind.supp --leak-check=full \
--show-leak-kinds=all src/test/test_bitcoin --log_level=test_suite
$ valgrind -v --leak-check=full src/bitcoind -printtoconsole
```
2018-03-22 18:34:27 +01:00
### Compiling for test coverage
2017-11-14 08:06:48 +01:00
LCOV can be used to generate a test coverage report based upon `make check`
execution. LCOV must be installed on your system (e.g. the `lcov` package
on Debian/Ubuntu).
To enable LCOV report generation during test runs:
```shell
./configure --enable-lcov
make
make cov
# A coverage report will now be accessible at `./test_bitcoin.coverage/index.html`.
```
2013-05-20 06:30:00 +02:00
Locking/mutex usage notes
2014-04-07 08:39:31 +02:00
-------------------------
2013-05-20 06:30:00 +02:00
The code is multi-threaded, and uses mutexes and the
2018-03-22 18:34:27 +01:00
`LOCK` and `TRY_LOCK` macros to protect data structures.
2013-05-20 06:30:00 +02:00
2018-03-22 18:34:27 +01:00
Deadlocks due to inconsistent lock ordering (thread 1 locks `cs_main` and then
`cs_wallet` , while thread 2 locks them in the opposite order: result, deadlock
as each waits for the other to release its lock) are a problem. Compile with
`-DDEBUG_LOCKORDER` (or use `--enable-debug` ) to get lock order inconsistencies
reported in the debug.log file.
2013-05-20 06:30:00 +02:00
Re-architecting the core code so there are better-defined interfaces
between the various components is a goal, with any necessary locking
2018-03-22 18:34:27 +01:00
done by the components (e.g. see the self-contained `CBasicKeyStore` class
and its `cs_KeyStore` lock for example).
2013-05-20 06:30:00 +02:00
Threads
2014-04-07 08:39:31 +02:00
-------
2013-05-20 06:30:00 +02:00
2014-03-05 21:23:39 +01:00
- ThreadScriptCheck : Verifies block scripts.
- ThreadImport : Loads blocks from blk*.dat files or bootstrap.dat.
2013-05-20 06:30:00 +02:00
- StartNode : Starts other threads.
2014-03-05 21:23:39 +01:00
- ThreadDNSAddressSeed : Loads addresses of peers from the DNS.
- ThreadMapPort : Universal plug-and-play startup/shutdown
2013-05-20 06:30:00 +02:00
- ThreadSocketHandler : Sends/Receives data from peers on port 8333.
2014-03-05 21:23:39 +01:00
- ThreadOpenAddedConnections : Opens network connections to added nodes.
2013-05-20 06:30:00 +02:00
- ThreadOpenConnections : Initiates new connections to peers.
2014-03-05 21:23:39 +01:00
- ThreadMessageHandler : Higher-level message handling (sending and receiving).
- DumpAddresses : Dumps IP addresses of nodes to peers.dat.
2013-05-20 06:30:00 +02:00
- ThreadRPCServer : Remote procedure call handler, listens on port 8332 for connections and services them.
2014-03-05 21:23:39 +01:00
- Shutdown : Does an orderly shutdown of everything.
2015-10-23 12:32:40 +02:00
Ignoring IDE/editor files
--------------------------
In closed-source environments in which everyone uses the same IDE it is common
to add temporary files it produces to the project-wide `.gitignore` file.
However, in open source software such as Bitcoin Core, where everyone uses
their own editors/IDE/tools, it is less common. Only you know what files your
editor produces and this may change from version to version. The canonical way
to do this is thus to create your local gitignore. Add this to `~/.gitconfig` :
```
[core]
excludesfile = /home/.../.gitignore_global
```
(alternatively, type the command `git config --global core.excludesfile ~/.gitignore_global`
on a terminal)
Then put your favourite tool's temporary filenames in that file, e.g.
```
# NetBeans
nbproject/
```
Another option is to create a per-repository excludes file `.git/info/exclude` .
These are not committed but apply only to one repository.
If a set of tools is used by the build system or scripts the repository (for
example, lcov) it is perfectly acceptable to add its files to `.gitignore`
and commit them.
2015-11-13 11:49:12 +01:00
Development guidelines
============================
A few non-style-related recommendations for developers, as well as points to
pay attention to for reviewers of Bitcoin Core code.
General Bitcoin Core
----------------------
- New features should be exposed on RPC first, then can be made available in the GUI
- *Rationale*: RPC allows for better automatic testing. The test suite for
the GUI is very limited
2015-11-23 21:11:53 +01:00
- Make sure pull requests pass Travis CI before merging
2015-11-13 11:49:12 +01:00
- *Rationale*: Makes sure that they pass thorough testing, and that the tester will keep passing
on the master branch. Otherwise all new pull requests will start failing the tests, resulting in
confusion and mayhem
2016-08-26 13:40:24 +02:00
2015-11-13 11:49:12 +01:00
- *Explanation*: If the test suite is to be updated for a change, this has to
2016-08-26 13:40:24 +02:00
be done first
2015-11-13 11:49:12 +01:00
Wallet
-------
2015-11-23 21:11:53 +01:00
- Make sure that no crashes happen with run-time option `-disablewallet` .
2015-11-13 11:49:12 +01:00
2015-11-23 21:11:53 +01:00
- *Rationale*: In RPC code that conditionally uses the wallet (such as
2015-11-13 11:49:12 +01:00
`validateaddress` ) it is easy to forget that global pointer `pwalletMain`
2017-08-16 17:26:07 +02:00
can be nullptr. See `test/functional/disablewallet.py` for functional tests
2015-11-13 11:49:12 +01:00
exercising the API with `-disablewallet`
- Include `db_cxx.h` (BerkeleyDB header) only when `ENABLE_WALLET` is set
- *Rationale*: Otherwise compilation of the disable-wallet build will fail in environments without BerkeleyDB
General C++
-------------
- Assertions should not have side-effects
2017-06-20 00:57:31 +02:00
- *Rationale*: Even though the source code is set to refuse to compile
2015-11-13 11:49:12 +01:00
with assertions disabled, having side-effects in assertions is unexpected and
makes the code harder to understand
2015-11-23 21:11:53 +01:00
- If you use the `.h` , you must link the `.cpp`
2015-11-13 11:49:12 +01:00
2015-11-23 21:11:53 +01:00
- *Rationale*: Include files define the interface for the code in implementation files. Including one but
2015-11-13 11:49:12 +01:00
not linking the other is confusing. Please avoid that. Moving functions from
the `.h` to the `.cpp` should not result in build errors
- Use the RAII (Resource Acquisition Is Initialization) paradigm where possible. For example by using
2016-06-08 22:26:18 +02:00
`unique_ptr` for allocations in a function.
2015-11-13 11:49:12 +01:00
- *Rationale*: This avoids memory and resource leaks, and ensures exception safety
C++ data structures
--------------------
2015-11-23 21:11:53 +01:00
- Never use the `std::map []` syntax when reading from a map, but instead use `.find()`
2015-11-13 11:49:12 +01:00
2015-11-23 21:11:53 +01:00
- *Rationale*: `[]` does an insert (of the default element) if the item doesn't
2015-11-13 11:49:12 +01:00
exist in the map yet. This has resulted in memory leaks in the past, as well as
2015-11-23 21:11:53 +01:00
race conditions (expecting read-read behavior). Using `[]` is fine for *writing* to a map
2015-11-13 11:49:12 +01:00
- Do not compare an iterator from one data structure with an iterator of
another data structure (even if of the same type)
- *Rationale*: Behavior is undefined. In C++ parlor this means "may reformat
the universe", in practice this has resulted in at least one hard-to-debug crash bug
2016-06-08 22:26:18 +02:00
- Watch out for out-of-bounds vector access. `&vch[vch.size()]` is illegal,
including `&vch[0]` for an empty vector. Use `vch.data()` and `vch.data() +
vch.size()` instead.
2015-11-13 11:49:12 +01:00
- Vector bounds checking is only enabled in debug mode. Do not rely on it
- Make sure that constructors initialize all fields. If this is skipped for a
good reason (i.e., optimization on the critical path), add an explicit
comment about this
- *Rationale*: Ensure determinism by avoiding accidental use of uninitialized
values. Also, static analyzers balk about this.
2017-08-22 22:50:03 +02:00
- By default, declare single-argument constructors `explicit` .
- *Rationale*: This is a precaution to avoid unintended conversions that might
arise when single-argument constructors are used as implicit conversion
functions.
2015-11-13 11:49:12 +01:00
- Use explicitly signed or unsigned `char` s, or even better `uint8_t` and
`int8_t` . Do not use bare `char` unless it is to pass to a third-party API.
This type can be signed or unsigned depending on the architecture, which can
lead to interoperability problems or dangerous conditions such as
out-of-bounds array accesses
- Prefer explicit constructions over implicit ones that rely on 'magical' C++ behavior
- *Rationale*: Easier to understand what is happening, thus easier to spot mistakes, even for those
that are not language lawyers
2018-02-14 22:30:55 +01:00
- Initialize all non-static class members where they are defined
- *Rationale*: Initializing the members in the declaration makes it easy to spot uninitialized ones,
and avoids accidentally reading uninitialized memory
```cpp
class A
{
uint32_t m_count{0};
}
```
2015-11-13 11:49:12 +01:00
Strings and formatting
------------------------
2015-11-23 21:11:53 +01:00
- Be careful of `LogPrint` versus `LogPrintf` . `LogPrint` takes a `category` argument, `LogPrintf` does not.
2015-11-13 11:49:12 +01:00
- *Rationale*: Confusion of these can result in runtime exceptions due to
formatting mismatch, and it is easy to get wrong because of subtly similar naming
2015-11-23 21:11:53 +01:00
- Use `std::string` , avoid C string manipulation functions
2015-11-13 11:49:12 +01:00
- *Rationale*: C++ string handling is marginally safer, less scope for
2015-11-23 21:11:53 +01:00
buffer overflows and surprises with `\0` characters. Also some C string manipulations
2015-11-13 11:49:12 +01:00
tend to act differently depending on platform, or even the user locale
2016-06-08 10:23:25 +02:00
- Use `ParseInt32` , `ParseInt64` , `ParseUInt32` , `ParseUInt64` , `ParseDouble` from `utilstrencodings.h` for number parsing
2015-11-13 11:49:12 +01:00
- *Rationale*: These functions do overflow checking, and avoid pesky locale issues
- For `strprintf` , `LogPrint` , `LogPrintf` formatting characters don't need size specifiers
- *Rationale*: Bitcoin Core uses tinyformat, which is type safe. Leave them out to avoid confusion
2016-09-22 16:39:25 +02:00
Variable names
--------------
2017-05-10 19:48:24 +02:00
Although the shadowing warning (`-Wshadow`) is not enabled by default (it prevents issues rising
from using a different variable with the same name),
please name variables so that their names do not shadow variables defined in the source code.
2016-09-22 16:39:25 +02:00
E.g. in member initializers, prepend `_` to the argument name shadowing the
member name:
```c++
class AddressBookPage
{
2018-02-14 22:30:55 +01:00
Mode m_mode;
2016-09-22 16:39:25 +02:00
}
AddressBookPage::AddressBookPage(Mode _mode) :
2018-02-14 22:30:55 +01:00
m_mode(_mode)
2016-09-22 16:39:25 +02:00
...
```
When using nested cycles, do not name the inner cycle variable the same as in
upper cycle etc.
2015-11-13 11:49:12 +01:00
Threads and synchronization
----------------------------
- Build and run tests with `-DDEBUG_LOCKORDER` to verify that no potential
deadlocks are introduced. As of 0.12, this is defined by default when
configuring with `--enable-debug`
- When using `LOCK` /`TRY_LOCK` be aware that the lock exists in the context of
the current scope, so surround the statement and the code that needs the lock
with braces
OK:
```c++
{
TRY_LOCK(cs_vNodes, lockNodes);
...
}
```
Wrong:
```c++
TRY_LOCK(cs_vNodes, lockNodes);
{
...
}
```
Source code organization
--------------------------
- Implementation code should go into the `.cpp` file and not the `.h` , unless necessary due to template usage or
when performance due to inlining is critical
- *Rationale*: Shorter and simpler header files are easier to read, and reduce compile time
2017-06-12 00:53:33 +02:00
- Every `.cpp` and `.h` file should `#include` every header file it directly uses classes, functions or other
definitions from, even if those headers are already included indirectly through other headers. One exception
is that a `.cpp` file does not need to re-include the includes already included in its corresponding `.h` file.
- *Rationale*: Excluding headers because they are already indirectly included results in compilation
failures when those indirect dependencies change. Furthermore, it obscures what the real code
dependencies are.
2015-11-13 11:49:12 +01:00
- Don't import anything into the global namespace (`using namespace ...`). Use
fully specified types such as `std::string` .
- *Rationale*: Avoids symbol conflicts
2017-05-31 22:21:25 +02:00
- Terminate namespaces with a comment (`// namespace mynamespace`). The comment
should be placed on the same line as the brace closing the namespace, e.g.
```c++
namespace mynamespace {
...
} // namespace mynamespace
namespace {
...
} // namespace
```
- *Rationale*: Avoids confusion about the namespace context
2017-08-15 19:45:16 +02:00
- Prefer `#include <primitives/transaction.h>` bracket syntax instead of
2017-12-16 13:29:26 +01:00
`#include "primitives/transactions.h"` quote syntax when possible.
2017-08-15 19:45:16 +02:00
- *Rationale*: Bracket syntax is less ambiguous because the preprocessor
searches a fixed list of include directories without taking location of the
source file into account. This allows quoted includes to stand out more when
the location of the source file actually is relevant.
2015-11-13 11:49:12 +01:00
GUI
-----
- Do not display or manipulate dialogs in model code (classes `*Model` )
- *Rationale*: Model classes pass through events and data from the core, they
should not interact with the user. That's where View classes come in. The converse also
holds: try to not directly access core data structures from Views.
2016-06-08 18:07:45 +02:00
2016-11-30 09:45:01 +01:00
Subtrees
----------
Several parts of the repository are subtrees of software maintained elsewhere.
Some of these are maintained by active developers of Bitcoin Core, in which case changes should probably go
directly upstream without being PRed directly against the project. They will be merged back in the next
subtree merge.
Others are external projects without a tight relationship with our project. Changes to these should also
be sent upstream but bugfixes may also be prudent to PR against Bitcoin Core so that they can be integrated
quickly. Cosmetic changes should be purely taken upstream.
There is a tool in contrib/devtools/git-subtree-check.sh to check a subtree directory for consistency with
its upstream repository.
Current subtrees include:
- src/leveldb
2018-02-20 23:25:24 +01:00
- Upstream at https://github.com/google/leveldb ; Maintained by Google, but
open important PRs to Core to avoid delay.
- **Note**: Follow the instructions in [Upgrading LevelDB ](#upgrading-leveldb ) when
merging upstream changes to the leveldb subtree.
2016-11-30 09:45:01 +01:00
- src/libsecp256k1
- Upstream at https://github.com/bitcoin-core/secp256k1/ ; actively maintaned by Core contributors.
- src/crypto/ctaes
- Upstream at https://github.com/bitcoin-core/ctaes ; actively maintained by Core contributors.
- src/univalue
- Upstream at https://github.com/jgarzik/univalue ; report important PRs to Core to avoid delay.
2018-02-20 23:25:24 +01:00
Upgrading LevelDB
---------------------
Extra care must be taken when upgrading LevelDB. This section explains issues
you must be aware of.
### File Descriptor Counts
In most configurations we use the default LevelDB value for `max_open_files` ,
which is 1000 at the time of this writing. If LevelDB actually uses this many
file descriptors it will cause problems with Bitcoin's `select()` loop, because
it may cause new sockets to be created where the fd value is >= 1024. For this
reason, on 64-bit Unix systems we rely on an internal LevelDB optimization that
uses `mmap()` + `close()` to open table files without actually retaining
references to the table file descriptors. If you are upgrading LevelDB, you must
sanity check the changes to make sure that this assumption remains valid.
In addition to reviewing the upstream changes in `env_posix.cc` , you can use `lsof` to
check this. For example, on Linux this command will show open `.ldb` file counts:
```bash
$ lsof -p $(pidof bitcoind) |\
awk 'BEGIN { fd=0; mem=0; } /ldb$/ { if ($4 == "mem") mem++; else fd++ } END { printf "mem = %s, fd = %s\n", mem, fd}'
mem = 119, fd = 0
```
The `mem` value shows how many files are mmap'ed, and the `fd` value shows you
many file descriptors these files are using. You should check that `fd` is a
small number (usually 0 on 64-bit hosts).
See the notes in the `SetMaxOpenFiles()` function in `dbwrapper.cc` for more
details.
### Consensus Compatibility
It is possible for LevelDB changes to inadvertently change consensus
compatibility between nodes. This happened in Bitcoin 0.8 (when LevelDB was
first introduced). When upgrading LevelDB you should review the upstream changes
to check for issues affecting consensus compatibility.
For example, if LevelDB had a bug that accidentally prevented a key from being
returned in an edge case, and that bug was fixed upstream, the bug "fix" would
be an incompatible consensus change. In this situation the correct behavior
would be to revert the upstream fix before applying the updates to Bitcoin's
copy of LevelDB. In general you should be wary of any upstream changes affecting
what data is returned from LevelDB queries.
2016-11-30 09:45:01 +01:00
2017-02-03 09:12:21 +01:00
Git and GitHub tips
2016-06-08 18:07:45 +02:00
---------------------
- For resolving merge/rebase conflicts, it can be useful to enable diff3 style using
`git config merge.conflictstyle diff3` . Instead of
< < <
yours
===
theirs
>>>
you will see
< < <
yours
|||
original
===
theirs
>>>
This may make it much clearer what caused the conflict. In this style, you can often just look
at what changed between *original* and *theirs* , and mechanically apply that to *yours* (or the other way around).
- When reviewing patches which change indentation in C++ files, use `git diff -w` and `git show -w` . This makes
the diff algorithm ignore whitespace changes. This feature is also available on github.com, by adding `?w=1`
at the end of any URL which shows a diff.
- When reviewing patches that change symbol names in many places, use `git diff --word-diff` . This will instead
of showing the patch as deleted/added *lines* , show deleted/added *words* .
- When reviewing patches that move code around, try using
`git diff --patience commit~:old/file.cpp commit:new/file/name.cpp` , and ignoring everything except the
moved body of code which should show up as neither `+` or `-` lines. In case it was not a pure move, this may
even work when combined with the `-w` or `--word-diff` options described above.
- When looking at other's pull requests, it may make sense to add the following section to your `.git/config`
file:
[remote "upstream-pull"]
fetch = +refs/pull/*:refs/remotes/upstream-pull/*
url = git@github.com:bitcoin/bitcoin.git
This will add an `upstream-pull` remote to your git repository, which can be fetched using `git fetch --all`
or `git fetch upstream-pull` . Afterwards, you can use `upstream-pull/NUMBER/head` in arguments to `git show` ,
`git checkout` and anywhere a commit id would be acceptable to see the changes from pull request NUMBER.
2017-04-26 07:10:44 +02:00
2017-09-22 22:10:25 +02:00
Scripted diffs
--------------
For reformatting and refactoring commits where the changes can be easily automated using a bash script, we use
scripted-diff commits. The bash script is included in the commit message and our Travis CI job checks that
the result of the script is identical to the commit. This aids reviewers since they can verify that the script
does exactly what it's supposed to do. It is also helpful for rebasing (since the same script can just be re-run
on the new master commit).
To create a scripted-diff:
- start the commit message with `scripted-diff:` (and then a description of the diff on the same line)
- in the commit message include the bash script between lines containing just the following text:
- `-BEGIN VERIFY SCRIPT-`
- `-END VERIFY SCRIPT-`
The scripted-diff is verified by the tool `contrib/devtools/commit-script-check.sh`
2018-01-24 17:07:25 +01:00
Commit [`bb81e173` ](https://github.com/bitcoin/bitcoin/commit/bb81e173 ) is an example of a scripted-diff.
2017-09-22 22:10:25 +02:00
2017-04-26 07:10:44 +02:00
RPC interface guidelines
--------------------------
A few guidelines for introducing and reviewing new RPC interfaces:
- Method naming: use consecutive lower-case names such as `getrawtransaction` and `submitblock`
- *Rationale*: Consistency with existing interface.
- Argument naming: use snake case `fee_delta` (and not, e.g. camel case `feeDelta` )
- *Rationale*: Consistency with existing interface.
- Use the JSON parser for parsing, don't manually parse integers or strings from
arguments unless absolutely necessary.
- *Rationale*: Introduces hand-rolled string manipulation code at both the caller and callee sites,
which is error prone, and it is easy to get things such as escaping wrong.
JSON already supports nested data structures, no need to re-invent the wheel.
2017-08-07 14:18:47 +02:00
- *Exception*: AmountFromValue can parse amounts as string. This was introduced because many JSON
2017-04-26 07:10:44 +02:00
parsers and formatters hard-code handling decimal numbers as floating point
values, resulting in potential loss of precision. This is unacceptable for
2017-08-07 14:18:47 +02:00
monetary values. **Always** use `AmountFromValue` and `ValueFromAmount` when
2017-04-26 07:10:44 +02:00
inputting or outputting monetary values. The only exceptions to this are
`prioritisetransaction` and `getblocktemplate` because their interface
is specified as-is in BIP22.
- Missing arguments and 'null' should be treated the same: as default values. If there is no
2017-08-15 05:32:38 +02:00
default value, both cases should fail in the same way. The easiest way to follow this
guideline is detect unspecified arguments with `params[x].isNull()` instead of
`params.size() <= x` . The former returns true if the argument is either null or missing,
while the latter returns true if is missing, and false if it is null.
2017-04-26 07:10:44 +02:00
- *Rationale*: Avoids surprises when switching to name-based arguments. Missing name-based arguments
are passed as 'null'.
- Try not to overload methods on argument type. E.g. don't make `getblock(true)` and `getblock("hash")`
do different things.
- *Rationale*: This is impossible to use with `bitcoin-cli` , and can be surprising to users.
- *Exception*: Some RPC calls can take both an `int` and `bool` , most notably when a bool was switched
to a multi-value, or due to other historical reasons. **Always** have false map to 0 and
true to 1 in this case.
- Don't forget to fill in the argument names correctly in the RPC command table.
- *Rationale*: If not, the call can not be used with name-based arguments.
- Set okSafeMode in the RPC command table to a sensible value: safe mode is when the
blockchain is regarded to be in a confused state, and the client deems it unsafe to
do anything irreversible such as send. Anything that just queries should be permitted.
- *Rationale*: Troubleshooting a node in safe mode is difficult if half the
RPCs don't work.
- Add every non-string RPC argument `(method, idx, name)` to the table `vRPCConvertParams` in `rpc/client.cpp` .
- *Rationale*: `bitcoin-cli` and the GUI debug console use this table to determine how to
convert a plaintext command line to JSON. If the types don't match, the method can be unusable
from there.
- A RPC method must either be a wallet method or a non-wallet method. Do not
2017-07-17 01:29:08 +02:00
introduce new methods such as `signrawtransaction` that differ in behavior
based on presence of a wallet.
2017-04-26 07:10:44 +02:00
- *Rationale*: as well as complicating the implementation and interfering
with the introduction of multi-wallet, wallet and non-wallet code should be
separated to avoid introducing circular dependencies between code units.
2017-08-25 16:52:35 +02:00
- Try to make the RPC response a JSON object.
- *Rationale*: If a RPC response is not a JSON object then it is harder to avoid API breakage if
new data in the response is needed.
2017-05-03 21:57:56 +02:00
- Wallet RPCs call BlockUntilSyncedToCurrentChain to maintain consistency with
`getblockchaininfo` 's state immediately prior to the call's execution. Wallet
RPCs whose behavior does *not* depend on the current chainstate may omit this
call.
- *Rationale*: In previous versions of Bitcoin Core, the wallet was always
in-sync with the chainstate (by virtue of them all being updated in the
same cs_main lock). In order to maintain the behavior that wallet RPCs
return results as of at least the highest best-known block an RPC
client may be aware of prior to entering a wallet RPC call, we must block
until the wallet is caught up to the chainstate as of the RPC call's entry.
This also makes the API much easier for RPC clients to reason about.
2018-03-15 21:37:57 +01:00
- Be aware of RPC method aliases and generally avoid registering the same
callback function pointer for different RPCs.
- *Rationale*: RPC methods registered with the same function pointer will be
considered aliases and only the first method name will show up in the
`help` rpc command list.
- *Exception*: Using RPC method aliases may be appropriate in cases where a
new RPC is replacing a deprecated RPC, to avoid both RPCs confusingly
showing up in the command list.