This fixes one of the last major layer violations in the networking stack.
The network side is no longer in charge of message serialization, so it is now
decoupled from Bitcoin structures. Only the header is serialized and attached
to the payload.
The changes here are dense and subtle, but hopefully all is more explicit
than before.
- CConnman is now in charge of sending data rather than the nodes themselves.
This is necessary because many decisions need to be made with all nodes in
mind, and a model that requires the nodes calling up to their manager quickly
turns to spaghetti.
- The per-node-serializer (ssSend) has been replaced with a (quasi-)const
send-version. Since the send version for serialization can only change once
per connection, we now explicitly tag messages with INIT_PROTO_VERSION if
they are sent before the handshake. With this done, there's no need to lock
for access to nSendVersion.
Also, a new stream is used for each message, so there's no need to lock
during the serialization process.
- This takes care of accounting for optimistic sends, so the
nOptimisticBytesWritten hack can be removed.
- -dropmessagestest and -fuzzmessagestest have not been preserved, as I suspect
they haven't been used in years.
nMaxInbound might very well be 0 or -1, if the user prefers to keep
a small number of maxconnections.
Note: nMaxInbound of -1 means that the user set maxconnections
to 8 or less, but we still want to keep an additional slot for
the feeler connection.
Add getNetworkActive()/setNetworkActive() method to client model.
Send network active status through NotifyNetworkActiveChanged.
Indicate in tool tip of gui status bar network indicator whether network activity is disabled.
Indicate in debug window whether network activity is disabled and add button to allow user to toggle network activity state.
Added the function SetNetworkActive() which when called with argument set to false disconnects all nodes and sets the flag fNetworkActive to false. As long as this flag is false no new connections are attempted and no incoming connections are accepted. Network activity is reenabled by calling the function with argument true.
4630479 Make dnsseed's definition of acute need include relevant services. (Gregory Maxwell)
9583477 Be more aggressive in connecting to peers with relevant services. (Gregory Maxwell)
We normally prefer to connect to peers offering the relevant services.
If we're not connected to enough peers with relevant services, we
probably don't know about them and could use dnsseed's help.
Only allow skipping relevant services until there are four outbound
connections up.
This avoids quickly filling up with peers lacking the relevant
services when addrman has few or none of them.
There are only a few uses of `insecure_random` outside the tests.
This PR replaces uses of insecure_random (and its accompanying global
state) in the core code with an FastRandomContext that is automatically
seeded on creation.
This is meant to be used for inner loops. The FastRandomContext
can be in the outer scope, or the class itself, then rand32() is used
inside the loop. Useful e.g. for pushing addresses in CNode or the fee
rounding, or randomization for coin selection.
As a context is created per purpose, thus it gets rid of
cross-thread unprotected shared usage of a single set of globals, this
should also get rid of the potential race conditions.
- I'd say TxMempool::check is not called enough to warrant using a special
fast random context, this is switched to GetRand() (open for
discussion...)
- The use of `insecure_rand` in ConnectThroughProxy has been replaced by
an atomic integer counter. The only goal here is to have a different
credentials pair for each connection to go on a different Tor circuit,
it does not need to be random nor unpredictable.
- To avoid having a FastRandomContext on every CNode, the context is
passed into PushAddress as appropriate.
There remains an insecure_random for test usage in `test_random.h`.
In principle, the checksums of P2P packets are simply 4-byte blobs which
are the first four bytes of SHA256(SHA256(payload)).
Currently they are handled as little-endian 32-bit integers half of the
time, as blobs the other half, sometimes copying the one to the other,
resulting in somewhat confused code.
This PR changes the handling to be consistent both at packet creation
and receiving, making it (I think) easier to understand.
After #8594 the addrFrom sent by a node is not used anymore at all,
so don't bother sending it.
Also mitigates the privacy issue in (#8616). It doesn't completely solve
the issue as GetLocalAddress is also called in AdvertiseLocal, but at
least when advertising addresses it stands out less as *our* address.
This was broken by 63cafa6329.
Note that while this fixes the settings, it doesn't fix the actual usage of
-maxuploadtarget completely, as there is currently a bug in the
nOptimisticBytesWritten accounting that causes a delayed response if the target
is reached. That bug will be addressed separately.
In the case of (for example) an already-running bitcoind, the shutdown sequence
begins before CConnman has been created, leading to a null-pointer dereference
when g_connman->Stop() is called.
Instead, Just let the CConnman dtor take care of stopping.
CConnman then passes the current best height into CNode at creation time.
This way CConnman/CNode have no dependency on main for height, and the signals
only move in one direction.
This also helps to prevent identity leakage a tiny bit. Before this change, an
attacker could theoretically make 2 connections on different interfaces. They
would connect fully on one, and only establish the initial connection on the
other. Once they receive a new block, they would relay it to your first
connection, and immediately commence the version handshake on the second. Since
the new block height is reflected immediately, they could attempt to learn
whether the two connections were correlated.
This is, of course, incredibly unlikely to work due to the small timings
involved and receipt from other senders. But it doesn't hurt to lock-in
nBestHeight at the time of connection, rather than letting the remote choose
the time.
This behavior seems to have been quite racy and broken.
Move nLocalHostNonce into CNode, and check received nonces against all
non-fully-connected nodes. If there's a match, assume we've connected
to ourself.
Tests if addresses are online or offline by briefly connecting to them. These short lived connections are referred to as feeler connections. Feeler connections are designed to increase the number of fresh online addresses in tried by selecting and connecting to addresses in new. One feeler connection is attempted on average once every two minutes.
This change was suggested as Countermeasure 4 in
Eclipse Attacks on Bitcoin’s Peer-to-Peer Network, Ethan Heilman,
Alison Kendler, Aviv Zohar, Sharon Goldberg. ePrint Archive Report
2015/263. March 2015.
9e9d644 net: fixup nits (Cory Fields)
8945384 net: Have LookupNumeric return a CService directly (Cory Fields)
21ba407 net: narrow include scope after moving to netaddress (Cory Fields)
21e5b96 net: move CNetAddr/CService/CSubNet out of netbase (Cory Fields)
1017b8a net: Add direct tests for new CSubNet constructors (Cory Fields)
b6c3ff3 net: Split resolving out of CSubNet (Cory Fields)
f96c7c4 net: Split resolving out of CService (Cory Fields)
31d6b1d net: Split resolving out of CNetAddr (Cory Fields)
9d4eb9a Do diskspace check before import thread is started (Pieter Wuille)
aa59f2e Add extra message to avoid a long 'Loading banlist' (Pieter Wuille)
0fd2a33 Use a signal to continue init after genesis activation (Pieter Wuille)
1a5a4e6 Randomize name lookup result in ConnectSocketByName (Pieter Wuille)
f9f5cfc Prevent duplicate connections where one is by name and another by ip (Pieter Wuille)
1111b80 Rework addnode behaviour (Pieter Wuille)
6ee7f05 Allow disconnecting a netgroup with only one member in eviction. (Gregory Maxwell)
5d0ca81 Add recently accepted blocks and txn to AttemptToEvictConnection. (Gregory Maxwell)
* Use CNode::addeName to track whether a connection to a name is already open
* A new connection to a previously-connected by-name addednode is only opened when
the previous one closes (even if the name starts resolving to something else)
* At most one connection is opened per addednode (even if the name resolves to multiple)
* Unify the code between ThreadOpenAddedNodeConnections and getaddednodeinfo
* Information about open connections is always returned, and the dns argument becomes a dummy
* An IP address and inbound/outbound is only reported for the (at most 1) open connection
eebc232 test: Add more test vectors for siphash (Wladimir J. van der Laan)
8884830 Use C++11 thread-safe static initializers (Pieter Wuille)
c31b24f Use 64-bit SipHash of netgroups in eviction (Pieter Wuille)
9bf156b Support SipHash with arbitrary byte writes (Pieter Wuille)
053930f Avoid recalculating vchKeyedNetGroup in eviction logic. (Patrick Strateman)
6182d10 Do not increment nAttempts by more than one for every Good connection. (Gregory Maxwell)
c769c4a Avoid counting failed connect attempts when probably offline. (Gregory Maxwell)
This reduces the rate of not founds by better matching the far
end expectations, it also improves privacy by removing the
ability to use getdata to probe for a node having a txn before
it has been relayed.
If a node is offline failed outbound connection attempts will crank up
the addrman counter and effectively blow away our state.
This change reduces the problem by only counting attempts made while
the node believes it has outbound connections to at least two
netgroups.
Connect and addnode connections are also not counted, as there is no
reason to unequally penalize them for their more frequent
connections -- though there should be no real effect from this
unless their addnode configureation is later removed.
Wasteful repeated connection attempts while only a few connections are
up are avoided via nLastTry.
This is still somewhat incomplete protection because our outbound
peers could be down but not timed out or might all be on 'local'
networks (although the requirement for multiple netgroups helps).
The ability to GETDATA a transaction which has not (yet) been relayed
is a privacy loss vector.
The use of the mempool for this was added as part of the mempool p2p
message and is only needed to fetch transactions returned by it.
5d5e7a0 net: No need to export ConnectNode (Cory Fields)
e9ed620 net: No need to export DumpBanlist (Cory Fields)
8b8f877 net: make Ban/Unban/ClearBan functionality consistent (Cory Fields)
cca221f net: Drop CNodeRef for AttemptToEvictConnection (Cory Fields)
563f375 net: use the exposed GetNodeSignals() rather than g_signals directly (Cory Fields)
9faa490 net: remove unused set (Cory Fields)
52cbce2 net: don't import std namespace (Cory Fields)
1475ecf Fix de-serialization bug where AddrMan is corrupted after exception * CAddrDB modified so that when de-serialization code throws an exception Addrman is reset to a clean state * CAddrDB modified to make unit tests possible * Regression test created to ensure bug is fixed * StartNode modifed to clear adrman if CAddrDB::Read returns an error code. (EthanHeilman)
- Ban/Unban/ClearBan call uiInterface.BannedListChanged() as necessary
- Ban/Unban/ClearBan sync to disk if the operation is user-invoked
- Mark node for disconnection automatically when banning
- Lock cs_vNodes while setting disconnected
- Don't spin in a tight loop while setting disconnected
Locking for each operation here is unnecessary, and solves the wrong problem.
Additionally, it introduces a problem when cs_vNodes is held in an owning
class, to which invididual CNodeRefs won't have access.
These should be weak pointers anyway, once vNodes contain shared pointers.
Rather than using a refcounting class, use a 3-step process instead.
1. Lock vNodes long enough to snapshot the fields necessary for comparing
2. Unlock and do the comparison
3. Re-lock and mark the resulting node for disconnection if it still exists
b559914 Move bloom and feerate filtering to just prior to tx sending. (Gregory Maxwell)
4578215 Return mempool queries in dependency order (Pieter Wuille)
ed70683 Handle mempool requests in send loop, subject to trickle (Pieter Wuille)
dc13dcd Split up and optimize transaction and block inv queues (Pieter Wuille)
f2d3ba7 Eliminate TX trickle bypass, sort TX invs for privacy and priority. (Gregory Maxwell)
* CAddrDB modified so that when de-serialization code throws an exception Addrman is reset to a clean state
* CAddrDB modified to make unit tests possible
* Regression test created to ensure bug is fixed
* StartNode modifed to clear adrman if CAddrDB::Read returns an error code.
Some developers clearly don't get this and have been posting
"improvements" that create clear vulnerabilities. It should
have been better explained in the code, since the design
is somewhat subtle and getting it right is important.
DumpBanList currently does this:
- with lock: take a copy of the banmap
- perform I/O (write out the banmap)
- with lock: mark the banmap non-dirty
If a new ban is added during the I/O operation, it may never be persisted to
disk.
Reorder operations so that the data to be persisted cannot be older than the
time at which the banmap was marked non-dirty.
This will avoid sending more pointless INVs around updates, and
prevents using filter updates to timetag transactions.
Also adds locking for fRelayTxes.
By eliminating queued entries from the mempool response and responding only at
trickle time, this makes the mempool no longer leak transaction arrival order
information (as the mempool itself is also sorted)-- at least no more than
relay itself leaks it.
Note: Some seeds aren't actually returning an IP for their name entries, so
they're being added to addrman with a source of [::].
This commit shouldn't change that behavior, for better or worse.
Previously we used the CInv that would be sent to the peer announcing the
transaction as the key, but using the txid instead allows us to decouple the
p2p layer from the application logic (which relies on this map to avoid
duplicate tx requests).
The "feefilter" p2p message is used to inform other nodes of your mempool min fee which is the feerate that any new transaction must meet to be accepted to your mempool. This will allow them to filter invs to you according to this feerate.
027fdb8 When/if the copyright line does not mention Bitcoin Core developers, add a second line to copyrights in -version, About dialog, and splash screen (Luke Dashjr)
cc2095e Rewrite FormatParagraph to handle newlines within input strings correctly (Luke Dashjr)
cddffaf Bugfix: Include COPYRIGHT_HOLDERS_SUBSTITUTION in Makefile substitutions so it gets passed to extract-strings correctly (Luke Dashjr)
29598e4 Move PACKAGE_URL to configure.ac (Luke Dashjr)
78ec83d splashscreen: Resize text to fit exactly (Luke Dashjr)
3cae140 Bugfix: Actually use _COPYRIGHT_HOLDERS_SUBSTITUTION everywhere (Luke Dashjr)
4d5a3df Bugfix: gitian-descriptors: Add missing python-setuptools requirement for OS X (biplist module) (Luke Dashjr)
e4ab5e5 Bugfix: Correct copyright year in Mac DMG background image (Luke Dashjr)
917b1d0 Set copyright holders displayed in notices separately from the package name (Luke Dashjr)
c39a6ff Travis & gitian-osx: Use depends for ds_store and mac_alias modules (Luke Dashjr)
902ccde depends: Add mac_alias to depends (Luke Dashjr)
82a2d98 depends: Add ds_store to depends (Cory Fields)
de619a3 depends: Pass PYTHONPATH along to configure (Cory Fields)
e611b6e macdeploy: Use rsvg-convert rather than cairosvg (Luke Dashjr)
63bcdc5 More complicated package name substitution for Mac deployment (Luke Dashjr)
1a6c67c Parameterise 2009 in translatable copyright strings (Luke Dashjr)
d5f4683 Unify package name to as few places as possible without major changes (Luke Dashjr)
This corrects a bug the case of tying group size where the code may
fail to select the group with the newest member. Since newest time
is the final selection criteria, failing to break ties on it
on the step before can undermine the final selection.
Tied netgroups are very common.
(cherry picked from commit 8e09f914f8ec66301257358b250e9a61befadd95)
With automatic tor HS support in place we should probably not be providing
absolute protection for local peers, since HS inbound could be used to
attack pretty easily. Instead, this counts on the latency metric inside
AttemptToEvictConnection to privilege actually local peers.
(cherry picked from commit 46dbcd4833115401fecbb052365b4c7725874414)
e8600c9 banlist (bugfix): allow CNode::SweepBanned() to run on interval (Philip Kaufmann)
2977c24 banlist: add more banlist infos to log / add GUI signal (Philip Kaufmann)
ce479aa banlist: better handling of banlist in StartNode() (Philip Kaufmann)
57c77fe banlist: update set dirty to be more fine grained (Philip Kaufmann)
