lbry-sdk/lbrynet/core/Session.py

173 lines
7 KiB
Python
Raw Normal View History

2015-08-20 17:27:15 +02:00
import logging
refactor lbrynet-daemon into modular components (#1164) * add daemon Component and ComponentManager classes * convert directory and SQLiteStorage setup to be a Component * support callbacks to component setups * Fixed typo in ComponentManager * convert wallet to be Component * Use storage from session. * Remove create_session internal function and PEP8 * Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py * Making DHT component(broken) * Refactored classes to reduce redundancy in getting config setting * DHT is now it's own component * Fixed `test_streamify` test * Fixed regression caused by removing `peer_manager` from session * refactor ComponentManager and Component to use instance instead of class methods * Hash announcer, file manager, stream identifier components * Query Handler and server components * Reflector Component * Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯) * All tests now passing * Pylint fixes * Oops(That's all you're gonna get :-P) * Making decorators(WIP, commit so that I don't lose work) * Decorator made and decorating of functions done(some other changes) * import fixes and removed temporary test function * Fixed new broken tests from daemon refactor * Sanitization of modules * Reworded errors * wallet unlock condition checks, fixed breaking changes * Rebased on amster and other crazy stuff * Started writing tests * Tests for component manager * Fix Daemon Tests * Fixed passing mutable args in init * Using constants instead of strings. Added CHANGELOG.md * Now components can be skipped by setting relevant config in file. * P-Y-L-I-N-T #angry_emoji
2018-07-05 21:21:52 +02:00
from twisted.internet import defer
from lbrynet.core.BlobManager import DiskBlobManager
from lbrynet.database.storage import SQLiteStorage
2015-08-20 17:27:15 +02:00
from lbrynet.core.RateLimiter import RateLimiter
from lbrynet.core.PaymentRateManager import BasePaymentRateManager, OnlyFreePaymentsManager
2015-08-20 17:27:15 +02:00
log = logging.getLogger(__name__)
class Session(object):
2016-11-03 20:22:16 +01:00
"""This class manages all important services common to any application that uses the network.
the hash announcer, which informs other peers that this peer is
associated with some hash. Usually, this means this peer has a
blob identified by the hash in question, but it can be used for
other purposes.
the peer finder, which finds peers that are associated with some
hash.
the blob manager, which keeps track of which blobs have been
downloaded and provides access to them,
the rate limiter, which attempts to ensure download and upload
rates stay below a set maximum
upnp, which opens holes in compatible firewalls so that remote
peers can connect to this peer.
"""
2017-04-10 16:51:49 +02:00
refactor lbrynet-daemon into modular components (#1164) * add daemon Component and ComponentManager classes * convert directory and SQLiteStorage setup to be a Component * support callbacks to component setups * Fixed typo in ComponentManager * convert wallet to be Component * Use storage from session. * Remove create_session internal function and PEP8 * Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py * Making DHT component(broken) * Refactored classes to reduce redundancy in getting config setting * DHT is now it's own component * Fixed `test_streamify` test * Fixed regression caused by removing `peer_manager` from session * refactor ComponentManager and Component to use instance instead of class methods * Hash announcer, file manager, stream identifier components * Query Handler and server components * Reflector Component * Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯) * All tests now passing * Pylint fixes * Oops(That's all you're gonna get :-P) * Making decorators(WIP, commit so that I don't lose work) * Decorator made and decorating of functions done(some other changes) * import fixes and removed temporary test function * Fixed new broken tests from daemon refactor * Sanitization of modules * Reworded errors * wallet unlock condition checks, fixed breaking changes * Rebased on amster and other crazy stuff * Started writing tests * Tests for component manager * Fix Daemon Tests * Fixed passing mutable args in init * Using constants instead of strings. Added CHANGELOG.md * Now components can be skipped by setting relevant config in file. * P-Y-L-I-N-T #angry_emoji
2018-07-05 21:21:52 +02:00
def __init__(self, blob_data_payment_rate, db_dir=None, node_id=None, dht_node_port=None,
2018-05-29 22:22:30 +02:00
known_dht_nodes=None, peer_finder=None, hash_announcer=None, blob_dir=None, blob_manager=None,
refactor lbrynet-daemon into modular components (#1164) * add daemon Component and ComponentManager classes * convert directory and SQLiteStorage setup to be a Component * support callbacks to component setups * Fixed typo in ComponentManager * convert wallet to be Component * Use storage from session. * Remove create_session internal function and PEP8 * Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py * Making DHT component(broken) * Refactored classes to reduce redundancy in getting config setting * DHT is now it's own component * Fixed `test_streamify` test * Fixed regression caused by removing `peer_manager` from session * refactor ComponentManager and Component to use instance instead of class methods * Hash announcer, file manager, stream identifier components * Query Handler and server components * Reflector Component * Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯) * All tests now passing * Pylint fixes * Oops(That's all you're gonna get :-P) * Making decorators(WIP, commit so that I don't lose work) * Decorator made and decorating of functions done(some other changes) * import fixes and removed temporary test function * Fixed new broken tests from daemon refactor * Sanitization of modules * Reworded errors * wallet unlock condition checks, fixed breaking changes * Rebased on amster and other crazy stuff * Started writing tests * Tests for component manager * Fix Daemon Tests * Fixed passing mutable args in init * Using constants instead of strings. Added CHANGELOG.md * Now components can be skipped by setting relevant config in file. * P-Y-L-I-N-T #angry_emoji
2018-07-05 21:21:52 +02:00
peer_port=None, use_upnp=True, rate_limiter=None, wallet=None, blob_tracker_class=None,
payment_rate_manager_class=None, is_generous=True, external_ip=None, storage=None,
dht_node=None, peer_manager=None):
2016-11-03 20:22:16 +01:00
"""@param blob_data_payment_rate: The default payment rate for blob data
2015-08-20 17:27:15 +02:00
@param db_dir: The directory in which levelDB files should be stored
2017-10-10 19:15:25 +02:00
@param node_id: The unique ID of this node
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param peer_manager: An object which keeps track of all known
peers. If None, a PeerManager will be created
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param dht_node_port: The port on which the dht node should
listen for incoming connections
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param known_dht_nodes: A list of nodes which the dht node
should use to bootstrap into the dht
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param peer_finder: An object which is used to look up peers
that are associated with some hash. If None, a
DHTPeerFinder will be used, which looks for peers in the
distributed hash table.
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param hash_announcer: An object which announces to other
peers that this peer is associated with some hash. If
None, and peer_port is not None, a DHTHashAnnouncer will
be used. If None and peer_port is None, a
DummyHashAnnouncer will be used, which will not actually
announce anything.
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param blob_dir: The directory in which blobs will be
stored. If None and blob_manager is None, blobs will be
stored in memory only.
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param blob_manager: An object which keeps track of downloaded
blobs and provides access to them. If None, and blob_dir
is not None, a DiskBlobManager will be used, with the
given blob_dir. If None and blob_dir is None, a
TempBlobManager will be used, which stores blobs in memory
only.
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param peer_port: The port on which other peers should connect
to this peer
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param use_upnp: Whether or not to try to open a hole in the
firewall so that outside peers can connect to this peer's
peer_port and dht_node_port
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param rate_limiter: An object which keeps track of the amount
of data transferred to and from this peer, and can limit
that rate if desired
2015-08-20 17:27:15 +02:00
2016-11-03 20:22:16 +01:00
@param wallet: An object which will be used to keep track of
expected payments and which will pay peers. If None, a
wallet which uses the Point Trader system will be used,
which is meant for testing only
2015-08-20 17:27:15 +02:00
"""
self.db_dir = db_dir
2017-10-10 19:15:25 +02:00
self.node_id = node_id
2015-08-20 17:27:15 +02:00
self.peer_manager = peer_manager
self.peer_finder = peer_finder
self.hash_announcer = hash_announcer
2015-08-20 17:27:15 +02:00
self.dht_node_port = dht_node_port
self.known_dht_nodes = known_dht_nodes
if self.known_dht_nodes is None:
self.known_dht_nodes = []
self.blob_dir = blob_dir
self.blob_manager = blob_manager
# self.blob_tracker = None
# self.blob_tracker_class = blob_tracker_class or BlobAvailabilityTracker
2015-08-20 17:27:15 +02:00
self.peer_port = peer_port
self.use_upnp = use_upnp
self.rate_limiter = rate_limiter
self.external_ip = external_ip
self.upnp_redirects = []
2015-08-20 17:27:15 +02:00
self.wallet = wallet
refactor lbrynet-daemon into modular components (#1164) * add daemon Component and ComponentManager classes * convert directory and SQLiteStorage setup to be a Component * support callbacks to component setups * Fixed typo in ComponentManager * convert wallet to be Component * Use storage from session. * Remove create_session internal function and PEP8 * Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py * Making DHT component(broken) * Refactored classes to reduce redundancy in getting config setting * DHT is now it's own component * Fixed `test_streamify` test * Fixed regression caused by removing `peer_manager` from session * refactor ComponentManager and Component to use instance instead of class methods * Hash announcer, file manager, stream identifier components * Query Handler and server components * Reflector Component * Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯) * All tests now passing * Pylint fixes * Oops(That's all you're gonna get :-P) * Making decorators(WIP, commit so that I don't lose work) * Decorator made and decorating of functions done(some other changes) * import fixes and removed temporary test function * Fixed new broken tests from daemon refactor * Sanitization of modules * Reworded errors * wallet unlock condition checks, fixed breaking changes * Rebased on amster and other crazy stuff * Started writing tests * Tests for component manager * Fix Daemon Tests * Fixed passing mutable args in init * Using constants instead of strings. Added CHANGELOG.md * Now components can be skipped by setting relevant config in file. * P-Y-L-I-N-T #angry_emoji
2018-07-05 21:21:52 +02:00
self.dht_node = dht_node
2015-08-20 17:27:15 +02:00
self.base_payment_rate_manager = BasePaymentRateManager(blob_data_payment_rate)
self.payment_rate_manager = OnlyFreePaymentsManager()
# self.payment_rate_manager_class = payment_rate_manager_class or NegotiatedPaymentRateManager
# self.is_generous = is_generous
self.storage = storage or SQLiteStorage(self.db_dir)
2015-08-20 17:27:15 +02:00
def setup(self):
"""Create the blob directory and database if necessary, start all desired services"""
2017-08-02 21:48:07 +02:00
log.debug("Starting session.")
2015-08-20 17:27:15 +02:00
refactor lbrynet-daemon into modular components (#1164) * add daemon Component and ComponentManager classes * convert directory and SQLiteStorage setup to be a Component * support callbacks to component setups * Fixed typo in ComponentManager * convert wallet to be Component * Use storage from session. * Remove create_session internal function and PEP8 * Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py * Making DHT component(broken) * Refactored classes to reduce redundancy in getting config setting * DHT is now it's own component * Fixed `test_streamify` test * Fixed regression caused by removing `peer_manager` from session * refactor ComponentManager and Component to use instance instead of class methods * Hash announcer, file manager, stream identifier components * Query Handler and server components * Reflector Component * Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯) * All tests now passing * Pylint fixes * Oops(That's all you're gonna get :-P) * Making decorators(WIP, commit so that I don't lose work) * Decorator made and decorating of functions done(some other changes) * import fixes and removed temporary test function * Fixed new broken tests from daemon refactor * Sanitization of modules * Reworded errors * wallet unlock condition checks, fixed breaking changes * Rebased on amster and other crazy stuff * Started writing tests * Tests for component manager * Fix Daemon Tests * Fixed passing mutable args in init * Using constants instead of strings. Added CHANGELOG.md * Now components can be skipped by setting relevant config in file. * P-Y-L-I-N-T #angry_emoji
2018-07-05 21:21:52 +02:00
if self.dht_node is not None:
if self.peer_manager is None:
self.peer_manager = self.dht_node.peer_manager
if self.peer_finder is None:
self.peer_finder = self.dht_node.peer_finder
2015-08-20 17:27:15 +02:00
refactor lbrynet-daemon into modular components (#1164) * add daemon Component and ComponentManager classes * convert directory and SQLiteStorage setup to be a Component * support callbacks to component setups * Fixed typo in ComponentManager * convert wallet to be Component * Use storage from session. * Remove create_session internal function and PEP8 * Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py * Making DHT component(broken) * Refactored classes to reduce redundancy in getting config setting * DHT is now it's own component * Fixed `test_streamify` test * Fixed regression caused by removing `peer_manager` from session * refactor ComponentManager and Component to use instance instead of class methods * Hash announcer, file manager, stream identifier components * Query Handler and server components * Reflector Component * Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯) * All tests now passing * Pylint fixes * Oops(That's all you're gonna get :-P) * Making decorators(WIP, commit so that I don't lose work) * Decorator made and decorating of functions done(some other changes) * import fixes and removed temporary test function * Fixed new broken tests from daemon refactor * Sanitization of modules * Reworded errors * wallet unlock condition checks, fixed breaking changes * Rebased on amster and other crazy stuff * Started writing tests * Tests for component manager * Fix Daemon Tests * Fixed passing mutable args in init * Using constants instead of strings. Added CHANGELOG.md * Now components can be skipped by setting relevant config in file. * P-Y-L-I-N-T #angry_emoji
2018-07-05 21:21:52 +02:00
d = self.storage.setup()
2015-08-20 17:27:15 +02:00
d.addCallback(lambda _: self._setup_other_components())
return d
def shut_down(self):
"""Stop all services"""
2017-08-02 21:48:07 +02:00
log.info('Stopping session.')
2015-08-20 17:27:15 +02:00
ds = []
# if self.blob_tracker is not None:
# ds.append(defer.maybeDeferred(self.blob_tracker.stop))
if self.rate_limiter is not None:
ds.append(defer.maybeDeferred(self.rate_limiter.stop))
if self.blob_manager is not None:
ds.append(defer.maybeDeferred(self.blob_manager.stop))
refactor lbrynet-daemon into modular components (#1164) * add daemon Component and ComponentManager classes * convert directory and SQLiteStorage setup to be a Component * support callbacks to component setups * Fixed typo in ComponentManager * convert wallet to be Component * Use storage from session. * Remove create_session internal function and PEP8 * Starting to convert session to its own component. Removed ref to `self.storage` from Daemon.py * Making DHT component(broken) * Refactored classes to reduce redundancy in getting config setting * DHT is now it's own component * Fixed `test_streamify` test * Fixed regression caused by removing `peer_manager` from session * refactor ComponentManager and Component to use instance instead of class methods * Hash announcer, file manager, stream identifier components * Query Handler and server components * Reflector Component * Fixed test_streamify(well Jack did, but ¯\_(ツ)_/¯) * All tests now passing * Pylint fixes * Oops(That's all you're gonna get :-P) * Making decorators(WIP, commit so that I don't lose work) * Decorator made and decorating of functions done(some other changes) * import fixes and removed temporary test function * Fixed new broken tests from daemon refactor * Sanitization of modules * Reworded errors * wallet unlock condition checks, fixed breaking changes * Rebased on amster and other crazy stuff * Started writing tests * Tests for component manager * Fix Daemon Tests * Fixed passing mutable args in init * Using constants instead of strings. Added CHANGELOG.md * Now components can be skipped by setting relevant config in file. * P-Y-L-I-N-T #angry_emoji
2018-07-05 21:21:52 +02:00
# if self.use_upnp is True:
# ds.append(defer.maybeDeferred(self._unset_upnp))
2015-08-20 17:27:15 +02:00
return defer.DeferredList(ds)
def _setup_other_components(self):
log.debug("Setting up the rest of the components")
2015-08-20 17:27:15 +02:00
if self.rate_limiter is None:
self.rate_limiter = RateLimiter()
if self.blob_manager is None:
if self.blob_dir is None:
raise Exception(
"TempBlobManager is no longer supported, specify BlobManager or db_dir")
2015-08-20 17:27:15 +02:00
else:
self.blob_manager = DiskBlobManager(self.blob_dir, self.storage, self.dht_node._dataStore)
2015-08-20 17:27:15 +02:00
# if self.blob_tracker is None:
# self.blob_tracker = self.blob_tracker_class(
# self.blob_manager, self.dht_node.peer_finder, self.dht_node
# )
# if self.payment_rate_manager is None:
# self.payment_rate_manager = self.payment_rate_manager_class(
# self.base_payment_rate_manager, self.blob_tracker, self.is_generous
# )
self.rate_limiter.start()
d = self.blob_manager.setup()
2016-06-04 20:28:32 +02:00
return d