Jack Robison
7720724ec0
reflect lbry_files in looping call in EncryptedFileManager
...
-remove ManagedEncryptedFileDownloader._reupload
-clean up reflector functions in Daemon, move to reflector.reupload
-check ConnectionLost in reflector client
-close sd blob file handle when it wont be sent (otherwise read handle stays open)
-log reflector sd info
-give reflector client factory a lbry file
2017-02-13 15:13:37 -05:00
Alex Grintsvayg
c7106dfc97
reduce loudness of reflector message
2017-02-13 12:12:32 -05:00
Jack Robison
777419b4a9
log traceback on failed blob upload
...
-move status message on connectionDone to client, indicate if blobs
were sent or not (and how many blobs reflector still needs, if any)
-only try uploading failed blob once after first failure, to prevent
indefinite retries
2017-02-08 16:18:34 -05:00
Jack Robison
f2ddc9bd98
fix recursion depth bug upon failed blob
2017-02-08 14:28:30 -05:00
Jack Robison
d053db8dfd
add uri to stream reflector for better logging
2017-02-08 14:28:16 -05:00
Jack Robison
a58a81a841
stopProducing in reflector client file_sender when uploading is done
...
this fixes exceptions.ValueError: I/O operation on closed file in
twisted.protocols.basic.ResumeProducing
2017-02-08 14:09:03 -05:00
Jack Robison
f614a13db4
update documentation
2017-02-06 15:56:00 -05:00
Jack Robison
6fae07d29e
reflector v2
...
-add {‘sd_blob_hash’: …, ‘sd_blob_size ‘: …} query type with
{‘send_sd_blob’: True/False, ‘needed_blobs’: []} response
this allows the reflector client to know how much of a stream reflector
already has covered, as to minimize the number of subsequent requests
and prevent streams from being partially reflected
-remove empty {} request
2017-02-06 15:55:46 -05:00
Jack Robison
74a872d763
fix pylint errors
2017-01-25 13:10:38 -05:00
Jack Robison
5fa2dfeca7
try reflecting all the blobs in a stream
...
previously only the sd blob was reflected, if the server indicated it
needed the blob then the rest of the stream would follow. this allowed
for many streams to be partially reflected, where for whatever reason
the connection was broken before the full upload was completed. this
meant that on a subsequent run, the client would falsely believe
reflector had the whole stream when it actually only had some portion
of it.
this solution isn’t ideal, I’m most of the way done with a better one,
but this can be deployed now.
2017-01-25 13:03:48 -05:00
Job Evers-Meltzer
fa8190356a
replace log_support.failure calls
2016-12-15 18:58:15 -06:00
Job Evers-Meltzer
c30ea04959
Remove commented code
...
This is the result of running eradicate (https://github.com/myint/eradicate )
on the code and double-checking the changes.
2016-12-15 18:58:06 -06:00
Job Evers-Meltzer
b64fa51567
fixup long lines
2016-12-07 09:38:34 -05:00
Job Evers-Meltzer
0941618048
move reflector blob client to its own module
2016-11-14 19:33:36 -06:00
Job Evers-Meltzer
85dee27c92
whitespace
2016-11-14 19:33:36 -06:00
Job Evers-Meltzer
61a354fa60
add bad-whitespace check to pylint
2016-11-09 11:27:07 -06:00
Jack
9de7354ea0
drop connection on failed read blob
2016-11-01 16:32:34 -04:00
Jack
cfbdda411a
whoops
2016-11-01 13:23:02 -04:00
Jack
96c4837cb8
clean up errback
2016-11-01 13:19:13 -04:00
Jack
4cffc32328
catch availability check error
2016-11-01 13:05:19 -04:00
Jack
110a9b1e0f
whoops
2016-10-28 15:56:22 -04:00
Jack
035a1cf758
feedback from job
2016-10-28 15:48:51 -04:00
Jack
369cd516c0
move availability check and fix from ManagedEncryptedFileDownloader to ReflectorAvailabilityHelper
2016-10-26 16:17:01 -04:00
Jack
3dd99fdc92
upload unavailable streams to reflector on startup
2016-10-20 15:40:35 -04:00
Job Evers-Meltzer
7dbeb1f2cf
Remove LBRY prefix from class and module names
...
This is the result of running:
grep -RlI LBRY . | xargs sed -i 's/LBRY\([A-Z]\)/\1/g'
2016-09-27 11:18:35 -07:00
Job Evers-Meltzer
550fd185a3
rename LBRYFile to EncryptedFile
2016-09-27 11:18:16 -07:00
Jack
aa66af7ba6
fix tests
2016-08-26 20:22:11 -04:00
Jack
be3633622c
whoops
2016-08-26 20:13:10 -04:00
Jack
b235f6fc6b
send all blobs to reflector
2016-08-26 19:58:53 -04:00
Jack
3d92413372
debug logging
2016-08-20 23:47:41 -04:00
Jack
5022ed2acc
blob_completed before moving to next one
...
otherwise blob files download to the blobfiles directory, but they are
never recorded in blobs.db
2016-08-20 23:47:12 -04:00
Jack
7a54894015
debugging stuff
2016-08-19 02:41:23 -04:00
Jack
8e7b8af180
call blob_completed when is_validated is true
2016-08-18 06:23:27 -04:00
Jack
c046cd02d5
call blob_completed after receiving blob
2016-08-18 05:58:13 -04:00
Jack
f5508fcdf1
typo
2016-08-18 05:36:17 -04:00
Job Evers-Meltzer
c7430f4ae9
better more logging on the server side
2016-08-11 05:25:45 +00:00
Job Evers-Meltzer
623fda3087
improve logging
2016-08-11 05:16:50 +00:00
Job Evers-Meltzer
80f27f2b29
Revert "more logging on the server side"
...
This reverts commit f83daa5a22
2016-08-11 05:11:18 +00:00
Job Evers-Meltzer
f83daa5a22
more logging on the server side
2016-08-11 04:45:25 +00:00
Job Evers-Meltzer
0a07c8f131
log on closed/lost connection
2016-08-10 19:39:28 -05:00
Job Evers-Meltzer
35481a92f5
actually cleanup patched time.time
2016-08-10 19:36:52 -05:00
Job Evers-Meltzer
1a42e43573
log all the things
2016-08-10 19:06:26 -05:00
Jack
132aa569fe
move run_reflector_server into settings file and add debug lines
2016-08-10 16:51:46 -04:00
Job Evers-Meltzer
67909724e7
fix up import paths
2016-08-10 07:44:41 -05:00
Jack
3fb4831804
raise exception when curr_pos > 100 and json fails to decode
2016-08-10 02:30:41 -04:00
Jack
b65570fb16
drop connection on response longer than 100 bytes that can't' be decoded
2016-08-09 19:53:13 -04:00
Jimmy Kiselak
7e2ad58edd
get reflector client and server working; add func test to prove it
2016-08-09 00:59:50 -04:00
Jimmy Kiselak
b7e2e87ac1
fill in much of the skeleton for the reflector protocol
2016-08-07 22:33:40 -04:00
Jimmy Kiselak
1dea20a358
include sd blob hashes in list of blob hashes to send via reflector
2016-08-03 23:11:29 -04:00
Jimmy Kiselak
191f661b35
skeleton for reflector protocol
2016-08-03 23:03:14 -04:00
