Victor Shyba
a533cda6f0
ES: all _hash to _id
2021-07-22 16:25:55 -03:00
Lex Berezhny
fe4b07b8ae
v0.101.0
2021-07-21 12:35:16 -04:00
Lex Berezhny
f9f2ccd904
revert version
2021-07-21 12:28:41 -04:00
Lex Berezhny
8d9073cd31
v0.101.0
2021-07-20 22:52:44 -04:00
Jack Robison
9a8f9f0a94
fix stuck notification due to mempool/notification race
2021-07-20 15:14:10 -04:00
belikor
6421cecafb
daemon: fix --claim_id with lbrynet claim search
...
For some reason, when using `claim_search`
with `--claim_id`, the arguments dictionary will also
contain `claim_ids` with an empty list, even if we didn't specify it.
```
lbrynet claim search --claim_id=8945573bcfcb7f8276187dfbb93545eac4ebf71a
```
Using both `claim_id` and `claim_ids` will raise a `ValueError`
exception so the daemon won't return a valid result
even if the claim ID is in fact valid.
So if `claim_id` exists, we need to discard `claim_ids`
if it is empty, before proceeding with the rest of the code.
On the other hand, if `claim_ids` is used, and `claim_id` is absent,
there will be no problem as `claim_id` won't be added to the dictionary.
```
lbrynet claim search --claim_ids=8945573bcfcb7f8276187dfbb93545eac4ebf71a
```
2021-07-19 22:24:43 -05:00
Alex Grin
74116cc550
Merge branch 'master' into propagate_external_ip_change
2021-07-19 14:12:19 -04:00
Victor Shyba
0ccafd5b53
make get_or_create_usable_address respect the generator lock
2021-07-19 14:09:52 -04:00
belikor
b7791d2845
exchange_rate_manager: raise exception if 'error' is in json_response
...
If the error is not handled, the running daemon will continuously
print the following error message:
```
Traceback (most recent call last):
File "lbry/extras/daemon/exchange_rate_manager.py", line 77, in get_rate
File "lbry/extras/daemon/exchange_rate_manager.py", line 189, in get_rate_from_response
KeyError: 0
```
This started happening when the UPBit exchange decided to delist
the LBC coin.
Normally `json_response` should be a dictionary, not a list,
so `json_response[0]` causes an error.
By checking for the `'error'` key, we can raise the proper exception.
Once this is done, the message will be a warning, not a traceback.
```
WARNING lbry.extras.daemon.exchange_rate_manager:92:
Failed to get exchange rate from UPbit: result not found
```
2021-07-19 13:41:49 -04:00
Victor Shyba
6254f53716
propagate external ip changes from upnp component to dht node protocol
2021-07-08 03:46:05 -03:00
Victor Shyba
3de0982a4a
limit request error logging to 16k
2021-07-07 18:39:38 -03:00
Victor Shyba
c2184fb3bf
run migration on history db open
2021-07-07 18:39:38 -03:00
Victor Shyba
919c09fcb0
add migration
2021-07-07 18:39:38 -03:00
Victor Shyba
1d9dbd40ec
increase flush counter to 32 bits
2021-07-07 18:39:38 -03:00
Victor Shyba
309e957a85
add concurrent_hub_requests conf
2021-06-24 21:21:19 -03:00
Victor Shyba
d7007e402e
move request semaphore to session and apply to all requests
2021-06-24 21:02:41 -03:00
Victor Shyba
91323a21cf
add hub_timeout and propagate it to network code
2021-06-24 21:02:41 -03:00
Lex Berezhny
fea893d76c
v0.100.0
2021-06-22 13:33:03 -04:00
Lex Berezhny
761bc6ba4c
revert release and fix test
2021-06-22 13:32:41 -04:00
Lex Berezhny
75172feb4e
v0.100.0
2021-06-22 12:53:23 -04:00
Lex Berezhny
3285fb1608
revert release
2021-06-22 12:52:48 -04:00
Lex Berezhny
03a4c6910d
v0.100.0
2021-06-22 12:51:36 -04:00
Lex Berezhny
485b958599
revert release
2021-06-22 12:50:11 -04:00
Lex Berezhny
da47ba2f67
v0.100.0
2021-06-22 11:11:02 -04:00
Lex Berezhny
c39195488a
bug fix
2021-06-22 11:07:58 -04:00
Lex Berezhny
0946c72b88
lint
2021-06-22 11:07:58 -04:00
Lex Berezhny
7d49b046d4
added support to config for determining if value is set and implemented hub selection logic
2021-06-22 11:07:58 -04:00
Lex Berezhny
5f0426c840
country jurisdiction added to hub UDP protocol
2021-06-22 11:07:58 -04:00
Lex Berezhny
73e239cc5f
client side hub discovery pub/sub and hub metadata stored, removed old peers implementation
2021-06-22 11:07:58 -04:00
Lex Berezhny
ad670f721a
working client peer hub
2021-06-22 11:07:58 -04:00
Lex Berezhny
028a4a70cf
wallet server federation, client portion
2021-06-22 11:07:58 -04:00
Lex Berezhny
e457b2f0d6
fix trending to use built-in sqlite instead of apsw
2021-06-15 18:22:42 -04:00
Lex Berezhny
c9cf7fd4d4
drop apsw in wallet.server.db.elasticsearch.sync
2021-06-15 18:22:42 -04:00
Lex Berezhny
25e16c3565
dropping apsw
2021-06-15 18:22:42 -04:00
Victor Shyba
fb438dc108
remove the unregister call
2021-06-04 12:47:16 -03:00
Victor Shyba
4e6b4f179b
add items() to LRUCache
2021-06-04 12:20:44 -03:00
Victor Shyba
00d038c8f3
add default parameter to pop on LRUCache
2021-06-04 12:15:47 -03:00
Victor Shyba
a9f6a68952
use LRU caches for DHT metrics
2021-06-04 11:54:37 -03:00
Victor Shyba
6c812f663e
drop value_type for support
2021-06-02 18:01:54 -03:00
Lex Berezhny
2d184d77b6
v0.99.0
2021-06-02 12:07:37 -04:00
Victor Shyba
bce299ccc7
fix docopt typo
2021-06-02 12:05:36 -04:00
Victor Shyba
a638aa9d53
add and test support for support_create anonymous --comment
2021-06-02 12:05:36 -04:00
Victor Shyba
67cce0ef7e
test+implement --comment for support_create
2021-06-02 12:05:36 -04:00
Victor Shyba
82f4267bf6
add comment property/setter to the signable support class
2021-06-02 12:05:36 -04:00
Victor Shyba
45a9ca29c4
update generated support protobuf with field
2021-06-02 12:05:36 -04:00
Victor Shyba
7f4e813277
document schema update process
2021-06-02 12:05:36 -04:00
Lex Berezhny
464cfd475e
properly format scripthash address on output
2021-06-02 11:34:21 -04:00
Lex Berezhny
fe469ae57f
create appropriate script for scripthash address
2021-06-02 11:34:21 -04:00
Lex Berezhny
550ef9a1c4
allows script addresses (beginning with r) to be used
2021-06-02 11:34:21 -04:00
Victor Shyba
3974df4a62
fix interaction between two modes
2021-05-27 20:14:12 -03:00
Victor Shyba
8c4b0037f5
API: add --remove_duplicates to claim_search
2021-05-27 20:14:12 -03:00
Victor Shyba
ca28de02d8
test and implementation for remove_duplicates on post-search filtering
2021-05-27 20:14:12 -03:00
Victor Shyba
bfc15ea029
handle limit being 0 and skip reordering if 0/none
2021-05-27 20:14:12 -03:00
Victor Shyba
6e8b8a5920
always call search_ahead
2021-05-27 20:14:12 -03:00
Victor Shyba
142d182bc1
if progress was made, retry without a delay
2021-05-27 18:24:58 -03:00
Victor Shyba
1437871d88
fix reflector client: only set completed when server says so
2021-05-27 18:24:58 -03:00
Victor Shyba
352bf69409
improve test
2021-05-27 18:24:58 -03:00
Victor Shyba
9bdf3d23e1
test bug 3296, failing
2021-05-27 18:24:58 -03:00
Victor Shyba
be8ecfa707
sort keys so helper scripts can send blobs using send_request
2021-05-27 18:24:58 -03:00
Lex Berezhny
51da0d0259
v0.98.0
2021-05-26 09:23:19 -04:00
Alex Grin
e1a44c93f8
Merge branch 'master' into fix-collectionUpdateWithReplace
2021-05-18 15:25:40 -04:00
Alex Grin
2c79c7e2f6
Merge branch 'master' into patch-1
2021-05-18 15:23:08 -04:00
Victor Shyba
09f6637fe0
remove unused multiprocessin.Manager
2021-05-17 15:07:32 -03:00
zeppi
2b950ff5dd
fix bug in collection_update --replace
2021-05-15 03:27:33 -03:00
Victor Shyba
ccadd88af5
fix cache call
2021-05-13 22:40:21 -03:00
Victor Shyba
cc02a0efc2
fix es migration bug, expand test case
2021-05-13 19:00:53 -03:00
Victor Shyba
43a1385b79
test sync helper
2021-05-13 19:00:53 -03:00
keikari
d9413039ec
Fix suggestion for issue #3240
...
L135: If `getattr()` returns `None`, use `""` instead to avoid error in issue #3240
2021-05-12 18:30:38 +03:00
Jack Robison
eba0c9be34
fix typo
2021-05-07 16:51:19 -04:00
Jack Robison
81ebde88db
resync ES search index on version bumps
...
-bump ES search index to version 1
2021-05-07 14:36:53 -04:00
Victor Shyba
a4058b84ce
clean out unused sharding
2021-05-07 15:03:37 -03:00
Victor Shyba
7bf211a52b
apply reposted_claim_type on es sync
2021-05-07 15:03:37 -03:00
Victor Shyba
d5f722792f
fix and test has_source for channel reposts
2021-05-07 15:03:37 -03:00
Victor Shyba
0f02906c9b
fix has_source for reposted channels
2021-05-07 15:03:37 -03:00
Victor Shyba
7554e6d7f9
remove dead code
2021-05-07 15:02:31 -03:00
Victor Shyba
cb8f26f177
remove broken feed
2021-05-07 15:02:31 -03:00
Jack Robison
b5dfce7861
Revert "finished switch from using hash # in URLs to colon :"
...
This reverts commit 888aa558
2021-05-07 11:31:28 -04:00
Jack Robison
2ca5a65544
Revert "FindShortestID updated"
...
This reverts commit 8f04a50c
2021-05-07 11:30:31 -04:00
Jack Robison
32f7ecb261
fix invalid claim fees breaking the block processor
2021-05-06 11:18:58 -04:00
Victor Shyba
fb77fde710
for debug, it is always whole page
2021-05-04 22:22:07 -03:00
Victor Shyba
3c67bb90d7
don't fail when a single one go on maintenance and set completion event regardless of failures
2021-05-04 22:22:07 -03:00
Victor Shyba
dabb168853
dont log full exceptions on simple connection errors
2021-05-04 22:22:07 -03:00
Victor Shyba
45e5b3b219
dont log full pages
2021-05-04 22:22:07 -03:00
Jack Robison
21d0038ff2
add timestamps to hub log
2021-04-28 16:47:00 -04:00
Jack Robison
c094d8f2e8
add ALLOW_LAN_UDP hub setting
2021-04-28 16:47:00 -04:00
Jack Robison
c465d6a6c2
ignore udp packets with low source ports
2021-04-28 16:47:00 -04:00
Lex Berezhny
73d35bc985
v0.97.0
2021-04-28 16:23:46 -04:00
Lex Berezhny
8f04a50ce1
FindShortestID updated
2021-04-28 16:21:01 -04:00
Lex Berezhny
888aa5586b
finished switch from using hash # in URLs to colon :
2021-04-28 16:21:01 -04:00
Lex Berezhny
99f56f5d22
v0.96.0
2021-04-28 15:26:58 -04:00
Victor Shyba
f0d334d3e2
refactor from review
2021-04-28 13:28:38 -03:00
Victor Shyba
5f829b048f
use separator to avoid cache key conflicts
2021-04-27 22:57:04 -03:00
Victor Shyba
1a961e66ff
invalidate short_id cache on new block
2021-04-27 22:57:04 -03:00
Victor Shyba
fdb0e22656
cache search_ahead
2021-04-27 22:57:04 -03:00
Victor Shyba
44bf4f3c8f
fix if statement from always evaluating a string
2021-04-27 17:10:04 -03:00
Lex Berezhny
b167c87267
v0.95.0
2021-04-23 14:55:38 -04:00
Lex Berezhny
2280fe8e8e
default has_source to 1
2021-04-23 14:54:51 -04:00
Lex Berezhny
575d6dcd2d
migration specifically for upgrading from client db v1.5 to v1.6
2021-04-23 14:54:51 -04:00
Lex Berezhny
f729490c6b
pending claims ordered towards top in claim_list
2021-04-23 11:00:58 -04:00
Lex Berezhny
3d4321ee38
added --has_source/--has_no_source filters to claim_list
2021-04-23 10:24:48 -04:00
Lex Berezhny
5e3a234cbe
v0.94.1
2021-04-16 11:18:24 -04:00
Lex Berezhny
e54c31d2d5
fix bug in how reserved balance is calculated
2021-04-16 11:17:51 -04:00
Lex Berezhny
3d241500cf
v0.94.0
2021-04-14 19:55:35 -04:00
Lex Berezhny
f8814881a1
ability to set sd_hash, file_name and file_hash when updating a stream claim
2021-04-14 19:52:50 -04:00
Victor Shyba
cc2852cd48
new implementation for limit_claims_per_channel
2021-04-14 18:32:16 -04:00
Lex Berezhny
3cfc292d84
lint
2021-04-14 11:24:58 -04:00
Lex Berezhny
6acf94a810
moved balance calculation to SQL
2021-04-14 11:24:58 -04:00
Jack Robison
31367fb4c4
show hostnames of spvs
2021-04-13 11:51:27 -04:00
Jack Robison
12d6074e3b
fix typing
2021-04-13 11:51:27 -04:00
Lex Berezhny
ff30386051
lint
2021-04-06 21:22:27 -04:00
shubhendra
601f99ac16
Remove unnecessary generator
...
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra
68399ca31c
Iterate dictionary directly
...
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra
2a6d7fd80f
Remove methods with unnecessary super delegation.
...
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra
4725f510d8
Remove unnecessary use of comprehension
...
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
shubhendra
be0ba22222
Remove unnecessary comprehension
...
Signed-off-by: shubhendra <withshubh@gmail.com>
2021-04-06 21:22:27 -04:00
Lex Berezhny
c8781392be
added unit test for Access-Control HTTP headers
2021-04-06 17:12:05 -04:00
John Leith
b97164fcfb
adding access control headers
2021-04-06 17:12:05 -04:00
Lex Berezhny
0dfb92281b
v0.93.0
2021-03-30 20:59:47 -04:00
Victor Shyba
46dd389d0d
add elasticsearch service to gitlab
2021-03-30 17:00:15 -04:00
Jack Robison
7832c62c5d
add ELASTIC_HOST and ELASTIC_PORT settings to hub
2021-03-30 12:48:13 -04:00
Lex Berezhny
d025ee9dbe
revert release
2021-03-30 11:29:17 -04:00
Lex Berezhny
a9a9cb4319
v0.93.0
2021-03-30 10:15:31 -04:00
Victor Shyba
aa727cb9b1
show channels regardless of no_source
2021-03-30 09:47:08 -04:00
Victor Shyba
b8c9a99f20
fix no_source for reposts
2021-03-30 09:47:08 -04:00
Lex Berezhny
aff995b0d0
temporary fix for mempool sync failing during reorg
2021-03-29 16:11:03 -04:00
Victor Shyba
5235a150b1
add prog name to sync arg parser
2021-03-24 17:07:17 -03:00
Victor Shyba
7df4cc44c4
fixes from review
2021-03-24 16:30:33 -03:00
Victor Shyba
7f5d88e95c
remove dead/broken/unused API
2021-03-19 19:58:13 -03:00
Victor Shyba
d09663c066
remove flush call
2021-03-19 19:58:13 -03:00
Victor Shyba
ef97c9b69f
torba-server -> hub
2021-03-19 19:58:13 -03:00
Victor Shyba
d855e6c8b1
move elasticsearch things into its own module
2021-03-19 19:58:13 -03:00
Victor Shyba
cd66f7eb43
if not no_totals, use default page size
2021-03-19 19:58:13 -03:00
Victor Shyba
a3e146dc68
sort on index time
2021-03-19 19:58:13 -03:00
Victor Shyba
b81305a4a9
index and allow has_source
2021-03-19 19:58:13 -03:00
Victor Shyba
73884b34bc
apply no_totals
2021-03-19 19:58:13 -03:00
Victor Shyba
6166a34db2
check cache item before locking
2021-03-19 19:58:13 -03:00
Victor Shyba
6fa7da4b1c
less slices
2021-03-19 19:58:13 -03:00
Victor Shyba
c3e426c491
fix search by channel for invalid channel
2021-03-19 19:58:13 -03:00
Victor Shyba
21e023f0db
fix search by channel
2021-03-19 19:58:13 -03:00
Victor Shyba
063be001b3
cache inner parsing
2021-03-19 19:58:13 -03:00
Victor Shyba
5dff02e8bc
on resolve, get all claims at once
2021-03-19 19:58:13 -03:00
Victor Shyba
60a59407d8
cache the encoded output instead
2021-03-19 19:58:13 -03:00
Victor Shyba
20a5aecfca
fix lib exception to asyncio TimeoutError
2021-03-19 19:58:13 -03:00
Victor Shyba
c2e7b5a67d
restore some of the interrupt metrics
2021-03-19 19:58:13 -03:00
Victor Shyba
8f32303d07
apply search timeout
2021-03-19 19:58:13 -03:00
Victor Shyba
891b1e7782
track results up to 200
2021-03-19 19:58:13 -03:00
Victor Shyba
f26394fd3b
report deletions on docs that doesnt exist, but dont raise
2021-03-19 19:58:13 -03:00
Victor Shyba
4d83d42b4c
fix equality instead of mod
2021-03-19 19:58:13 -03:00
Victor Shyba
57f1108df2
fix query being json serializable
2021-03-19 19:58:13 -03:00
Victor Shyba
2641a9abe5
make better resolve cache
2021-03-19 19:58:13 -03:00
Victor Shyba
6b193ab350
make indexing cooperative
2021-03-19 19:58:13 -03:00
Victor Shyba
b1bb37511c
use right key on cache
2021-03-19 19:58:13 -03:00
Victor Shyba
319187d6d6
log mempool task exceptions
2021-03-19 19:58:13 -03:00
Victor Shyba
02eb789f84
caching for resolve
2021-03-19 19:58:13 -03:00
Victor Shyba
5a9338a27f
use a dict on set_reference
2021-03-19 19:58:13 -03:00
Victor Shyba
eb6924277f
round time to 10 minutes and fetch referenced by id
2021-03-19 19:58:13 -03:00
Victor Shyba
bd8f371fd5
bump referenced rows query limit up
2021-03-19 19:58:13 -03:00
Victor Shyba
1783ff2845
dont delete claims on reorg
2021-03-19 19:58:13 -03:00
Victor Shyba
d388527ffa
log indexing errors
2021-03-19 19:58:13 -03:00
Victor Shyba
19494088bd
generate from queue
2021-03-19 19:58:13 -03:00
Victor Shyba
920dad524a
simplify sync and use asyncio Queue instead
2021-03-19 19:58:13 -03:00
Victor Shyba
ec89bcac8e
improve sync script for no-downtime maintenance
2021-03-19 19:58:13 -03:00
Victor Shyba
a916c1f4ad
check if db file exists before sync
2021-03-19 19:58:13 -03:00
Victor Shyba
a9a0ac92d7
ignore unset flag
2021-03-19 19:58:13 -03:00
Victor Shyba
da8a8bd1ef
filter+fts and tests for edge cases
2021-03-19 19:58:13 -03:00
Victor Shyba
67817005b5
check ES synced without a process and wait for ES
2021-03-19 19:58:13 -03:00
Jack Robison
24d11de5a7
torba-elastic-sync
2021-03-19 19:58:13 -03:00
Victor Shyba
e12fab90d1
docker compose update
2021-03-19 19:58:13 -03:00
Victor Shyba
0a194b5b01
claim_ids query
2021-03-19 19:58:13 -03:00
Victor Shyba
8d028adc53
be a writer by default
2021-03-19 19:58:13 -03:00
Victor Shyba
dfca15395e
claim id is also a keyword
2021-03-19 19:58:13 -03:00
Victor Shyba
e21f2362fe
apply reorg deletion as well
2021-03-19 19:58:13 -03:00
Victor Shyba
1ce328e8a9
cache signature inspection
2021-03-19 19:58:13 -03:00
Victor Shyba
038a5f999f
cache encoded headers
2021-03-19 19:58:13 -03:00
Victor Shyba
5d3704c7ea
reader mode
2021-03-19 19:58:13 -03:00
Victor Shyba
87037c06c9
remove reader code
2021-03-19 19:58:13 -03:00
Victor Shyba
dd412c0f50
delete sqlite fts
2021-03-19 19:58:13 -03:00
Victor Shyba
bf44befff6
backport fixes from server
2021-03-19 19:58:13 -03:00
Victor Shyba
e61874bb6f
only repeat search if it has blocked items
2021-03-19 19:58:13 -03:00
Victor Shyba
1e5331768f
fix some of the tests
2021-03-19 19:58:13 -03:00
Victor Shyba
ec9a3a4f7c
do not page filtered
2021-03-19 19:58:13 -03:00
Victor Shyba
e439a3a8dc
advanced resolve
2021-03-19 19:58:13 -03:00
Victor Shyba
19f70d7a11
create changelog trigger
2021-03-19 19:58:13 -03:00
Victor Shyba
afe7ed5b05
adjust size
2021-03-19 19:58:13 -03:00
Victor Shyba
d4bf004d74
use a thread pool to sync changes
2021-03-19 19:58:13 -03:00
Victor Shyba
e4d06a088b
include the channel being filtered/blocked
2021-03-19 19:58:13 -03:00
Victor Shyba
0929088b12
missing refresh step
2021-03-19 19:58:13 -03:00
Victor Shyba
7b4838fc9b
dont update more than 400 items a time
2021-03-19 19:58:13 -03:00
Victor Shyba
0cf9533248
narrow update by query
2021-03-19 19:58:13 -03:00
Victor Shyba
84ff0b8a9f
general timeout
2021-03-19 19:58:13 -03:00
Victor Shyba
d467dcfeaf
increase sync queue
2021-03-19 19:58:13 -03:00
Victor Shyba
8e68ba4751
fix join, refresh before update
2021-03-19 19:58:13 -03:00
Victor Shyba
0f2a85ba9f
simplify sync
2021-03-19 19:58:13 -03:00
Victor Shyba
7674a0a91e
backport fixes from testing server
2021-03-19 19:58:13 -03:00
Victor Shyba
5bc1a66572
32 slices and add censor type to fields
2021-03-19 19:58:13 -03:00
Victor Shyba
9b56067213
raise request timeout for content filtering
2021-03-19 19:58:13 -03:00
Victor Shyba
9a9df2fc3c
apply filtering only to whats unfiltered
2021-03-19 19:58:13 -03:00
Victor Shyba
9989d8d1d4
refresh after delete
2021-03-19 19:58:13 -03:00
Victor Shyba
f9471f297e
apply filter and block from ES script lang
2021-03-19 19:58:13 -03:00
Victor Shyba
146b693e4a
exclude title and description
2021-03-19 19:58:13 -03:00
