Commit graph

33 commits

Author SHA1 Message Date
Niko Storni
f1ad94d29d fix es storage object 2019-10-30 17:57:40 +01:00
Niko Storni
e7cf20eda4 address review comments 2019-10-15 00:04:47 +02:00
Niko Storni
9934258046 remove unused promises 2019-10-14 16:40:19 +02:00
Niko Storni
7bdf527ec9 feedback on drained queue 2019-10-14 16:40:19 +02:00
Niko Storni
42f2a6386c possibly improve memory management 2019-10-14 16:40:19 +02:00
Niko Storni
79b9eda568 don't spam slack 2019-10-14 16:40:19 +02:00
Niko Storni
c485b89972 construct object from mysql 2019-10-14 16:40:19 +02:00
Niko Storni
02f322059d parse json 2019-10-14 16:40:19 +02:00
Niko Storni
469ac7329a add debugging 2019-10-14 16:40:19 +02:00
Niko Storni
ba258cae03 replace API with db connector 2019-10-14 16:40:19 +02:00
Mark Beamer Jr
4b88db1f55
Cleanup the sync
Add syncstate to status call
Remove old sync logic

Signed-off-by: Mark Beamer Jr <markbeamerjr@gmail.com>
2019-09-21 18:47:23 -04:00
Mark Beamer Jr
42bec75ef9
Set last sync time to start of sync when doing a long running sync.
Signed-off-by: Mark Beamer Jr <markbeamerjr@gmail.com>
2019-09-21 18:09:11 -04:00
Mark Beamer Jr
162107239b
Add capability to restart sync where it left off by storing state of the sync. Set to update
Signed-off-by: Mark Beamer Jr <markbeamerjr@gmail.com>
2019-09-21 17:25:18 -04:00
Mark Beamer Jr
c67f298ee6
Add channel id to claim if exists for more precise filtering 2019-09-19 22:15:52 -04:00
Thomas Zarebczan
c002a09d7e fix: eslint problems 2019-09-18 12:47:21 -04:00
Niko Storni
b873d93bf6 .io to .com 2019-07-08 20:00:10 +02:00
Mark Beamer Jr
db39a6c47f
skip bad claim when processing to ensure all future claims make it into elastic search. 2019-06-22 16:37:39 -04:00
Mark Beamer Jr
11d000119b
check for null value to notify lbry team. 2019-06-22 16:17:40 -04:00
Mark Beamer Jr
ea2fcb56ab
log error when step in sync process fails. 2019-06-21 00:11:16 -04:00
Mark Beamer Jr
4f5206d18e
Check for Claim field 2019-06-20 23:26:36 -04:00
Mark Beamer Jr
5a9ce272b9
Added reverted changes back in 2019-02-26 08:50:26 -05:00
Mark
79a2470504
Revert "added channel effective amount to the claim weight." 2019-02-24 22:05:39 -05:00
Mark Beamer Jr
101e55d924
added channel effective amount to the claim weight. 2019-02-24 17:03:50 -05:00
Mark Beamer Jr
81986315cb
Batched the chainquery calls for lighthouse in case it needs to grab all 700K in a redo. 2019-02-24 16:57:56 -05:00
Mark Beamer Jr
965746b9d2
added slack integration 2018-11-04 01:01:23 -04:00
filipnyquist
6b3269bfe3 Added the breaking change of #93. Added the change, do not merge until the chainquery change has been merged and deployed. 2018-07-30 15:19:29 +02:00
filipnyquist
4423fc2cfc This PR adds channels into the index, this combined with the added channel filtering parameter allows us to search specific channels for content.
While adding the code for filtering I split the search query into individual parts so it easier can be edited and read.
2018-07-23 12:06:47 +02:00
Mark Beamer Jr
cd0f980904 -Refactored handling of blocked claims.
-Changed deletes to be queued and refactored.
-Added logging.
-Removed commented debug code.
-Added Spent and Expired bid_state handling.
2018-05-26 22:56:09 -04:00
Mark Beamer Jr
78e26b2e6e -Parameterized elastic search logging to a const. 2018-05-26 22:49:29 -04:00
Mark Beamer Jr
bb8525f69e -Fixed bug that causes completionmapper parse error in elastic search. This causes an entire batch to fail. Batches are up to 500 so if 1 fails they all fail. The impact is that many claims were not indexed in elastic search. 2018-05-26 22:47:26 -04:00
Mark Beamer Jr
4e1808c585 -fixed util import for chainquery sync
-added https for blocked files api
-added comma to commented section so query is intact if uncommented.
2018-05-15 00:03:07 -04:00
Mark Beamer Jr
c6893991c3 -removed comma on last column of query.
-removed useless console log.
-added more understandable logging for which claims are pushed to elastic search.
2018-05-14 21:28:14 -04:00
Mark Beamer Jr
b1c7f8b8ca -chainquery integration. Instead of using lbrycrd directly, use the chainquery app as the intermediary. 2018-05-12 19:35:56 -04:00