LBRYCommunity/herald.go
1
0
Fork 0
Code Issues 16 Pull requests Packages Projects Releases 14 Wiki Activity

Add subscribe/unsubscribe RPCs. Add session, sessionManager, and serve JSON RPC (without HTTP). #66

Merged
moodyjon merged 16 commits from blockchain_rpc2 into master 2022-10-04 16:05:06 +02:00
Conversation 2 Commits 16 Files changed 11 +1058 -156
moodyjon commented 2022-09-28 02:03:01 +02:00 (Migrated from github.com)
Copy link

Fixes #63

I've added a new argument --json-rpc-http-port which will get you the original gorilla/rpc server started on the specified port. Gorilla/rpc serves JSONRPC in the body of HTTP POST. The pure JSON server (TCP without HTTP) listens on port 50001 by default.

I've been manually testing with netcat (nc) like this:

swdev1@Jonathans-Mac-mini herald % stdbuf -i0 -oL -eL  nc localhost 50001
{"jsonrpc":"1.0", "method":"blockchain.address.subscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":40}
{"id":40,"result":[""],"error":null}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":1}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":2}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":3}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":4}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":5}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":6}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":7}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":8}
{"jsonrpc":"1.0", "method":"blockchain.address.unsubscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":41}
{"id":41,"result":null,"error":null}
Fixes #63 I've added a new argument `--json-rpc-http-port` which will get you the original gorilla/rpc server started on the specified port. Gorilla/rpc serves JSONRPC in the body of HTTP POST. The pure JSON server (TCP without HTTP) listens on port 50001 by default. I've been manually testing with netcat (nc) like this: ``` swdev1@Jonathans-Mac-mini herald % stdbuf -i0 -oL -eL nc localhost 50001 {"jsonrpc":"1.0", "method":"blockchain.address.subscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":40} {"id":40,"result":[""],"error":null} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":1} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":2} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":3} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":4} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":5} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":6} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":7} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":8} {"jsonrpc":"1.0", "method":"blockchain.address.unsubscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":41} {"id":41,"result":null,"error":null} ```
moodyjon (Migrated from github.com) reviewed 2022-09-28 17:13:03 +02:00
server/session.go
moodyjon (Migrated from github.com) commented 2022-09-28 17:13:03 +02:00
Copy link

This is one of the biggest obstacles to limiting the number of go-routines. I have to call ServeCodec() in a new go-routine in order to allow more than one session at a time, and I don't know how to control/track this routine.

This is one of the biggest obstacles to limiting the number of go-routines. I have to call `ServeCodec()` in a new go-routine in order to allow more than one session at a time, and I don't know how to control/track this routine.
jeffreypicard (Migrated from github.com) reviewed 2022-09-28 17:24:02 +02:00
server/jsonrpc_service.go
jeffreypicard (Migrated from github.com) commented 2022-09-28 16:16:43 +02:00
Copy link

This looks like an infinite for that only triggers if JSONRPCPort != 0, and then breaks unconditionally? Wouldn't an if statement do the same and be less awkward?

This looks like an infinite for that only triggers if JSONRPCPort != 0, and then breaks unconditionally? Wouldn't an if statement do the same and be less awkward?
jeffreypicard (Migrated from github.com) commented 2022-09-28 16:23:42 +02:00
Copy link

Same thing here, is this for needed?

Same thing here, is this for needed?
moodyjon (Migrated from github.com) reviewed 2022-09-28 18:27:56 +02:00
server/jsonrpc_service.go
moodyjon (Migrated from github.com) commented 2022-09-28 18:27:56 +02:00
Copy link

I found that break only works inside for. That's the reason. I wanted to bail out of the block of code setting up the pure JSON server if anything failed so that it would continue on and set up the alternate HTTP-based server.

I found that `break` only works inside `for`. That's the reason. I wanted to bail out of the block of code setting up the pure JSON server if anything failed so that it would continue on and set up the alternate HTTP-based server.
moodyjon commented 2022-09-29 19:17:31 +02:00 (Migrated from github.com)
Copy link

Testing showed a problem with how I compute hashX values for addresses. Either the DB produced by python scribe does not contain anything for HashXStatus, or I'm not computing hashX key the exact same way as python.

Otherwise ready for review.

Testing showed a problem with how I compute hashX values for addresses. Either the DB produced by python scribe does not contain anything for HashXStatus, or I'm not computing hashX key the exact same way as python. Otherwise ready for review.
moodyjon commented 2022-09-29 22:16:56 +02:00 (Migrated from github.com)
Copy link

Got it! 1eb645a0b9

The HashXStatus, HashXMempoolStatus tables are not populated unless scribe --index_address_statuses option was used.

Got it! https://github.com/lbryio/herald.go/pull/66/commits/1eb645a0b95dc0420e8de47e860748aed5729111 The HashXStatus, HashXMempoolStatus tables are not populated unless scribe `--index_address_statuses` option was used.
moodyjon (Migrated from github.com) reviewed 2022-09-30 00:23:29 +02:00
db/db_get.go
@ -269,0 +283,4 @@
Prefix: []byte{prefixes.HashXStatus},
HashX: hashX,
}
rawKey := key.PackKey()
moodyjon (Migrated from github.com) commented 2022-09-30 00:23:29 +02:00
Copy link

@jackrobison Can you confirm that the lookup should go to HashXMempoolStatus first, return if found, then go to HashXStatus?

@jackrobison Can you confirm that the lookup should go to HashXMempoolStatus first, return if found, then go to HashXStatus?
jeffreypicard (Migrated from github.com) reviewed 2022-10-03 13:01:57 +02:00
server/session.go
jeffreypicard (Migrated from github.com) commented 2022-10-03 12:59:12 +02:00
Copy link

Interesting way to keep this running forever, I think it spawns a new goroutine every time, it might be better to just use this in an intfinite loop with a sleep, and maybe parametarize the polling time?

Interesting way to keep this running forever, I think it spawns a new goroutine every time, it might be better to just use this in an intfinite loop with a sleep, and maybe parametarize the polling time?
moodyjon (Migrated from github.com) reviewed 2022-10-03 20:22:33 +02:00
db/db_get.go
@ -269,0 +283,4 @@
Prefix: []byte{prefixes.HashXStatus},
HashX: hashX,
}
rawKey := key.PackKey()
moodyjon (Migrated from github.com) commented 2022-10-03 20:22:33 +02:00
Copy link

Resolved at meeting.

Resolved at meeting.
moodyjon (Migrated from github.com) reviewed 2022-10-03 20:40:01 +02:00
server/session.go
moodyjon (Migrated from github.com) commented 2022-10-03 20:40:01 +02:00
Copy link

Using Ticker object now. The Ticker.Reset() method can be used in unit tests to adjust frequency up/down.

Using Ticker object now. The Ticker.Reset() method can be used in unit tests to adjust frequency up/down.
jeffreypicard (Migrated from github.com) approved these changes 2022-10-04 06:10:48 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
jeffreypicard
Labels
Clear labels   
consider soon

Discuss this issue at the next planning meeting, then remove this label

  
documentation

Improvements or additions to documentation

  
good first issue

Good for newcomers

  
hacktoberfest

Welcome to Hacktoberfest

  
help wanted

Extra attention is needed

  
priority: blocker

Issue is blocking release, do ASAP

  
priority: high

Work needs to be moved into sprint ASAP

  
priority: low

Work should be done but can stay in the backlog for now

  
priority: medium

Work needs to be done within 2-3 sprints

  
type: bug

Existing functionality is wrong or broken

  
type: discussion

A conversation. No specific changes requested

  
type: improvement

Existing (or partially existing) functionality needs to be changed

  
type: new feature

New functionality that does not exist yet

  
type: refactor

Minimal user-visible changes, but significant internal work

  
type: task

Either work that's not related to the code, or a small chore that does not fit into other categories

No labels
consider soon
documentation
good first issue
hacktoberfest
help wanted
priority: blocker
priority: high
priority: low
priority: medium
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/herald.go#66
No description provided.
Delete branch "blockchain_rpc2"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?