Add subscribe/unsubscribe RPCs. Add session, sessionManager, and serve JSON RPC (without HTTP). #66

Merged
moodyjon merged 16 commits from blockchain_rpc2 into master 2022-10-04 16:05:06 +02:00
moodyjon commented 2022-09-28 02:03:01 +02:00 (Migrated from github.com)

Fixes #63

I've added a new argument --json-rpc-http-port which will get you the original gorilla/rpc server started on the specified port. Gorilla/rpc serves JSONRPC in the body of HTTP POST. The pure JSON server (TCP without HTTP) listens on port 50001 by default.

I've been manually testing with netcat (nc) like this:

swdev1@Jonathans-Mac-mini herald % stdbuf -i0 -oL -eL  nc localhost 50001
{"jsonrpc":"1.0", "method":"blockchain.address.subscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":40}
{"id":40,"result":[""],"error":null}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":1}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":2}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":3}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":4}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":5}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":6}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":7}
{"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":8}
{"jsonrpc":"1.0", "method":"blockchain.address.unsubscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":41}
{"id":41,"result":null,"error":null}
Fixes #63 I've added a new argument `--json-rpc-http-port` which will get you the original gorilla/rpc server started on the specified port. Gorilla/rpc serves JSONRPC in the body of HTTP POST. The pure JSON server (TCP without HTTP) listens on port 50001 by default. I've been manually testing with netcat (nc) like this: ``` swdev1@Jonathans-Mac-mini herald % stdbuf -i0 -oL -eL nc localhost 50001 {"jsonrpc":"1.0", "method":"blockchain.address.subscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":40} {"id":40,"result":[""],"error":null} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":1} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":2} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":3} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":4} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":5} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":6} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":7} {"method":"blockchain.address.subscribe","params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm","66616b655f737461747573"]],"id":8} {"jsonrpc":"1.0", "method":"blockchain.address.unsubscribe", "params":[["bNe63fYgYNA85ZQ56p7MwBtuCL7MXPRfrm"]], "id":41} {"id":41,"result":null,"error":null} ```
moodyjon (Migrated from github.com) reviewed 2022-09-28 17:13:03 +02:00
moodyjon (Migrated from github.com) commented 2022-09-28 17:13:03 +02:00

This is one of the biggest obstacles to limiting the number of go-routines. I have to call ServeCodec() in a new go-routine in order to allow more than one session at a time, and I don't know how to control/track this routine.

This is one of the biggest obstacles to limiting the number of go-routines. I have to call `ServeCodec()` in a new go-routine in order to allow more than one session at a time, and I don't know how to control/track this routine.
jeffreypicard (Migrated from github.com) reviewed 2022-09-28 17:24:02 +02:00
jeffreypicard (Migrated from github.com) commented 2022-09-28 16:16:43 +02:00

This looks like an infinite for that only triggers if JSONRPCPort != 0, and then breaks unconditionally? Wouldn't an if statement do the same and be less awkward?

This looks like an infinite for that only triggers if JSONRPCPort != 0, and then breaks unconditionally? Wouldn't an if statement do the same and be less awkward?
jeffreypicard (Migrated from github.com) commented 2022-09-28 16:23:42 +02:00

Same thing here, is this for needed?

Same thing here, is this for needed?
moodyjon (Migrated from github.com) reviewed 2022-09-28 18:27:56 +02:00
moodyjon (Migrated from github.com) commented 2022-09-28 18:27:56 +02:00

I found that break only works inside for. That's the reason. I wanted to bail out of the block of code setting up the pure JSON server if anything failed so that it would continue on and set up the alternate HTTP-based server.

I found that `break` only works inside `for`. That's the reason. I wanted to bail out of the block of code setting up the pure JSON server if anything failed so that it would continue on and set up the alternate HTTP-based server.
moodyjon commented 2022-09-29 19:17:31 +02:00 (Migrated from github.com)

Testing showed a problem with how I compute hashX values for addresses. Either the DB produced by python scribe does not contain anything for HashXStatus, or I'm not computing hashX key the exact same way as python.

Otherwise ready for review.

Testing showed a problem with how I compute hashX values for addresses. Either the DB produced by python scribe does not contain anything for HashXStatus, or I'm not computing hashX key the exact same way as python. Otherwise ready for review.
moodyjon commented 2022-09-29 22:16:56 +02:00 (Migrated from github.com)

Got it! 1eb645a0b9

The HashXStatus, HashXMempoolStatus tables are not populated unless scribe --index_address_statuses option was used.

Got it! https://github.com/lbryio/herald.go/pull/66/commits/1eb645a0b95dc0420e8de47e860748aed5729111 The HashXStatus, HashXMempoolStatus tables are not populated unless scribe `--index_address_statuses` option was used.
moodyjon (Migrated from github.com) reviewed 2022-09-30 00:23:29 +02:00
@ -269,0 +283,4 @@
Prefix: []byte{prefixes.HashXStatus},
HashX: hashX,
}
rawKey := key.PackKey()
moodyjon (Migrated from github.com) commented 2022-09-30 00:23:29 +02:00

@jackrobison Can you confirm that the lookup should go to HashXMempoolStatus first, return if found, then go to HashXStatus?

@jackrobison Can you confirm that the lookup should go to HashXMempoolStatus first, return if found, then go to HashXStatus?
jeffreypicard (Migrated from github.com) reviewed 2022-10-03 13:01:57 +02:00
jeffreypicard (Migrated from github.com) commented 2022-10-03 12:59:12 +02:00

Interesting way to keep this running forever, I think it spawns a new goroutine every time, it might be better to just use this in an intfinite loop with a sleep, and maybe parametarize the polling time?

Interesting way to keep this running forever, I think it spawns a new goroutine every time, it might be better to just use this in an intfinite loop with a sleep, and maybe parametarize the polling time?
moodyjon (Migrated from github.com) reviewed 2022-10-03 20:22:33 +02:00
@ -269,0 +283,4 @@
Prefix: []byte{prefixes.HashXStatus},
HashX: hashX,
}
rawKey := key.PackKey()
moodyjon (Migrated from github.com) commented 2022-10-03 20:22:33 +02:00

Resolved at meeting.

Resolved at meeting.
moodyjon (Migrated from github.com) reviewed 2022-10-03 20:40:01 +02:00
moodyjon (Migrated from github.com) commented 2022-10-03 20:40:01 +02:00

Using Ticker object now. The Ticker.Reset() method can be used in unit tests to adjust frequency up/down.

Using Ticker object now. The Ticker.Reset() method can be used in unit tests to adjust frequency up/down.
jeffreypicard (Migrated from github.com) approved these changes 2022-10-04 06:10:48 +02:00
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/herald.go#66
No description provided.