LBRYCommunity/lbcwallet
1
0
Fork 0
Code Issues 5 Pull requests Projects Releases 14 Packages Wiki Activity
lbcwallet/rpc/documentation/serverchanges.md
Josh Rickmar 497ffc11f0 Modernize the RPC server. 
This is a rather monolithic commit that moves the old RPC server to
its own package (rpc/legacyrpc), introduces a new RPC server using
gRPC (rpc/rpcserver), and provides the ability to defer wallet loading
until request at a later time by an RPC (--noinitialload).

The legacy RPC server remains the default for now while the new gRPC
server is not enabled by default.  Enabling the new server requires
setting a listen address (--experimenalrpclisten).  This experimental
flag is used to effectively feature gate the server until it is ready
to use as a default.  Both RPC servers can be run at the same time,
but require binding to different listen addresses.

In theory, with the legacy RPC server now living in its own package it
should become much easier to unit test the handlers.  This will be
useful for any future changes to the package, as compatibility with
Core's wallet is still desired.

Type safety has also been improved in the legacy RPC server.  Multiple
handler types are now used for methods that do and do not require the
RPC client as a dependency.  This can statically help prevent nil
pointer dereferences, and was very useful for catching bugs during
refactoring.

To synchronize the wallet loading process between the main package
(the default) and through the gRPC WalletLoader service (with the
--noinitialload option), as well as increasing the loose coupling of
packages, a new wallet.Loader type has been added.  All creating and
loading of existing wallets is done through a single Loader instance,
and callbacks can be attached to the instance to run after the wallet
has been opened.  This is how the legacy RPC server is associated with
a loaded wallet, even after the wallet is loaded by a gRPC method in a
completely unrelated package.

Documentation for the new RPC server has been added to the
rpc/documentation directory.  The documentation includes a
specification for the new RPC API, addresses how to make changes to
the server implementation, and provides short example clients in
several different languages.

Some of the new RPC methods are not implementated exactly as described
by the specification.  These are considered bugs with the
implementation, not the spec.  Known bugs are commented as such.
2016-01-29 11:18:26 -05:00

3.8 KiB
Raw Blame History

Making API Changes

This document describes the process of how btcwallet developers must make changes to the RPC API and server. Due to the use of gRPC and Protocol Buffers for the RPC implementation, changes to this API require extra dependencies and steps before changes to the server can be implemented.

Requirements

  • The Protocol Buffer compiler protoc installed with support for the proto3 language

    The protoc tool is part of the Protocol Buffers project. This can be installed from source, from an official binary release, or through an operating system's package manager.

  • The gRPC protoc plugin for Go

    This plugin is written in Go and can be installed using go get:

    go get github.com/golang/protobuf/protoc-gen-go

  • Knowledge of Protocol Buffers version 3 (proto3)

Note that a full installation of gRPC Core is not required, and only the protoc compiler and Go plugins are necessary. This is due to the project using a pure Go gRPC implementation instead of wrapping the C library from gRPC Core.

Step 1: Modify the .proto

Once the developer dependencies have been met, changes can be made to the API by modifying the Protocol Buffers descriptor file api.proto.

The API is versioned according to the rules of Semantic Versioning 2.0. After any changes, bump the API version in the API specification and add the changes to the spec.

Unless backwards compatibility is broken (and the version is bumped to represent this change), message fields must never be removed or changed, and new fields must always be appended.

It is forbidden to use the required attribute on a message field as this can cause errors during parsing when the new API is used by an older client. Instead, the (implicit) optional attribute is used, and the server implementation must return an appropiate error if the new request field is not set to a valid value.

Step 2: Compile the .proto

Once changes to the descriptor file and API specification have been made, the protoc compiler must be used to compile the descriptor into a Go package. This code contains interfaces (stubs) for each service (to be implemented by the wallet) and message types used for each RPC. This same code can also be imported by a Go client that then calls same interface methods to perform RPC with the wallet.

By committing the autogenerated package to the project repo, the proto3 compiler and plugin are not needed by users installing the project by source or by other developers not making changes to the RPC API.

A sh shell script is included to compile the Protocol Buffers descriptor. It must be run from the rpc directory.

$ sh regen.sh

If a sh shell is unavailable, the command can be run manually instead (again from the rpc directory).

protoc -I. api.proto --go_out=plugins=grpc:walletrpc

TODO(jrick): This step could be simplified and be more portable by putting the commands in a Go source file and executing them with go generate. It should, however, only be run when API changes are performed (not with go generate ./... in the project root) since not all developers are expected to have protoc installed.

Step 3: Implement the API change in the RPC server

After the Go code for the API has been regenated, the necessary changes can be implemented in the rpcserver package.

Additional Resources