Update README!

This commit is contained in:
Daniel Krol 2022-06-15 22:24:23 -04:00
parent 64597dcc42
commit 21c1ed01de
2 changed files with 165 additions and 105 deletions

View file

@ -1,17 +1,15 @@
# Test Client
A couple example flows so it's clear how it works.
A couple example flows so it's clear how it works. We're assuming that we're starting with a fresh DB on the server, and that we've created two wallets on the SDK: `"test_wallet_1"` and `"test_wallet_2"`.
## Initial setup and account recovery
Set up two clients with the same account (which won't exist on the server yet).
Set up a client for each wallet, but with the same sync account (which won't exist on the server yet). This will simulate clients on two different computers.
```
>>> from test_client import Client
>>> c1 = Client()
>>> c2 = Client()
>>> c1.set_account("joe2@example.com", "123abc2")
>>> c2.set_account("joe2@example.com", "123abc2")
>>> c1 = Client("joe2@example.com", "123abc2", 'test_wallet_1')
>>> c2 = Client("joe2@example.com", "123abc2", 'test_wallet_2')
```
Register the account on the server with one of the clients.
@ -25,31 +23,36 @@ Now that the account exists, grab an auth token with both clients.
```
>>> c1.get_auth_token()
Got auth token: b646a357038d394ef7b70f350c666aeb29e24072c7eeaaad4fb6759c1fca281a
Got auth token: 3d98076fda58400f3dbd5ea6511184507d5f8637bd5549e5cb0cc9cdbb7102e5
>>> c2.get_auth_token()
Got auth token: bd24ae67cb1bde2da8a27f2c2e5ec8ff1f9bebccb11e16dd35aea31bf422133c
Got auth token: 1385a51bf3ba86a3e1f412908c3b2165cc399e06692a2dc602f5e17fe2c7993c
```
## Syncing
Create a new wallet + metadata (we'll wrap it in a struct we'll call `WalletState` in this client) and POST them to the server. The metadata (as of now) in the walletstate is only `sequence`. This increments for every POSTed wallet. This is bookkeeping to prevent certain syncing errors.
Create a new wallet + metadata (we'll wrap it in a struct we'll call `WalletState` in this client) using `init_wallet_state` and POST them to the server. The metadata (as of now) in the walletstate is only `sequence`. `sequence` is an integer that increments for every POSTed wallet. This is bookkeeping to prevent certain syncing errors.
Note that after POSTing, it says it "got" a new wallet. This is because the POST endpoint also returns the latest version. The purpose of this will be explained in "Conflicts" below.
_Note that after POSTing, it says it "got" a new wallet. This is because the POST endpoint also returns the latest version. The purpose of this will be explained in "Conflicts" below._
```
>>> c1.new_wallet_state()
>>> c1.update_wallet()
>>> c1.init_wallet_state()
Wallet not found
No wallet found on the server for this account. Starting a new one.
>>> c1.update_remote_wallet()
Successfully updated wallet state on server
Got new walletState:
WalletState(sequence=1, encrypted_wallet='-')
WalletState(sequence=1, encrypted_wallet='czo4MTkyOjE2OjE6ew8QGI/89sz70Oud6NljymaLSUCyNSBYwpTCBZu9gMbwXYuDKqB4YnZeYRJdHhXz+9NQ9qSkRUIPHQ4m6f38R38KvXCE5raRnnozrnmDOt/eGFUl9XYMrFcYqgqYSCxb1kTcWS1cWkkOO6TtrjeBKuc+qriKZr9ggk1pnLmnKQc=')
'Success'
```
With the other client, GET it from the server. Note that both clients have the same data now.
Now, call `init_wallet_state` with the other client. This time, `init_wallet_state` will GET the wallet from the server. In general, `init_wallet_state` is used to set up a new client; first it checks the server, then failing that, it initializes it locally. (In a real client, it would save the walletstate to disk, and `init_wallet_state` would check there before checking the server).
(There are a few potential unresolved issues surrounding this related to sequence of events. Check comments on `init_wallet_state`. SDK again works around them with the timestamps.)
```
>>> c2.get_wallet()
>>> c2.init_wallet_state()
Got latest walletState:
WalletState(sequence=1, encrypted_wallet='-')
WalletState(sequence=1, encrypted_wallet='czo4MTkyOjE2OjE6ew8QGI/89sz70Oud6NljymaLSUCyNSBYwpTCBZu9gMbwXYuDKqB4YnZeYRJdHhXz+9NQ9qSkRUIPHQ4m6f38R38KvXCE5raRnnozrnmDOt/eGFUl9XYMrFcYqgqYSCxb1kTcWS1cWkkOO6TtrjeBKuc+qriKZr9ggk1pnLmnKQc=')
```
## Updating
@ -57,39 +60,47 @@ WalletState(sequence=1, encrypted_wallet='-')
Push a new version, GET it with the other client. Even though we haven't edited the encrypted wallet yet, we can still increment the sequence number.
```
>>> c2.update_wallet()
>>> c2.update_remote_wallet()
Successfully updated wallet state on server
Got new walletState:
WalletState(sequence=2, encrypted_wallet='-')
>>> c1.get_wallet()
WalletState(sequence=2, encrypted_wallet='czo4MTkyOjE2OjE6DAT6j6JSp0by78XpOOMtGroxFUX5vh6X+oXhIVlHVhvmVgp+09vWt7IP/IGofP4Ua7Dggr9iyxF4A3F9tSNgxKrev08eMP+8W2LAwk3jTAtZPoh5vtz/20tJFWOw+Y+s00NRNXcDeT8GjZvgTfawy+k7WKQMt6ryW6c8liORDfA=')
'Success'
>>> c1.get_remote_wallet()
Got latest walletState:
WalletState(sequence=2, encrypted_wallet='-')
WalletState(sequence=2, encrypted_wallet='czo4MTkyOjE2OjE6DAT6j6JSp0by78XpOOMtGroxFUX5vh6X+oXhIVlHVhvmVgp+09vWt7IP/IGofP4Ua7Dggr9iyxF4A3F9tSNgxKrev08eMP+8W2LAwk3jTAtZPoh5vtz/20tJFWOw+Y+s00NRNXcDeT8GjZvgTfawy+k7WKQMt6ryW6c8liORDfA=')
'Success'
```
## Wallet Changes
For demo purposes, this test client represents each change to the wallet by appending segments separated by `:` so that we can more easily follow the history. (The real app will not actually edit the wallet in the form of an append log.)
We'll track changes to the wallet by changing and looking at preferences in the locally saved wallet. We see that both clients have settings blank. We change a preference on one client:
```
>>> c1.cur_encrypted_wallet()
'-'
>>> c1.change_encrypted_wallet()
>>> c1.cur_encrypted_wallet()
'-:a776'
>>> c1.get_preferences()
{'animal': '', 'car': ''}
>>> c2.get_preferences()
{'animal': '', 'car': ''}
>>> c1.set_preference('animal', 'cow')
{'animal': 'cow'}
>>> c1.get_preferences()
{'animal': 'cow', 'car': ''}
```
The wallet is synced between the clients.
The wallet is synced between the clients. The client with the changed preference sends its wallet to the server, and the other one GETs it locally.
```
>>> c1.update_wallet()
>>> c1.update_remote_wallet()
Successfully updated wallet state on server
Got new walletState:
WalletState(sequence=3, encrypted_wallet='-:a776')
>>> c2.get_wallet()
WalletState(sequence=3, encrypted_wallet='czo4MTkyOjE2OjE6SQ/9PBDeOs8sOG+QDnOBmgbOKJDUx3TJD1p9r/bIuD2R5lamKmn1UKz/fQynLJexPJj3QCJP5u5OKTZDMBhY6HF5qBV2ndnWmPLjB40KlGj7jjZJaETEMktyJjjKdLbsV8nKLpnB2KpyYZejJVppBS+DRswAFByTE6c5E+8FJ3TTPXhzTvE9L3RqvetQEUxn')
'Success'
>>> c2.get_remote_wallet()
Got latest walletState:
WalletState(sequence=3, encrypted_wallet='-:a776')
>>> c2.cur_encrypted_wallet()
'-:a776'
WalletState(sequence=3, encrypted_wallet='czo4MTkyOjE2OjE6SQ/9PBDeOs8sOG+QDnOBmgbOKJDUx3TJD1p9r/bIuD2R5lamKmn1UKz/fQynLJexPJj3QCJP5u5OKTZDMBhY6HF5qBV2ndnWmPLjB40KlGj7jjZJaETEMktyJjjKdLbsV8nKLpnB2KpyYZejJVppBS+DRswAFByTE6c5E+8FJ3TTPXhzTvE9L3RqvetQEUxn')
'Success'
>>> c2.get_preferences()
{'animal': 'cow', 'car': ''}
```
## Merging Changes
@ -97,73 +108,97 @@ WalletState(sequence=3, encrypted_wallet='-:a776')
Both clients create changes. They now have diverging wallets.
```
>>> c1.change_encrypted_wallet()
>>> c2.change_encrypted_wallet()
>>> c1.cur_encrypted_wallet()
'-:a776:8fc8'
>>> c2.cur_encrypted_wallet()
'-:a776:2433'
>>> c1.set_preference('car', 'Audi')
{'car': 'Audi'}
>>> c2.set_preference('animal', 'horse')
{'animal': 'horse'}
>>> c1.get_preferences()
{'animal': 'cow', 'car': 'Audi'}
>>> c2.get_preferences()
{'animal': 'horse', 'car': ''}
```
One client POSTs its change first.
```
>>> c1.update_wallet()
>>> c1.update_remote_wallet()
Successfully updated wallet state on server
Got new walletState:
WalletState(sequence=4, encrypted_wallet='-:a776:8fc8')
WalletState(sequence=4, encrypted_wallet='czo4MTkyOjE2OjE62uWympFofMnLmZSGGPTC5qctGKlWkan/DmOFLVZHktzqY9OndxhY3VCr5QBMXOGyn/Y321zNtL6YNfA+gs3Ov6qhzcneERHJM3ByySjMPwEds4NVDctKW4HAXggZIA1xhga1XlNggrBXlu09Sqro9zEbJdrBwJQI6BeuZHpH2eaJBDI73ljTWgtqoIeLg1WA')
'Success'
```
The other client pulls that change, and _merges_ those changes on top of the changes it had saved locally.
The other client pulls that change, and _merges_ those changes on top of the changes it had saved locally. For now, the SDK merges the preferences based on timestamps internal to the wallet.
The _merge base_ that a given client uses is the last version that it successfully got from or POSTed to the server. You can see the merge base here: `"-:a776"`, the first part of the wallet which both clients had in common before the merge.
Eventually, the client will be responsible (or at least more responsible) for merging. At this point, the _merge base_ that a given client will use is the last version that it successfully GETed from POSTed to the server. It's the last common version between the client merging and the client that created the wallet version on the server.
```
>>> c2.get_wallet()
>>> c2.get_remote_wallet()
Got latest walletState:
WalletState(sequence=4, encrypted_wallet='-:a776:8fc8')
>>> c2.cur_encrypted_wallet()
'-:a776:8fc8:2433'
WalletState(sequence=4, encrypted_wallet='czo4MTkyOjE2OjE62uWympFofMnLmZSGGPTC5qctGKlWkan/DmOFLVZHktzqY9OndxhY3VCr5QBMXOGyn/Y321zNtL6YNfA+gs3Ov6qhzcneERHJM3ByySjMPwEds4NVDctKW4HAXggZIA1xhga1XlNggrBXlu09Sqro9zEbJdrBwJQI6BeuZHpH2eaJBDI73ljTWgtqoIeLg1WA')
'Success'
>>> c2.get_preferences()
{'animal': 'horse', 'car': 'Audi'}
```
Finally, the client with the merged wallet pushes it to the server, and the other client GETs the update.
```
>>> c2.update_wallet()
>>> c2.update_remote_wallet()
Successfully updated wallet state on server
Got new walletState:
WalletState(sequence=5, encrypted_wallet='-:a776:8fc8:2433')
>>> c1.get_wallet()
WalletState(sequence=5, encrypted_wallet='czo4MTkyOjE2OjE6ngb8TU1FyKgmzyHLQ8c30yOg/kVFNSDbtquXHKs16vEIQta3zrJLnGiY0WoiXx8Ul4uvYLK1riNaoo+OfZYtJjtpYLWf1oGdn0PDq0ZCHhK6GcX2Zbz/YQEdPcOvDeENjxZ4Pq2qoZYSDcPvwOgbvO2FSOK27OhCWHCA/9LbzAu6Suq6RS3i2p2TpmUHtz2H')
'Success'
>>> c1.get_remote_wallet()
Got latest walletState:
WalletState(sequence=5, encrypted_wallet='-:a776:8fc8:2433')
>>> c1.cur_encrypted_wallet()
'-:a776:8fc8:2433'
WalletState(sequence=5, encrypted_wallet='czo4MTkyOjE2OjE6ngb8TU1FyKgmzyHLQ8c30yOg/kVFNSDbtquXHKs16vEIQta3zrJLnGiY0WoiXx8Ul4uvYLK1riNaoo+OfZYtJjtpYLWf1oGdn0PDq0ZCHhK6GcX2Zbz/YQEdPcOvDeENjxZ4Pq2qoZYSDcPvwOgbvO2FSOK27OhCWHCA/9LbzAu6Suq6RS3i2p2TpmUHtz2H')
'Success'
>>> c1.get_preferences()
{'animal': 'horse', 'car': 'Audi'}
```
Note that we're sidestepping the question of merging different changes to the same preference. The SDK resolves this, again, by timestamps. But ideally we would resolve such an issue with a user interaction (particularly if one of the changes involves _deleting_ the preference altogether). Using timestamps as the SDK does is a holdover from the current system, so we won't distract ourselves by demonstrating it here.
## Conflicts
A client cannot POST if it is not up to date. It needs to merge in any new changes on the server before POSTing its own changes. For convenience, if a conflicting POST request is made, the server responds with the latest version of the wallet state (just like a GET request). This way the client doesn't need to make a second request to perform the merge.
(If a non-conflicting POST request is made, it responds with the same wallet state that the client just POSTed, as it is now the server's current wallet state)
So for example, let's say we create diverging changes in the wallets:
```
>>> c2.change_encrypted_wallet()
>>> c2.update_wallet()
>>> _ = c2.set_preference('animal', 'beaver')
>>> _ = c1.set_preference('car', 'Toyota')
>>> c2.get_preferences()
{'animal': 'beaver', 'car': 'Audi'}
>>> c1.get_preferences()
{'animal': 'horse', 'car': 'Toyota'}
```
We try to POST both of them to the server, but the second one fails because of the conflict. Instead, merges the two locally:
```
>>> c2.update_remote_wallet()
Successfully updated wallet state on server
Got new walletState:
WalletState(sequence=6, encrypted_wallet='-:a776:8fc8:2433:0FdD')
>>> c1.change_encrypted_wallet()
>>> c1.update_wallet()
WalletState(sequence=6, encrypted_wallet='czo4MTkyOjE2OjE6MnLcl2+VTv8B9MIKJjpwptjF8Ws6NfhFkFBnsTDy8arv7akMSV/jojkvz2bJzOjX+iAKiY0+FKgD2akONsUnQqF95pnbr+TPnpbFxS4TLFUWxbpJMm7+r3FZiOauMZ6ewBfBq3vzI2UA2o3RrSxzucKZ6ZcgZqJsKCnk+rCj/ADmrUJb01kwB6WDATcMlG5A')
'Success'
>>> c1.update_remote_wallet()
Wallet state out of date. Getting updated wallet state. Try posting again after this.
Got new walletState:
WalletState(sequence=6, encrypted_wallet='-:a776:8fc8:2433:0FdD')
WalletState(sequence=6, encrypted_wallet='czo4MTkyOjE2OjE6MnLcl2+VTv8B9MIKJjpwptjF8Ws6NfhFkFBnsTDy8arv7akMSV/jojkvz2bJzOjX+iAKiY0+FKgD2akONsUnQqF95pnbr+TPnpbFxS4TLFUWxbpJMm7+r3FZiOauMZ6ewBfBq3vzI2UA2o3RrSxzucKZ6ZcgZqJsKCnk+rCj/ADmrUJb01kwB6WDATcMlG5A')
'Success'
>>> c1.get_preferences()
{'animal': 'beaver', 'car': 'Toyota'}
```
Now the merge is complete, and the client can make a second POST request containing the merged wallet.
Now that the merge is complete, the client can make a second POST request containing the merged wallet.
```
>>> c1.update_wallet()
>>> c1.update_remote_wallet()
Successfully updated wallet state on server
Got new walletState:
WalletState(sequence=7, encrypted_wallet='-:a776:8fc8:2433:0FdD:BA43')
WalletState(sequence=7, encrypted_wallet='czo4MTkyOjE2OjE6uexO9yl0JVsKFo6WeJGOsJ/sm1RJPc+NwLxniaE744lVEihK2HyNxDVbcAFEMxn/vKXgFKtzLV/D7eAzeGrSIyQR5v3YeZrTWPcRzK79rJgHzjcZpjKpytcDMZp2lB5cRHkNg7u8qAa2DnbebMXU0CKblTL++IIteU+CzyuTdW1Uoj4cEOsy6G8HwrZc5drf')
'Success'
```

View file

@ -1,6 +1,13 @@
# Generate the README since I want real behavior interspersed with comments
# Come to think of it, this is accidentally a pretty okay integration test for client and server
# NOTE - delete the database before running this, or else you'll get an error for registering. also we want the wallet to start empty
# NOTE - in the SDK, create wallets called "test_wallet_1" and "test_wallet_2"
from test_client import LBRYSDK
# reset all of the preferences so we can run our example
for wallet in ['test_wallet_1', 'test_wallet_2']:
for pref_key in ['car', 'animal']:
LBRYSDK.set_preference(wallet, pref_key, '')
def code_block(code):
print ("```")
@ -16,20 +23,18 @@ def code_block(code):
print("""# Test Client
A couple example flows so it's clear how it works.
A couple example flows so it's clear how it works. We're assuming that we're starting with a fresh DB on the server, and that we've created two wallets on the SDK: `"test_wallet_1"` and `"test_wallet_2"`.
""")
print("""## Initial setup and account recovery
Set up two clients with the same account (which won't exist on the server yet).
Set up a client for each wallet, but with the same sync account (which won't exist on the server yet). This will simulate clients on two different computers.
""")
code_block("""
from test_client import Client
c1 = Client()
c2 = Client()
c1.set_account("joe2@example.com", "123abc2")
c2.set_account("joe2@example.com", "123abc2")
c1 = Client("joe2@example.com", "123abc2", 'test_wallet_1')
c2 = Client("joe2@example.com", "123abc2", 'test_wallet_2')
""")
print("""
@ -52,22 +57,24 @@ c2.get_auth_token()
print("""
## Syncing
Create a new wallet + metadata (we'll wrap it in a struct we'll call `WalletState` in this client) and POST them to the server. The metadata (as of now) in the walletstate is only `sequence`. This increments for every POSTed wallet. This is bookkeeping to prevent certain syncing errors.
Create a new wallet + metadata (we'll wrap it in a struct we'll call `WalletState` in this client) using `init_wallet_state` and POST them to the server. The metadata (as of now) in the walletstate is only `sequence`. `sequence` is an integer that increments for every POSTed wallet. This is bookkeeping to prevent certain syncing errors.
Note that after POSTing, it says it "got" a new wallet. This is because the POST endpoint also returns the latest version. The purpose of this will be explained in "Conflicts" below.
_Note that after POSTing, it says it "got" a new wallet. This is because the POST endpoint also returns the latest version. The purpose of this will be explained in "Conflicts" below._
""")
code_block("""
c1.new_wallet_state()
c1.update_wallet()
c1.init_wallet_state()
c1.update_remote_wallet()
""")
print("""
With the other client, GET it from the server. Note that both clients have the same data now.
Now, call `init_wallet_state` with the other client. This time, `init_wallet_state` will GET the wallet from the server. In general, `init_wallet_state` is used to set up a new client; first it checks the server, then failing that, it initializes it locally. (In a real client, it would save the walletstate to disk, and `init_wallet_state` would check there before checking the server).
(There are a few potential unresolved issues surrounding this related to sequence of events. Check comments on `init_wallet_state`. SDK again works around them with the timestamps.)
""")
code_block("""
c2.get_wallet()
c2.init_wallet_state()
""")
print("""
@ -77,30 +84,32 @@ Push a new version, GET it with the other client. Even though we haven't edited
""")
code_block("""
c2.update_wallet()
c1.get_wallet()
c2.update_remote_wallet()
c1.get_remote_wallet()
""")
print("""
## Wallet Changes
For demo purposes, this test client represents each change to the wallet by appending segments separated by `:` so that we can more easily follow the history. (The real app will not actually edit the wallet in the form of an append log.)
We'll track changes to the wallet by changing and looking at preferences in the locally saved wallet. We see that both clients have settings blank. We change a preference on one client:
""")
code_block("""
c1.cur_encrypted_wallet()
c1.change_encrypted_wallet()
c1.cur_encrypted_wallet()
c1.get_preferences()
c2.get_preferences()
c1.set_preference('animal', 'cow')
c1.get_preferences()
""")
print("""
The wallet is synced between the clients.
The wallet is synced between the clients. The client with the changed preference sends its wallet to the server, and the other one GETs it locally.
""")
code_block("""
c1.update_wallet()
c2.get_wallet()
c2.cur_encrypted_wallet()
c1.update_remote_wallet()
c2.get_remote_wallet()
c2.get_preferences()
""")
print("""
@ -109,13 +118,11 @@ print("""
Both clients create changes. They now have diverging wallets.
""")
merge_base = c2.cur_encrypted_wallet()
code_block("""
c1.change_encrypted_wallet()
c2.change_encrypted_wallet()
c1.cur_encrypted_wallet()
c2.cur_encrypted_wallet()
c1.set_preference('car', 'Audi')
c2.set_preference('animal', 'horse')
c1.get_preferences()
c2.get_preferences()
""")
print("""
@ -123,18 +130,18 @@ One client POSTs its change first.
""")
code_block("""
c1.update_wallet()
c1.update_remote_wallet()
""")
print("""
The other client pulls that change, and _merges_ those changes on top of the changes it had saved locally.
The other client pulls that change, and _merges_ those changes on top of the changes it had saved locally. For now, the SDK merges the preferences based on timestamps internal to the wallet.
The _merge base_ that a given client uses is the last version that it successfully got from or POSTed to the server. You can see the merge base here: `"%s"`, the first part of the wallet which both clients had in common before the merge.
""" % merge_base)
Eventually, the client will be responsible (or at least more responsible) for merging. At this point, the _merge base_ that a given client will use is the last version that it successfully GETed from POSTed to the server. It's the last common version between the client merging and the client that created the wallet version on the server.
""")
code_block("""
c2.get_wallet()
c2.cur_encrypted_wallet()
c2.get_remote_wallet()
c2.get_preferences()
""")
print("""
@ -142,30 +149,48 @@ Finally, the client with the merged wallet pushes it to the server, and the othe
""")
code_block("""
c2.update_wallet()
c1.get_wallet()
c1.cur_encrypted_wallet()
c2.update_remote_wallet()
c1.get_remote_wallet()
c1.get_preferences()
""")
# Make sure the next preference changes have a later timestamp!
import time
time.sleep(3)
print("""
Note that we're sidestepping the question of merging different changes to the same preference. The SDK resolves this, again, by timestamps. But ideally we would resolve such an issue with a user interaction (particularly if one of the changes involves _deleting_ the preference altogether). Using timestamps as the SDK does is a holdover from the current system, so we won't distract ourselves by demonstrating it here.
## Conflicts
A client cannot POST if it is not up to date. It needs to merge in any new changes on the server before POSTing its own changes. For convenience, if a conflicting POST request is made, the server responds with the latest version of the wallet state (just like a GET request). This way the client doesn't need to make a second request to perform the merge.
(If a non-conflicting POST request is made, it responds with the same wallet state that the client just POSTed, as it is now the server's current wallet state)
So for example, let's say we create diverging changes in the wallets:
""")
code_block("""
c2.change_encrypted_wallet()
c2.update_wallet()
c1.change_encrypted_wallet()
c1.update_wallet()
_ = c2.set_preference('animal', 'beaver')
_ = c1.set_preference('car', 'Toyota')
c2.get_preferences()
c1.get_preferences()
""")
print("""
Now the merge is complete, and the client can make a second POST request containing the merged wallet.
We try to POST both of them to the server, but the second one fails because of the conflict. Instead, merges the two locally:
""")
code_block("""
c1.update_wallet()
c2.update_remote_wallet()
c1.update_remote_wallet()
c1.get_preferences()
""")
print("""
Now that the merge is complete, the client can make a second POST request containing the merged wallet.
""")
code_block("""
c1.update_remote_wallet()
""")