LBRYCommunity/lbry.tech
1
0
Fork 0
Code Issues 13 Pull requests 8 Projects Releases Packages Wiki Activity

integrate spec.lbry.io as iframed page at /spec. Fixes #11. #233

Merged
lyoshenka merged 6 commits from spec into master 2019-01-25 17:43:02 +01:00
Conversation 0 Commits 6 Files changed 8 +4 -362
5 changed files with 4 additions and 362 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files
Showing only changes of commit c672feb4aa - Show all commits

5
app/data/redirects.json
View file

@ -7,5 +7,8 @@
"/tour": "/playground",
"/repository-standards": "/resources/repository-standards",
"/resources/signing-claim": "/resources/claim-signing",
"/whitepaper": "https://spec.lbry.io"
"/whitepaper": "https://spec.lbry.io",
"/resources/uri": "/spec#urls",
"/resources/schema": "/spec#metadata",
"/resources/lbry-claimtrie": "/spec#claimtrie"
}

2
documents/resources.md
View file

@ -6,8 +6,6 @@ title: Resources
## Additional Resources
- [LBRY URIs](/resources/uri)
- [LBRY Claim Metadata Schema](/resources/schema)
- [LBRY Merkle Claim Trie](/resources/claimtrie)
- [LBRY Consensus Algorithm](/resources/consensus)
- [LBRY Glossary](/glossary)

74
documents/resources/lbry-claimtrie.md
View file

@ -1,74 +0,0 @@
# LBRY ClaimTrie
This document describes the implementation detail of the ClaimTrie in LBRY. The ClaimTrie is the data structure which LBRY uses to store claims to names. It uses a [Trie](https://en.wikipedia.org/wiki/Trie) to efficiently store all claimed names, which can then be hashed the same way a [Merkle tree](https://en.wikipedia.org/wiki/Merkle_tree) is hashed. The root hash of the ClaimTrie is stored in the blockheader of each LBRY block, enabling nodes in the LBRY network to efficiently and securely validate the state of the ClaimTrie.
Bids to claim a name must win out against other claims for the same name before they can be inserted into the ClaimTrie. The short summary is that the bid with the most LBRY credits assigned to it will win the right to claim a name, but the implementation detail is more involved and this is what we aim to cover in this document. Bids to claim a name have four properties tied to it :
1. *Name* : The name is a human-readable address and is the property that the bids compete to obtain.
2. *Value* : The value is the data that is attached to the name.
3. *Quantity* : The quantity is the number of LBRY credits assigned to the bid.
4. *Claim Id* : A unique ID used to identify the bid.
There are also three different bid types: claim, update, and support.
1. *Claim*: A claim represents new bids for a name. If a user wants to make a claim to a brand new name, or submit a competing claim to an existing name, this bid type is used.
2. *Support*: A support adds to the total quantity of credits assigned to any bid by referring to a bid's Claim Id. A support bid can be made by anyone on any bid. It does not have its own Value or its own Claim Id, but it does contain the Claim Id of the bid that it is supporting.
3. *Update*: An update can modify the value and the quantity for a pre-existing claim without changing the Claim Id or the name that it is bidding on. Since the Claim Id of the original bid is not changed, an updated bid will still retain all the supports attached to the original bid.
## ClaimTrie Bid States
This section describes how bids are processed by the ClaimTrie in order to determine which bids have won the rights to claim a particular name. There are 6 states a bid can be in, and they are explained below.
1. *Not accepted*: This bid is in a transaction which has not yet been included in a block which has been included in the blockchain.
2. *Accepted*: This bid has been accepted into the blockchain. This happens when the transaction containing the TXout which contains the bid is included in a block which is included in the blockchain.
3. *Active*: This bid is capable of controlling a name. Active bids must be in the "accepted" state and not "expired" or "spent". Bids are "active" when either of the two conditions below is met:
* The current block height exceeds the height of the block at which the bid became accepted plus the activation delay for the name as calculated at either the block at which the bid was accepted or any block after the bid was accepted. The activation delay is calculated as follows:
* If, immediately before this block was included in the blockchain, there were no 'active' bids for the name and therefore no 'controlling' bids, the delay is 0.
* If there is a "controlling" bid for the name: Delay = (HeightB - HeightA) / 32
* HeightA = the most recent height at which the bid controlling the name changed
* HeightB = the current height
* Maximum delay is 7 days of blocks at 2.5 min/block (or 4032 blocks). Thus maximum delay can be reached in 224 (7x32) days.
* The bid's Claim Id matches the Claim Id of the bid which was the controlling bid immediately before the block containing this bid was included in the blockchain. In other words, it is either an update to the previous controlling bid or an update to an update to the previous controlling bid if the bid was updated twice in this block, etc.
4. *Controlling*: This bid currently controls the name. When clients ask which bid controls the name as of the current block, this is the bid that will be returned. Must be in the "active" state and only one bid for any name may be in this state. A support cannot be in the "controlling" state. To determine which "active" bid is the "controlling" bid for each name:
* Add the quantity of each 'active' bid to the quantity of all 'active' supports for that bid, and take whichever is greatest. If two bids have the same quantity, older bids take precedence over newer bids.
* If the bid with the greatest amount does not have the same claimID as the bid which was 'controlling' prior to including the current block, change the delay for the name as of the current block to 0, redetermine which bids and supports should be active, and then perform the previous calculation again.
* At this point, the bid calculated to have the greatest amount behind it is the 'controlling' bid as of this block
5. *Spent*: A transaction has been included in the blockchain which spends the TXout which contains the bid. Must be in the 'accepted' state.
6. *Expired*: All bids 'expire' regardless of what state they are in when the current block height exceeds the height of the block at which the bid was accepted plus 2102400 blocks, or 3650 days or 10 years considering a 2.5 minute block time. (Prior to block 400155 this was set to 262974 blocks, or 456 days, which was changed with a [hardfork](https://github.com/lbryio/lbrycrd/pull/137). Updated claims will restart the expiration timer at the block height of the update.
## ClaimTrie Transaction Implementation
This section describes how the three ClaimTrie bid types are implemented as transactions on the blockchain. Readers should have prior knowledge of Bitcoin [transactions](https://en.bitcoin.it/wiki/Transaction) and the Bitcoin [scripting system](https://en.bitcoin.it/wiki/Script). LBRY supports three op codes that do not exist in Bitcoin: OP_CLAIM_NAME, OP_SUPPORT_CLAIM, and OP_UPDATE_CLAIM (in Bitcoin they are respectively OP_NOP6, OP_NOP7, and OP_NOP8). Each op code will push a zero on to the execution stack, and in addition, will trigger the ClaimTrie to perform calculations necessary for each bid type. Below are the three supported transactions scripts using these op codes.
```python
OP_CLAIM_NAME <Name> <Value> OP_2DROP OP_DROP [script pubkey]
OP_UPDATE_CLAIM <Name> <ClaimId> <Value> OP_2DROP OP_2DROP [script pubkey]
OP_SUPPORT_CLAIM <Name> <ClaimId> OP_2DROP OP_DROP [script pubkey]
```
[script pubkey] can be any valid Bitcoin payout script. Thus it can be something like a standard "pay to pubkey" script to a user controlled address. Also note that the zero pushed onto the stack by the ClaimTrie op codes, and the ClaimTrie vectors, are all dropped by the preceding OP_2DROP and OP_DROP. This means that ClaimTrie transactions exist as prefixes to Bitcoin payout scripts and can be spent in the same way as is expected in Bitcoin.
For example, a claim transaction using a pay to pubkey script will have the below full payout script. Let's also say that this claim is for the name "Fruit" to be set to value "Apple".
```python
OP_CLAIM_NAME <Fruit> <Apple> OP_2DROP OP_DROP OP_DUP OP_HASH160 <LBRY_Address_A> OP_EQUALVERIFY OP_CHECKSIG
```
Like any standard Bitcoin transaction output script, it will be associated with a transaction hash and a transaction output index. The transaction hash and transaction output index is concatenated and hashed using RIPEMD-160 to create the Claim Id for this claim. For the example above, let's say it has a Claim Id X. A support for this bid will have the below full payout script.
```python
OP_SUPPORT_CLAIM <Fruit> <X> OP_2DROP OP_DROP OP_DUP OP_HASH160 <LBRY_Address_B> OP_EQUALVERIFY OP_CHECKSIG
```
And now let's say we want to update the original claim to change the value to "Banana". An update transaction has a special requirement in that it must spend the existing claim that it wishes to update in its redeem script. Otherwise, it will be considered invalid and will not make it into the ClaimTrie. Thus it will have the below redeem script to spend the claim created to set name "Fruit" to "Apple". Note that this is identical to the standard way of redeeming a "pay to pubkey" script in Bitcoin.
```python
<Signature> <Public_key_for_LBRY_Address_A>
```
And the payout script for the update transaction is below.
```python
OP_UPDATE_CLAIM <Fruit> <X> <Banana> OP_2DROP OP_2DROP OP_DUP OP_HASH160 <LBRY_Address_C> OP_EQUALVERIFY OP_CHECKSIG
```

185
documents/resources/schema.md
View file

@ -1,185 +0,0 @@
# LBRY Claim Metadata Schema
The schema defines the structure of the data that is stored in claims in the LBRY blockchain. It has several goals:
- **Extensibility**. The schema could grow to encompass thousands of fields for dozens of types of content. It should be easy to modify the schema while maintaining backward compatibility. Blockchain data is permanent and cannot be migrated, so any selected data structure will have to be maintained forever.
- **Compactness**. Blockchain space is expensive. Data should be stored as compactly as possible.
- **Cross-language Interop**. These definitions will be used by many projects written in different languages.
## [Claim](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/claim.proto)
A `Claim` is the top level schema for everything that is published to the blockchain.
```protobuf
message Claim {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
enum ClaimType {
UNKNOWN_CLAIM_TYPE = 0;
streamType = 1;
certificateType = 2;
}
required ClaimType claimType = 2;
optional Stream stream = 3;
optional Certificate certificate = 4;
optional Signature publisherSignature = 5;
}
```
## Content
### [Stream](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/stream.proto)
All content claims have a Stream field, which includes the content-specific information (e.g. a description of the content, instructions for downloading the content, etc).
```protobuf
message Stream {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required Metadata metadata = 2;
required Source source = 3;
}
```
### [Metadata](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/metadata.proto)
`Metadata` provides information about a piece of content, such as the title, description, and price.
```protobuf
message Metadata {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
_0_0_2 = 2;
_0_0_3 = 3;
_0_1_0 = 4;
}
required Version version = 1;
enum Language {
UNKNOWN_LANGUAGE = 0;
en = 1;
}
required Language language = 2;
required string title = 3;
required string description = 4;
required string author = 5;
required string license = 6;
required bool nsfw = 7;
optional Fee fee = 8;
optional string thumbnail = 9;
optional string preview = 10;
optional string licenseUrl = 11;
}
```
### [Fee](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/fee.proto)
A `Fee` defines the prices for accessing a piece of content.
```protobuf
message Fee {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
enum Currency {
UNKNOWN_CURRENCY = 0;
LBC = 1;
BTC = 2;
USD = 3;
}
required Currency currency = 2;
required bytes address = 3;
required float amount = 4;
}
```
### [Source](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/source.proto)
A `Source` contains information on how to download a stream. Only the LBRY data network is supported at the moment, but other sources may be added in the future.
```protobuf
message Source {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
enum SourceTypes {
UNKNOWN_SOURCE_TYPE = 0;
lbry_sd_hash = 1;
}
required SourceTypes sourceType = 2;
required bytes source = 3;
required string contentType = 4;
}
```
## Channels
Channels are the identity mechanism in LBRY. They are constructed out of Certificates and Signatures. Both utilize a KeyType:
```protobuf
enum KeyType {
UNKNOWN_PUBLIC_KEY_TYPE = 0;
NIST256p = 1;
NIST384p = 2;
SECP256k1 = 3;
}
```
### [Certificate](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/certificate.proto)
Creating a channel involves making a `certificateType` claim. This claim contains the public key for a channel. It must include a Certificate field:
```protobuf
message Certificate {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required KeyType keyType = 2;
required bytes publicKey = 4;
}
```
### [Signature](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/signature.proto)
Publishing a claim to a channel simply means that the claim is signed using the private key for a channel. This is done by including a Signature field in a Claim:
```protobuf
message Signature {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required KeyType signatureType = 2;
required bytes signature = 3;
required bytes certificateId = 4;
}
```

100
documents/resources/uri.md
View file

@ -1,100 +0,0 @@
# URI
## Regex
If you are a robot and prefer regexes to English, here's the full regex for `lbry://` URIs:
```
(?P<uri>
^
(?P<protocol>lbry\:\/\/)?
(?P<content_or_channel_name>
(?P<content_name>[a-zA-Z0-9\-]+)
|
(?P<channel_name>\@[a-zA-Z0-9\-]{4,})
)
(?P<modifier>
(?:\#(?P<claim_id>[0-9a-f]{1,40}))
|
(?:\$(?P<bid_position>\-?[1-9][0-9]*))
|
(?:\:(?P<claim_sequence>\-?[1-9][0-9]*))
)?
(?:\/(?P<path>[a-zA-Z0-9\-]+))?
$
)
```
## Protocol
The LBRY protocol is called `lbry`. URIs using the protocol must start with `lbry://`.
## Reserved characters
- CHANNEL_CHAR = `@`
- CLAIM_ID_CHAR = `#`
- CLAIM_SEQUENCE_CHAR = `:`
- BID_POSITION_CHAR = `$`
- PATH_CHAR = `/`
- QUERY_CHAR = `?`
## Names
Names may contain English letters (upper and lower case), numbers, and hyphens.
### Content Name
`content_name` is the name of a piece of content.
### Channel Name
`channel_name` is the name of a channel (aka publisher identity). It must start with CHANNEL_CHAR,
followed by at least 4 name characters.
## Modifiers
Only one modifier is allowed at a time.
### Claim ID
`claim_id` is a hex string identifying a claim.
A claim id is prefixed with the CLAIM_ID_CHAR.
Partial claim ids are allowed (same is git hashes), and
will resolve to the oldest claim whose id starts with the given characters.
### Claim Sequence
`claim_sequence` is a positive integer (>= 1) that resolves to the Nth claim for a given name.
A claim sequence is prefixed with the CLAIM_SEQUENCE_CHAR.
All valid claims are considered in the order that they appear in the blockchain.
Nonwinning claims are included.
For example, `lbry://@chan:1` resolves to the oldest valid claim for `@chan`, even if that claim is no longer the winning claim for `@chan`.
Negative claim sequence numbers will be supported eventually.
### Bid Position
_not implemented yet_
`bid_position` is a positive integer (>= 1) that resolves to the Nth highest-bid claim for a given name.
A bid position is prefixed with the BID_POSITION_CHAR.
All valid claims are considered, in order from highest bid to lowest bid, with ties being broken by claim age.
Nonwinning claims are included.
For example, `lbry://@chan$1` always resolves to the current winning claim for `@chan`. `@chan` and `@chan$1` are equivalent.
Negative bid position numbers will be supported eventually.
## Path
`path` is a Unix-style path that resolves to a claim within a channel.
A path is prefixed with PATH_CHAR.
Only paths one level deep are currently supported.
Only channel claims may have a path.
For example, `lbry://@chan/snaps_from_last_night` resolves to the claim for `snaps_from_last_night` that is signed by `@chan`.
## Query Params
_not implemented yet_