` in this script may be the same as the address in the original transaction, or it may be a new address.
#### Proof of Payment
No system can strongly enforce digital intellectual property rights, especially not a decentralized one. Therefore, the protocol must be able to produce evidence that differentiates legitimate and illegitimate use. In LBRY, this is done via blockchain transactions and proofs of payment.
A proof of payment has two components:
1. A transaction on the blockchain that spends credits to the fee address for a claim (the transaction must send a number of credits equal to or greater than the fee amount for the claim).
2. Proof that a client knows the private key of the address that the transaction spends from.
To prove 1, it is sufficient to provide the transaction ID and input index of the spend. Proving 2 requires signing a nonce using the associated private key.
Verifying a proof of payment is done as follows:
1. Look up the fee amount and fee address of the claim that the proof is for.
2. Use the transaction ID from the proof to find the transaction. Verify that it spends the correct amount to the correct address.
3. Use the public key from the transaction output to verify the signed nonce.
The protocol is likely to be extended in the future to enable stricter proofs of payment.
### Consensus
In addition to the stake-related changes described above, LBRY makes changes to the following blockchain consensus rules.
#### Block Timing
The target block time was lowered from 10 minutes to 2.5 minutes to facilitate faster transaction confirmation.
#### Difficulty Adjustment
The proof-of-work target is adjusted every block to better adapt to sudden changes in hash rate. The exact adjustment algorithm can be seen [here](https://github.com/lbryio/lbrycrd/blob/master/src/lbry.cpp).
#### Block Hash Algorithm
LBRY uses a combination of SHA-256, SHA-512, and RIPEMD-160. The exact hashing algorithm can be seen [here](https://github.com/lbryio/lbrycrd/blob/master/src/hash.cpp#L18).
#### Block Rewards
The block reward schedule was adjusted to provide an initial testing period, a quick ramp-up to max block rewards, then a logarithmic decay to 0. The source for the algorithm is [here](https://github.com/lbryio/lbrycrd/blob/master/src/main.cpp#L1594).
#### Addresses
The address version byte is set to `0x55` for standard (pay-to-public-key-hash) addresses and `0x7a` for multisig (pay-to-script-hash) addresses. P2PKH addresses start with the letter `b`, and P2SH addresses start with `r`.
All the chain parameters are defined [here](https://github.com/lbryio/lbrycrd/blob/master/src/chainparams.cpp).
## Metadata
Metadata is structured information about a stream or channel separate from the content itself (e.g. the title, language, media type, etc.). It is stored in the blockchain as the [value property](#claim-properties) of a claim.
Metadata is stored in a serialized binary format using [Protocol Buffers](https://developers.google.com/protocol-buffers/). This allows for metadata to be:
- **Extensibile**. Metadata can encompass thousands of fields for dozens of types of content. It must be efficient to both modify the structure and maintain backward compatibility.
- **Compact**. Blockchain space is expensive. Data must be stored as compactly as possible.
- **Interoperabile**. Metadata will be used by many projects written in different languages.
The serialized metadata may be cryptographically signed to indicate membership in a channel. See [Channels](#channels) for more info.
### Specification
The metadata specification is designed to grow and change frequently. The full specification is not detailed here. The [types](https://github.com/lbryio/types) repository is considered the precise specification.
Instead, let's look at an example and some key fields.
#### Example {#metadata-example}
Here’s some example metadata:
```
{
"stream": {
"title": "What is LBRY?",
"author": "Samuel Bryan",
"description": "What is LBRY? An introduction with Alex Tabarrok",
"language": "en",
"license": "Public Domain",
"thumbnail": "https://s3.amazonaws.com/files.lbry.io/logo.png",
"mediaType": "video/mp4",
"streamHash": "232068af6d51325c4821ac897d13d7837265812164021ec832cb7f18b9caf6c77c23016b31bac9747e7d5d9be7f4b752"
}
}
```
Note: Some fields are omitted.
### Key Fields
Some important metadata fields are highlighted below.
#### Stream Hash
A unique identifier that is used to locate and fetch the content from the data network. More in [Data](#data).
#### Fee
Information on how to pay for the content. It includes the address that will receive the payment (the _fee address_), the amount to be paid, and the currency.
Example fee:
```
"fee": {
"address":"bNz8Va7xMyK9eHA5APzLph6cCTjBtGgmDN",
"amount":"99.95",
"currency":"LBC"
}
```
#### Title, Author, Description
Basic information about the stream.
#### Language
The [ISO 639-1](https://www.iso.org/iso-639-language-codes.html) two-letter code for the language of the stream.
#### Thumbnail
A URL to be used to display an image associated with the content.
#### Media Type
The media type of the item as [defined](https://www.iana.org/assignments/media-types/media-types.xhtml) by the IANA.
### Channels (Identities) {#channels}
Channels are the unit of identity. A channel is a claim for a name beginning with `@` that contains a metadata structure for identity rather than content. Included in the metadata is the channel's public key. Here's an example:
```
"claimID": "6e56325c5351ceda2dd0795a30e864492910ccbf",
"name": "@lbry",
"amount": 6.26,
"value": {
"channel": {
"keyType": "SECP256k1",
"publicKey": "3056301006072a8648ce3d020106052b8104000a03420004180488ffcb3d1825af538b0b952f0eba6933faa6d8229609ac0aeadfdbcf49C59363aa5d77ff2b7ff06cddc07116b335a4a0849b1b524a4a69d908d69f1bcebb"
}
}
```
Claims published to a channel contain a signature made with the corresponding private key. A valid signature proves channel membership.
The purpose of channels is to allow content to be clustered under a single pseudonym or identity. This allows publishers to easily list all their content, maintain attribution, and build their brand.
#### Signing
A claim is considered part of a channel when its metadata is signed by the channel's private key. Here's the structure of a signed metadata value:
field | size | description
:--- | :--- | :---
Version | 1 byte | Format version. See [Format Versions](#format-versions).
Channel ID | 20 bytes | Claim ID of the channel claim that contains the matching public key. _Skip this field if there is no signature._
Signature | 64 bytes | The signature. _Skip this field if there is no signature._
Payload | variable | The protobuf-encoded metadata.
##### Format Versions
The following formats are supported:
format | description
:--- | :---
`00000000` | No signature.
`00000001` | Signature using ECDSA SECP256k1 key and SHA-256 hash.
##### Signing Process
1. Encode the metadata using protobuf.
2. Hash the encoded claim using SHA-256.
3. Sign the hash using the private key associated with the channel.
4. Append all the values (the version, the claim ID of the corresponding channel claim, the signature, and the protobuf-encoded metadata).
##### Signature Validation
1. Split out the version from the rest of the data.
1. Check the version field. If it indicates that there is no signature, then no validation is necessary.
1. Split out the channel ID and signature from the rest of the data.
1. Look up the channel claim to ensure it exists and contains a public key.
1. Use the public key to verify the signature.
### Validation {#metadata-validation}
The blockchain treats metadata as an opaque series of bytes. Clients should not trust the metadata they read from the blockchain. Each client is responsible for correctly encoding and decoding the metadata, and for validating its structure and signatures. This allows evolution of the metadata definition without changes to blockchain consensus rules.
## Data
Files published using LBRY are stored in a distributed fashion by the clients participating in the network. Each file is split into many small pieces. Each piece is encrypted and [announced](#announce) to the network. The pieces may also be uploaded to other hosts on the network that specialize in rehosting content.
The purpose of this process is to enable file storage and access without relying on centralized infrastructure, and to create a marketplace for data that allows hosts to be paid for their services. The design is strongly influenced by the [BitTorrent protocol](https://en.wikipedia.org/wiki/BitTorrent).
### Encoding
Content on LBRY is encoded to facilitate distribution.
#### Blobs
The smallest unit of data is called a _blob_. A blob is a chunk of data up to 2MiB in size. Each blob is indexed by its _blob hash_, which is a SHA-384 hash of the blob. Addressing blobs by their hashes protects against naming collisions and ensures that data cannot be accidentally or maliciously modified.
#### Streams
Multiple blobs are combined into a _stream_. A stream may be a book, a movie, a CAD file, etc. All content on the network is shared as streams. Every stream begins with the _manifest blob_, followed by one or more _content blobs_. The content blobs hold the actual content of the stream. The manifest blob contains information necessary to find the content blobs and decode them into a file. This includes the hashes of the content blobs, their order in the stream, and cryptographic material for decrypting them.
Content blobs are encrypted using AES-256 in CBC mode and PKCS7 padding. In order to keep each encrypted blob at 2MiB max, a blob can hold at most 2097151 bytes (2MiB minus 1 byte) of plaintext data. The source code for the exact algorithm is available [here](https://github.com/lbryio/lbry.go/blob/master/stream/blob.go). The encryption key and the initialization vectors for each blob are stored in the manifest blob.
The blob hash of the manifest blob is called the _stream hash_. It uniquely identifies each stream.
#### Manifest Contents
A manifest blob's contents are encoded using [canonical JSON encoding](http://wiki.laptop.org/go/Canonical_JSON). The JSON encoding must be canonical to support consistent hashing and validation. Here's an example manifest:
```
{"blobs":[{"blob_hash":"a6daea71be2bb89fab29a2a10face08143411a5245edcaa5efff48c2e459e7ec01ad20edfde6da43a932aca45b2cec61","iv":"ef6caef207a207ca5b14c0282d25ce21","length":2097152},{"blob_hash":"bf2717e2c445052366d35bcd58edb108cbe947af122d8f76b4856db577aeeaa2def5b57dbb80f7b1531296bd3e0256fc","iv":"a37b291a37337fc1ff90ae655c244c1d","length":2097152},...,{"blob_hash":"322973617221ddfec6e53bff4b74b9c21c968cd32ba5a5094d84210e660c4b2ed0882b114a2392a08b06183f19330aaf","iv": "a00f5f458695bdc9d50d3dbbc7905abc","length":600160}],"filename":"6b706a7977755477704d632e6d7034","key":"94d89c0493c576057ac5f32eb0871180","version":1}
```
Here's the same manifest, with whitespace added for readability:
```
{
"blobs":[
{
"blobHash":"a6daea71be2bb89fab29a2a10face08143411a5245edcaa5efff48c2e459e7ec01ad20edfde6da43a932aca45b2cec61",
"iv":"ef6caef207a207ca5b14c0282d25ce21",
"length":2097152
},
{
"blobHash":"bf2717e2c445052366d35bcd58edb108cbe947af122d8f76b4856db577aeeaa2def5b57dbb80f7b1531296bd3e0256fc",
"iv":"a37b291a37337fc1ff90ae655c244c1d",
"length":2097152
},
...,
{
"blobHash":"322973617221ddfec6e53bff4b74b9c21c968cd32ba5a5094d84210e660c4b2ed0882b114a2392a08b06183f19330aaf",
"iv": "a00f5f458695bdc9d50d3dbbc7905abc",
"length": 600160
}
],
"filename":"6b706a7977755477704d632e6d7034",
"key":"94d89c0493c576057ac5f32eb0871180",
"version":1
}
```
The `blobs` field is an ordered list of blobs in the stream. Each item in the list has the blob hash for that blob, the hex-encoded initialization vector used to create the blob, and the length of the encrypted blob (not the original file chunk).
The `filename` is the hex-encoded name of the original file.
The `key` field contains the hex-encoded _stream key_, which is used to decrypt the blobs in the stream. This field is optional. The stream key may instead be stored by a third party and made available to a client when presented with proof that the content was purchased.
The `version` field is always 1. It is intended to signal structure changes in future versions of this protocol.
Every stream must have at least two blobs - the manifest blob and a content blob. Consequently, zero-length streams are not allowed.
#### Stream Encoding
A file must be encoded into a stream before it can be published. Encoding involves breaking the file into chunks, encrypting the chunks into content blobs, and creating the manifest blob. Here are the steps:
##### Setup
1. Generate a random 32-byte stream key. This key will be used to encrypt each content blob in the stream.
##### Content Blobs
1. Break the file into chunks of at most 2097151 bytes.
1. Generate a random 32-byte initialization vector (IV) for each chuck.
1. Pad each chunk using PKCS7 padding.
1. Encrypt each chunk with AES-CBC using the stream key and the IV for that chunk.
1. An encrypted chunk is a blob.
##### Manifest Blob
1. Fill in the manifest data as described in the [Manifest Contents](#manifest-contents).
2. Encode the data using the canonical JSON encoding.
3. Compute the stream hash.
An implementation of this process is available [here](https://github.com/lbryio/lbry.go/tree/master/stream).
#### Stream Decoding
Decoding a stream is like encoding in reverse, and with the added step of verifying that the expected blob hashes match the actual data.
1. Verify that the hash of the manifest blob and matches the stream hash.
2. Parse the JSON in manifest blob.
3. Verify the hashes of the content blobs.
4. Decrypt and remove the padding from each content blob using the stream key and IVs in the manifest.
5. Concatenate the decrypted chunks in order.
### Announce
After a stream is encoded, it must be _announced_ to the network. Announcing is the process of letting other nodes on the network know that a client has content available for download. LBRY tracks announced content using a distributed hash table.
#### Distributed Hash Table
_Distributed hash tables_ (or DHTs) are an effective way to build a peer-to-peer content network. LBRY's DHT implementation follows the [Kademlia](https://pdos.csail.mit.edu/~petar/papers/maymounkov-kademlia-lncs.pdf)
specification fairly closely, with some modifications.
A distributed hash table is a key-value store that is spread over multiple nodes in a network. Nodes may join or leave the network anytime, with no central coordination necessary. Nodes communicate with each other using a peer-to-peer protocol to advertise what data they have and what they are best positioned to store.
When a host connects to the DHT, it announces the hash for every blob it wishes to share. Downloading a blob from the network requires querying the DHT for a list of hosts that announced that blob’s hash (called _peers_), then requesting the blob from the peers directly.
#### Announcing to the DHT
A host announces a hash to the DHT in two steps. First, the host looks for nodes that are closest to the target hash. Then the host asks those nodes to store the fact that the host has the target hash available for download.
Finding the closest nodes is done via iterative `FindNode` DHT requests. The host starts with the closest nodes it knows about and sends a `FindNode(target_hash)` request to each of them. If any of the requests return nodes that are closer to the target hash, the host sends `FindNode` requests to those nodes to try to get even closer. When the `FindNode` requests no longer return nodes that are closer, the search ends.
Once the search is over, the host sends a `Store(target_hash)` request to the closest several nodes it found. The nodes receiving this request store the fact that the host is a peer for the target hash.
### Download
A client wishing to download a stream must first query the DHT to find peers hosting the blobs in that stream, then contact those peers to download the blobs directly.
#### Querying the DHT
Querying works almost the same way as announcing. A client looking for a target hash starts by sending iterative `FindValue(target_hash)` requests to the nodes it knows that are closest to the target hash. If a node receives a `FindValue` request and knows of any peers for the target hash, it responds with a list of those peers. Otherwise, it responds with the closest nodes to the target hash that it knows about. The client then queries those closer nodes using the same `FindValue` call. This way, each call either finds the client some peers, or brings it closer to finding those peers. If no peers are found and no closer nodes are being returned, the client determines that the target hash is not available and gives up.
#### Blob Exchange Protocol
Downloading a blob from a peer is governed by the _Blob Exchange Protocol_. It is used by hosts and clients to exchange blobs and check data pricing and blob availability. The protocol is an RPC protocol using Protocol Buffers and the gRPC framework. It has five types of requests.
##### PriceCheck
PriceCheck gets the price that the server is charging for data transfer. It returns the price in LBC per KB.
##### DownloadCheck
DownloadCheck checks whether the server has certain blobs available for download. For each hash in the request, the server returns a true or false to indicate whether the blob is available.
##### Download
Download requests the blob for a given hash. The response contains the blob, its hash, and the address where to send payment for the data transfer. If the blob is not available on the server, the response instead contains an error.
##### UploadCheck
UploadCheck asks the server whether blobs can be uploaded to it. For each hash in the request, the server returns a true or false to indicate whether it would accept a given blob for upload. In addition, if any of the hashes in the request is a stream hash and the server has the manifest blob for that stream but is missing some content blobs, it may include the hashes of those content blobs in the response.
##### Upload
Upload sends a blob to the server. If uploading many blobs, the client should use the UploadCheck request to check which blobs the server actually needs. This avoids needlessly uploading blobs that the server already has. If a client tries to upload too many blobs that the server does not want, the server may consider it a denial of service attack.
The protocol methods and message types are defined in detail [here](https://github.com/lbryio/lbry.go/blob/master/blobex/blobex.proto).
### Reflectors and Data Markets
In order for a client to download content, there must be hosts online that have the content the client wants, when the client wants it. To incentivize the continued hosting of data, the blob exchange protocol supports data upload and payment for data. _Reflectors_ are hosts that accept data uploads. They rehost (reflect) the uploaded data and charge for downloads.
Using a reflector is optional, but most publishers will probably choose to use them. Doing so obviates the need for the publisher's server to be online and connectable, which can be especially useful for mobile clients or those behind a firewall.
The current version of the protocol does not support sophisticated price negotiation between clients and hosts. The host simply chooses the price it wants to charge. Clients check this price before downloading, and pay the price after the download is complete. Future protocol versions will include more options for price negotiation, as well as stronger proofs of payment.
## Appendix
### Claim Activation Example
Here is a step-by-step example to illustrate how competing claims activate and are ordered. All stakes are for the same name.
**Block 13:** Claim A for 10LBC is accepted. It is the first claim, so it immediately becomes active and controlling.
State: A(10) is controlling
**Block 1001:** Claim B for 20LBC is accepted. Its activation height is `1001 + min(4032, floor((1001-13) / 32)) = 1001 + 30 = 1031`.
State: A(10) is controlling, B(20) is accepted.
**Block 1010:** Support X for 14LBC for claim A is accepted. Since it is a support for the controlling claim, it activates immediately.
State: A(10+14) is controlling, B(20) is accepted.
**Block 1020:** Claim C for 50LBC is accepted. The activation height is `1020 + min(4032, floor((1020-13) / 32)) = 1020 + 31 = 1051`.
State: A(10+14) is controlling, B(20) is accepted, C(50) is accepted.
**Block 1031:** Claim B activates. It has 20LBC, while claim A has 24LBC (10 original + 14 from support X). There is no takeover, and claim A remains controlling.
State: A(10+14) is controlling, B(20) is active, C(50) is accepted.
**Block 1040:** Claim D for 300LBC is accepted. The activation height is `1040 + min(4032, floor((1040-13) / 32)) = 1040 + 32 = 1072`.
State: A(10+14) is controlling, B(20) is active, C(50) is accepted, D(300) is accepted.
**Block 1051:** Claim C activates. It has 50LBC, while claim A has 24LBC, so a takeover is initiated. The takeover height for this name is set to 1051, and therefore the activation delay for all the claims becomes `min(4032, floor((1051-1051) / 32)) = 0`. All the claims become active. The totals for each claim are recalculated, and claim D becomes controlling because it has the highest total.
State: A(10+14) is active, B(20) is active, C(50) is active, D(300) is controlling.
### URL Resolution Examples
Suppose the following names were claimed in the following order and no other claims exist.
Channel Name | Stream Name | Claim ID | Amount
:--- | :--- | :--- | :---
_--_ | apple | 690eea | 1
_--_ | banana | 714a3f | 2
_--_ | cherry | bfaabb | 100
_--_ | apple | 690eea | 10
@Arthur | _--_ | b7bab5 | 1
@Bryan | _--_ | 0da517 | 1
@Chris | _--_ | b3f7b1 | 1
@Chris | banana | fc861c | 1
@Arthur | apple | 37ee1 | 20
@Bryan | cherry | a18bca | 10
@Chris | _--_ | 005a7d | 100
@Arthur | cherry | d39aa0 | 20
Here is how the following URLs resolve:
URL | Claim ID
:--- | :---
`lbry://apple` | a37ee1
`lbry://banana` | 714a3f
`lbry://@Chris` | 005a7d
`lbry://@Chris/banana` | _not found_ (the controlling `@Chris` does not have a `banana`)
`lbry://@Chris:1/banana` | fc861c
`lbry://@Chris:#fc8/banana` | fc861c
`lbry://cherry` | bfaabb
`lbry://@Arthur/cherry` | d39aa0
`lbry://@Bryan` | 0da517
`lbry://banana$1` | 714a3f
`lbry://banana$2` | fc861c
`lbry://banana$3` | _not found_
`lbry://@Arthur:1` | b7bab5
### Additional Resources
- [lbry.tech](https://lbry.tech) is LBRY's website designed specifically for anyone capable of understanding this document.
- [github.com/lbryio](https://github.com/lbryio) contains MIT-licensed source code and implementation details for everything explained by this document.
- [lbry.io](https://lbry.io) is LBRY's consumer, creator, and end-user website.
- [lbry.io/get](https://lbry.io/get) provides downloads for LBRY's end-user applications. They're significantly less stuffy than this spec.
- [lbry.tech/resources](https://lbry.tech/resources) provides API specifications, additional resources, and other items of technical interest.
- [lbry.tech/contribute](https://lbry.tech/contribute) explains how you can get involved. If you're still reading at this point then you're probably interested, but if you're on the fence then [please watch this video](http://spee.ch/doit.mp4).
- [lbry.tech/community](https://lbry.tech/community) lists how you can interact with us, other users, and keep up-to-date.
++
:+++++
+++++++++
'++++++++++++`
.++++++',++++++++`
+++++++ .++++++++.
;++++++: `++++++++,
+++++++ ++++++++,
+++++++` ++++++++:
,++++++' ++++++++;
+++++++ ++++++++'
'++++++, ++++++++'
`+++++++ '++++++++
+++++++ '++++++++
:++++++; ;++++++++
+++++++ :++++++++
'++++++. ,++++++++
.++++++' ,++++++++`
+++++++ .++++++++`
;++++++, `+++++++
`+++++++ `+++++
+++++++` ++++
:++++++; +++++
+++++++ ,++++++
'++++++. +++++++
.++++++' '++++++,
+++++++ `+++++++
+++++: +++++++
++++ ;++++++:
++++ +++++++
++++ ++ +++++++`
++++ ++++ ,++++++' .:
++++ ++++++ +++++++ :'++++++++++
++++ ++++++++ '++++++. ++++++++++
++++ :++++++++ .++++++' .+++++++++
++++ :++++++++ +++++++ ++++++++.
++++ ,++++++++ ;++++++: :++++++++
++++ ,++++++++ `+++++++ +++++++++,
++++ ,++++++++ +++++++` +++++++++++
+++++. .++++++++` :++++++; ,+++++++ +++;
+++++++. .++++++++` +++++++ +++++++ ++
++++++++, `++++++++` '++++++. '++++++: ,'
++++++++, `++++++++` .++++++' .+++++++
++++++++, `++++++++. +++++++ +++++++`
++++++++, ++++++++.'++++++, ;++++++;
++++++++: +++++++++++++ `+++++++
++++++++: +++++++++` +++++++.
++++++++: +++++; ,++++++'
++++++++: ++ +++++++
++++++++; '++++++,
'+++++++; .+++++++
'+++++++; +++++++
'+++++++; ;++++++:
'+++++++' `+++++++
'+++++++' +++++++`
'+++++++' :++++++;
;+++++++' +++++++
;+++++++++++++.
;+++++++++'
;++++++
:++,
---
_[Edit this on Github](https://github.com/lbryio/spec/blob/master/index.md)_