Compare commits

...

42 commits

Author SHA1 Message Date
Alex Grin
0983f336f3
Merge pull request #31 from Pigges/patch-1 2022-01-03 14:10:40 -05:00
Pigges
1e7563bf96
Update index.md
Changed in the Stakes section where there apparently where two a "A _stake_ is a a single "
2022-01-02 16:25:05 +01:00
Alex Grin
54f1477e6f
Merge pull request #29 from tsheaff/typos-2021-09-23 2021-09-24 10:51:12 -04:00
Tyler Sheaffer
0239bafaed fix small typos on stream decoding steps 2021-09-23 13:58:59 -10:00
Tyler Sheaffer
242485b59e Remove 4th newlines to increase consistency 2021-09-23 13:58:46 -10:00
Tyler Sheaffer
c47376250b Fix Extensible and Interoperable typos 2021-09-23 13:58:27 -10:00
Alex Grin
536239a1df
fix typo
fixes #28
2021-08-30 12:51:47 -04:00
Alex Grintsvayg
569ee3a9db
pgp 2021-05-03 18:06:39 -04:00
Alex Grintsvayg
bf4a17d373
Revert "Fix grammar: add missing determiner."
This reverts commit 96e673a344.
2021-04-05 15:31:45 -04:00
Alex Grin
2dc8d00124
Merge pull request #26 from stefansundin/master 2021-04-05 15:31:21 -04:00
Stefan Sundin
795122bc02 Fix grammar: add comma after conjunctive adverb. 2021-04-02 16:57:50 -07:00
Stefan Sundin
a80cad7064 Fix grammar: it's -> its. 2021-04-02 16:57:50 -07:00
Stefan Sundin
96e673a344 Fix grammar: add missing determiner. 2021-04-02 16:57:11 -07:00
Stefan Sundin
885df96b8f Fix typo: charater -> character. 2021-04-02 16:50:09 -07:00
Alex Grin
63244fec75
Merge pull request #24 from eatdostacos/patch-1 2020-06-03 09:12:19 -04:00
Alex Grin
bfe024b69a
Update index.md 2020-06-03 09:11:40 -04:00
eatdostacos
4343ad23a3
Update status and some other small changes 2020-06-01 22:16:37 -07:00
Alex Grintsvayg
86543087f9
fix broken lbrycrd links
fixes #22
2020-04-29 20:28:35 -04:00
Thomas Zarebczan
a2a62cf65d
Merge pull request #23 from half-duplex/patch-1
Fix `apple` Claim ID
2020-04-26 22:51:34 -04:00
mal
b0b73cdf5a
Fix apple Claim ID 2020-04-27 02:26:14 +00:00
Alex Grintsvayg
500066a889
update spec to deprecate #, replace it with :, and replace that with * 2020-02-28 17:06:30 -05:00
Thomas Zarebczan
e442497799
Merge pull request #20 from ykris45/patch-1
update year
2020-02-03 17:04:16 -05:00
YULIUS KURNIAWAN KRISTIANTO
73e091b39c
Update LICENSE 2020-02-03 04:09:54 +07:00
Alex Grintsvayg
01f66f90f9
pdf update 2019-12-09 15:38:30 -05:00
Alex Grintsvayg
d0a6fd3cac
Merge branch 'signing'
* signing:
  clarify claim signing. fixes #16
2019-12-09 15:37:09 -05:00
Alex Grintsvayg
1639fbc400
clarify claim signing. fixes #16 2019-12-04 16:19:15 -05:00
Alex Grintsvayg
5dfdaad1aa
minor style improvements 2019-10-16 16:46:39 -04:00
Thomas Zarebczan
b8b1287190
Merge pull request #18 from StrikerRUS/patch-1
bump year in license
2019-10-15 11:51:10 -04:00
Nikita Titov
a94cea6735
bump year in license 2019-10-08 23:00:04 +03:00
Thomas Zarebczan
aea931f545
Merge pull request #14 from m160/patch-1
Update index.md
2019-08-21 22:18:24 +02:00
Michael H
7e99003625
Update index.md 2019-06-21 19:04:26 +02:00
Alex Grintsvayg
02273d189d
explain old expiration behavior 2019-05-22 17:04:20 -04:00
Alex Grintsvayg
6b11b95f20
pdf update 2019-05-08 13:37:46 -04:00
Alex Grin
7a12a32457
Update index.md 2019-05-07 12:48:10 -04:00
Alex Grintsvayg
c46e37dd34
lbry.com switch 2019-03-18 14:27:55 -04:00
Niko
99d6b6d7a5 Update CNAME 2019-03-18 14:18:06 -04:00
Alex Grintsvayg
ffa196a55d
update pdf 2019-02-22 14:53:17 -05:00
Alex Grintsvayg
b96e5e8665
add justification 2019-02-20 11:10:59 -05:00
Alex Grintsvayg
34f2373aea
fix external link icon 2019-02-19 16:11:45 -05:00
Alex Grintsvayg
935719b272
words 2019-02-19 15:45:38 -05:00
Alex Grintsvayg
eeab53c485
additional resources. closes #10 2019-02-19 13:06:49 -05:00
Alex Grintsvayg
cad57bd3f0
rename bid position. we don't use 'bid' anywhere else 2019-02-19 13:06:49 -05:00
8 changed files with 104 additions and 83 deletions

2
CNAME
View file

@ -1 +1 @@
spec.lbry.io spec.lbry.com

View file

@ -1,6 +1,6 @@
The MIT License (MIT) The MIT License (MIT)
Copyright (c) 2015-2018 LBRY Inc Copyright (c) 2015-2020 LBRY Inc
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the
"Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish,

View file

@ -1,5 +1,5 @@
title: LBRY Protocol Specification title: LBRY Protocol Specification
email: grin@lbry.io email: grin@lbry.com
baseurl: "" # the subpath of your site, e.g. /blog baseurl: "" # the subpath of your site, e.g. /blog
url: "" # the base hostname & protocol for your site, e.g. http://example.com url: "" # the base hostname & protocol for your site, e.g. http://example.com

View file

@ -15,7 +15,7 @@
<main> <main>
<h1>LBRY: A Decentralized Digital Content Marketplace</h1> <h1>LBRY: A Decentralized Digital Content Marketplace</h1>
<div class="byline"> <div class="byline">
Alex Grintsvayg (<a href="mailto:grin@lbry.io">grin@lbry.io</a>), Jeremy Kauffman (<a href="mailto:jeremy@lbry.io">jeremy@lbry.io</a>) Alex Grintsvayg (<a href="mailto:grin@lbry.com">grin@lbry.com</a>), Jeremy Kauffman (<a href="mailto:jeremy@lbry.com">jeremy@lbry.com</a>)
</div> </div>
<a href="/lbry-spec.pdf" class="pdf-hide" target="_blank" rel="noopener">PDF Version</a> <a href="/lbry-spec.pdf" class="pdf-hide" target="_blank" rel="noopener">PDF Version</a>
<div class="toc-menu pdf-hide">Menu</div> <div class="toc-menu pdf-hide">Menu</div>

View file

@ -106,16 +106,18 @@ h1 {
} }
h2 { h2 {
font-size: 3rem; font-size: 3rem;
border-bottom: solid 1px black;
padding-bottom: 5px;
} }
h3 { h3 {
font-size: 2.25rem; font-size: 2.5rem;
} }
h4 { h4 {
font-size: 2rem; font-size: 2rem;
margin-bottom: 1.5rem; margin-bottom: 1.5rem;
} }
h5 { h5 {
font-size: 1.75rem; font-size: 1.5rem;
margin-bottom: 1rem; margin-bottom: 1rem;
} }
@ -124,7 +126,7 @@ a {
color: #0074D9; color: #0074D9;
} }
a.external-link::after { a.external-link::after {
content: url('data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 576 512" color="#0074D9" width="13px" height="13px" class="ext-link-icon"><path fill="currentColor" d="M576 24v127.984c0 21.461-25.96 31.98-40.971 16.971l-35.707-35.709-243.523 243.523c-9.373 9.373-24.568 9.373-33.941 0l-22.627-22.627c-9.373-9.373-9.373-24.569 0-33.941L442.756 76.676l-35.703-35.705C391.982 25.9 402.656 0 424.024 0H552c13.255 0 24 10.745 24 24zM407.029 270.794l-16 16A23.999 23.999 0 0 0 384 303.765V448H64V128h264a24.003 24.003 0 0 0 16.97-7.029l16-16C376.089 89.851 365.381 64 344 64H48C21.49 64 0 85.49 0 112v352c0 26.51 21.49 48 48 48h352c26.51 0 48-21.49 48-48V287.764c0-21.382-25.852-32.09-40.971-16.97z" class=""></path></svg>'); content: url('data:image/svg+xml;utf8,<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 576 512" color="rgb(0,116,217)" width="13px" height="13px" class="ext-link-icon"><path fill="currentColor" d="M576 24v127.984c0 21.461-25.96 31.98-40.971 16.971l-35.707-35.709-243.523 243.523c-9.373 9.373-24.568 9.373-33.941 0l-22.627-22.627c-9.373-9.373-9.373-24.569 0-33.941L442.756 76.676l-35.703-35.705C391.982 25.9 402.656 0 424.024 0H552c13.255 0 24 10.745 24 24zM407.029 270.794l-16 16A23.999 23.999 0 0 0 384 303.765V448H64V128h264a24.003 24.003 0 0 0 16.97-7.029l16-16C376.089 89.851 365.381 64 344 64H48C21.49 64 0 85.49 0 112v352c0 26.51 21.49 48 48 48h352c26.51 0 48-21.49 48-48V287.764c0-21.382-25.852-32.09-40.971-16.97z" class=""></path></svg>');
padding-left: 3px; padding-left: 3px;
} }
@ -183,9 +185,18 @@ table {
table tr { table tr {
border-bottom: 1px solid #dee2e6; border-bottom: 1px solid #dee2e6;
} }
td {
border-left: solid 1px #dee2e6;
border-right: solid 1px #dee2e6;
}
th, td { th, td {
padding-left: 5px; padding-left: 5px;
} }
th {
font-size: 1.5rem;
font-weight: normal;
text-transform: uppercase;
}
table thead tr { table thead tr {
border-bottom: 2px solid #dee2e6; border-bottom: 2px solid #dee2e6;
} }

146
index.md
View file

@ -8,19 +8,23 @@ LBRY is a protocol for accessing and publishing digital content in a global, dec
Clients can use LBRY to publish, host, find, download, and pay for content — books, movies, music, or anything else that can be represented as a stream of bits. The protocol is permissionless and censorship-resistant, which means that participation is open to everyone and no one can unilaterally block or remove content. Clients can use LBRY to publish, host, find, download, and pay for content — books, movies, music, or anything else that can be represented as a stream of bits. The protocol is permissionless and censorship-resistant, which means that participation is open to everyone and no one can unilaterally block or remove content.
LBRY is a step forward from previous generations of decentralized networks, which provide no discovery or payment mechanisms. For creators, LBRY is unparalleled in trust and earning potential. For consumers, LBRY is the first system that provides end-to-end digital content consumption that does not require trusting a third-party. For the world, LBRY is designed to engender the most complete catalog of information to ever exist, and to be controlled by the only party that could be trusted with such monumental responsibility: no one. Before LBRY, publishers had to choose between a centralized host such as Amazon or Youtube, or a protocol like Bittorrent. Centralized platforms suffer from several problems because their incentives are not aligned with the incentives of their users. Hosts engage in rent-seeking behavior, often extracting 30-55% of creator profits. They enforce opaque and arbitrary rules on creators, and change those rules without warning or community input. They choose to censor content at the behest of repressive regimes around the world, in exchange for access to more users and higher profits for themselves.
### Assumptions Bittorrent does not have these faults, but it has problems of its own. It is only useful if one already knows the infohash of the content they seek, and there is no way to discover these hashes within the protocol. Even using an external search engine does not provide a comprehensive list of what is available on the network. There are no incentives for users to seed content, and Bittorrent largely works because users earn status through private communities, are nice, or simply fail to understand what their client is doing. Finally, a lot of content on BitTorrent infringes on copyright, which taints the protocol's public perception and overshadows the many positives it has.
This document assumes that the reader is familiar with distributed hash tables (DHTs), the BitTorrent protocol, Bitcoin, and blockchain technology in general. It does not attempt to document these technologies or explain how they work. The [Bitcoin developer reference](https://bitcoin.org/en/developer-reference) and [BitTorrent protocol specification](http://www.bittorrent.org/beps/bep_0003.html) are recommended for anyone wishing to understand the technical details. LBRY offers a significant improvement over both options. It uses a blockchain to provide the good parts of a centralized host (a single place to store data, find interesting content, build a brand, and get rewarded for contributing), while removing the downsides (opaque and arbitrary rules, rent extraction, censorship). It is public and no one can be censored or blocked from using it. Its rules are clearly defined and cannot be changed without community consensus. The blockchain records everything that is published to LBRY, so interesting content is easy to find and infringing content is difficult to hide. Accessing the blockchain data is free, the costs for downloading content are transparent, and publishers earn 100% of the price they set.
### Status
LBRY has been in public use since June 2016. As of May 2020, over 3.3 million pieces of digital content have been published via the protocol. Millions of users access this content each month, downloading and uploading terabytes of data. Graphical browsers and wallets are available for all major operating systems and can be downloaded at [lbry.com/get](https://lbry.com/get).
### Overview ### Overview
This document defines the LBRY protocol, its components, and how they fit together. LBRY consists of several discrete components that are used together in order to provide the end-to-end capabilities of the protocol. There are two distributed data stores (blockchain and DHT), a peer-to-peer protocol for exchanging data, and specifications for data structure, encoding, and retrieval. This document defines the LBRY protocol, its components, and how they fit together. LBRY consists of several discrete components that are used together in order to provide the end-to-end capabilities of the protocol. There are two distributed data stores (blockchain and DHT), a peer-to-peer protocol for exchanging data, and specifications for data structure, encoding, and retrieval.
### Status ### Assumptions
LBRY has been in public use since June 2016. As of February 2019, approximately 750,000 pieces of digital content have been published via the protocol. Tens of thousands of users access hundreds of thousands of pieces of content each month, downloading and uploading terrabytes of data. Graphical browsers and wallets are available for all major operating systems and can be downloaded on LBRY's user-facing portal at [lbry.io/get](https://lbry.io/get). This document assumes that the reader is familiar with distributed hash tables (DHTs), the BitTorrent protocol, Bitcoin, and blockchain technology in general. It does not attempt to document these technologies or explain how they work. The [Bitcoin developer reference](https://bitcoin.org/en/developer-reference) and [BitTorrent protocol specification](http://www.bittorrent.org/beps/bep_0003.html) are recommended for anyone wishing to understand the technical details.
### Conventions and Terminology ### Conventions and Terminology
@ -60,7 +64,7 @@ LBRY has been in public use since June 2016. As of February 2019, approximately
## Blockchain ## Blockchain
The LBRY blockchain is a public, proof-of-work blockchain. The design is based on the [Bitcoin](https://bitcoin.org/bitcoin.pdf) blockchain, with substantial modifications. This document does not cover or specify any aspects of LBRY that are identical to Bitcoin and instead focuses on the differences, primarily the claim operations and claimtrie. The LBRY blockchain is a public proof-of-work blockchain. The design is based on [Bitcoin](https://bitcoin.org/bitcoin.pdf), with substantial modifications. This document does not cover or specify any aspects of LBRY that are identical to Bitcoin and instead focuses on the differences, primarily the claim operations and claimtrie.
Our blockchain serves three key purposes: Our blockchain serves three key purposes:
@ -71,7 +75,7 @@ Our blockchain serves three key purposes:
### Stakes ### Stakes
A _stake_ is a a single entry in the blockchain that commits credits toward a name. The two types of stakes are [_claims_](#claims) and [_supports_](#supports). A _stake_ is a single entry in the blockchain that commits credits toward a name. The two types of stakes are [_claims_](#claims) and [_supports_](#supports).
All stakes have these properties: All stakes have these properties:
@ -85,14 +89,7 @@ All stakes have these properties:
#### Claims #### Claims
A _claim_ is a stake that stores metadata. There are two types of claims: A _claim_ is a stake that stores metadata. There are two types of claims. _Stream claims_ declare the availability, access method, and publisher of a stream. _Channel claims_ create a pseudonym that can be used as the publisher of stream claims.
<dl>
<dt>stream claim</dt>
<dd>Declares the availability, access method, and publisher of a stream.</dd>
<dt>channel claim</dt>
<dd>Creates a pseudonym that can be declared as the publisher of stream claims.</dd>
</dl>
##### Claim Properties ##### Claim Properties
@ -182,7 +179,7 @@ The _root hash_ is the hash of the root node. It is stored in the header of each
Multiple claims can exist for the same name. They are all stored in the leaf node for that name. See [Claim Ordering](#claim-ordering) Multiple claims can exist for the same name. They are all stored in the leaf node for that name. See [Claim Ordering](#claim-ordering)
For more details on the specific claimtrie implementation, see [the source code](https://github.com/lbryio/lbrycrd/blob/master/src/claimtrie.cpp). For more details on the specific claimtrie implementation, see [the source code](https://github.com/lbryio/lbrycrd/blob/v0.17.3.2/src/claimtrie.cpp).
#### Statuses {#stake-statuses} #### Statuses {#stake-statuses}
@ -259,6 +256,10 @@ See the [example](#claim-activation-example) in the appendix for more informatio
Names in the claimtrie are normalized when performing any comparisons. This is necessary to avoid confusion due to Unicode equivalence or casing. When names are being compared, they are first converted using [Unicode Normalization Form D](http://unicode.org/reports/tr15/#Norm_Forms) (NFD), then lowercased using the en_US locale. This means names are effectively case-insensitive. Since claims competing for the same name are stored in the same node in the claimtrie, names are also normalized to determine the claimtrie path to the node. Names in the claimtrie are normalized when performing any comparisons. This is necessary to avoid confusion due to Unicode equivalence or casing. When names are being compared, they are first converted using [Unicode Normalization Form D](http://unicode.org/reports/tr15/#Norm_Forms) (NFD), then lowercased using the en_US locale. This means names are effectively case-insensitive. Since claims competing for the same name are stored in the same node in the claimtrie, names are also normalized to determine the claimtrie path to the node.
#### Expiration
In an earlier version of the protocol, stakes would expire (i.e. automatically become abandoned) 262974 blocks after they were accepted. A hard fork was deployed that effectively disables expiration. Any stakes that expired before the fork took effect are treated as if they were abandoned. For details see [this pull request](https://github.com/lbryio/lbrycrd/pull/137).
### URLs ### URLs
URLs are memorable references to claims. All URLs: URLs are memorable references to claims. All URLs:
@ -302,23 +303,25 @@ lbry://@lbry/meet-lbry
A claim for this name with this claim ID. Partial prefix matches are allowed (see [URL Resolution](#url-resolution)). A claim for this name with this claim ID. Partial prefix matches are allowed (see [URL Resolution](#url-resolution)).
``` ```
lbry://meet-lbry#7a0aa95c5023c21c098 lbry://meet-lbry:7a0aa95c5023c21c098
lbry://meet-lbry#7a lbry://meet-lbry:7a
lbry://@lbry#3f/meet-lbry lbry://@lbry:3f/meet-lbry
``` ```
##### Claim Sequence *Note: in a previous version of this spec, the `#` character was used to signify the claim ID portion of the url. This character is now deprecated and will stop being supported in the future.*
The _n_th accepted claim for this name. _n_ must be a positive number. This can be used to resolve claims in the order in which they were made, rather than by the amount of credits backing a claim. ##### Sequence
The _n_th accepted claim for this name. _n_ must be a positive number. This can be used to reference claims in the order in which they were made, rather than by the amount of credits backing a claim.
``` ```
lbry://meet-lbry:1 lbry://meet-lbry*1
lbry://@lbry:1/meet-lbry lbry://@lbry*1/meet-lbry
``` ```
##### Bid Position ##### Amount Order
The _n_th claim for this name, ordered by total amount (highest first). _n_ must be a positive number. This is useful for resolving non-winning bids in bid order. The _n_th claim for this name, ordered by total amount (highest first). _n_ must be a positive number. This is useful for resolving non-controlling claims that may become controlling.
``` ```
lbry://meet-lbry$2 lbry://meet-lbry$2
@ -353,10 +356,10 @@ ChannelClaimNameAndModifier ::= ChannelClaimName Modifier?
StreamClaimName ::= NameChar+ StreamClaimName ::= NameChar+
ChannelClaimName ::= '@' NameChar+ ChannelClaimName ::= '@' NameChar+
Modifier ::= ClaimID | ClaimSequence | BidPosition Modifier ::= ClaimID | Sequence | AmountOrder
ClaimID ::= '#' Hex+ ClaimID ::= ':' Hex+
ClaimSequence ::= ':' PositiveNumber Sequence ::= '*' PositiveNumber
BidPosition ::= '$' PositiveNumber AmountOrder ::= '$' PositiveNumber
Query ::= '?' QueryParameterList Query ::= '?' QueryParameterList
QueryParameterList ::= QueryParameter ( '&' QueryParameterList )* QueryParameterList ::= QueryParameter ( '&' QueryParameterList )*
@ -371,7 +374,7 @@ PositiveNumber ::= PositiveDigit Digit*
HexAlpha ::= [abcdef] HexAlpha ::= [abcdef]
Hex ::= (Digit | HexAlpha)+ Hex ::= (Digit | HexAlpha)+
NameChar ::= Char - [=&#:$@%?/] /* any character that is not reserved */ NameChar ::= Char - [=&#:*$@%?/] /* any character that is not reserved */
Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] /* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */ Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x10000-#x10FFFF] /* any Unicode character, excluding the surrogate blocks, FFFE, and FFFF. */
``` ```
@ -387,13 +390,13 @@ Return the controlling claim for the name. Stream claims and channel claims are
Get all claims for the claim name whose IDs start with the given `ClaimID`. Sort the claims in ascending order by block height and position within the block. Return the first claim. Get all claims for the claim name whose IDs start with the given `ClaimID`. Sort the claims in ascending order by block height and position within the block. Return the first claim.
##### ClaimSequence ##### Sequence
Get all claims for the claim name. Sort the claims in ascending order by block height and position within the block. Return the _n_th claim, where _n_ is the given `ClaimSequence` value. Get all claims for the claim name. Sort the claims in ascending order by block height and position within the block. Return the _n_th claim, where _n_ is the given `Sequence` value.
##### BidPosition ##### AmountOrder
Get all claims for the claim name. Sort the claims in descending order by total effective amount. Return the _n_th claim, where _n_ is the given `BidSequence` value. Get all claims for the claim name. Sort the claims in descending order by total effective amount. Return the _n_th claim, where _n_ is the given `AmountOrder` value.
##### ChannelClaimName and StreamClaimName ##### ChannelClaimName and StreamClaimName
@ -453,7 +456,7 @@ OP_CLAIM_NAME Fruit Apple OP_2DROP OP_DROP OP_DUP OP_HASH160 <address> OP_EQUALV
##### OP\_UPDATE\_CLAIM ##### OP\_UPDATE\_CLAIM
`OP_UPDATE_CLAIM` updates a claim by replacing its metadata. An update transaction has an added requirement that it must spend the output for the existing claim that it wishes to update. Otherwise, it is considered invalid and will not make it into the claimtrie. Thus it must have the following redeem script: `OP_UPDATE_CLAIM` updates a claim by replacing its metadata. An update transaction has an added requirement that it must spend the output for the existing claim that it wishes to update. Otherwise, it is considered invalid and will not make it into the claimtrie. Thus, it must have the following redeem script:
``` ```
<signature> <pubKeyForPreviousAddress> <signature> <pubKeyForPreviousAddress>
@ -509,21 +512,21 @@ The target block time was lowered from 10 minutes to 2.5 minutes to facilitate f
#### Difficulty Adjustment #### Difficulty Adjustment
The proof-of-work target is adjusted every block to better adapt to sudden changes in hash rate. The exact adjustment algorithm can be seen [here](https://github.com/lbryio/lbrycrd/blob/master/src/lbry.cpp). The proof-of-work target is adjusted every block to better adapt to sudden changes in hash rate. The exact adjustment algorithm can be seen [here](https://github.com/lbryio/lbrycrd/blob/v0.17.3.2/src/lbry.cpp).
#### Block Hash Algorithm #### Block Hash Algorithm
LBRY uses a combination of SHA-256, SHA-512, and RIPEMD-160. The exact hashing algorithm can be seen [here](https://github.com/lbryio/lbrycrd/blob/master/src/hash.cpp#L18). LBRY uses a combination of SHA-256, SHA-512, and RIPEMD-160. The exact hashing algorithm can be seen [here](https://github.com/lbryio/lbrycrd/blob/v0.17.3.2/src/hash.cpp#L15).
#### Block Rewards #### Block Rewards
The block reward schedule was adjusted to provide an initial testing period, a quick ramp-up to max block rewards, then a logarithmic decay to 0. The source for the algorithm is [here](https://github.com/lbryio/lbrycrd/blob/master/src/main.cpp#L1594). The block reward schedule was adjusted to provide an initial testing period, a quick ramp-up to max block rewards, then a logarithmic decay to 0. The source for the algorithm is [here](https://github.com/lbryio/lbrycrd/blob/v0.17.3.2/src/validation.cpp#L1025).
#### Addresses #### Addresses
The address version byte is set to `0x55` for standard (pay-to-public-key-hash) addresses and `0x7a` for multisig (pay-to-script-hash) addresses. P2PKH addresses start with the letter `b`, and P2SH addresses start with `r`. The address version byte is set to `0x55` for standard (pay-to-public-key-hash) addresses and `0x7a` for multisig (pay-to-script-hash) addresses. P2PKH addresses start with the letter `b`, and P2SH addresses start with `r`.
All the chain parameters are defined [here](https://github.com/lbryio/lbrycrd/blob/master/src/chainparams.cpp). All the chain parameters are defined [here](https://github.com/lbryio/lbrycrd/blob/v0.17.3.2/src/chainparams.cpp).
## Metadata ## Metadata
@ -532,9 +535,9 @@ Metadata is structured information about a stream or channel separate from the c
Metadata is stored in a serialized binary format using [Protocol Buffers](https://developers.google.com/protocol-buffers/). This allows for metadata to be: Metadata is stored in a serialized binary format using [Protocol Buffers](https://developers.google.com/protocol-buffers/). This allows for metadata to be:
- **Extensibile**. Metadata can encompass thousands of fields for dozens of types of content. It must be efficient to both modify the structure and maintain backward compatibility. - **Extensible**. Metadata can encompass thousands of fields for dozens of types of content. It must be efficient to both modify the structure and maintain backward compatibility.
- **Compact**. Blockchain space is expensive. Data must be stored as compactly as possible. - **Compact**. Blockchain space is expensive. Data must be stored as compactly as possible.
- **Interoperabile**. Metadata will be used by many projects written in different languages. - **Interoperable**. Metadata will be used by many projects written in different languages.
The serialized metadata may be cryptographically signed to indicate membership in a channel. See [Channels](#channels) for more info. The serialized metadata may be cryptographically signed to indicate membership in a channel. See [Channels](#channels) for more info.
@ -627,15 +630,15 @@ The purpose of channels is to allow content to be clustered under a single pseud
#### Signing #### Signing
A claim is considered part of a channel when its metadata is signed by the channel's private key. Here's the structure of a signed metadata value: A claim is considered part of a channel when the metadata in its value is signed by the channel's private key. Here's the structure of a signed value:
field | size | description field | size | description
:--- | :--- | :--- :--- | :--- | :---
Version | 1 byte | Format version. See [Format Versions](#format-versions). Version | 1 byte | Format version. See [Format Versions](#format-versions).
Channel ID | 20 bytes | Claim ID of the channel claim that contains the matching public key. _Skip this field if there is no signature._ Channel Claim ID | 20 bytes | Claim ID of the channel claim that contains the matching public key. _Skip this field if there is no signature._
Signature | 64 bytes | The signature. _Skip this field if there is no signature._ Signature | 64 bytes | The signature. _Skip this field if there is no signature._
Payload | variable | The protobuf-encoded metadata. Metadata | variable | The protobuf-encoded metadata.
##### Format Versions ##### Format Versions
@ -650,9 +653,13 @@ format | description
##### Signing Process ##### Signing Process
1. Encode the metadata using protobuf. 1. Encode the metadata using protobuf.
2. Hash the encoded claim using SHA-256. 1. Create the payload to be signed by concatenating the following:
3. Sign the hash using the private key associated with the channel. - The outpoint hash the first transaction input of this claim's transaction. This is in the payload to prevent replay attacks.
4. Append all the values (the version, the claim ID of the corresponding channel claim, the signature, and the protobuf-encoded metadata). - The claim ID of the channel. See <a href="#stake-identifier-generation">this</a> for more on outpoints and claim IDs.
- The encoded metadata
1. Hash the payload using SHA-256.
1. Sign the payload hash using the private key associated with the channel.
1. Concatenate the version, the channel claim ID, the payload signature, and the protobuf-encoded metadata. This is the claim value.
##### Signature Validation ##### Signature Validation
@ -660,7 +667,8 @@ format | description
1. Check the version field. If it indicates that there is no signature, then no validation is necessary. 1. Check the version field. If it indicates that there is no signature, then no validation is necessary.
1. Split out the channel ID and signature from the rest of the data. 1. Split out the channel ID and signature from the rest of the data.
1. Look up the channel claim to ensure it exists and contains a public key. 1. Look up the channel claim to ensure it exists and contains a public key.
1. Use the public key to verify the signature. 1. Create the payload hash as described <a href="#signing-process">above</a>.
1. Use the public key to verify that the payload hash signature is valid.
### Validation {#metadata-validation} ### Validation {#metadata-validation}
@ -668,7 +676,6 @@ format | description
The blockchain treats metadata as an opaque series of bytes. Clients should not trust the metadata they read from the blockchain. Each client is responsible for correctly encoding and decoding the metadata, and for validating its structure and signatures. This allows evolution of the metadata definition without changes to blockchain consensus rules. The blockchain treats metadata as an opaque series of bytes. Clients should not trust the metadata they read from the blockchain. Each client is responsible for correctly encoding and decoding the metadata, and for validating its structure and signatures. This allows evolution of the metadata definition without changes to blockchain consensus rules.
## Data ## Data
Files published using LBRY are stored in a distributed fashion by the clients participating in the network. Each file is split into many small pieces. Each piece is encrypted and [announced](#announce) to the network. The pieces may also be uploaded to other hosts on the network that specialize in rehosting content. Files published using LBRY are stored in a distributed fashion by the clients participating in the network. Each file is split into many small pieces. Each piece is encrypted and [announced](#announce) to the network. The pieces may also be uploaded to other hosts on the network that specialize in rehosting content.
@ -742,7 +749,6 @@ The `version` field is always 1. It is intended to signal structure changes in f
Every stream must have at least two blobs - the manifest blob and a content blob. Consequently, zero-length streams are not allowed. Every stream must have at least two blobs - the manifest blob and a content blob. Consequently, zero-length streams are not allowed.
#### Stream Encoding #### Stream Encoding
A file must be encoded into a stream before it can be published. Encoding involves breaking the file into chunks, encrypting the chunks into content blobs, and creating the manifest blob. Here are the steps: A file must be encoded into a stream before it can be published. Encoding involves breaking the file into chunks, encrypting the chunks into content blobs, and creating the manifest blob. Here are the steps:
@ -754,7 +760,7 @@ A file must be encoded into a stream before it can be published. Encoding involv
##### Content Blobs ##### Content Blobs
1. Break the file into chunks of at most 2097151 bytes. 1. Break the file into chunks of at most 2097151 bytes.
1. Generate a random 32-byte initialization vector (IV) for each chuck. 1. Generate a random 32-byte initialization vector (IV) for each chunk.
1. Pad each chunk using PKCS7 padding. 1. Pad each chunk using PKCS7 padding.
1. Encrypt each chunk with AES-CBC using the stream key and the IV for that chunk. 1. Encrypt each chunk with AES-CBC using the stream key and the IV for that chunk.
1. An encrypted chunk is a blob. 1. An encrypted chunk is a blob.
@ -774,14 +780,13 @@ An implementation of this process is available [here](https://github.com/lbryio/
Decoding a stream is like encoding in reverse, and with the added step of verifying that the expected blob hashes match the actual data. Decoding a stream is like encoding in reverse, and with the added step of verifying that the expected blob hashes match the actual data.
1. Verify that the hash of the manifest blob and matches the stream hash. 1. Verify that the hash of the manifest blob matches the stream hash.
2. Parse the JSON in manifest blob. 2. Parse the JSON in the manifest blob.
3. Verify the hashes of the content blobs. 3. Verify the hashes of the content blobs.
4. Decrypt and remove the padding from each content blob using the stream key and IVs in the manifest. 4. Decrypt and remove the padding from each content blob using the stream key and IVs in the manifest.
5. Concatenate the decrypted chunks in order. 5. Concatenate the decrypted chunks in order.
### Announce ### Announce
After a stream is encoded, it must be _announced_ to the network. Announcing is the process of letting other nodes on the network know that a client has content available for download. LBRY tracks announced content using a distributed hash table. After a stream is encoded, it must be _announced_ to the network. Announcing is the process of letting other nodes on the network know that a client has content available for download. LBRY tracks announced content using a distributed hash table.
@ -841,8 +846,6 @@ Upload sends a blob to the server. If uploading many blobs, the client should us
The protocol methods and message types are defined in detail [here](https://github.com/lbryio/lbry.go/blob/master/blobex/blobex.proto). The protocol methods and message types are defined in detail [here](https://github.com/lbryio/lbry.go/blob/master/blobex/blobex.proto).
### Reflectors and Data Markets ### Reflectors and Data Markets
In order for a client to download content, there must be hosts online that have the content the client wants, when the client wants it. To incentivize the continued hosting of data, the blob exchange protocol supports data upload and payment for data. _Reflectors_ are hosts that accept data uploads. They rehost (reflect) the uploaded data and charge for downloads. In order for a client to download content, there must be hosts online that have the content the client wants, when the client wants it. To incentivize the continued hosting of data, the blob exchange protocol supports data upload and payment for data. _Reflectors_ are hosts that accept data uploads. They rehost (reflect) the uploaded data and charge for downloads.
@ -858,25 +861,25 @@ The current version of the protocol does not support sophisticated price negotia
Here is a step-by-step example to illustrate how competing claims activate and are ordered. All stakes are for the same name. Here is a step-by-step example to illustrate how competing claims activate and are ordered. All stakes are for the same name.
**Block 13:** Claim A for 10LBC is accepted. It is the first claim, so it immediately becomes active and controlling. **Block 13:** Claim A for 10 LBC is accepted. It is the first claim, so it immediately becomes active and controlling.
<br>State: A(10) is controlling <br>State: A(10) is controlling
**Block 1001:** Claim B for 20LBC is accepted. Its activation height is `1001 + min(4032, floor((1001-13) / 32)) = 1001 + 30 = 1031`. **Block 1001:** Claim B for 20 LBC is accepted. Its activation height is `1001 + min(4032, floor((1001-13) / 32)) = 1001 + 30 = 1031`.
<br>State: A(10) is controlling, B(20) is accepted. <br>State: A(10) is controlling, B(20) is accepted.
**Block 1010:** Support X for 14LBC for claim A is accepted. Since it is a support for the controlling claim, it activates immediately. **Block 1010:** Support X for 14 LBC for claim A is accepted. Since it is a support for the controlling claim, it activates immediately.
<br>State: A(10+14) is controlling, B(20) is accepted. <br>State: A(10+14) is controlling, B(20) is accepted.
**Block 1020:** Claim C for 50LBC is accepted. The activation height is `1020 + min(4032, floor((1020-13) / 32)) = 1020 + 31 = 1051`. **Block 1020:** Claim C for 50 LBC is accepted. The activation height is `1020 + min(4032, floor((1020-13) / 32)) = 1020 + 31 = 1051`.
<br>State: A(10+14) is controlling, B(20) is accepted, C(50) is accepted. <br>State: A(10+14) is controlling, B(20) is accepted, C(50) is accepted.
**Block 1031:** Claim B activates. It has 20LBC, while claim A has 24LBC (10 original + 14 from support X). There is no takeover, and claim A remains controlling. **Block 1031:** Claim B activates. It has 20 LBC, while claim A has 24 LBC (10 original + 14 from support X). There is no takeover, and claim A remains controlling.
<br>State: A(10+14) is controlling, B(20) is active, C(50) is accepted. <br>State: A(10+14) is controlling, B(20) is active, C(50) is accepted.
**Block 1040:** Claim D for 300LBC is accepted. The activation height is `1040 + min(4032, floor((1040-13) / 32)) = 1040 + 32 = 1072`. **Block 1040:** Claim D for 300 LBC is accepted. The activation height is `1040 + min(4032, floor((1040-13) / 32)) = 1040 + 32 = 1072`.
<br>State: A(10+14) is controlling, B(20) is active, C(50) is accepted, D(300) is accepted. <br>State: A(10+14) is controlling, B(20) is active, C(50) is accepted, D(300) is accepted.
**Block 1051:** Claim C activates. It has 50LBC, while claim A has 24LBC, so a takeover is initiated. The takeover height for this name is set to 1051, and therefore the activation delay for all the claims becomes `min(4032, floor((1051-1051) / 32)) = 0`. All the claims become active. The totals for each claim are recalculated, and claim D becomes controlling because it has the highest total. **Block 1051:** Claim C activates. It has 50 LBC, while claim A has 24 LBC, so a takeover is initiated. The takeover height for this name is set to 1051, and therefore the activation delay for all the claims becomes `min(4032, floor((1051-1051) / 32)) = 0`. All the claims become active. The totals for each claim are recalculated, and claim D becomes controlling because it has the highest total.
<br>State: A(10+14) is active, B(20) is active, C(50) is active, D(300) is controlling. <br>State: A(10+14) is active, B(20) is active, C(50) is active, D(300) is controlling.
### URL Resolution Examples ### URL Resolution Examples
@ -893,7 +896,7 @@ _--_ | apple | 690eea | 10
@Bryan | _--_ | 0da517 | 1 @Bryan | _--_ | 0da517 | 1
@Chris | _--_ | b3f7b1 | 1 @Chris | _--_ | b3f7b1 | 1
@Chris | banana | fc861c | 1 @Chris | banana | fc861c | 1
@Arthur | apple | 37ee1 | 20 @Arthur | apple | a37ee1 | 20
@Bryan | cherry | a18bca | 10 @Bryan | cherry | a18bca | 10
@Chris | _--_ | 005a7d | 100 @Chris | _--_ | 005a7d | 100
@Arthur | cherry | d39aa0 | 20 @Arthur | cherry | d39aa0 | 20
@ -906,15 +909,22 @@ URL | Claim ID
`lbry://banana` | 714a3f `lbry://banana` | 714a3f
`lbry://@Chris` | 005a7d `lbry://@Chris` | 005a7d
`lbry://@Chris/banana` | _not found_ (the controlling `@Chris` does not have a `banana`) `lbry://@Chris/banana` | _not found_ (the controlling `@Chris` does not have a `banana`)
`lbry://@Chris:1/banana` | fc861c `lbry://@Chris*1/banana` | fc861c
`lbry://@Chris:#fc8/banana` | fc861c `lbry://@Chris:fc8/banana` | fc861c
`lbry://cherry` | bfaabb `lbry://cherry` | bfaabb
`lbry://@Arthur/cherry` | d39aa0 `lbry://@Arthur/cherry` | d39aa0
`lbry://@Bryan` | 0da517 `lbry://@Bryan` | 0da517
`lbry://banana$1` | 714a3f `lbry://banana$1` | 714a3f
`lbry://banana$2` | fc861c `lbry://banana$2` | fc861c
`lbry://banana$3` | _not found_ `lbry://banana$3` | _not found_
`lbry://@Arthur:1` | b7bab5 `lbry://@Arthur*1` | b7bab5
### Additional Resources
- [lbry.tech](https://lbry.tech) is designed specifically for a technical audience. There you will find detailed explanations and examples, ways to interact with the LBRY community, and guidance for anyone who wants to get more involved.
- [github.com/lbryio](https://github.com/lbryio) contains MIT-licensed source code and implementation details for the protocol and related software.
- [lbry.com](https://lbry.com) is an application built on top of LBRY. It's one example of the type of end-to-end experience that is possible to create using this technology.
<pre class="pdf-hide" style="font: 10px/5px monospace;overflow:hidden;text-align: center;margin: 10rem 0"> <pre class="pdf-hide" style="font: 10px/5px monospace;overflow:hidden;text-align: center;margin: 10rem 0">
@ -987,4 +997,4 @@ URL | Claim ID
--- ---
_[Edit this on Github](https://github.com/lbryio/spec/blob/master/index.md)_ _[Improve this page on Github](https://github.com/lbryio/spec/blob/master/index.md)_

Binary file not shown.

View file

@ -7,7 +7,7 @@ This repo contains the official protocol specification for LBRY. It is built wit
## Usage ## Usage
The spec is available at https://spec.lbry.io. The spec is available at https://spec.lbry.com.
To view the content locally, you must have [Ruby](https://www.ruby-lang.org/en/documentation/installation/) and [bundler](https://bundler.io#getting-started) installed. Then run `bundle install` and `bundle exec jekyll serve`, and open `localhost:4000` in your browser. To view the content locally, you must have [Ruby](https://www.ruby-lang.org/en/documentation/installation/) and [bundler](https://bundler.io#getting-started) installed. Then run `bundle install` and `bundle exec jekyll serve`, and open `localhost:4000` in your browser.
@ -20,7 +20,7 @@ To edit the spec:
- make changes to the markdown - make changes to the markdown
- refresh `localhost:4000` in your browser to see the effect of your changes. - refresh `localhost:4000` in your browser to see the effect of your changes.
Contributions to this project are welcome, encouraged, and compensated. For more details, [click here](https://lbry.io/faq/contributing). Contributions to this project are welcome, encouraged, and compensated. For more details, [click here](https://lbry.com/faq/contributing).
## License ## License
@ -30,9 +30,9 @@ This project is MIT licensed. For the full license, see [LICENSE](LICENSE).
## Security ## Security
We take security seriously. Please contact security@lbry.io regarding any security issues. [Our GPG key is here](https://lbry.io/faq/gpg-key) if you need it. We take security seriously. Please contact security@lbry.com regarding any security issues. [Our PGP key is here](https://lbry.com/faq/pgp-key) if you need it.
## Contact ## Contact
The primary contact for this project is [@lyoshenka](https://github.com/lyoshenka) (grin@lbry.io). The primary contact for this project is [@lyoshenka](https://github.com/lyoshenka) (grin@lbry.com).