LBRYCommunity/lbry.tech
1
0
Fork 0
Code Issues 13 Pull requests 8 Packages Projects Releases Wiki Activity

Moved Markdown files to /documents and included resource docs from master branch

Browse source
This commit is contained in:
ポール ウェッブ 2018-07-13 13:03:21 -05:00
parent f6eaf4fb13
commit 29c817e2d2
17 changed files with 1202 additions and 19 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

12
config.js
View file

@ -8,15 +8,5 @@ module.exports = exports = {
docsBranch: "master/content",
editLinkText: "Edit this page on GitHub"
},
ga: "UA-60403362-1",
markdown: {
config: md => md.use(require("markdown-it-wikilinks")({
makeAllLinksAbsolute: true,
baseURL: "/glossary.html#",
uriSuffix: "",
htmlAttributes: {
class: "wikilink"
}
}))
}
ga: "UA-60403362-1"
};

2
views/pages/build.md → documents/build.md
View file

@ -40,7 +40,7 @@ Almost all web, desktop, and mobile applications will use the [lbry daemon](http
#### iOS Applications
It is not currently possible to use LBRY on iOS. [There is a bounty](lbry.io/bounty/ios-daemon) for this.
It is not currently possible to use LBRY on iOS. [There is a bounty](https://lbry.io/bounty/ios-daemon) for this.
### Blockchain and Wallet Level (Full Node Applications)

72
documents/claimtrie.md Normal file
View file

@ -0,0 +1,72 @@
The Merkle Claim Trie
==============
How the data structure that organizes claims by names works, how proofs are generated/verified and how consensus on the state of the trie is represented.
For looking into how claims end up in the trie, [read that instead](https://lbry.io/faq/claimtrie-implementation).
## The Trie
A Trie is an ordered tree data structure. Think of it as a hash map or dictionary where the keys are ordered and organized in prefixes. It's used for storing claims by name and looks (a bit) like that:
![Wikipedia claimtrie](https://upload.wikimedia.org/wikipedia/commons/thumb/b/be/Trie_example.svg/400px-Trie_example.svg.png)
You can read more on it [here](https://en.wikipedia.org/wiki/Trie), but for understanding the claimtrie let's just think of it as a mapping of names and claims where you can easily find which character of a name preffix forms a collision. Like the first `t` for `{to, tea, ted, ten}` and the `e` in `te` for `{tea, ted, ten}`.
## Consensus
Each block header holds an extra 256 bits value calculated out of the root node of the claim trie at that block height. It's called `nameclaimroot` and is influenced by all children nodes as we will see next. If a blockchain network peer disagrees that a claim name was accepted or who is the winner of each name, its `nameclaimroot` will differ and the block won't form the same chain as the ones that accepted the official rules. This is the same for the traditional Merkle Root, which is the root of the [Merkle Tree](https://bitcoin.org/en/glossary/merkle-tree), formed by transactions in a block.
## What's in a leaf?
The leaf currently holds the winner of that name. Its formed by the transaction hash, output number of the claim in that transaction and the height it was accepted.
### Generating the leaf hash
So, let's suppose that the winner claim of `mindblown` name was made at transaction output `1` of the transaction hash `67ad533eb2676c9d36bfa100092af5358de747e08ef928c0c54a8b3891c2b76b` and included in the Trie at height `102`.
1. The transaction hash is converted from [RPC byte order](https://bitcoin.org/en/glossary/rpc-byte-order) to [internal byte order](https://bitcoin.org/en/glossary/internal-byte-order).
2. The output number as a simple string.
3. The height as a big endian 64 bits value.
4. The node hash is calculated as the double sha-256 hash of the double sha-256 hash of the internal byte order representation of the transaction hash concatenated with the double sha-256 hash of the output number representation concatenated with the double sha-256 hash of the height.
This is better represented in the simple python script below:
```python
import struct
from hashlib import sha256
from binascii import unhexlify, hexlify
name = "mindblown".encode()
tx = '67ad533eb2676c9d36bfa100092af5358de747e08ef928c0c54a8b3891c2b76b'
nout = 1
at_height = 102
sha256d = lambda x: sha256(sha256(x).digest()).digest()
def hash_leaf(tx, nout, height):
raw_tx = unhexlify(tx)[::-1]
raw_nout = str(nout).encode()
raw_height = struct.pack('>Q', height)
return sha256d(sha256d(raw_tx) + sha256d(raw_nout) + sha256d(raw_height))
print("leaf hash is {}".format(hexlify(hash_leaf(tx, nout, at_height)[::-1])))
```
## How the root hash is calculated?
Let's start with a Claim Trie holding a single claim.
The claim is named `mindblown` and was included at block 102 with the same details as the last section. It's actually a real claim as [you can see in the block explorer](https://explorer.lbry.io/blocks/102). This block has the first claim ever, so the claimtrie was the simple case being explained in here.
We start with the leaf hash:
1. Hash the leaf hash using double sha-256.
2. For each character of the name (nodes of the trie), the hash of a node is the double sha-256 of the node's character concatenated with the children hash.
Continuing with the Python script from the last section, this is how to calculate the root of a claim trie holding a single name:
```python
def hash_single_claim_trie(name, claim_tx, claim_nout, claim_height):
final_hash = sha256d(hash_leaf(claim_tx, claim_nout, claim_height))
for character in reversed(name):
final_hash = sha256d(bytearray([character]) + final_hash)
return final_hash
print("root hash is {}".format(hexlify(hash_single_claim_trie(name, tx, nout, at_height)[::-1])))
```
## What if there are more leafs?
Just concatenate the node character with the childrens hashes sha256d(character + leftmost child hash + ... + rightmost child hash ).
TO BE IMPROVED

0
views/pages/community.md → documents/community.md
View file

0
views/pages/contribute.md → documents/contribute.md
View file

9
documents/encrypt-lbrycrd.md Normal file
View file

@ -0,0 +1,9 @@
*Note: The below instructions are the [LBRYCrd Full Blockchain wallet](https://github.com/lbryio/lbrycrd) and not the default wallet that ships with the LBRY App. We are still working on an encryption solution for this.*
You can use `lbrycrd-cli encryptwallet <passphrase>` to encrypt your wallet.
You can use `lbrycrd-cli walletpassphrase <passphrase> <timeout>` to temporarily unlock the wallet. The <timeout> parameter is in seconds.
For example, `lbrycrd-cli walletpassphrase 'open sesame 321' 300` would unlock your wallet for five minutes, assuming your passphrase was `open sesame 321`. (In reality, you should choose a harder-to-guess passphrase than that.)
If you set <timeout> too low, it might expire before you get done using your wallet. If you set it too high, you might forget that you left your wallet unlocked.

343
documents/glossary.md Normal file
View file

@ -0,0 +1,343 @@
---
title: LBRY Glossary
---
This glossary will help you understand exact meaning of LBRY and blockchain related terms.
We encourage the submission of changes and additions to this glossary.
### Blob
A Binary Large Object (BLOB) is a collection of binary data stored as a single entity in a database management system. When files are uploaded to the LBRY peer to peer network, they are broken down into 2MB encrypted blobs which are then shared to other peers.
### Block
A data structure that consists of a *block header* and a *merkle tree* of transactions. Each block (except for *genesis block*) references one previous block thus forming a tree called the *blockchain*. Block can be thought of as a group of transactions with a timestamp and a *proof-of-work* attached.
### Block Header
A data structure containing a previous block hash, a hash of a merkle tree of transactions, a timestamp, a *difficulty* and a *nonce*.
### Block Height
A sequence number of a block in the blockchain. Height 0 refers to the *genesis block*. Several blocks may share the same height (see *Orphan*), but only one of them belongs to the *main chain*. Block height is used in *Lock time*.
### Blockchain
A public ledger of all confirmed transactions in a form of a tree of all valid *blocks* (including *orphans*). Most of the time, "blockchain" means the *main chain*, a single most *difficult* chain of blocks. Blockchain is updated by *mining* blocks with new transactions. *Unconfirmed transactions* are not part of the blockchain. If some clients disagree on which chain is main or which blocks are valid, a *fork* happens.
### Chainquery
Chainquery provides a SQLized view of the LBRY blockchain. The model of Chainquery at its foundation consists of the fundamental data types found in the block chain. This information is then expounded on with additional columns and tables that make querying the data much easier. Chainquery consists of 4 main parts. The API Server, the Daemon, the Job Scheduler, and the upgrade manager.
### Change
Informal name for a portion of a *transaction output* that is returned to a sender as a "change" after spending that output. Since *transaction outputs* cannot be partially spent, one can spend 1 BTC out of 3 BTC output only be creating two new outputs: a "payment" output with 1 BTC sent to a payee address, and a "change" output with remaining 2 BTC (minus *transaction fees*) sent to the payer's addresses.
### Channel Claim / Signature
Creating a channel claim certificate allows you to group and identify claims based on an identity. A certificate is used to sign the claims and ensure uniqueness along with the claim ID. See [channel signing](https://github.com/lbryio/lbryschema/blob/master/docs/signing.md) for more information
### Claim
A claim (ClaimTrie) is the data structure which LBRY uses to store claims to names. It uses a Trie to efficiently store all claimed names, which can then be hashed the same way a Merkle Tree is hashed. The root hash of the ClaimTrie is stored in the blockheader of each LBRY block, enabling nodes in the LBRY network to efficiently and securely validate the state of the ClaimTrie. [Read more](https://lbry.io/faq/claimtrie-implementation)
### Claim Deposit
When creating a channel claim or publishing content onto the LBRY blockchain, a small amount (or more) LBC must be deposited to reserve the name space in the claimtrie. See our [naming documentation](https://lbry.io/faq/naming) for more information.
### Claim Support
A special type of transaction that includes claim information, a LBC address and a LBC value. Supports to one's own address increase the bid value of a claim and can be revoked anytime. Supports to an outside address also increase the value, but can only be revoked by the receiver (tip mechanism).
### Cold Storage
A collective term for various security measures to reduce the risk of remote access to the private keys. It could be a normal computer disconnected from the internet, or a dedicated hardware wallet, or a USB stick with a wallet file, or a *paper wallet*.
### Confirmed Transaction
Transaction that has been included in the blockchain. Probability of transaction being rejected is measured in a number of confirmations. See *Confirmation Number*.
### Confirmation Number
Confirmation number is a measure of probability that transaction could be rejected from the *main chain*. "Zero confirmations" means that transaction is *unconfirmed* (not in any block yet). One confirmation means that the transaction is included in the latest block in the main chain. Two confirmations means the transaction is included in the block right before the latest one. And so on. Probability of transaction being reversed (*"double spent"*) is diminishing exponentially with more blocks added "on top" of it.
### Dewey
[A library classification system](https://en.wikipedia.org/wiki/Dewey_Decimal_Classification) and also the name of the smallest unit used in transactions. 1 LBRY Credit (LBC) is equal to 100 million deweys.
### Difficulty
Difficulty is a measure of how difficult it is to find a new block compared to the easiest it can ever be. By definition, it is a maximum *target* divided by the current target. Difficulty is used in two LBRY rules: 1) every block must be meet difficulty target to ensure 2.5 minute interval between blocks and 2) transactions are considered confirmed only when belonging to a *main chain* which is the one with the biggest cumulative difficulty of all blocks.
### Depth
Depth refers to a place in the blockchain. A transaction with 6 *confirmations* can also be called "6 blocks deep".
### Deterministic Wallet
A collective term for different ways to generate a sequence of *private keys* and/or *public keys*. Deterministic wallet does not need a *Key Pool*. The simplest form of a deterministic wallet is based on hashing a secret string concatenated with a key number. For each number the resulting hash is used as a private key (public key is derived from it). More complex scheme uses *elliptic curve arithmetic* to derive sequences of public and private keys separately which allows generating new *addresses* for every payment request without storing private keys on a web server. [More information on Bitcoin Wiki](https://en.bitcoin.it/wiki/Deterministic_wallet). See also *Wallet*.
### Dust
A transaction output that is smaller than a typically fee required to spend it. This is not a strict part of the protocol, as any amount more than zero is valid.
### ECDSA
Stands for *Elliptic Curve Digital Signature Algorithm*. Used to verify transaction ownership when making a transfer of bitcoins. See *Signature*.
### Elliptic Curve Arithmetic
A set of mathematical operations defined on a group of points on a 2D elliptic curve. LBRY, similar to the Bitcoin protocol, uses predefined curve [secp256k1](https://en.bitcoin.it/wiki/Secp256k1). Here's the simplest possible explanation of the operations: you can add and subtract points and multiply them by an integer. Dividing by an integer is computationally infeasible (otherwise cryptographic signatures won't work). The private key is a 256-bit integer and the public key is a product of a predefined point G ("generator") by that integer: A = G * a. Associativity law allows implementing interesting cryptographic schemes like Diffie-Hellman key exchange (ECDH): two parties with private keys *a* and *b* may exchange their public keys *A* and *B* to compute a shared secret point C: C = A * b = B * a because (G * a) * b == (G * b) * a. Then this point C can be used as an AES encryption key to protect their communication channel.
### Fork
Refers either to a fork of a source code or, more often, to a split of the blockchain when two different parts of the network see different main chains. In a sense, fork occurs every time two blocks of the same height are created at the same time. Both blocks always have the different hashes (and therefore different difficulty), so when a node sees both of them, it will always choose the most difficult one. However, before both blocks arrive to a majority of nodes, two parts of the network will see different blocks as tips of the main chain.
### Full Node
A *node* which implements all of LBRY blockchain and does not require trusting any external service to validate transactions. It is able to download and validate the entire *blockchain*. All full nodes implement the same peer-to-peer messaging protocol to exchange transactions and blocks, but that is not a requirement. A full node may receive and validate data using any protocol and from any source. However, the highest security is achieved by being able to communicate as fast as possible with as many nodes as possible.
### Genesis Block
A very first block in the blockchain with hard-coded contents and a all-zero reference to a previous block. The LBRY genesis block was released on [28 Oct 2015](https://explorer.lbry.io/blocks/0) and the first block mined was on [23 Jun 2016](https://explorer.lbry.io/blocks/1).
### Hard Fork
Some people use term *hard fork* to stress that changing LBRY protocol requires overwhelming majority to agree with it, or some noticeable part of the economy will continue with original blockchain following the old rules. See *Fork* and *Soft Fork* for further discussion.
### Hash Function
LBRY POW calculation uses three cryptographic hash functions: SHA-512, SHA-256 and RIPEMD-160. Click [here](https://lbry.io/faq/proof-algorithm) for more details.
### Hashrate
A measure of mining hardware performance expressed in hashes per second (GH/s). Click [here](https://www.tokens24.com/cryptopedia/basics/bitcoin-hash-rate) for more details.
### Key
Could mean an ECDSA public or private key, or AES symmetric encryption key. AES is not used in the protocol itself (only to encrypt the ECDSA keys and other sensitive data), so usually the word *key* means an ECDSA key. When talking about *keys*, people usually mean private keys as public key can always be derived from a private one. See also *Private Key* and *Public Key*.
### Key fee
The content price, set by the publisher, in order to download a claim. The key fee is paid once any part of the data is able to be downloaded.
### LBC
The currency code for 1 LBRY Credit (defined as 100 000 000 *deweys*).
### LBRY Credits
LBRY Credits (LBC) is the cryptocurrency used to make digital transactions (payments, tips, claims) on the LBRY blockchain.
## LBRY Daemon
The daemon combines various components to provide a single API across the LBRY ecosystem in order to interact with the blockchain and datanetwork.
### LBRY Redux
[lbry-redux](https://github.com/lbryio/lbry-redux) is a module which contains common React and redux code shared between lbry-desktop and lbry-android.
### LBRY Protocol
LBRY is an open-source protocol providing distribution, discovery, and purchase of digital content (data) via a decentralized network. It utilizes the LBRY blockchain as a global namespace and database of digital content. Blockchain entries contain searchable content metadata, identities, and rights and access rules. LBRY also provides a data network that consists of peers uploading and downloading data from other peers, possibly in exchange for payments, and a distributed hash table, used by peers to discover other peers.
### LBRY Reference Application
For most users, LBRY will be a place where they can find great videos, music, ebooks, and more. A vast digital library that is available on all of your devices. But LBRY is many components working together. The LBRY app is a graphical browser for the decentralized content marketplace provided by the LBRY protocol. It is essentially the lbry daemon bundled with an UI using Electron.
### LBRYCrd
LBRYcrd uses a blockchain similar to bitcoin's to implement an index and payment system for content on the LBRY network. It is a fork of Core.
### LbryumX Server
A LbryumX-server for the LbryumX client. LbryumX is an extension of electrumx that provides the server side of LBRY Electrum Protocol. It sits between the LBRY daemon and LBRYCrd to provide SPV access to/from clients.
### Lighthouse
[Lighthouse](https://github.com/lbryio/lighthouse) is a lightning-fast advanced search engine API for publications on the LBRYcrd with autocomplete capabilities. The official lighthouse instance is live at https://lighthouse.lbry.io
### Lightweight client
Comparing to a *full node*, lightweight node does not store the whole blockchain and thus cannot fully verify any transaction. There are two kinds of lightweight nodes: those fully trusting an external service to determine wallet balance and validity of transactions (e.g. *blockchain.info*) and the apps implementing *Simplified Payment Verification* (SPV). SPV clients do not need to trust any particular service, but are more vulnerable to a *51% attack* than full nodes. See *Simplified Payment Verification* for more info.
### Mainnet
Main LBRY network and its blockchain. The term is mostly used in comparison to *testnet*.
### Main Chain
A part of the blockchain which a node considers the most difficult (see *difficulty*). All nodes store all valid blocks, including *orphans* and recompute the total difficulty when receiving another block. If the newly arrived block or blocks do not extend existing main chain, but create another one from some previous block, it is called *reorganization*.
### Merkle Tree
Merkle tree is an abstract data structure that organizes a list of data items in a tree of their hashes (like in Git, Mercurial or ZFS). In LBRY the merkle tree is used only to organize transactions within a block (the block header contains only one hash of a tree) so that full nodes may prune fully spent transactions to save disk space. Click [here](https://en.bitcoin.it/wiki/Protocol_documentation#Merkle_Trees) for more details.
### Mempool
A technical term for a collection of unconfirmed transactions stored by a node until they either expire or get included in the main chain. When *reorganization* happens, transactions from orphaned blocks either become invalid (if already included in the *main chain*) or moved to a pool of unconfirmed transactions. By default, *bitcoind* nodes throw away unconfirmed transactions after 24 hours.
### Mining
A process of finding valid *hashes* of a block header by iterating millions of variants of block headers (using *nonce* and *extra nonce*) in order to find a hash lower than the *target* (see also *difficulty*). The process needs to determine a single global history of all transactions (grouped in blocks). Mining consumes time and electricity and nowadays the difficulty is so big, that energy-wise it's not even profitable to mine using video graphics cards. Mining is paid for by *transaction fees* and by block *rewards* (newly generated coins, hence the term "mining").
### Mining Pool
A service that allows separate owners of mining hardware to split the reward proportionally to submitted work. Since probability of finding a valid block hash is proportional to miner's *hashrate*, small individual miners may work for months before finding a big per-block reward. Mining pools allow more steady stream of smaller income. Pool owner determines the block contents and distributes ranges of *nonce* values between its workers. Normally, mining pools are centralized. P2Pool is a fully decentralized pool.
### Miner
A person, a software or a hardware that performs *mining*.
### M-of-N Multi-signature Transaction
A transaction that can be spent using M signatures when N public keys are required (M is less or equal to N). Multi-signature transactions that only contain one *OP_CHECKMULTISIG* opcode and N is 3, 2 or 1 are considered *standard*.
### Node
Node, or client, is a computer on the network that speaks LBRY message protocol (exchanging transactions and blocks). There are *full nodes* that are capable of validating the entire blockchain and *lightweight nodes*, with reduced functionality. Wallet applications that speak to a server are not considered nodes.
### Nonce
Stands for "number used once". A 32-bit number in a *block header* which is iterated during a search for proof-of-work. Each time the nonce is changed, the *hash* of the block header is recalculated. If nonce overflows before valid proof-of-work is found, an *extra nonce* is incremented and placed in the *coinbase* script. Alternatively, one may change a merkle tree of transactions or a timestamp.
### Opcode
8-bit code of a *script* operation. Codes from 0x01 to 0x4B (decimal 75) are interpreted as a length of data to be pushed on the stack of the interpreter (data bytes follow the opcode). Other codes are either do something interesting, or disabled and cause transaction verification to fail, or do nothing (reserved for future use). LBRY implemented special op codes for storing and updating of claim data.
### Orphan, Orphaned Block
A valid block that is no longer a part of a *main chain*. Usually happens when two or more blocks of the same *height* are produced at the same time. When one of them becomes a part of the main chain, others are considered "orphaned". Orphans also may happen when the blockchain is *forked* due to an attack (see *51% attack*) or a bug. Then a chain of several blocks may become abandoned. Usually a transaction is included in all blocks of the same height, so its *confirmation* is not delayed and there is no *double spend*. See also *Fork*.
### Outpoint
An outpoint, as referenced in API documentation and elsewhere, is the most specific identification for a particular version of a claim (a claim may be updated and will be referenced by a new outpoint). The outpoint is the concatenation of the transaction id and nout (position in the transaction). Outpoint example: `f6dea4ad26fd526b77935969a17b081342fc92d68b3a1daf69d4a3378657c2fc:0`
### Peer
A peer is one instance of a client running on a computer on the Internet to which other clients connect and transfer data. Depending on context, "peer" can refer either to any client in the swarm or more specifically to a downloader, a client that has only parts of the file.
### Pay-to-Script Hash
A type of the *script* and *address* that allows sending bitcoins to arbitrary complex scripts using a compact hash of that script. This allows payer to pay much smaller *transaction fees* and not wait very long for a *non-standard* transaction to get included in the blockchain. Then the actual script matching the hash must be provided by the payee when redeeming the funds. P2SH addresses are encoded in *Base58Check* just like regular public keys and start with number "3".
### Paper Wallet
A form of *cold storage* where a *private key* for LBRY Credits *address* is printed on a piece of paper (with or without encryption) and then all traces of the key are removed from the computer where it was generated. To redeem bitcoins, a key must be imported in the wallet application so it can sign a transaction. See also *Casascius Coins*.
### Proof-of-Work (PoW)
A number that is provably hard to compute. That is, it takes measurable amount of time and/or computational power (energy) to produce. In LBRY, similar to Bitcoin, it is a *hash* of a *block header*. A block is considered valid only if its hash is lower than the current *target* (roughly, starts with a certain amount of zero bits). Each block refers to a previous block thus accumulating previous proof-of-work and forming a *blockchain*.
Proof-of-work is not the only requirement, but an important one to make sure that it is economically infeasible to produce an alternative history of transactions with the same accumulated work. Each client can independently consider the most difficult chain of valid blocks as the "true" history of transactions, without need to trust any source that provides the blocks.
Note that owning a very large amount of computational power does not override other rules enforced by every client. Ill-formed blocks or blocks containing invalid transactions are rejected no matter how difficult they were to produce.
### Private Key (Privkey)
A 256-bit number used in *ECDSA* algorithm to create transaction *signatures* in order to prove ownership of certain amount of credits. Can also be used in arbitrary *elliptic curve arithmetic* operations. Private keys are stored within *wallet* applications and are usually encrypted with a pass phrase. Private keys may be completely random (see *Key Pool*) or generated from a single secret number ("seed"). See also *Deterministic Wallet*.
### Public Key (Pubkey)
A 2D point on an elliptic curve [secp256k1](https://en.bitcoin.it/wiki/Secp256k1) that is produced by multiplying a predefined "generator" point by a *private key*. Usually it is represented by a pair of 256-bit numbers ("uncompressed public key"), but can also be compressed to just one 256-bit number (at the slight expense of CPU time to decode an uncompressed number). A special hash of a public key is called *address*. Typical LBRY transactions contain public keys or addresses in the output scripts and *signatures* in the input scripts.
### Reflector
A reflector cluster to accept LBRY content for hosting en masse, rehost the content, and make money on data fees (currently disabled). This code includes Go implementations of the LBRY peer protocol, reflector protocol, and DHT.
### Reorg, Reorganization
An event in the *node* when one or more blocks in the *main chain* become *orphaned*. Usually, newly received blocks are extending existing main chain. Sometimes (4-6 times a week) a couple of blocks of the same *height* are produced almost simultaneously and for a short period of time some nodes may see one block as a tip of the main chain which will be eventually replaced by a more difficult block(s). Each transaction in the orphaned blocks either becomes invalid (if already included in the main chain block) or becomes *unconfirmed* and moved to the *mempool*. In case of a major bug or a *51% attack*, reorganization may involve reorganizing more than one block.
### Resolve
The resolve API command returns all available information about a claim or channel.
### Reward
Amount of newly generated LBRY credits that a *miner* may claim in a new block. The first transaction in the block allows miner to claim currently allowed reward as well as all *transaction fees* from all transactions in the block. For security reasons, rewards cannot be *spent* before 100 blocks built on top of the current block.
### Stream Descriptor (SD) Blob
The initial blob of a stream, which contains encryption information as well as points to other blobs required for a stream.
### Script
A compact turing-incomplete programming language used in transaction *inputs* and *outputs*. Scripts are interpreted by a Forth-like stack machine: each operation manipulates data on the stack. Most scripts follow the standard pattern and verify the digital *signature* provided in the transaction *input* against a *public key* provided in the previous transaction's *output*. Both signatures and public keys are provided using scripts. Scripts may contain complex conditions, but can never change amounts being transferred. Amount is stored in a separate field in a *transaction output*.
### Signature
A sequence of bytes that proves that a piece of data is acknowledged by a person holding a certain *public key*. LBRY, like Bitcoin, uses *ECDSA* for signing transactions. Amounts of credits are sent through a chain of transactions: from one to another. Every transaction must provide a signature matching a public key defined in the previous transaction. This way only a proper owner of a secret *private key* associated with a given public key can spend credits further.
### Simplified Payment Verification (SPV)
A scheme to validate transactions without storing the whole blockchain (only block headers) and without trusting any external service. Every transaction must be present with all its parent and sibling hashes in a *merkle tree* up to the root. SPV client trusts the most *difficult* chain of block headers and can validate if the transaction indeed belongs to a certain block header. Since SPV does not validate all transactions, a *51% attack* may not only cause a *double spend* (like with *full nodes*), but also make a completely invalid payment with credits created from nowhere. However, this kind of attack is very costly and probably more expensive than a product in question.
### Secret key
Either the *Private Key* or an encryption key used in encrypted *wallets*. LBRY protocol does not use encryption anywhere, so *secret key* typically means a *private key* used for signing transactions.
### Soft Fork
Sometimes the *soft fork* refers to an important change of software behavior that is not a *hard fork* (e.g. changing *mining fee* policy). See also *Hard Fork* and *Fork*.
### Spam
Incorrect peer-to-peer messages (like sending invalid transactions) may be considered a denial of service attack (see *DoS*). Valid transactions sending very tiny amounts and/or having low *mining fees* are called *Dust* by some people. The protocol itself does not define which transactions are not worth relaying or mining, it's a decision of every individual node. Any valid transaction in the blockchain must be accepted by the node if it wishes to accept the remaining blocks, so transaction censorship only means increased confirmation delays. Individual payees may also blacklist certain addresses (refuse to accept payments from some addresses), but that's too easy to work around using *mixing*.
### Spent Output
A transaction *output* can be spent only once: when another valid transaction makes a reference to this output from its own input. When another transaction attempts to spend the same output, it will be rejected by the nodes already seeing the first transaction. Blockchain as a *proof-of-work* scheme allows every node to agree on which transaction was indeed the first one. The whole transaction is considered spent when all its outputs are spent.
### Stream
Streaming media is multimedia that is constantly received by and presented to an end-user while being delivered by a provider. In LBRY, streams as associated with claim data in order to provide the capability to download files over a Peer to Peer network.
### Target
A 256-bit number that puts an upper limit for a block header hash to be valid. The lower the target is, the higher the *difficulty* to find a valid hash. The maximum (easiest) target is 0x00000000FFFF0000000000000000000000000000000000000000000000000000. The difficulty and the target are adjusted every 2016 blocks (approx. 2 weeks) to keep interval between the blocks close to 10 minutes.
### Takeover Period
In order to take over a claim at an existing vanity URL, the bid must be higher and takeover period must pass. In simple terms, the longer the claim is held, the longer the takeover period. For each month held, a day is added to the take over period for a maximum of 7 days. See [Claimtrie Bid States section here](https://lbry.io/faq/claimtrie-implementation) for more information.
### Testnet
A set of parameters used for testing a LBRY network. Testnet is like *mainnet*, but has a different genesis block (it was reset several times, the latest testnet is *testnet3*). Testnet uses slightly different *address* format to avoid confusion with main LBRY addresses and all nodes are relaying and mining non-standard transactions.
### Transaction
A chunk of binary data that describes how credits are moved from one owner to another. Transactions are stored in the *blockchain*. Every transaction (except for *coinbase* transactions) has a reference to one or more previous transactions (*inputs*) and one or more rules on how to spend these credits further (*outputs*). See *Transaction Input* and *Transaction Output* for more info.
### Transaction Fee
Also known as "miners' fee", an amount that an author of transaction pays to a miner who will include the transaction in a block. The fee is expressed as difference between the sum of all *input* amounts and a sum of all *output* amounts. Unlike traditional payment systems, miners do not explicitly require fees and most miners allow free transactions. All miners are competing between each other for the fees and all transactions are competing for a place in a block. There are soft rules encoded in most clients that define minimum fees per kilobyte to relay or mine a transaction (mostly to prevent *DoS* and *spam*). Typically, the fee affects the priority of a transaction.
### Transaction Input
A part of a transaction that contains a reference to a previous transaction's *output* and a *script* that can prove ownership of that output. The script usually contains a *signature* and thus called *scriptSig*. Inputs spend previous outputs completely. So if one needs to pay only a portion of some previous output, the transaction should include extra *change* output that sends the remaining portion back to its owner (on the same or different address). *Coinbase* transactions contain only one input with a zeroed reference to a previous transaction and an arbitrary data in place of script.
### Transaction Output
An output contains an amount to be sent and a *script* that allows further spending. The script typically contains a *public key* (or an *address*, a hash of a public key) and a signature verification *opcode*. Only an owner of a corresponding *private key* is able to create another transaction that sends that amount further to someone else. In every transaction, the sum of output amounts must be equal or less than a sum of all input amounts. See also *Change*.
### Unconfirmed Transaction
Transaction that is not included in any block. Also known as "0-confirmation" transaction. Unconfirmed transactions are *relayed* by the nodes and stay in their *mempools*. Unconfirmed transaction stays in the pool until the node decides to throw it away, finds it in the blockchain, or includes it in the blockchain itself (if it's a miner). See also *Confirmation Number*.
### UTXO Set
A collection of *Unspent Transaction Outputs*. Typically used in discussions on optimizing an ever-growing index of *transaction outputs* that are not yet *spent*. The index is important to efficiently validate newly created transactions. Even if the rate of the new transactions remains constant, the time required to locate and verify unspent outputs grows.
### Wallet
An application or a service that helps keeping private keys for signing transactions. Wallet does not keep LBRY credits themselves (they are recorded in *blockchain*). "Storing LBC" usually means storing the keys.
### 51% Attack
Also known as >50% attack or a *double spend* attack. An attacker can make a payment, wait till the merchant accepts some number of *confirmations* and provides the service, then starts mining a parallel chain of blocks starting with a block before the transaction. This parallel blockchain then includes another transaction that spends the same *outputs* on some other address. When the parallel chain becomes more *difficult*, it is considered a *main chain* by all nodes and the original transaction becomes invalid. Having more than a half of total *hashrate* guarantees possibility to overtake chain of any length, hence the name of an attack (strictly speaking, it is "more than 50%", not 51%). Also, even 40% of hashrate allows making a double spend, but the chances are less than 100% and are diminishing exponentially with the number of confirmations that the merchant requires.
About
-----
Portions of this glossary originated from: [https://github.com/oleganza/bitcoin-papers/blob/master/BitcoinGlossary.md](https://github.com/oleganza/bitcoin-papers/blob/master/BitcoinGlossary.md)

75
documents/lbry-claimtrie.md Normal file
View file

@ -0,0 +1,75 @@
## Introduction
This document describes the implementation detail of the ClaimTrie in LBRY. The ClaimTrie is the data structure which LBRY uses to store claims to names. It uses a [Trie](https://en.wikipedia.org/wiki/Trie) to efficiently store all claimed names, which can then be hashed the same way a [Merkle Tree](https://en.wikipedia.org/wiki/Merkle_tree) is hashed. The root hash of the ClaimTrie is stored in the blockheader of each LBRY block, enabling nodes in the LBRY network to efficiently and securely validate the state of the ClaimTrie.
Bids to claim a name must win out against other claims for the same name before they can be inserted into the ClaimTrie. The short summary is that the bid with the most LBRY credits assigned to it will win the right to claim a name, but the implementation detail is more involved and, this is what we aim to cover in this document. Bids to claim a name have four properties tied to it :
1. *Name* : The name is a human-readable address and is the property that the bids compete to obtain.
2. *Value* : The value is the data that is attached to the name.
3. *Quantity* : The quantity is the number of LBRY credits assigned to the bid.
4. *Claim Id* : A unique ID used to identify the bid.
There are also three different bid types: claim, update, and support.
1. *Claim*: A claim represent new bids for a name. If a user wants to make a claim to a brand new name, or submit a competing claim to an existing name, this bid type is used.
2. *Support*: A support adds to the total quantity of credits assigned to any bid by referring to a bid's Claim Id. A support bid can be made by anyone on any bid. It does not have its own Value or its own Claim Id, but it does contain the Claim Id of the bid that it is supporting.
3. *Update*: An update can modify the value and the quantity for a pre-existing claim without changing the Claim Id or the name that it is bidding on. Since the Claim Id of the original bid is not changed, an updated bid will still retain all the supports attached to the original bid.
## ClaimTrie Bid States
This section describes how bids are processed by the ClaimTrie in order to determine which bids have won the rights to claim a particular name. There are 6 states a bid can be in, and they are explained below.
1. *Not accepted*: This bid is in a transaction which has not yet been included in a block which has been included in the blockchain.
2. *Accepted*: This bid has been accepted into the blockchain. This happens when the transaction containing the TXout which contains the bid is included in a block which is included in the blockchain.
3. *Active*: This bid is capable of controlling a name. Active bids must be in the "accepted" state and not "expired" or "spent". Bids are "active" when either of the two conditions below is met:
* The current block height exceeds the height of the block at which the bid became accepted plus the activation delay for the name as calculated at either the block at which the bid was accepted or any block after the bid was accepted. The activation delay is calculated as follows:
* If, immediately before this block was included in the blockchain, there were no ‘active’ bids for the name and therefore no ‘controlling’ bids, the delay is 0.
* If there is a "controlling" bid for the name: Delay = (HeightB - HeightA) / 32
* HeightA = the most recent height at which the bid controlling the name changed
* HeightB = the current height
* Maximum delay is 7 days of blocks at 2.5 min/block (or 4032 blocks). Thus maximum delay can be reached in 224 (7x32) days.
* The bid’s Claim Id matches the Claim Id of the bid which was the controlling bid immediately before the block containing this bid was included in the blockchain. In other words, it is either an update to the previous controlling bid or update to update to the previous controlling bid if the bid was updated twice in this block, etc.
4. *Controlling*: This bid currently controls the name. When clients ask which bid controls the name as of the current block, this is the bid that will be returned. Must be in the "active" state and only one bid for any name may be in this state. A support cannot be in the "controlling" state. To determine which "active" bid is the "controlling" bid for each name:
* Add the quantity of each ‘active’ bid to the quantity of all ‘active’ supports for that bid, and take whichever is greatest. If two bids have the same quantity, older bids take precedence over newer bids.
* If the bid with the greatest amount does not have the same claimID as the bid which was ‘controlling’ prior to including the current block, change the delay for the name as of the current block to 0, redetermine which bids and supports should be active, and then perform the previous calculation again.
* At this point, the bid calculated to have the greatest amount behind it is the ‘controlling’ bid as of this block
5. *Spent*: A transaction has been included in the blockchain which spends the TXout which contains the bid. Must be in the ‘accepted’ state.
6. *Expired*: All bids ‘expire’ regardless of what state they are in when the current block height exceeds the height of the block at which the bid was accepted plus 52416 blocks, or 91 days ( currently this is set to 262974 blocks, or 456 days, which will be fixed in a future hard fork ). Updated claims will restart the expiration timer at the block height of the update.
## ClaimTrie Transaction Implementation
This section describes how the three ClaimTrie bid types are implemented as transactions on the blockchain. Readers should have prior knowledge of Bitcoin [transactions](https://en.bitcoin.it/wiki/Transaction) and the Bitcoin [scripting system](https://en.bitcoin.it/wiki/Script). LBRY supports three op codes that do not exist in Bitcoin: OP_CLAIM_NAME, OP_SUPPORT_CLAIM, and OP_UPDATE_CLAIM (in Bitcoin they are respectively OP_NOP6, OP_NOP7, and OP_NOP8). Each op code will push a zero on to the execution stack, and in addition, will trigger the ClaimTrie to perform calculations necessary for each bid type. Below are the three supported transactions scripts using these op codes.
```python
OP_CLAIM_NAME <Name> <Value> OP_2DROP OP_DROP [script pubkey]
OP_UPDATE_CLAIM <Name> <ClaimId> <Value> OP_2DROP OP_2DROP [script pubkey]
OP_SUPPORT_CLAIM <Name> <ClaimId> OP_2DROP OP_DROP [script pubkey]
```
[script pubkey] can be any valid Bitcoin payout script, thus it can be something like a standard "pay to pubkey" script to a user controlled address. Also note that the zero pushed onto the stack by the ClaimTrie op codes, and the ClaimTrie vectors, are all dropped by the preceding OP_2DROP and OP_DROP. This means that ClaimTrie transactions exist as prefixes to Bitcoin payout scripts and can be spent in the same way as is expected in Bitcoin.
For example, a claim transaction using a pay to pubkey script will have the below full payout script. Let's also say that this claim is for the name "Fruit" to be set to value "Apple".
```python
OP_CLAIM_NAME <Fruit> <Apple> OP_2DROP OP_DROP OP_DUP OP_HASH160 <LBRY_Address_A> OP_EQUALVERIFY OP_CHECKSIG
```
Like any standard Bitcoin transaction output script, it will be associated with a transaction hash and a transaction output index. The transaction hash and transaction output index is concatenated and hashed using RIPEMD-160 to create the Claim Id for this claim. For the example above, let's say it has a Claim Id X. A support for this bid will have the below full payout script.
```python
OP_SUPPORT_CLAIM <Fruit> <X> OP_2DROP OP_DROP OP_DUP OP_HASH160 <LBRY_Address_B> OP_EQUALVERIFY OP_CHECKSIG
```
And now let's say we want to update the original claim to change the value to "Banana". An update transaction has a special requirement that it must spend the existing claim that it wishes to update in its redeem script. Otherwise, it will be considered invalid and will not make it into the ClaimTrie. Thus it will have the below redeem script to spend the claim created to set name "Fruit" to "Apple". Note that this is identical to the standard way of redeeming a "pay to pubkey" script in Bitcoin.
```python
<Signature> <Public_key_for_LBRY_Address_A>
```
And the payout script for the update transaction is below.
```python
OP_UPDATE_CLAIM <Fruit> <X> <Banana> OP_2DROP OP_2DROP OP_DUP OP_HASH160 <LBRY_Address_C> OP_EQUALVERIFY OP_CHECKSIG
```

0
views/pages/overview.md → documents/overview.md
View file

19
documents/pow.md Normal file
View file

@ -0,0 +1,19 @@
## The LBRY Proof of Work (POW) Algorithm
LBRY uses [proof of work](https://en.bitcoin.it/wiki/Proof_of_work) the same way that Bitcoin does. The
only difference is the hash function. LBRY uses a slightly different algorithm that achieves the same ends but slightly delayed the development of a GPU miner and gave early adopters a chance to mine without specialized hardware.
LBRY's algorithm is
```python
intermediate = sha512(sha256(sha256(data))) # compute the sha512() of the double-sha256() of the data
left = ripemd(intermediate[:len(intermediate)/2]) # separately ripemd160 the left half
right = ripemd(intermediate[len(intermediate)/2:]) # and the right half
proof = sha256(sha256(left + right)) # concatenate the two halves, and double-sha256() it again
```
For comparison, Bitcoin's algorithm is...
```python
proof = sha256(sha256(data))
```

202
documents/regtest-setup.md Normal file
View file

@ -0,0 +1,202 @@
## Why use a regtest server
A regtest server provides for a way to instantly generate blocks so that transactions can be instantaneous, so ultimately no waiting for confirmations from the blockchain. Also, no problem if you accidentally corrupt your wallet, since no real funds are lost! Delete the files and setup a new one.
## Setup
To begin setting up the network, there are a few things you need.
You'll need a Linux or a Mac distribution to run all this. A virtual machine is fine.
Note: These instructions specifically were tested on Ubuntu version 16.04.
### Virtual Environment
First up it's a good idea to create a Python virtual environment. This requires you to have a functional python2.7 setup, with the Python package manager `pip` installed. To create a new virtual environment in a folder `lbry-env`, run this:
`virtualenv -p /usr/bin/python2.7 lbry-env`
To enter the environment, run:
`source lbry-env/bin/activate`.
### lbrycrd
You need to download a build of `lbrycrd` from [here](https://github.com/lbryio/lbrycrd/releases/), no installation required. To configure `lbrycrd` you need to create a file at `~/.lbrycrd/lbrycrd.conf`,
containing the following.
```ini
rpcuser=test
rpcpassword=test
rpcport=18332
regtest=1
server=1
txindex=1
daemon=1
listen=0
discover=0
```
### lbryum-server
To install lbryum-server, you first need to install the package `leveldb`. After that, download the source from [here](https://github.com/lbryio/lbryum-server/releases), and run the following _not_ inside the environment.
```bash
cd lbryum-server
sudo pip2 install -r requirements.txt
```
If you're not running debian/\*buntu or a derivative of those, you need to edit the `configure` file a bit. In line 11, remove the `apt-get` line and manually install the required packages. In line 51, change `adduser` to `useradd` and on the same line, change `--disabled-password` to `-p !`.
```bash
sudo ./configure
sudo python2 setup.py install
```
The `sudo ./configure` command creates a new user in the system by the name "lbryum", which is the user through which we'll be the running the server. lbryum-server also need W/R access to `/var/lbryum-server`
To do that run:
```bash
sudo chown -R lbryum /var/lbryum-server
```
When installed, append/use the following config options to the `/etc/lbryum.conf` file.
```ini
[lbrycrdd]
lbrycrdd_host = localhost
lbrycrdd_port = 18332
# user and password from lbrycrd.conf
lbrycrdd_user = test
lbrycrdd_password = test
[network]
type=lbrycrd_regtest
```
### lbryum
To install lbryum, first, download the source from [here](https://github.com/lbryio/lbryum/releases). To install it, run the following inside the virtual environment.
```bash
cd lbryum
pip2 install -r requirements.txt
pip2 install -e .
```
After installation completes, you must set the config option for lbryum using:
```bash
lbryum setconfig default_servers '{ "localhost": { "t": "50001" }}'
lbryum setconfig chain 'lbrycrd_regtest'
```
Alternatively, you can create a file `touch ~/.lbryum/config` and paste the following config:
```json
{
"chain": "lbrycrd_regtest",
"default_servers": {
"localhost": {
"t": "50001"
}
}
}
```
### lbry
Download source from [here](https://github.com/lbryio/lbry/releases), and run the following inside the environment.
```bash
cd lbry
pip2 install -r requirements.txt
pip2 install -e .
mkdir ~/.lbrynet
touch ~/.lbrynet/daemon_settings.yml
```
Append the following in the newly created `~/.lbrynet/daemon_settings.yml` file
```yml
blockchain_name: lbrycrd_regtest
lbryum_servers:
- localhost:50001
reflect_uploads: false
share_usage_data: false
use_upnp: false
```
### Last step
Go to the `lbryum` folder once again and run:
```bash
pip2 install -e .
```
This is to ensure that `lbrynet-daemon` uses the correct wallet.
## Firing up the regtest server
### Wallet backup
To start off, if you've already used LBRY on your machine, you need to backup the wallet by copying the folders `~/.lbrynet` and `~/.lbryum`, then delete them to start from fresh. Run
`mkdir ~/.lbryum`
Now it should be all set-up, just execute the commands in the following order, and the regtest server should be good to go.
### 1) lbrycrd
To run the `lbrycrd` daemon, run the following in the `lbrycrd` folder.
`./lbrycrdd`
To generate blocks, run `./lbrycrd-cli generate <num_of_blocks>`
You'll need to generate some blocks to get the network going. Start off by generating at least 100.
`./lbrycrd-cli generate 173`
If you'd prefer a more verbose output from lbrycrdd, run lbrycrd using
`./lbrycrdd -printtoconsole`
### 2) lbryum-server
To run the server, run:
```bash
sudo runuser -l lbryum -c 'lbryum-server --conf=/etc/lbryum.conf'
```
Note: conf flag can be left out if the config is in the default directory(default: `/etc/lbryum.conf`)
### 3) lbryum
To run the lbryum, run:
```bash
lbryum daemon start
```
Generate some more blocks, get a wallet address by running:
`lbryum getunusedaddress`
and then send some credits to your wallet by doing
`./lbrycrd-cli sendtoaddress <address> <num_of_credits>`
### 4) lbry
You can now run `lbrynet-daemon`, and it should connect to the `lbryum`. Now you can use the regtest stack as you would normally use lbryum.
## Shutdown
To stop the network, run `lbrynet-cli daemon_stop`, `lbryum daemon stop`, and kill the `lbryum-server` process and stop lbrycrd by `lbrycrdd-cli stop`. If you want to use your wallet and the official servers again, backup the new regtest wallet, and replace it with your own.
## Note 1
You need to generate a few blocks everytime you make a new transaction in the form of send, receive, claim, update, publish, support, tip etc. for it to show up in the daemon and lbryum etc.
## Note 2
If something goes wrong and you get a "Block not found" error, remember to delete `/var/lbryum-server` before trying again.
## Cheatsheet
#### Required processes in the correct order
```bash
lbrycrdd
sudo runuser -l lbryum -c 'lbryum-server --conf=/etc/lbryum.conf'
lbryum daemon start
lbrynet-daemon
```
#### Generate blocks
```bash
lbrycrd-cli generate 5
```
#### Get a wallet address
```bash
lbryum getunsusedaddress
```
#### Send credits from lbrycrd to your wallet
```bash
lbrycrd-cli sendtoaddress <address> <num_of_credits>
```

101
documents/repository-standards.md Normal file
View file

@ -0,0 +1,101 @@
# LBRY Repository Documentation Standards
This document outlines the standards for all public, open-source repositories used by LBRY.
The goal of this document is to ensure that LBRY documentation is welcoming, informative, comprehensive, and well-written.
Each codebase should include the following documents, committed as markdown files at the top level of the repository.
## README.md
This document exists to introduce a project to a new visitor. It may also serve as a reference document for existing contributors. This document should include the following:
### Title / Heading
* The title (name) of the project at the top as an h1
* Any build status badges as appropriate immediately below the title
* A one or two sentence description of the project, below the title or status badges. The description should mention the language, and any key frameworks (e.g. "lbry" should mention it uses both Python and Twisted)
* A screenshot, image or gif of the project below the description (host it on spee.ch!)
* A table of contents if the document is long (can be one line of links to sections).
### Installation
* A single header labeled "Install" should be the next header as an h2
* Installation means installation for users. It should cover how to have the software on their computer in a user-friendly fashion, not how to run from source. E.g. it should link binaries if they exist.
* If the project is a library only intended to be used in other projects, it should still exist but can be extremely succinct (e.g. pip install lbryschema)
* If the project is not designed to be installed directly or used as a library in other projects, it should state as such, e.g. "This project is not designed to be installed directly. Continue reading below to learn how to use this project."
### Usage
* A single header labeled "Usage" should be the next header as an h2
* The Usage section should explain how to run or launch the project if applicable.
* The Usage section should include examples of some basic commands, if applicable.
* In the case of a library, it should provide a few basic examples of how the library would be used.
* Usage should always be included, even if very simple. For example, "Double click the installed application to browse with the LBRY network."
### Running from Source
* This section covers how to run the project from source and/or with the intent of working directly on it.
* It can have a Prerequisites section for what is required to run from source.
* It is okay to assume some basic assumptions about what people know about the language or tools the project uses. However, it’s good when possible to assume very little, and provide links to the user for how to get the prerequisites.
* For prerequisites that it is not safe to assume knowledge of, the project should list the explicit command to add the prerequisite, or link to instructions on how to add the prerequisite.
* If there are operating system specific instructions, these should be broken out into separate headings for each operating system.
* It is okay to assume Linux and Unix are "first class citizens". While we want to support all OSes, Windows instructions and considerations can be secondary.
* Include a step that explains how to verify you are running from source directly.
* It is okay to point to a INSTALL.md file if the installation steps are relatively lenghty, which should reside in the root folder.
### Contributing
* A single header labeled "Contributing" should appear as an h2
* This should be the same copy: "Contributions to this project are welcome, encouraged, and compensated. For more details, see [CONTRIBUTING.md](*CONTRIBUTING.md*)"
* If CONTRIBUTING.md does not exist in the project, it should link to [https://lbry.io/faq/contributing](https://lbry.io/faq/contributing) (soon to be lbry.tech/contributing)
### (Additional Headings)
* Additional headings as appropriate will typically exist after the above and below the subsequent areas.
* These areas should cover anything else appropriate or helpful to introduce a project to a new user.
### License
* A single header labeled "License" should appear as an h2
* Assuming a standard license, this should be the same two sentences: "This project is X licensed. For the full license, see [LICENSE]."
### Security
* "We take security seriously. Please contact [security@lbry.io](mailto:security@lbry.io) regarding any security issues. Our PGP key is [here](https://keybase.io/lbry/key.asc) if you need it."
### Contact
* A single header labeled "Contact" should appear as an h2
* This should be the same or similar copy: "The primary contact for this project is [@XXX](https://github.com/@XXX) ([xxx@lbry.io](mailto:xxx@lbry.io))"
### Additional Info and Links
* (optional)
* References to any additional documentation, such as generated API docs.
## CONTRIBUTING.md
This document explains anything a visitor would need to know to contribute to the project.
This document should cover the following:
* First, it should contain a single sentence: "This project follows the global contributing standards for all LBRY projects, to read those go < here >".
* A "Code Overview" section explaining some basic design choices and how to begin stepping through the code. An example would be explaining that Daemon.py is the primary entry point for the daemon code, and one can begin to trace through the code by looking for jsonrpc_xxx, where xxx is one of the api calls listed [here](https://lbry.io/api).
* A "Testing" section, explaining how to run tests, and that tests are necessary.
* Information on how to submit pull requests, and what to expect afterwards. (e.g. a link to our [branching doc](https://github.com/lbryio/lbry/wiki/Branching-and-Merging), commands to run before submitting PR, tests must pass, changelog entry, etc). If you find this gets repetitive, it may be best to link to a global doc.
* Anything else a new visitor to a repository should know about contributing to that specific repository (linting, generating documentation, etc).
## LICENSE
Every repository should have a LICENSE file stating the license. The default license we use is MIT, and we license all code as MIT whenever possible.
Some code may use external libraries that prevent MIT licensing. If adding a license to a project for the first time that uses 3rd-party code, please ensure it is okay to actually MIT license it.
## ISSUE_TEMPLATE.md
A template for issues should exist to guide users in correctly filing them.
## Style and Formatting Notes
- Rely on autowrap instead of manually breaking up paragraphs with carriage return.

130
documents/schema.md Normal file
View file

@ -0,0 +1,130 @@
## [Claim](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/claim.proto)
Claims have the encompassing schema:
message Claim {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
enum ClaimType {
UNKNOWN_CLAIM_TYPE = 0;
streamType = 1;
certificateType = 2;
}
required ClaimType claimType = 2;
optional Stream stream = 3;
optional Certificate certificate = 4;
optional Signature publisherSignature = 5;
}
## [Stream](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/stream.proto)
Claims of streamType have a `stream`:
message Stream {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required Metadata metadata = 2;
required Source source = 3;
}
## [Metadata](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/metadata.proto)
Streams have `metadata` describing their content:
message Metadata {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
_0_0_2 = 2;
_0_0_3 = 3;
_0_1_0 = 4;
}
enum Language {
UNKNOWN_LANGUAGE = 0;
en = 1;
}
required Version version = 1;
required Language language = 2;
required string title = 3;
required string description = 4;
required string author = 5;
required string license = 6;
required bool nsfw = 7;
optional Fee fee = 8;
optional string thumbnail = 9;
optional string preview = 10;
optional string licenseUrl = 11;
}
## [Fee](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/fee.proto)
Metadata may include a fee to access the decryption key:
message Fee {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
enum Currency {
UNKNOWN_CURRENCY = 0;
LBC = 1;
BTC = 2;
USD = 3;
}
required Version version = 1;
required Currency currency = 2;
required bytes address = 3;
required float amount = 4;
}
## [Source](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/source.proto)
Streams have a `source` to download:
message Source {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
enum SourceTypes {
UNKNOWN_SOURCE_TYPE = 0;
lbry_sd_hash = 1;
}
required SourceTypes sourceType = 2;
required bytes source = 3;
required string contentType = 4;
}
## [Certificate](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/certificate.proto)
Claims of certificateType have a `certificate`:
message Certificate {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required KeyType keyType = 2;
required bytes publicKey = 4;
}
## [Signature](https://github.com/lbryio/lbryschema/blob/master/lbryschema/proto/signature.proto)
Claims may be signed using the private key to a Certificate public key:
message Signature {
enum Version {
UNKNOWN_VERSION = 0;
_0_0_1 = 1;
}
required Version version = 1;
required KeyType signatureType = 2;
required bytes signature = 3;
required bytes certificateId = 4;
}

129
documents/signing-claim.md Normal file
View file

@ -0,0 +1,129 @@
### Signing a claim
Reilly wants to publish _Terror on the Midway_ to the channel he claimed `lbry://@FleischerSuperman`. He picks the name "terroronthemidway", and fills in the information for the claim:
{
"claimType": "streamType",
"stream": {
"metadata": {
"author": "Paramount Pictures",
"description": "The episode in the series of Fleischer Studios-produced Superman serials. Subsequent episodes were produced by Famous Studios.",
"language": "en",
"license": "Public Domain",
"licenseUrl": "",
"nsfw": false,
"preview": "",
"thumbnail": "https://s3.amazonaws.com/files.lbry.io/thumbnails/superman1940-thumb.png",
"title": "Terror on the Midway - Superman Ep 9",
"version": "_0_1_0"
},
"source": {
"contentType": "video/mp4",
"source": "9b70337f51fe9a4481504059b4220ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea",
"sourceType": "lbry_sd_hash",
"version": "_0_0_1"
},
"version": "_0_0_1"
},
"version": "_0_0_1"
}
This is serialized as:
080110011ae6020801129e02080410011a24546572726f72206f6e20746865204d6964776179202d2053757065726d616e2045702039227f54686520657069736f6
46520696e2074686520736572696573206f6620466c656973636865722053747564696f732d70726f64756365642053757065726d616e2073657269616c732e2053
756273657175656e7420657069736f64657320776572652070726f64756365642062792046616d6f7573202053747564696f732e2a12506172616d6f756e7420506
9637475726573320d5075626c696320446f6d61696e38004a4868747470733a2f2f73332e616d617a6f6e6177732e636f6d2f66696c65732e6c6272792e696f2f74
68756d626e61696c732f73757065726d616e313934302d7468756d622e706e6752005a001a41080110011a309b70337f51fe9a4481504059b4220ad4f87378d59ec
c87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea2209766964656f2f6d7034
To publish the uri `lbry://@FleischerSuperman/terroronthemidway`, Reilly must sign the SHA256 of the the combined claim address, claim value, and certificate claim id.
He starts by generating an address for the new name claim:
bEGGwBFf39ek6ASJV5YwiUPvDqtpgczCVZ
His claim for the certificate at the uri `lbry://@FleischerSuperman` has a `claim_id` of
2996b9a087c18456402b57cba6085b2a8fcc136d
It contains a der-encoded SECP256k1 public key
{
"certificate": {
"keyType": "SECP256k1",
"publicKey": "3056301006072a8648ce3d020106052b8104000a03420004180488ffcb3d1825af538b0b952f0eba6933faa6d8229609ac0aeadfdbcf49
c59363aa5d77ff2b7ff06cddc07116b335a4a0849b1b524a4a69d908d69f1bcebb",
"version": "_0_0_1"
}
### Signing the claim in detail
Reilly decodes and combines the claim address, the serialized claim value, and the claim id of
the certificate claim. As hex, the combined value is:
5510d538a8ea31210a2a65cfcdd985e3e7bbc42e7c2cecb7bc080110011ae6020801129e02080410011a24546572726f72206f6e20746865204d696477617920
2d2053757065726d616e2045702039227f54686520657069736f646520696e2074686520736572696573206f6620466c656973636865722053747564696f732d
70726f64756365642053757065726d616e2073657269616c732e2053756273657175656e7420657069736f64657320776572652070726f647563656420627920
46616d6f7573202053747564696f732e2a12506172616d6f756e74205069637475726573320d5075626c696320446f6d61696e38004a4868747470733a2f2f73
332e616d617a6f6e6177732e636f6d2f66696c65732e6c6272792e696f2f7468756d626e61696c732f73757065726d616e313934302d7468756d622e706e6752
005a001a41080110011a309b70337f51fe9a4481504059b4220ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea2209766964
656f2f6d70342996b9a087c18456402b57cba6085b2a8fcc136d
Then he takes the sha256 of the combined string, giving:
dea44974ace1893f304cae4073af06a7a6cbb209f97cf8ad5f322216f044304e
He signs (RFC 6979) this hash using the private key to the previously shown certificate, giving the signature:
bf82d53143155bb0cac1fd3d917c000322244b5ad17e7865124db2ed33812ea66c9b0c3f390a65a9e2d452e315e91ae695642847d88e90348ef3c1fa283a36a8
Now he add this signature to the claim:
{
"claimType": "streamType",
"publisherSignature": {
"certificateId": "2996b9a087c18456402b57cba6085b2a8fcc136d",
"signature": "bf82d53143155bb0cac1fd3d917c000322244b5ad17e7865124db2ed33812ea66c9b0c3f390a65a9e2d452e315e91ae695642847d88e90348ef3c1fa283a36a8",
"signatureType": "SECP256k1",
"version": "_0_0_1"
},
"stream": {
"metadata": {
"author": "Paramount Pictures",
"description": "The episode in the series of Fleischer Studios-produced Superman serials. Subsequent episodes were produced by Famous Studios.",
"language": "en",
"license": "Public Domain",
"licenseUrl": "",
"nsfw": false,
"preview": "",
"thumbnail": "https://s3.amazonaws.com/files.lbry.io/thumbnails/superman1940-thumb.png",
"title": "Terror on the Midway - Superman Ep 9",
"version": "_0_1_0"
},
"source": {
"contentType": "video/mp4",
"source": "9b70337f51fe9a4481504059b4220ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea",
"sourceType": "lbry_sd_hash",
"version": "_0_0_1"
},
"version": "_0_0_1"
},
"version": "_0_0_1"
}
Serialized, the signed claim is represented as:
080110011ae6020801129e02080410011a24546572726f72206f6e20746865204d6964776179202d2053757065726d616e2045702039227f5468652065706973
6f646520696e2074686520736572696573206f6620466c656973636865722053747564696f732d70726f64756365642053757065726d616e2073657269616c73
2e2053756273657175656e7420657069736f64657320776572652070726f64756365642062792046616d6f7573202053747564696f732e2a12506172616d6f75
6e74205069637475726573320d5075626c696320446f6d61696e38004a4868747470733a2f2f73332e616d617a6f6e6177732e636f6d2f66696c65732e6c6272
792e696f2f7468756d626e61696c732f73757065726d616e313934302d7468756d622e706e6752005a001a41080110011a309b70337f51fe9a4481504059b422
0ad4f87378d59ecc87bd924c3f0f23da9442b9f75ffc091b65deefe92477a86a31ea2209766964656f2f6d70342a5c080110031a40bf82d53143155bb0cac1fd
3d917c000322244b5ad17e7865124db2ed33812ea66c9b0c3f390a65a9e2d452e315e91ae695642847d88e90348ef3c1fa283a36a822142996b9a087c1845640
2b57cba6085b2a8fcc136d
Now he can put this value in a claim transaction, and broadcast the claim!
So long as his certificate claim is winning, his publication can be resolved by `lbry://@FleischerSuperman/terroronthemidway`
Even if his name is outbid, the publication can be resolved by the channel claim id at the uri: `lbry://@FleischerSuperman#2996b9a087c18456402b57cba6085b2a8fcc136d/terroronthemidway`

101
documents/uri.md Normal file
View file

@ -0,0 +1,101 @@
//this doc has been moved to lbry.tech repo, will clean up later.
# URI
## Regex
If you are a robot and prefer regexes to English, here's the full regex for lbry:// URIs:
```
(?P<uri>
^
(?P<protocol>lbry\:\/\/)?
(?P<content_or_channel_name>
(?P<content_name>[a-zA-Z0-9\-]+)
|
(?P<channel_name>\@[a-zA-Z0-9\-]{4,})
)
(?P<modifier>
(?:\#(?P<claim_id>[0-9a-f]{1,40}))
|
(?:\$(?P<bid_position>\-?[1-9][0-9]*))
|
(?:\:(?P<claim_sequence>\-?[1-9][0-9]*))
)?
(?:\/(?P<path>[a-zA-Z0-9\-]+))?
$
)
```
## Protocol
The LBRY protocol is called `lbry`. URIs using the protocol must start with `lbry://`.
## Reserved characters
- CHANNEL_CHAR = '@'
- CLAIM_ID_CHAR = '#'
- CLAIM_SEQUENCE_CHAR = ':'
- BID_POSITION_CHAR = '$'
- PATH_CHAR = '/'
- QUERY_CHAR = '?'
## Names
Names may contain English letters (upper and lower case), numbers, and hyphens.
### Content Name
`content_name` is the name of piece of content.
### Channel Name
`channel_name` is the name of a channel (aka publisher identity). It must start with CHANNEL_CHAR,
followed by at least 4 name characters.
## Modifiers
Only one modifier is allowed at a time.
### Claim ID
`claim_id` is a hex string identifying a claim.
A claim id is prefixed with the CLAIM_ID_CHAR.
Partial claim ids are allowed (same is git hashes), and
will resolve to the oldest claim who's id starts with the given characters.
### Claim Sequence
`claim_sequence` is a positive integer (>= 1) that resolves to the Nth claim for a given name.
A claim sequence is prefixed with the CLAIM_SEQUENCE_CHAR.
All valid claims are considered, in the order that they appear in the blockchain.
Nonwinning claims are included.
For example, `lbry://@chan:1` resolves to the oldest valid claim for `@chan`, even if that claim is no longer the winning claim for `@chan`.
Negative claim sequence numbers will be supported eventually.
### Bid Position
_not implemented yet_
`bid_position` is a positive integer (>= 1) that resolves to the Nth highest-bid claim for a given name.
A bid position is prefixed with the BID_POSITION_CHAR.
All valid claims are considered, in order from highest bid to lowest bid, with ties being broken by claim age.
Nonwinning claims are included.
For example, `lbry://@chan$1` always resolves to the current winning claim for `@chan`. `@chan` and `@chan$1` are equivalent.
Negative bid position numbers will be supported eventually.
## Path
`path` is a Unix-style path that resolves to a claim within a channel.
A path is prefixed with PATH_CHAR.
Only paths one level deep are currently supported.
Only channel claims may have a path.
For example, `lbry://@chan/snaps_from_last_night` resolves to the claim for `snaps_from_last_night` that is signed by `@chan`
## Query Params
_not implemented yet_

2
package.json
View file

@ -33,7 +33,9 @@
"heroku-ssl-redirect": "0.0.4",
"make-promises-safe": "^1.1.0",
"markdown-it": "^8.4.1",
"markdown-it-anchor": "^5.0.2",
"markdown-it-sup": "^1.0.0",
"markdown-it-wikilinks": "^1.0.1",
"nanohtml": "^1.2.4",
"redis": "^2.8.0",
"relative-date": "^1.1.3",

24
views/pages/page.js
View file

@ -11,10 +11,21 @@ const fm = require("front-matter");
const fs = require("graceful-fs");
const html = require("choo-async/html");
const local = require("app-root-path").require;
const md = require("markdown-it")({
html: true,
typographer: true
}).use(require("markdown-it-sup"));
}).use(require("markdown-it-sup"))
.use(require("markdown-it-anchor"))
.use(require("markdown-it-wikilinks")({
makeAllLinksAbsolute: true,
baseURL: "/glossary#",
uriSuffix: "",
htmlAttributes: {
class: "wikilink"
}
}));
const raw = require("nanohtml/raw");
@ -24,7 +35,7 @@ const raw = require("nanohtml/raw");
const page = () => async (state, emit) => { // eslint-disable-line
const path = state.params.wildcard;
if (!fs.existsSync(`${__dirname}/${path}.md`)) {
if (!fs.existsSync(`./documents/${path}.md`)) {
return html`
<article class="page" itemtype="http://schema.org/BlogPosting">
<header class="page__header">
@ -44,7 +55,7 @@ const page = () => async (state, emit) => { // eslint-disable-line
`;
}
const markdownFile = fs.readFileSync(`${__dirname}/${path}.md`, "utf-8");
const markdownFile = fs.readFileSync(`./documents/${path}.md`, "utf-8");
const markdownFileDetails = fm(markdownFile);
const renderedMarkdown = md.render(partialFinder(markdownFileDetails.body));
@ -86,15 +97,14 @@ function partialFinder(markdownBody) {
const fileExistsTest = exists(`./views/partials/${filename}.js`); // `local` results in error if used here and file !exist
if (fileExistsTest) {
const something = local(`/views/partials/${filename}.js`);
const partialFunction = local(`/views/partials/${filename}.js`);
if (filename === "ecosystem") {
const Ecosystem = new something;
const Ecosystem = new partialFunction;
markdownBody = markdownBody.replace(partial, Ecosystem.render());
// console.log(new something);
}
else markdownBody = markdownBody.replace(partial, something);
else markdownBody = markdownBody.replace(partial, partialFunction);
}
}