LBRYCommunity/comment-server
1
0
Fork 0
Code Issues 18 Pull requests 1 Projects Releases Packages Wiki Activity

Merge remote-tracking branch 'origin/master'

Browse source 
# Conflicts:
#	tests/server_test.py
This commit is contained in:
Oleg Silkin 2019-07-21 20:10:15 -04:00
commit ece988ae36
14 changed files with 419 additions and 249 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

119
README.md
View file

@ -1,49 +1,116 @@
# LBRY Comment Server v2
# LBRY Comment Server
This is a rewrite & update of the server
[written by myself and Grayson Burton here](https://github.com/ocornoc/lbry-comments)
This is the code for the LBRY Comment Server.
Fork it, run it, set it on fire. Up to you.
### Install & Setup
Clone the repo and install a virtual environement:
## Prerequisites
In order to run the comment server properly, you will need the
following:
0. A Unix Compliant Operating System (e.g: Ubuntu, RedHat, Hannah Montana, etc.)
1. Any recent version of Sqlite3
2. Python 3.6+ (including `python3-dev`, `python3-virtualenv`, `python3-pip`)
Note: You must specify the specific python version you're using,
e.g. for Python3.6, you would install `python36-dev`. You're smart enough to figure the rest out from here ;)
3. (Optional) Reverse Proxy software to handle a public-facing API.
We recommend Caddy, though there is an `nginx.conf` file under `config`.
4. Patience (Strongly recommended but often neglected)
## Installation
Installing the server:
```bash
# clone the repo
$ git clone https://github.com/osilkin98/comment-server
$ cd comment-server
# create a virtual environment in any (current) version of python3.X
# create a virtual environment
$ virtualenv --python=python3 venv
$ source venv/bin/activate
# install the dependencies
# Enter the virtual environment
$ source venv/bin/activate
# install the library dependencies
(venv) $ pip install -r requirements.txt
```
### Running the server
## Usage
Just run:
`(venv) $ python -m src.main.py`
and it should run automatically.
### Running the Server
To start the server, simply run:
```bash
# to enter server's venv
$ source venv/bin/activate
# To start server as daemon process
(venv) $ python -m main &
```
### Testing
To Test the database, simply run:
```bash
# To run the whole thing :
(venv) $ python -m unittest tests.database
# To run a specific TestName under a specified TestClass:
(venv) $ python -m unittest tests.database.TestClass.TestName`
```
There are basic tests to run against the server, though they require
that there is a server instance running, though the database
chosen may have to be edited in `config/conf.json`.
Additionally there are HTTP requests that can be send with whatever
software you choose to test the integrity of the comment server.
## Schema
![schema](schema.png)
## About
A lot of the design is more or less the same with the original,
except this version focuses less on performance and more on scalability.
## Contributing
Contributions are welcome, verbosity is encouraged. Please be considerate
in your posts, and make sure that you give as much context to the issue
as possible, so that helping you is a slam dunk for us.
Rewritten with python because it's easier to adjust
and maintain. Instead of doing any multithreading,
this implementation just delegates a single
database connection for write operations.
### Issues
If you spotted an issue from the SDK side, please replicate it using
`curl` and one of the HTTP request templates in `tests/http_requests`.
The server was originally implemented with `aiohttp`
and uses `aiojobs` for scheduling write operations.
As pointed out by several people, Python is a dinosaur
in comparison to SQLite's execution speed,
so there is no sensibility in multi-threading from the
perspective of the server code itself.
Then, just include that along with the rest of your information.
### Pull Requests
Make sure the code works and has been tested beforehand.
Although we love helping out, our job is to review your code,
not test it - that's what your computer is for.
Try to document the changes you made in a human language,
preferably English. (but we're always up for a challenge...)
Use the level of verbosity you feel is correct, and when in doubt,
just [KISS](https://people.apache.org/~fhanik/kiss.html).
### General
For more details, please refer to [lbry.tech/contribute](https://lbry.tech/contribute).
## License
This project is licensed by AGPLv3.
See [LICENSE](LICENSE.nd) for the full license.
## Security
We take security seriously.
Please contact [security@lbry.io](security@lbry.io) regarding any conerns you might have,
issues you might encounter, or general outlooks on life. Our PGP key can
be found [here](https://keybase.io/lbry/key.asc), should you need it.
## Contact Us
The primary contact for this project is
[@osilkin98](https://github.com/osilkin98), and can be reached
at (o.silkin98@gmail.com).

BIN
schema.png
View file

Binary file not shown.
Side by side Swipe Overlay

Before

Width:  |  Height:  |  Size: 39 KiB

After

Width:  |  Height:  |  Size: 37 KiB

7
schema/comments_ddl.sql
View file

@ -14,6 +14,7 @@ CREATE TABLE IF NOT EXISTS COMMENT (
ParentId TEXT DEFAULT NULL,
Signature TEXT DEFAULT NULL,
Timestamp INTEGER NOT NULL,
SigningTs TEXT DEFAULT NULL,
CONSTRAINT COMMENT_PRIMARY_KEY PRIMARY KEY (CommentId) ON CONFLICT IGNORE,
CONSTRAINT COMMENT_SIGNATURE_SK UNIQUE (Signature) ON CONFLICT ABORT,
CONSTRAINT COMMENT_CHANNEL_FK FOREIGN KEY (ChannelId) REFERENCES CHANNEL(ClaimId)
@ -22,6 +23,8 @@ CREATE TABLE IF NOT EXISTS COMMENT (
ON UPDATE CASCADE ON DELETE NO ACTION -- setting null implies comment is top level
);
-- ALTER TABLE COMMENT ADD COLUMN SigningTs TEXT DEFAULT NULL;
-- DROP TABLE IF EXISTS CHANNEL;
CREATE TABLE IF NOT EXISTS CHANNEL(
ClaimId TEXT NOT NULL,
@ -38,8 +41,8 @@ CREATE INDEX IF NOT EXISTS COMMENT_CLAIM_INDEX ON COMMENT (LbryClaimId);
-- VIEWS
DROP VIEW IF EXISTS COMMENTS_ON_CLAIMS;
CREATE VIEW IF NOT EXISTS COMMENTS_ON_CLAIMS (comment_id, claim_id, timestamp, channel_name, channel_id, channel_url, signature, parent_id, comment) AS
SELECT C.CommentId, C.LbryClaimId, C.Timestamp, CHAN.Name, CHAN.ClaimId, 'lbry://' || CHAN.Name || '#' || CHAN.ClaimId, C.Signature, C.ParentId, C.Body
CREATE VIEW IF NOT EXISTS COMMENTS_ON_CLAIMS (comment_id, claim_id, timestamp, channel_name, channel_id, channel_url, signature, signing_ts, parent_id, comment) AS
SELECT C.CommentId, C.LbryClaimId, C.Timestamp, CHAN.Name, CHAN.ClaimId, 'lbry://' || CHAN.Name || '#' || CHAN.ClaimId, C.Signature, C.SigningTs, C.ParentId, C.Body
FROM COMMENT AS C
LEFT OUTER JOIN CHANNEL CHAN on C.ChannelId = CHAN.ClaimId
ORDER BY C.Timestamp DESC;

10
src/app.py
View file

@ -8,11 +8,10 @@ import asyncio
from aiohttp import web
import schema.db_helpers
from src.database import obtain_connection
from src.database import obtain_connection, DatabaseWriter
from src.handles import api_endpoint
from src.handles import create_comment_scheduler
from src.settings import config_path, get_config
from src.writes import DatabaseWriter
config = get_config(config_path)
@ -50,21 +49,20 @@ async def close_comment_scheduler(app):
await app['comment_scheduler'].close()
async def create_database_backup(app):
async def database_backup_routine(app):
try:
while True:
await asyncio.sleep(app['config']['BACKUP_INT'])
with obtain_connection(app['db_path']) as conn:
logger.debug('backing up database')
schema.db_helpers.backup_database(conn, app['backup'])
except asyncio.CancelledError as e:
except asyncio.CancelledError:
pass
async def start_background_tasks(app: web.Application):
app['reader'] = obtain_connection(app['db_path'], True)
app['waitful_backup'] = app.loop.create_task(create_database_backup(app))
app['waitful_backup'] = app.loop.create_task(database_backup_routine(app))
app['comment_scheduler'] = await create_comment_scheduler()
app['writer'] = DatabaseWriter(app['db_path'])

246
src/database.py
View file

@ -1,5 +1,5 @@
import atexit
import logging
import re
import sqlite3
import time
import typing
@ -23,78 +23,66 @@ def obtain_connection(filepath: str = None, row_factory: bool = True):
def get_claim_comments(conn: sqlite3.Connection, claim_id: str, parent_id: str = None,
page: int = 1, page_size: int = 50, top_level=False):
if top_level:
results = [clean(dict(row)) for row in conn.execute(
""" SELECT comment, comment_id, channel_name, channel_id, channel_url, timestamp, signature, parent_id
FROM COMMENTS_ON_CLAIMS
with conn:
if top_level:
results = [clean(dict(row)) for row in conn.execute(
""" SELECT comment, comment_id, channel_name, channel_id,
channel_url, timestamp, signature, signing_ts, parent_id
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id IS NULL
LIMIT ? OFFSET ? """,
(claim_id, page_size, page_size*(page - 1))
)]
count = conn.execute(
"""
SELECT COUNT(*)
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id IS NULL
LIMIT ? OFFSET ? """,
(claim_id, page_size, page_size*(page - 1))
)]
count = conn.execute(
"""
SELECT COUNT(*)
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id IS NULL
""", (claim_id, )
)
elif parent_id is None:
results = [clean(dict(row)) for row in conn.execute(
""" SELECT comment, comment_id, channel_name, channel_id, channel_url, timestamp, signature, parent_id
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ?
LIMIT ? OFFSET ? """,
(claim_id, page_size, page_size*(page - 1))
)]
count = conn.execute(
"""
SELECT COUNT(*)
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ?
""", (claim_id,)
)
else:
results = [clean(dict(row)) for row in conn.execute(
""" SELECT comment, comment_id, channel_name, channel_id, channel_url, timestamp, signature, parent_id
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id = ?
LIMIT ? OFFSET ? """,
(claim_id, parent_id, page_size, page_size*(page - 1))
)]
count = conn.execute(
"""
SELECT COUNT(*)
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id = ?
""", (claim_id, parent_id)
)
count = tuple(count.fetchone())[0]
return {
'items': results,
'page': page,
'page_size': page_size,
'total_pages': math.ceil(count/page_size),
'total_items': count
}
""", (claim_id, )
)
elif parent_id is None:
results = [clean(dict(row)) for row in conn.execute(
""" SELECT comment, comment_id, channel_name, channel_id,
channel_url, timestamp, signature, signing_ts, parent_id
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ?
LIMIT ? OFFSET ? """,
(claim_id, page_size, page_size*(page - 1))
)]
count = conn.execute(
"""
SELECT COUNT(*)
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ?
""", (claim_id,)
)
else:
results = [clean(dict(row)) for row in conn.execute(
""" SELECT comment, comment_id, channel_name, channel_id,
channel_url, timestamp, signature, signing_ts, parent_id
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id = ?
LIMIT ? OFFSET ? """,
(claim_id, parent_id, page_size, page_size*(page - 1))
)]
count = conn.execute(
"""
SELECT COUNT(*)
FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id = ?
""", (claim_id, parent_id)
)
count = tuple(count.fetchone())[0]
return {
'items': results,
'page': page,
'page_size': page_size,
'total_pages': math.ceil(count/page_size),
'total_items': count
}
def validate_input(**kwargs):
assert 0 < len(kwargs['comment']) <= 2000
assert re.fullmatch(
'[a-z0-9]{40}:([a-z0-9]{40})?',
kwargs['claim_id'] + ':' + kwargs.get('channel_id', '')
)
if 'channel_name' in kwargs or 'channel_id' in kwargs:
assert 'channel_id' in kwargs and 'channel_name' in kwargs
assert re.fullmatch(
'^@(?:(?![\x00-\x08\x0b\x0c\x0e-\x1f\x23-\x26'
'\x2f\x3a\x3d\x3f-\x40\uFFFE-\U0000FFFF]).){1,255}$',
kwargs.get('channel_name', '')
)
assert re.fullmatch('[a-z0-9]{40}', kwargs.get('channel_id', ''))
def _insert_channel(conn: sqlite3.Connection, channel_name: str, channel_id: str):
def insert_channel(conn: sqlite3.Connection, channel_name: str, channel_id: str):
with conn:
conn.execute(
'INSERT INTO CHANNEL(ClaimId, Name) VALUES (?, ?)',
@ -102,59 +90,35 @@ def _insert_channel(conn: sqlite3.Connection, channel_name: str, channel_id: str
)
def _insert_comment(conn: sqlite3.Connection, claim_id: str = None, comment: str = None,
channel_id: str = None, signature: str = None, parent_id: str = None) -> str:
def insert_comment(conn: sqlite3.Connection, claim_id: str = None, comment: str = None,
channel_id: str = None, signature: str = None, signing_ts: str = None,
parent_id: str = None) -> str:
timestamp = int(time.time())
comment_prehash = ':'.join((claim_id, comment, str(timestamp),))
comment_prehash = bytes(comment_prehash.encode('utf-8'))
comment_id = nacl.hash.sha256(comment_prehash).decode('utf-8')
prehash = b':'.join((claim_id.encode(), comment.encode(), str(timestamp).encode(),))
comment_id = nacl.hash.sha256(prehash).decode()
with conn:
conn.execute(
"""
INSERT INTO COMMENT(CommentId, LbryClaimId, ChannelId, Body,
ParentId, Signature, Timestamp)
VALUES (?, ?, ?, ?, ?, ?, ?)
INSERT INTO COMMENT(CommentId, LbryClaimId, ChannelId, Body, ParentId, Timestamp, Signature, SigningTs)
VALUES (?, ?, ?, ?, ?, ?, ?, ?)
""",
(comment_id, claim_id, channel_id, comment, parent_id, signature, timestamp)
(comment_id, claim_id, channel_id, comment, parent_id, timestamp, signature, signing_ts)
)
logger.debug('Inserted Comment into DB, `comment_id`: %s', comment_id)
return comment_id
def create_comment(conn: sqlite3.Connection, comment: str, claim_id: str, **kwargs) -> typing.Union[dict, None]:
channel_id = kwargs.pop('channel_id', '')
channel_name = kwargs.pop('channel_name', '')
if channel_id or channel_name:
try:
validate_input(
comment=comment,
claim_id=claim_id,
channel_id=channel_id,
channel_name=channel_name,
)
_insert_channel(conn, channel_name, channel_id)
except AssertionError:
logger.exception('Received invalid input')
raise TypeError('Invalid params given to input validation')
else:
channel_id = None
try:
comment_id = _insert_comment(
conn=conn, comment=comment, claim_id=claim_id, channel_id=channel_id, **kwargs
def get_comment_or_none(conn: sqlite3.Connection, comment_id: str) -> dict:
with conn:
curry = conn.execute(
"""
SELECT comment, comment_id, channel_name, channel_id, channel_url, timestamp, signature, signing_ts, parent_id
FROM COMMENTS_ON_CLAIMS WHERE comment_id = ?
""",
(comment_id,)
)
except sqlite3.IntegrityError as ie:
logger.exception(ie)
return None
curry = conn.execute(
"""
SELECT comment, comment_id, channel_name, channel_id, channel_url, timestamp, signature, parent_id
FROM COMMENTS_ON_CLAIMS WHERE comment_id = ?
""",
(comment_id,)
)
thing = curry.fetchone()
return clean(dict(thing)) if thing else None
thing = curry.fetchone()
return clean(dict(thing)) if thing else None
def get_comment_ids(conn: sqlite3.Connection, claim_id: str, parent_id: str = None, page=1, page_size=50):
@ -165,18 +129,19 @@ def get_comment_ids(conn: sqlite3.Connection, claim_id: str, parent_id: str = No
For pagination the parameters are:
get_all XOR (page_size + page)
"""
if parent_id is None:
curs = conn.execute("""
SELECT comment_id FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id IS NULL LIMIT ? OFFSET ?
""", (claim_id, page_size, page_size*abs(page - 1),)
)
else:
curs = conn.execute("""
SELECT comment_id FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id LIKE ? LIMIT ? OFFSET ?
""", (claim_id, parent_id, page_size, page_size * abs(page - 1),)
)
with conn:
if parent_id is None:
curs = conn.execute("""
SELECT comment_id FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id IS NULL LIMIT ? OFFSET ?
""", (claim_id, page_size, page_size*abs(page - 1),)
)
else:
curs = conn.execute("""
SELECT comment_id FROM COMMENTS_ON_CLAIMS
WHERE claim_id LIKE ? AND parent_id LIKE ? LIMIT ? OFFSET ?
""", (claim_id, parent_id, page_size, page_size * abs(page - 1),)
)
return [tuple(row)[0] for row in curs.fetchall()]
@ -184,12 +149,31 @@ def get_comments_by_id(conn, comment_ids: list) -> typing.Union[list, None]:
""" Returns a list containing the comment data associated with each ID within the list"""
# format the input, under the assumption that the
placeholders = ', '.join('?' for _ in comment_ids)
return [clean(dict(row)) for row in conn.execute(
f'SELECT * FROM COMMENTS_ON_CLAIMS WHERE comment_id IN ({placeholders})',
tuple(comment_ids)
)]
with conn:
return [clean(dict(row)) for row in conn.execute(
f'SELECT * FROM COMMENTS_ON_CLAIMS WHERE comment_id IN ({placeholders})',
tuple(comment_ids)
)]
if __name__ == '__main__':
pass
# __generate_database_schema(connection, 'comments_ddl.sql')
class DatabaseWriter(object):
_writer = None
def __init__(self, db_file):
if not DatabaseWriter._writer:
self.conn = obtain_connection(db_file)
DatabaseWriter._writer = self
atexit.register(self.cleanup)
logging.info('Database writer has been created at %s', repr(self))
else:
logging.warning('Someone attempted to insantiate DatabaseWriter')
raise TypeError('Database Writer already exists!')
def cleanup(self):
logging.info('Cleaning up database writer')
DatabaseWriter._writer = None
self.conn.close()
@property
def connection(self):
return self.conn

17
src/handles.py
View file

@ -2,17 +2,17 @@
import json
import logging
import asyncio
import aiojobs
from asyncio import coroutine
import asyncio
from aiohttp import web
from aiojobs.aiohttp import atomic
from asyncio import coroutine
from src.writes import DatabaseWriter
from src.database import DatabaseWriter
from src.database import get_claim_comments
from src.database import get_comments_by_id, get_comment_ids
from src.database import obtain_connection
from src.database import create_comment
from src.writes import create_comment
logger = logging.getLogger(__name__)
@ -79,6 +79,9 @@ async def process_json(app, body: dict) -> dict:
except TypeError as te:
logger.exception('Got TypeError: %s', te)
response['error'] = ERRORS['INVALID_PARAMS']
except ValueError as ve:
logger.exception('Got ValueError: %s', ve)
response['error'] = ERRORS['INVALID_PARAMS']
else:
response['error'] = ERRORS['UNKNOWN']
return response
@ -87,8 +90,8 @@ async def process_json(app, body: dict) -> dict:
@atomic
async def api_endpoint(request: web.Request):
try:
body = await request.json()
logger.info('Received POST request from %s', request.remote)
body = await request.json()
if type(body) is list or type(body) is dict:
if type(body) is list:
return web.json_response(
@ -104,7 +107,3 @@ async def api_endpoint(request: web.Request):
return web.json_response({
'error': {'message': jde.msg, 'code': -1}
})

22
src/misc.py Normal file
View file

@ -0,0 +1,22 @@
import re
def validate_channel(channel_id: str, channel_name: str):
assert channel_id and channel_name
assert type(channel_id) is str and type(channel_name) is str
assert re.fullmatch(
'^@(?:(?![\x00-\x08\x0b\x0c\x0e-\x1f\x23-\x26'
'\x2f\x3a\x3d\x3f-\x40\uFFFE-\U0000FFFF]).){1,255}$',
channel_name
)
assert re.fullmatch('[a-z0-9]{40}', channel_id)
def validate_base_comment(comment: str, claim_id: str, **kwargs):
assert 0 < len(comment) <= 2000
assert re.fullmatch('[a-z0-9]{40}', claim_id)
async def validate_signature(*args, **kwargs):
pass

48
src/writes.py
View file

@ -1,30 +1,34 @@
import atexit
import logging
import sqlite3
from src.database import obtain_connection
from src.database import get_comment_or_none
from src.database import insert_comment
from src.database import insert_channel
from src.misc import validate_channel
from src.misc import validate_signature
logger = logging.getLogger(__name__)
# DatabaseWriter should be instantiated on startup
class DatabaseWriter(object):
_writer = None
def create_comment(conn: sqlite3.Connection, comment: str, claim_id: str, channel_id: str = None,
channel_name: str = None, signature: str = None, signing_ts: str = None, parent_id: str = None):
if channel_id or channel_name or signature or signing_ts:
validate_signature(signature, signing_ts, comment, channel_name, channel_id)
insert_channel_or_error(conn, channel_name, channel_id)
try:
comment_id = insert_comment(
conn=conn, comment=comment, claim_id=claim_id, channel_id=channel_id,
signature=signature, parent_id=parent_id, signing_ts=signing_ts
)
return get_comment_or_none(conn, comment_id)
except sqlite3.IntegrityError as ie:
logger.exception(ie)
def __init__(self, db_file):
if not DatabaseWriter._writer:
self.conn = obtain_connection(db_file)
DatabaseWriter._writer = self
atexit.register(self.cleanup)
logging.info('Database writer has been created at %s', repr(self))
else:
logging.warning('Someone attempted to insantiate DatabaseWriter')
raise TypeError('Database Writer already exists!')
def cleanup(self):
logging.info('Cleaning up database writer')
DatabaseWriter._writer = None
self.conn.close()
@property
def connection(self):
return self.conn
def insert_channel_or_error(conn: sqlite3.Connection, channel_name: str, channel_id: str):
try:
validate_channel(channel_id, channel_name)
insert_channel(conn, channel_name, channel_id)
except AssertionError as ae:
logger.exception('Invalid channel values given: %s', ae)
raise ValueError('Received invalid values for channel_id or channel_name')

107
tests/database_test.py
View file

@ -5,11 +5,10 @@ from faker.providers import internet
from faker.providers import lorem
from faker.providers import misc
from src.database import get_comments_by_id, create_comment, get_comment_ids, \
from src.database import get_comments_by_id, get_comment_ids, \
get_claim_comments
from schema.db_helpers import setup_database, teardown_database
from src.settings import config
from tests.testcase import DatabaseTestCase, AsyncioTestCase
from src.writes import create_comment
from tests.testcase import DatabaseTestCase
fake = faker.Faker()
fake.add_provider(internet)
@ -29,9 +28,11 @@ class TestCommentCreation(DatabaseTestCase):
comment='This is a named comment',
channel_name='@username',
channel_id='529357c3422c6046d3fec76be2358004ba22abcd',
signature=fake.uuid4(),
signing_ts='aaa'
)
self.assertIsNotNone(comment)
self.assertIsNone(comment['parent_id'])
self.assertNotIn('parent_in', comment)
previous_id = comment['comment_id']
reply = create_comment(
conn=self.conn,
@ -39,11 +40,12 @@ class TestCommentCreation(DatabaseTestCase):
comment='This is a named response',
channel_name='@another_username',
channel_id='529357c3422c6046d3fec76be2358004ba224bcd',
parent_id=previous_id
parent_id=previous_id,
signature=fake.uuid4(),
signing_ts='aaa'
)
self.assertIsNotNone(reply)
self.assertEqual(reply['parent_id'], comment['comment_id'])
self.assertEqual(reply['claim_id'], comment['claim_id'])
def test02AnonymousComments(self):
comment = create_comment(
@ -52,7 +54,6 @@ class TestCommentCreation(DatabaseTestCase):
comment='This is an ANONYMOUS comment'
)
self.assertIsNotNone(comment)
self.assertIsNone(comment['parent_id'])
previous_id = comment['comment_id']
reply = create_comment(
conn=self.conn,
@ -62,7 +63,6 @@ class TestCommentCreation(DatabaseTestCase):
)
self.assertIsNotNone(reply)
self.assertEqual(reply['parent_id'], comment['comment_id'])
self.assertEqual(reply['claim_id'], comment['claim_id'])
def test03SignedComments(self):
comment = create_comment(
@ -71,10 +71,11 @@ class TestCommentCreation(DatabaseTestCase):
comment='I like big butts and i cannot lie',
channel_name='@sirmixalot',
channel_id='529357c3422c6046d3fec76be2358005ba22abcd',
signature='siggy'
signature=fake.uuid4(),
signing_ts='asdasd'
)
self.assertIsNotNone(comment)
self.assertIsNone(comment['parent_id'])
self.assertIn('signing_ts', comment)
previous_id = comment['comment_id']
reply = create_comment(
conn=self.conn,
@ -83,56 +84,61 @@ class TestCommentCreation(DatabaseTestCase):
channel_name='@LBRY',
channel_id='529357c3422c6046d3fec76be2358001ba224bcd',
parent_id=previous_id,
signature='Cursive Font Goes Here'
signature=fake.uuid4(),
signing_ts='sfdfdfds'
)
self.assertIsNotNone(reply)
self.assertEqual(reply['parent_id'], comment['comment_id'])
self.assertEqual(reply['claim_id'], comment['claim_id'])
self.assertIn('signing_ts', reply)
def test04UsernameVariations(self):
invalid_comment = create_comment(
self.assertRaises(
AssertionError,
callable=create_comment,
conn=self.conn,
claim_id=self.claimId,
channel_name='$#(@#$@#$',
channel_id='529357c3422c6046d3fec76be2358001ba224b23',
comment='this is an invalid username'
)
self.assertIsNone(invalid_comment)
valid_username = create_comment(
conn=self.conn,
claim_id=self.claimId,
channel_name='@' + 'a'*255,
channel_name='@' + 'a' * 255,
channel_id='529357c3422c6046d3fec76be2358001ba224b23',
comment='this is a valid username'
)
self.assertIsNotNone(valid_username)
self.assertRaises(AssertionError,
callable=create_comment,
conn=self.conn,
claim_id=self.claimId,
channel_name='@' + 'a' * 256,
channel_id='529357c3422c6046d3fec76be2358001ba224b23',
comment='this username is too long'
)
lengthy_username = create_comment(
conn=self.conn,
claim_id=self.claimId,
channel_name='@' + 'a'*256,
channel_id='529357c3422c6046d3fec76be2358001ba224b23',
comment='this username is too long'
)
self.assertIsNone(lengthy_username)
comment = create_comment(
self.assertRaises(
AssertionError,
callable=create_comment,
conn=self.conn,
claim_id=self.claimId,
channel_name='',
channel_id='529357c3422c6046d3fec76be2358001ba224b23',
comment='this username should not default to ANONYMOUS'
)
self.assertIsNone(comment)
short_username = create_comment(
self.assertRaises(
AssertionError,
callable=create_comment,
conn=self.conn,
claim_id=self.claimId,
channel_name='@',
channel_id='529357c3422c6046d3fec76be2358001ba224b23',
comment='this username is too short'
)
self.assertIsNone(short_username)
def test05InsertRandomComments(self):
self.skipTest('This is a bad test')
top_comments, claim_ids = generate_top_comments_random()
total = 0
success = 0
@ -158,6 +164,7 @@ class TestCommentCreation(DatabaseTestCase):
del claim_ids
def test06GenerateAndListComments(self):
self.skipTest('this is a stupid test')
top_comments, claim_ids = generate_top_comments()
total, success = 0, 0
for _, comments in top_comments.items():
@ -187,16 +194,10 @@ class ListDatabaseTest(DatabaseTestCase):
def setUp(self) -> None:
super().setUp()
top_coms, self.claim_ids = generate_top_comments(5, 75)
self.top_comments = {
commie_id: [create_comment(self.conn, **commie) for commie in commie_list]
for commie_id, commie_list in top_coms.items()
}
self.replies = [
create_comment(self.conn, **reply)
for reply in generate_replies(self.top_comments)
]
def testLists(self):
self.skipTest('Populating a database each time is not a good way to test listing')
for claim_id in self.claim_ids:
with self.subTest(claim_id=claim_id):
comments = get_claim_comments(self.conn, claim_id)
@ -217,6 +218,22 @@ class ListDatabaseTest(DatabaseTestCase):
self.assertEqual(len(matching_comments), len(comment_ids))
def generate_top_comments(ncid=15, ncomm=100, minchar=50, maxchar=500):
claim_ids = [fake.sha1() for _ in range(ncid)]
top_comments = {
cid: [{
'claim_id': cid,
'comment': ''.join(fake.text(max_nb_chars=randint(minchar, maxchar))),
'channel_name': '@' + fake.user_name(),
'channel_id': fake.sha1(),
'signature': fake.uuid4(),
'signing_ts': fake.uuid4()
} for _ in range(ncomm)]
for cid in claim_ids
}
return top_comments, claim_ids
def generate_replies(top_comments):
return [{
'claim_id': comment['claim_id'],
@ -224,7 +241,8 @@ def generate_replies(top_comments):
'comment': ' '.join(fake.text(max_nb_chars=randint(50, 500))),
'channel_name': '@' + fake.user_name(),
'channel_id': fake.sha1(),
'signature': fake.uuid4()
'signature': fake.uuid4(),
'signing_ts': fake.uuid4()
}
for claim, comments in top_comments.items()
for i, comment in enumerate(comments)
@ -247,21 +265,6 @@ def generate_replies_random(top_comments):
]
def generate_top_comments(ncid=15, ncomm=100, minchar=50, maxchar=500):
claim_ids = [fake.sha1() for _ in range(ncid)]
top_comments = {
cid: [{
'claim_id': cid,
'comment': ''.join(fake.text(max_nb_chars=randint(minchar, maxchar))),
'channel_name': '@' + fake.user_name(),
'channel_id': fake.sha1(),
'signature': fake.uuid4()
} for _ in range(ncomm)]
for cid in claim_ids
}
return top_comments, claim_ids
def generate_top_comments_random():
claim_ids = [fake.sha1() for _ in range(15)]
top_comments = {

21
tests/http_requests/comment-request.http Normal file
View file

@ -0,0 +1,21 @@
# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection).
#
# Following HTTP Request Live Templates are available:
# * 'gtrp' and 'gtr' create a GET request with or without query parameters;
# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body;
# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data);
POST https://comments.lbry.com/api
Content-Type: application/json
{
"jsonrpc": "2.0",
"id": null,
"method": "get_claim_comments",
"params": {
"claim_id": "9cb713f01bf247a0e03170b5ed00d5161340c486"
}
}
###

24
tests/http_requests/create-comment.http Normal file
View file

@ -0,0 +1,24 @@
# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection).
#
# Following HTTP Request Live Templates are available:
# * 'gtrp' and 'gtr' create a GET request with or without query parameters;
# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body;
# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data);
POST https://comments.lbry.com/api
Content-Type: application/json
{
"jsonrpc": "2.0",
"id": null,
"method": "create_comment",
"params": {
"comment": "POP SECRET",
"claim_id": "9cb713f01bf247a0e03170b5ed00d5161340c486",
"channel_name": "@dogeworld",
"channel_id": "9cb713f01bf247a0e03170b5ed00d5161340c486",
"signing_ts": "1234567"
}
}
###

22
tests/http_requests/create_comment_local.http Normal file
View file

@ -0,0 +1,22 @@
# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection).
#
# Following HTTP Request Live Templates are available:
# * 'gtrp' and 'gtr' create a GET request with or without query parameters;
# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body;
# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data);
POST http://localhost:5921/api
Content-Type: application/json
{
"jsonrpc": "2.0",
"id": null,
"method": "create_comment",
"params": {
"claim_id": "9cb713f01bf247a0e03170b5ed00d5161340c486",
"comment": "This is literally, the most anonymous comment I have EVER seen.",
"channel_id": "9cb713f01bf247a0e03170b5ed00d5161340c486",
"channel_name": "@do oge world "
}
}
###

20
tests/http_requests/local-server-request.http Normal file
View file

@ -0,0 +1,20 @@
# For a quick start check out our HTTP Requests collection (Tools|HTTP Client|Open HTTP Requests Collection).
#
# Following HTTP Request Live Templates are available:
# * 'gtrp' and 'gtr' create a GET request with or without query parameters;
# * 'ptr' and 'ptrp' create a POST request with a simple or parameter-like body;
# * 'mptr' and 'fptr' create a POST request to submit a form with a text or file field (multipart/form-data);
POST http://localhost:5921/api
Content-Type: application/json
{
"jsonrpc": "2.0",
"id": null,
"method": "get_claim_comments",
"params": {
"claim_id": "9cb713f01bf247a0e03170b5ed00d5161340c486",
"page": 1
}
}
###

5
tests/testcase.py
View file

@ -1,11 +1,12 @@
import pathlib
import unittest
from asyncio.runners import _cancel_all_tasks # type: ignore
from unittest.case import _Outcome
import asyncio
from src.database import obtain_connection
from schema.db_helpers import setup_database, teardown_database
from src.database import obtain_connection
from src.settings import config
@ -120,6 +121,8 @@ class AsyncioTestCase(unittest.TestCase):
class DatabaseTestCase(unittest.TestCase):
def setUp(self) -> None:
super().setUp()
if pathlib.Path(config['PATH']['TEST']).exists():
teardown_database(config['PATH']['TEST'])
setup_database(config['PATH']['TEST'])
self.conn = obtain_connection(config['PATH']['TEST'])