Blobs upload proposal #48

New issue

Open

opened 2021-01-11 16:24:02 +01:00 by nikooo777 · 0 comments

nikooo777 commented

2021-01-11 16:24:02 +01:00

(Migrated from github.com)

send sdblob in this format:

type request struct {
	hash string
	sdblob []byte
}

server checks the blocklist to see if it wants the stream
server checks if the sdblob is already in our system
3.1) if the sd blob exists, respond with a list of blobs it needs
3.2) if the sd blob doesn't exist, store the sd blob, save the blobs in the DB with is_stored=0, respond with a list of the blobs it needs (all of them)
client sends one blob at a time with the following format

type request struct {
	hash string
	blob []byte
	size int	
}

server checks that the uploaded blob has the correct hash and size, saves the blob to S3, sets is_stored=1 in the database, responds with a success
optionally we should add an endpoint that accepts a list of sdblob hashes and returns whether the sdblob is needed or not and if the sdblob is already stored, a list of blobs that are missing
the server thus has to implement the following endpoints:
/sdblob POST
accepts:

type request struct {
	hash string
	sdblob []byte
}

returns:

type response struct{
	success bool
	error *error
	type data struct {
		required_blobs []string
	}
}

/blob POST
accepts:

type request struct {
	hash string
	blob []byte
	size int	
}

returns:

type response struct{
	success bool
	error *error
}

/sdblobs POST
accepts:

type request struct {
	hashes []string
}

returns:

type response struct{
	success bool
	error *error
	type data struct {
		required_sdblobs []string
		required_blobs []struct{
			sdhash string
			missing_blobs []string
		}
	}
}

1) send sdblob in this format: ``` type request struct { hash string sdblob []byte } ``` 2) server checks the blocklist to see if it wants the stream 3) server checks if the sdblob is already in our system 3.1) if the sd blob exists, respond with a list of blobs it needs 3.2) if the sd blob doesn't exist, store the sd blob, save the blobs in the DB with is_stored=0, respond with a list of the blobs it needs (all of them) 4) client sends one blob at a time with the following format ``` type request struct { hash string blob []byte size int } ``` 5) server checks that the uploaded blob has the correct hash and size, saves the blob to S3, sets is_stored=1 in the database, responds with a success 6) optionally we should add an endpoint that accepts a list of sdblob hashes and returns whether the sdblob is needed or not and if the sdblob is already stored, a list of blobs that are missing the server thus has to implement the following endpoints: `/sdblob POST` accepts: ``` type request struct { hash string sdblob []byte } ``` returns: ``` type response struct{ success bool error *error type data struct { required_blobs []string } } ``` `/blob POST` accepts: ``` type request struct { hash string blob []byte size int } ``` returns: ``` type response struct{ success bool error *error } ``` `/sdblobs POST` accepts: ``` type request struct { hashes []string } ``` returns: ``` type response struct{ success bool error *error type data struct { required_sdblobs []string required_blobs []struct{ sdhash string missing_blobs []string } } } ```

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/reflector.go#48

No description provided.