rocksdb #29
6
db/db.go
|
@ -332,8 +332,10 @@ func IterCF(db *grocksdb.DB, opts *IterOptions) <-chan *prefixes.PrefixRowKV {
|
||||||
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
|
|||||||
}
|
}
|
||||||
|
|
||||||
go func() {
|
go func() {
|
||||||
defer it.Close()
|
defer func() {
|
||||||
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
|
|||||||
defer close(ch)
|
it.Close()
|
||||||
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
|
|||||||
|
close(ch)
|
||||||
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
|
|||||||
|
}()
|
||||||
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
|
|||||||
|
|
||||||
var prevKey []byte
|
var prevKey []byte
|
||||||
// FIXME: There's messy uses of kv being nil / not nil here.
|
// FIXME: There's messy uses of kv being nil / not nil here.
|
||||||
|
|
||||||
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
no need to start these with N no need to start these with N
and many of these are unused and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families typo: Famlies -> Families
this can be unexported this can be unexported
still need this? still need this?
typo typo
👀 :eyes:
I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation. I agree the "N" can go. I copied these all over from the Python. where I believe they are similarly unused, because I thought it might be the defacto documentation.
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this. ```
// Valid returns false only when an Iterator has iterated past either the
// first or the last key in the database.
func (iter *Iterator) Valid() bool {
return C.rocksdb_iter_valid(iter.c) != 0
}
```
If our iterator was to try to go off the end of the db, or the front if we're going backwards, we need to check this.
|
13
db/db_get.go
|
@ -44,6 +44,7 @@ func (db *ReadOnlyDBColumnFamily) GetBlockHash(height uint32) ([]byte, error) {
|
||||||
key := prefixes.NewBlockHashKey(height)
|
key := prefixes.NewBlockHashKey(height)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
@ -64,6 +65,7 @@ func (db *ReadOnlyDBColumnFamily) GetHeader(height uint32) ([]byte, error) {
|
||||||
key := prefixes.NewHeaderKey(height)
|
key := prefixes.NewHeaderKey(height)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
@ -127,6 +129,7 @@ func (db *ReadOnlyDBColumnFamily) GetClaimsInChannelCount(channelHash []byte) (u
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
|
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, err
|
return 0, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
@ -188,6 +191,7 @@ func (db *ReadOnlyDBColumnFamily) GetRepost(claimHash []byte) ([]byte, error) {
|
||||||
key := prefixes.NewRepostKey(claimHash)
|
key := prefixes.NewRepostKey(claimHash)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
@ -234,6 +238,7 @@ func (db *ReadOnlyDBColumnFamily) GetChannelForClaim(claimHash []byte, txNum uin
|
||||||
key := prefixes.NewClaimToChannelKey(claimHash, txNum, position)
|
key := prefixes.NewClaimToChannelKey(claimHash, txNum, position)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
@ -300,6 +305,7 @@ func (db *ReadOnlyDBColumnFamily) GetSupportAmount(claimHash []byte) (uint64, er
|
||||||
key := prefixes.NewSupportAmountKey(claimHash)
|
key := prefixes.NewSupportAmountKey(claimHash)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, err
|
return 0, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
@ -322,6 +328,7 @@ func (db *ReadOnlyDBColumnFamily) GetTxHash(txNum uint32) ([]byte, error) {
|
||||||
key := prefixes.NewTxHashKey(txNum)
|
key := prefixes.NewTxHashKey(txNum)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -355,6 +362,7 @@ func (db *ReadOnlyDBColumnFamily) GetActivationFull(txNum uint32, postition uint
|
||||||
key := prefixes.NewActivationKey(typ, txNum, postition)
|
key := prefixes.NewActivationKey(typ, txNum, postition)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, err
|
return 0, err
|
||||||
}
|
}
|
||||||
|
@ -379,6 +387,7 @@ func (db *ReadOnlyDBColumnFamily) GetCachedClaimTxo(claim []byte, useCache bool)
|
||||||
key := prefixes.NewClaimToTXOKey(claim)
|
key := prefixes.NewClaimToTXOKey(claim)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -418,6 +427,7 @@ func (db *ReadOnlyDBColumnFamily) GetControllingClaim(name string) (*prefixes.Cl
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
log.Println(hex.EncodeToString(rawKey))
|
log.Println(hex.EncodeToString(rawKey))
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
log.Printf("slice: %#v", slice)
|
log.Printf("slice: %#v", slice)
|
||||||
log.Printf("err: %#v", err)
|
log.Printf("err: %#v", err)
|
||||||
|
|
||||||
|
@ -468,6 +478,7 @@ func (db *ReadOnlyDBColumnFamily) GetTxCount(height uint32) (*prefixes.TxCountVa
|
||||||
key := prefixes.NewTxCountKey(height)
|
key := prefixes.NewTxCountKey(height)
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
|
@ -490,6 +501,7 @@ func (db *ReadOnlyDBColumnFamily) GetDBState() (*prefixes.DBStateValue, error) {
|
||||||
key := prefixes.NewDBStateKey()
|
key := prefixes.NewDBStateKey()
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
@ -546,6 +558,7 @@ func (db *ReadOnlyDBColumnFamily) GetCachedClaimHash(txNum uint32, position uint
|
||||||
rawKey := key.PackKey()
|
rawKey := key.PackKey()
|
||||||
|
|
||||||
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
slice, err := db.DB.GetCF(db.Opts, handle, rawKey)
|
||||||
|
defer slice.Free()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
} else if slice.Size() == 0 {
|
} else if slice.Size() == 0 {
|
||||||
|
|
|
@ -108,9 +108,12 @@ func (opts *IterOptions) ReadRow(prevKey *[]byte) *prefixes.PrefixRowKV {
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
}
|
}
|
||||||
|
|
||||||
key := it.Key()
|
key := it.Key()
|
||||||
|
defer key.Free()
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
keyData := key.Data()
|
keyData := key.Data()
|
||||||
keyLen := len(keyData)
|
keyLen := len(keyData)
|
||||||
|
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
value := it.Value()
|
value := it.Value()
|
||||||
|
defer value.Free()
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
valueData := value.Data()
|
valueData := value.Data()
|
||||||
valueLen := len(valueData)
|
valueLen := len(valueData)
|
||||||
|
|
||||||
|
@ -156,9 +159,6 @@ func (opts *IterOptions) ReadRow(prevKey *[]byte) *prefixes.PrefixRowKV {
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
key.Free()
|
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
value.Free()
|
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
|
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
|||||||
kv := &prefixes.PrefixRowKV{
|
kv := &prefixes.PrefixRowKV{
|
||||||
Key: outKey,
|
Key: outKey,
|
||||||
Value: outValue,
|
Value: outValue,
|
||||||
|
|
||||||
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
should this be named iterator.go or iteroptions.go? it only has options stuff in it should this be named iterator.go or iteroptions.go? it only has options stuff in it
or maybe just options.go or maybe just options.go
|
2
main.go
|
@ -5,6 +5,8 @@ import (
|
||||||
"fmt"
|
"fmt"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
_ "net/http/pprof"
|
||||||
|
|
||||||
prolly here just for profiling, right? prolly here just for profiling, right?
yeah yeah
|
|||||||
pb "github.com/lbryio/hub/protobuf/go"
|
pb "github.com/lbryio/hub/protobuf/go"
|
||||||
"github.com/lbryio/hub/server"
|
"github.com/lbryio/hub/server"
|
||||||
"github.com/lbryio/lbry.go/v2/extras/util"
|
"github.com/lbryio/lbry.go/v2/extras/util"
|
||||||
|
|
|
@ -150,7 +150,7 @@ func (s *Server) Run() {
|
||||||
// initializes everything. It loads information about previously known peers,
|
// initializes everything. It loads information about previously known peers,
|
||||||
// creates needed internal data structures, and initializes goroutines.
|
// creates needed internal data structures, and initializes goroutines.
|
||||||
func MakeHubServer(ctx context.Context, args *Args) *Server {
|
func MakeHubServer(ctx context.Context, args *Args) *Server {
|
||||||
grpcServer := grpc.NewServer(grpc.NumStreamWorkers(10))
|
grpcServer := grpc.NewServer(grpc.NumStreamWorkers(0))
|
||||||
|
|
||||||
multiSpaceRe, err := regexp.Compile(`\s{2,}`)
|
multiSpaceRe, err := regexp.Compile(`\s{2,}`)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|
no need to start these with N
and many of these are unused
is this really no big deal? seems like an error to me but idk rocksdb
typo: Famlies -> Families
this can be unexported
still need this?
typo
👀