Commit graph

20 commits

Author SHA1 Message Date
Brannon King
bea5c62644 [lbry] renamed package to lbcutil, updated refs to it 2021-12-14 22:08:50 -08:00
Oliver Gugger
12d7d04b31
multi: fix linter issues 2021-09-28 10:30:21 +02:00
Conner Fromknecht
9e5f4b9a99 gcs/gcs: use sort.Slice instead of sort.Sort, remove uint64Slice
This commit removes the uint64Slice type and performs sorting during
filter construction and ZipMatchAny using sort.Slice. The benchmarks
indicated that this speeds up BuildGCSFilter and ZipMatchAny by 10-12%,
likely to the overhead of needing to resolve the sort.Interface methods.

The benchmarks indicate that this improvement is not present for the
smallest query size in our benchmarks, i.e. 100 elements, but the
reduction is only about 3%. This would indicate the at these small
values, the use of reflection is actually slightly slower than interface
method resolution in total.
2019-04-25 16:57:16 -07:00
Conner Fromknecht
8cea9a2e28 gcs/gcs: ensure match zero hash 2019-04-25 16:42:51 -07:00
Conner Fromknecht
784cec0650 gcs/gcs: adds HashMatchAny for large filter queries 2019-01-11 20:11:46 -08:00
Olaoluwa Osuntokun
d3e82fcd5d gcs: switch basic filter to index prev output scripts of all inputs 2018-07-06 17:32:55 -05:00
Olaoluwa Osuntokun
0ecd90b8d6 gcs: update fp and modulus values based on recent optimality analysis
In this commit, we decrease the default fp rate to 19, or 1/2^19. We do
this as recent analysis by sipa on the bitcoin dev mailing list has
shown that optimally, we can use a value of 2^19 for the fp rate, while
use n=1.497137*2^P rather than n directly. As a result, we can shrink
the filter size by quite a bit, while still maintaining a sane false
positive value.
2018-07-06 17:32:55 -05:00
Olaoluwa Osuntokun
45edb4b6e5
gcs: fix stray import 2018-05-22 17:48:27 -07:00
Jim Posen
cbc2d0fee6 Allow construction of empty filters. 2018-05-15 19:14:18 -07:00
Jim Posen
884680ddbd Serialize filter with N as a VarInt instead of fixed-size. 2018-05-15 19:14:18 -07:00
Olaoluwa Osuntokun
54b6d534d2 gcs: extract fast reduce into distinct func, add comments 2018-05-15 19:14:18 -07:00
Alex
dabaf053db gcs: update modulo algorithm to fast range per gmaxwell's suggestion
The BIP will now specify that the fast range described at the URL
below is used instead of modulo:

https://lemire.me/blog/2016/06/27/a-fast-alternative-to-the-modulo-reduction/
2018-05-15 19:14:18 -07:00
Olaoluwa Osuntokun
e14ee486e6 gcs: fix constant overflow for 32-bit systems 2018-05-15 19:14:18 -07:00
Olaoluwa Osuntokun
18097a52c4 gcs: explicitly optimize inner loop of gcs filter computation 2018-05-15 19:14:18 -07:00
Olaoluwa Osuntokun
c59d7e8fba gcs: pre-allocate capacity of slice for uncompressed filter values
This change will reduce the total number of allocations required to
create/store the allocated filter as we’ll now perform a _single_
allocation, rather than one each time the the dynamically size slice
reaches capacity.
2018-05-15 19:14:18 -07:00
Olaoluwa Osuntokun
0f2eb80fdb gcs: add some line spacing, wrap comments to 80 characters 2018-05-15 19:14:18 -07:00
Alex
f74edf2c4d gcs: allow optional de/serialization of N and P parameters 2018-05-15 19:14:18 -07:00
Alex
b21739c0e2 Fix issues found by golint without changing API. 2018-05-15 19:14:18 -07:00
Alex
684cf07725 Rename gcsFilter type to GCSFilter to export it. 2018-05-15 19:14:18 -07:00
Alex
3069dcf175 gcs: add package for building and using Golomb-coded set filters 2018-05-15 19:14:18 -07:00