LBRYCommunity/herald.go
1
0
Fork 0
Code Issues 16 Pull requests Packages Projects Releases 14 Wiki Activity

Problems building with RocksDB v7.0.3 installed (MacOS) #44

New issue
Open
opened 2022-07-29 23:08:31 +02:00 by moodyjon · 1 comment
moodyjon commented 2022-07-29 23:08:31 +02:00 (Migrated from github.com)
Copy link

Filing this for informational purposes, as the real problem is how homebrew package manager gives no option to install a lower version of rocksdb (e.g. 6.29.5). Basically your only choice is to install the latest version.

Mac-mini herald % env CGO_LDFLAGS="-L/opt/homebrew/lib -lrocksdb -lstdc++ -lm -lz -lsnappy -llz4 -lzstd" CGO_CFLAGS="-I/opt/homebrew/include/rocksdb" go build .
# github.com/linxGnu/grocksdb
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:2291:13: could not determine kind of name for C.rocksdb_options_get_base_background_compactions
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1129:17: could not determine kind of name for C.rocksdb_options_get_hard_rate_limit
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1143:14: could not determine kind of name for C.rocksdb_options_get_rate_limit_delay_max_milliseconds
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1387:20: could not determine kind of name for C.rocksdb_options_get_skip_log_error_on_recovery
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1115:17: could not determine kind of name for C.rocksdb_options_get_soft_rate_limit
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:2286:2: could not determine kind of name for C.rocksdb_options_set_base_background_compactions
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1124:2: could not determine kind of name for C.rocksdb_options_set_hard_rate_limit
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:755:2: could not determine kind of name for C.rocksdb_options_set_max_mem_compaction_level
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1313:2: could not determine kind of name for C.rocksdb_options_set_purge_redundant_kvs_while_flush
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1137:2: could not determine kind of name for C.rocksdb_options_set_rate_limit_delay_max_milliseconds
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1380:2: could not determine kind of name for C.rocksdb_options_set_skip_log_error_on_recovery
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1110:2: could not determine kind of name for C.rocksdb_options_set_soft_rate_limit
../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1183:2: could not determine kind of name for C.rocksdb_options_set_table_cache_remove_scan_count_limit

I worked around this by upgrading grocksdb linkage to match the rocksdb that I have at the moment.

(lbry-venv-3.9) Mac-mini lbry-sdk % brew info rocksdb
rocksdb: stable 7.4.3 (bottled), HEAD
Embeddable, persistent key-value store for fast storage
https://rocksdb.org/
/opt/homebrew/Cellar/rocksdb/7.0.3 (138 files, 37.7MB) *
  Poured from bottle on 2022-04-29 at 16:21:30
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/rocksdb.rb
License: GPL-2.0-only or Apache-2.0
==> Dependencies
Build: cmake ✔
Required: gflags ✔, lz4 ✔, snappy ✔, zstd ✔
==> Options
--HEAD
	Install HEAD version
==> Analytics
install: 1,498 (30 days), 2,737 (90 days), 13,789 (365 days)
install-on-request: 1,246 (30 days), 2,388 (90 days), 11,101 (365 days)
build-error: 0 (30 days)

Here is my patch to make my current rocksdb v7.0.3 work:

diff --git a/go.mod b/go.mod
index bedf5f8..297e910 100644
--- a/go.mod
+++ b/go.mod
@@ -8,7 +8,7 @@ require (
        github.com/ReneKroon/ttlcache/v2 v2.8.1
        github.com/akamensky/argparse v1.2.2
        github.com/lbryio/lbry.go/v3 v3.0.1-beta
-       github.com/linxGnu/grocksdb v1.6.42
+       github.com/linxGnu/grocksdb v1.7.0
        github.com/olivere/elastic/v7 v7.0.24
        github.com/prometheus/client_golang v1.11.0
        github.com/prometheus/client_model v0.2.0
diff --git a/go.sum b/go.sum
index 6b69c85..195387a 100644
--- a/go.sum
+++ b/go.sum
@@ -357,6 +357,8 @@ github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-b
 github.com/lightstep/lightstep-tracer-go v0.18.1/go.mod h1:jlF1pusYV4pidLvZ+XD0UBX0ZE6WURAspgAczcDHrL4=
 github.com/linxGnu/grocksdb v1.6.42 h1:nJLoXFuzwBwQQQrXTUgRGRz1QRm7y8pR6CNV/gwrbqs=
 github.com/linxGnu/grocksdb v1.6.42/go.mod h1:JcMMDBFaDNhRXFYcYXmgQwb/RarSld1PulTI7UzE+w0=
+github.com/linxGnu/grocksdb v1.7.0 h1:UyFDykX0CUfxDN10cqlFho/rwt9K6KoDaLXL9Ej5z9g=
+github.com/linxGnu/grocksdb v1.7.0/go.mod h1:JcMMDBFaDNhRXFYcYXmgQwb/RarSld1PulTI7UzE+w0=
 github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ=
 github.com/lyoshenka/bencode v0.0.0-20180323155644-b7abd7672df5/go.mod h1:H0aPCWffGOaDcjkw1iB7W9DVLp6GXmfcJY/7YZCWPA4=
 github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ=
Filing this for informational purposes, as ***the real problem is how homebrew package manager gives no option to install a lower version of rocksdb (e.g. 6.29.5)***. Basically your only choice is to install the latest version. ``` Mac-mini herald % env CGO_LDFLAGS="-L/opt/homebrew/lib -lrocksdb -lstdc++ -lm -lz -lsnappy -llz4 -lzstd" CGO_CFLAGS="-I/opt/homebrew/include/rocksdb" go build . # github.com/linxGnu/grocksdb ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:2291:13: could not determine kind of name for C.rocksdb_options_get_base_background_compactions ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1129:17: could not determine kind of name for C.rocksdb_options_get_hard_rate_limit ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1143:14: could not determine kind of name for C.rocksdb_options_get_rate_limit_delay_max_milliseconds ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1387:20: could not determine kind of name for C.rocksdb_options_get_skip_log_error_on_recovery ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1115:17: could not determine kind of name for C.rocksdb_options_get_soft_rate_limit ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:2286:2: could not determine kind of name for C.rocksdb_options_set_base_background_compactions ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1124:2: could not determine kind of name for C.rocksdb_options_set_hard_rate_limit ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:755:2: could not determine kind of name for C.rocksdb_options_set_max_mem_compaction_level ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1313:2: could not determine kind of name for C.rocksdb_options_set_purge_redundant_kvs_while_flush ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1137:2: could not determine kind of name for C.rocksdb_options_set_rate_limit_delay_max_milliseconds ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1380:2: could not determine kind of name for C.rocksdb_options_set_skip_log_error_on_recovery ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1110:2: could not determine kind of name for C.rocksdb_options_set_soft_rate_limit ../go/pkg/mod/github.com/linx!gnu/grocksdb@v1.6.42/options.go:1183:2: could not determine kind of name for C.rocksdb_options_set_table_cache_remove_scan_count_limit ``` I worked around this by upgrading grocksdb linkage to match the rocksdb that I have at the moment. ``` (lbry-venv-3.9) Mac-mini lbry-sdk % brew info rocksdb rocksdb: stable 7.4.3 (bottled), HEAD Embeddable, persistent key-value store for fast storage https://rocksdb.org/ /opt/homebrew/Cellar/rocksdb/7.0.3 (138 files, 37.7MB) * Poured from bottle on 2022-04-29 at 16:21:30 From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/rocksdb.rb License: GPL-2.0-only or Apache-2.0 ==> Dependencies Build: cmake ✔ Required: gflags ✔, lz4 ✔, snappy ✔, zstd ✔ ==> Options --HEAD Install HEAD version ==> Analytics install: 1,498 (30 days), 2,737 (90 days), 13,789 (365 days) install-on-request: 1,246 (30 days), 2,388 (90 days), 11,101 (365 days) build-error: 0 (30 days) ``` Here is my patch to make my current rocksdb v7.0.3 work: ``` diff --git a/go.mod b/go.mod index bedf5f8..297e910 100644 --- a/go.mod +++ b/go.mod @@ -8,7 +8,7 @@ require ( github.com/ReneKroon/ttlcache/v2 v2.8.1 github.com/akamensky/argparse v1.2.2 github.com/lbryio/lbry.go/v3 v3.0.1-beta - github.com/linxGnu/grocksdb v1.6.42 + github.com/linxGnu/grocksdb v1.7.0 github.com/olivere/elastic/v7 v7.0.24 github.com/prometheus/client_golang v1.11.0 github.com/prometheus/client_model v0.2.0 diff --git a/go.sum b/go.sum index 6b69c85..195387a 100644 --- a/go.sum +++ b/go.sum @@ -357,6 +357,8 @@ github.com/lightstep/lightstep-tracer-common/golang/gogo v0.0.0-20190605223551-b github.com/lightstep/lightstep-tracer-go v0.18.1/go.mod h1:jlF1pusYV4pidLvZ+XD0UBX0ZE6WURAspgAczcDHrL4= github.com/linxGnu/grocksdb v1.6.42 h1:nJLoXFuzwBwQQQrXTUgRGRz1QRm7y8pR6CNV/gwrbqs= github.com/linxGnu/grocksdb v1.6.42/go.mod h1:JcMMDBFaDNhRXFYcYXmgQwb/RarSld1PulTI7UzE+w0= +github.com/linxGnu/grocksdb v1.7.0 h1:UyFDykX0CUfxDN10cqlFho/rwt9K6KoDaLXL9Ej5z9g= +github.com/linxGnu/grocksdb v1.7.0/go.mod h1:JcMMDBFaDNhRXFYcYXmgQwb/RarSld1PulTI7UzE+w0= github.com/lyft/protoc-gen-validate v0.0.13/go.mod h1:XbGvPuh87YZc5TdIa2/I4pLk0QoUACkjt2znoq26NVQ= github.com/lyoshenka/bencode v0.0.0-20180323155644-b7abd7672df5/go.mod h1:H0aPCWffGOaDcjkw1iB7W9DVLp6GXmfcJY/7YZCWPA4= github.com/magiconair/properties v1.8.0/go.mod h1:PppfXfuXeibc/6YijjN8zIbojt8czPbwD3XqdrwzmxQ= ```
moodyjon commented 2022-07-29 23:12:38 +02:00 (Migrated from github.com)
Copy link

To find the right version for you, see:

https://github.com/linxGnu/grocksdb/releases

To find the right version for you, see: https://github.com/linxGnu/grocksdb/releases
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
master
server-xxx-endpoints
blockchain_rpc2
blockchain_rpc1
add_staked_amount
v0.2022.10.05.1
v0.2022.10.04.1
v0.2022.08.16.1
v0.2022.08.09.1
v0.2022.01.21.1
v0.2021.12.18.1
v0.2021.12.06.1
range-queries
v0.2021.11.05.1
v0.2021.10.25.1
v0.2021.10.16.1
leveldb-hub
v0.2021.08.24.1
v0.2021.08.24-beta
v0.2021.07.24-beta
v0.2021.07.14.1
v0.2021.07.14
v0.2021.06.14-beta
Labels
Clear labels   
consider soon

Discuss this issue at the next planning meeting, then remove this label

  
documentation

Improvements or additions to documentation

  
good first issue

Good for newcomers

  
hacktoberfest

Welcome to Hacktoberfest

  
help wanted

Extra attention is needed

  
priority: blocker

Issue is blocking release, do ASAP

  
priority: high

Work needs to be moved into sprint ASAP

  
priority: low

Work should be done but can stay in the backlog for now

  
priority: medium

Work needs to be done within 2-3 sprints

  
type: bug

Existing functionality is wrong or broken

  
type: discussion

A conversation. No specific changes requested

  
type: improvement

Existing (or partially existing) functionality needs to be changed

  
type: new feature

New functionality that does not exist yet

  
type: refactor

Minimal user-visible changes, but significant internal work

  
type: task

Either work that's not related to the code, or a small chore that does not fit into other categories

No labels
consider soon
documentation
good first issue
hacktoberfest
help wanted
priority: blocker
priority: high
priority: low
priority: medium
type: bug
type: discussion
type: improvement
type: new feature
type: refactor
type: task
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: LBRYCommunity/herald.go#44
No description provided.
Delete branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?