LBRYCommunity/lbry-rocksdb
1
0
Fork 0
Code Issues 6 Pull requests 2 Projects Releases Packages Wiki Activity
269 commits 6 branches 5 tags 721 KiB
Commit graph

132 commits

Author SHA1 Message Date
iFA
734a8b6337
Update options.pxd 2020-03-12 20:49:29 +01:00
iFA
ac49a391af
Update options.pxd 2019-04-22 20:46:01 +02:00
iFA
57397cd077
Update _rocksdb.pyx 2019-04-22 20:45:29 +02:00
Ming-Hsuan-Tu
ee1bc5a0b9
Merge pull request #40 from fish2000/patch-2 
Allow `rocksdb.DB` instances to be manually closed (second revision PR)
 2019-02-02 12:45:08 +08:00
twmht
ccd9f21ea1 temporary remove some test options 2019-02-02 12:39:17 +08:00
Alexander Böhn
952e7bb4ab
Allow rocksdb.DB instances to be manually closed 
While `delete rocks_ptr` is a deterministic operation in C++, the Python analogue `del rocks_handle` is not – disposal and finalization of the Rocks database are entirely dependent on the Python garbage collector (q.v. the `python-rocksdb` tests themselves, which call `gc.collect()` after `del rocks_handle` to attempt to force the destructor to run). This change exposes a method to trigger the destruction of the underlying Rocks database pointer (deterministic!) through the Python Rocks handle; existing code will not need to be changed, as the Python object destructor (non-deterministic!) will now call this method.

This is the second revision of this PR – it resolves the first revision, #39.
 2019-02-01 11:36:36 -05:00
twmht
0c809e1d2b change default compaction_pri 2019-02-01 21:57:23 +08:00
twmht
32a3dca280 remove full_scan_mode 2019-02-01 21:56:58 +08:00
Jason Fried
2a66e20ca3 Column Family Support 
Add support for Column Families in a runtime safe way.
Add unittests to test functionality
Insure all unittests are passing.
Cleaned up unittests to not use a fixed directory in tmp, but use tempfile
 2018-11-06 02:26:47 +00:00
Ming-Hsuan-Tu
4d18d4be42
Merge branch 'master' into feature-fixSnappyCompressionTest 2018-02-15 15:56:05 +08:00
Mehdi Abaakouk
e4c0de9455 Allow to compile the extension everywhere 
This change:
* uses to setuptools Cython automatic extension build system.
* Add tox.ini to run tests and build docs into virtualenv
* Add .travis.yaml and Dockerfile to run tests in CI
* Change requirements to ensure:
  - Cython and setuptools are installed before we build the Cython
    extension
  - tests dependencies are not installed by default
  - doc dependencies are explicit
* Add missing lz4 library
* Allow to build the module with any librocksdb headers (no-rtti)

Closes #15
 2018-02-09 09:21:56 +01:00
George Mossessian
17277d3dd0 allow snappy_compression as a default option in test_options.py::TestOptions::test_simple 2017-06-28 15:04:33 -07:00
twmht
0dbe3eca5b add testcase for memtable 2017-06-19 13:59:31 +08:00
twmht
8382523ac4 fix memtable_factory crash 2017-06-19 13:58:37 +08:00
twmht
f18c09e1cd use pytest 2017-04-25 19:33:42 +08:00
twmht
5610d9474d update the usage of default operators 2017-04-25 08:25:02 +08:00
twmht
79b0f8d6fb add seekForPrev 2017-04-23 19:32:41 +08:00
twmht
9d87eb5bdc add compaction_pri 2017-04-17 00:00:44 +08:00
twmht
9fcafffbf4 add default merge operator 2017-04-16 14:13:01 +08:00
twmht
58ba17ac90 Merge options source_compaction_factor, max_grandparent_overlap_bytes and expanded_compaction_factor into max_compaction_bytes 2017-04-08 00:18:38 +08:00
twmht
056d62d43d now support rocksdb 5.3.0 2017-03-24 00:56:24 +08:00
hofmockel
01f13577ad Add "row_cache" to options. 2015-08-30 12:21:40 +02:00
hofmockel
6b54dc9e64 No need for the 'get_ob' methods on PyCache. 2015-08-30 12:20:45 +02:00
hofmockel
167ee99f33 Change API of compact_range to be compatible with the change of rocksdb. 2015-08-27 21:45:09 +02:00
hofmockel
8fba5ffa1d Wrap the RepairDB function. 2015-08-15 16:34:35 +02:00
hofmockel
aee8dbe540 Remove prints from the tests. 2015-07-05 14:16:20 +02:00
hofmockel
2ef04007c4 Add lz4 compression type. 2015-07-05 13:52:17 +02:00
hofmockel
302d1a6440 Make it possible to iterate over a WriteBatch. 2015-07-05 13:35:15 +02:00
hofmockel
cd545c7ac0 Update the declaration with rocksdb. 2015-04-12 14:09:02 +02:00
hofmockel
8b26e1e4b0 Get rid of depricated way to create backupengine. 2015-04-12 14:08:30 +02:00
hofmockel
0c13b55e55 Remove rm_scan_count_limit from Cache creation. 2015-04-12 13:58:22 +02:00
hofmockel
b9f06fe785 Remove table_cache_remove_scan_count_limit because it is also removed from rocksdb. 2015-04-12 13:51:28 +02:00
hofmockel
2cbeb48c17 Type has changed with rocksdb 3.6 2014-10-22 20:58:59 +02:00
hofmockel
9cf520a77b Update the tests. 2014-10-22 20:54:25 +02:00
hofmockel
87a7ddfe1c Move filter_policy to block_based_table_factory. 2014-10-22 09:43:47 +02:00
hofmockel
5550ed91e7 Move the block_cache* from options to block_based_table_factory. 2014-10-22 09:41:33 +02:00
hofmockel
14dd1c785a Move the 'simple' block based table options to the block_table_factory. 2014-10-22 09:35:17 +02:00
hofmockel
3358119f22 Remove deprecated options. 2014-10-22 09:31:46 +02:00
hofmockel
8a0bd3a24e Change include path and get rid of the warning. 2014-08-22 20:08:30 +02:00
hofmockel
e3aa17dc3a Fix unittest 2014-08-22 20:08:20 +02:00
hofmockel
a3072c79b3 Document the new init-methods for the SST-table-builders 2014-08-22 19:58:17 +02:00
hofmockel
13518d2680 Remove TotalOrderPlainTableFactory. It was also removed in rocksdb. 2014-08-22 19:15:46 +02:00
hofmockel
f4c5aed418 Make the 'compact_range' call available in python 
(cherry picked from commit c6e4014136)
 2014-05-31 20:25:06 +02:00
hofmockel
f897bf4911 Adapt to the changes of 'prefix seek api' 
See https://github.com/facebook/rocksdb/wiki/Prefix-Seek-API-Changes
 2014-05-31 20:21:42 +02:00
hofmockel
6ceb6514b5 Allow configuration of universal style compaction 2014-04-29 17:46:21 +02:00
hofmockel
1cb9ec4ee1 Allow it to configure the memtable representation 2014-04-28 20:32:33 +02:00
hofmockel
482379cf94 Add support for the 'PlainTableFactories' 2014-04-27 19:20:30 +02:00
hofmockel
5dfc823ac3 Change .pxd files to reflect changed rocksdb API's 2014-04-27 16:39:50 +02:00
hofmockel
bfb370fdc8 Add new option called verify_checksums_in_compaction 2014-04-27 16:39:17 +02:00
hofmockel
9d11bb0fb2 Get rid of some warnings 2014-04-01 21:32:18 +02:00
hofmockel
98301a8ccb Be nice and mark them as virutal 2014-04-01 21:31:03 +02:00
hofmockel
dca6b61bbb Update the default values for some options 2014-04-01 21:25:52 +02:00
hofmockel
098f9578d6 Change prefix_extractor to smart-pointer instead of raw 2014-04-01 21:24:18 +02:00
hofmockel
f16daf0a41 Add python wrapper for BackupEngine 2014-02-02 17:43:00 +01:00
hofmockel
f0e0512a5f Add definitions to wrap BackupEngine 2014-02-02 17:43:00 +01:00
hofmockel
8ef2a13476 Better error messages 2014-01-26 10:50:44 +01:00
hofmockel
44cd1978af Make tests python3 ready 2014-01-26 10:45:36 +01:00
hofmockel
41fc293cb8 Better error reporting for a python prefix extractor 2014-01-26 10:45:06 +01:00
hofmockel
f16932b325 Small code reformat 2014-01-26 09:49:07 +01:00
hofmockel
0dca8c379e Better error reporting in python filter policy 2014-01-26 09:47:04 +01:00
hofmockel
9133c81f9e Get rid of the second context, not needed it is always the same 2014-01-24 17:18:29 +01:00
hofmockel
a83a15e1a8 Better error reporting in python comparators 
* Written to the rocksdb log-file
* raise an exception back to rocksdb
 2014-01-24 17:11:41 +01:00
hofmockel
b600d3371f Get access to the dedicated logger of a DB 2014-01-24 17:06:26 +01:00
hofmockel
43a6997362 Fix deadlock on shutdown by releasing the GIL during del 
On delete rocksdb waits for the background thread to finish.
However the background threads needs the GIL to execute python-code
(for example comparator)
=>
* main thread has GIL
* main thread waits for background thread
* background thread tries to get GIL
which means deadlock
 2014-01-24 17:03:14 +01:00
hofmockel
c8b92d5adb Fore future use, prevent options beeing shared with other DB objects 
For better logging I'm going to inject the rocksdb info_logger into the C++ Wrapper classes
=> The classes on the options object have a member to a DB specific logger
=> This c++ classes can only belong to a SINGLE db
=> For simplicity make this requirement also for the options object itself
 2014-01-23 08:53:14 +01:00
hofmockel
a4c93f7144 __dealloc__ is also called if __cinit__ raises, be more robust in freeing 2014-01-23 08:34:26 +01:00
hofmockel
6ad545cea0 Fix possible crash if snapshot and DB take part in cyclic gargabe collection 
The cyclic garbage collector may choose this snappshot object to break the
cycle. In that case tp_clear will remove the reference to self.db.
So if __dealloc__ of the snapshot is called, self.db is not valid anymore
 2014-01-23 08:10:51 +01:00
hofmockel
a1cb7f9efc Raise error if offset+size is too big 2014-01-21 17:34:01 +01:00
hofmockel
36eb7024d3 Make 'prefix_extractor' active 2014-01-21 17:26:30 +01:00
hofmockel
6f452968bc PySliceTransfrom to bridge python and c++ 2014-01-21 13:15:26 +01:00
hofmockel
40c87ae7ed C callbacks to C++ for wrapping SliceTransfrom 2014-01-21 13:15:26 +01:00
hofmockel
c7d192128b Call it.Prev() without the GIL 2014-01-19 13:45:02 +01:00
hofmockel
ccfa2b522e Iterator notifies errors via 'it.status()' 2014-01-19 13:41:49 +01:00
hofmockel
0d20a2564a Prevent crash if python code is executed in a rocksdb background thread 2014-01-19 12:35:36 +01:00
hofmockel
ecb6e26546 Release the GIL if calls into rocksdb are made 2014-01-18 12:24:49 +01:00
hofmockel
86e6aef6cb Tag all the C++ functions with nogil and except+ 2014-01-17 20:06:17 +01:00
hofmockel
79e8f5da8c Use bytes for the internal representation of a WriteBatch 2014-01-17 07:39:33 +01:00
hofmockel
38f9c6b57f Fix import to support python3 2014-01-16 21:33:37 +01:00
hofmockel
7d61b63b79 Separate strictly between bytes/unicode 
which makes py3 support possible
 2014-01-16 21:32:00 +01:00
hofmockel
6277f9ab5c Support unicode objects for paths 
Use sys.getfilesystemencoding() for encoding
 2014-01-16 08:51:01 +01:00
hofmockel
b4fb12589d Fix refcount error and use cythons <bytes> to convert a python object to char* 2014-01-15 09:53:27 +01:00
hofmockel
77c384b6b0 Initial commit of the driver 2014-01-13 19:52:22 +01:00