Document the changes done at the options.

This commit is contained in:
hofmockel 2014-10-22 20:53:51 +02:00
parent 87a7ddfe1c
commit 7e1df8bec1
3 changed files with 70 additions and 101 deletions

View file

@ -109,45 +109,6 @@ Options object
| *Type:* ``int`` | *Type:* ``int``
| *Default:* ``5000`` | *Default:* ``5000``
.. py:attribute:: block_cache
Control over blocks (user data is stored in a set of blocks, and
a block is the unit of reading from disk).
If not ``None`` use the specified cache for blocks.
If ``None``, rocksdb will automatically create and use an 8MB internal cache.
| *Type:* Instace of :py:class:`rocksdb.LRUCache`
| *Default:* ``None``
.. py:attribute:: block_cache_compressed
If not ``None`` use the specified cache for compressed blocks.
If ``None``, rocksdb will not use a compressed block cache.
| *Type:* Instace of :py:class:`rocksdb.LRUCache`
| *Default:* ``None``
.. py:attribute:: block_size
Approximate size of user data packed per block. Note that the
block size specified here corresponds to uncompressed data. The
actual size of the unit read from disk may be smaller if
compression is enabled. This parameter can be changed dynamically.
| *Type:* ``int``
| *Default:* ``4096``
.. py:attribute:: block_restart_interval
Number of keys between restart points for delta encoding of keys.
This parameter can be changed dynamically. Most clients should
leave this parameter alone.
| *Type:* ``int``
| *Default:* ``16``
.. py:attribute:: compression .. py:attribute:: compression
Compress blocks using the specified compression algorithm. Compress blocks using the specified compression algorithm.
@ -156,15 +117,6 @@ Options object
| *Type:* Member of :py:class:`rocksdb.CompressionType` | *Type:* Member of :py:class:`rocksdb.CompressionType`
| *Default:* :py:attr:`rocksdb.CompressionType.snappy_compression` | *Default:* :py:attr:`rocksdb.CompressionType.snappy_compression`
.. py:attribute:: whole_key_filtering
If ``True``, place whole keys in the filter (not just prefixes).
This must generally be true for gets to be efficient.
| *Type:* ``bool``
| *Default:* ``True``
.. py:attribute:: num_levels .. py:attribute:: num_levels
Number of levels for this database Number of levels for this database
@ -315,15 +267,6 @@ Options object
| *Type:* ``bool`` | *Type:* ``bool``
| *Default:* ``False`` | *Default:* ``False``
.. py:attribute:: db_stats_log_interval
This number controls how often a new scribe log about
db deploy stats is written out.
-1 indicates no logging at all.
| *Type:* ``int``
| *Default:* ``1800``
.. py:attribute:: db_log_dir .. py:attribute:: db_log_dir
This specifies the info LOG dir. This specifies the info LOG dir.
@ -346,16 +289,6 @@ Options object
| *Type:* ``unicode`` | *Type:* ``unicode``
| *Default:* ``""`` | *Default:* ``""``
.. py:attribute:: disable_seek_compaction
Disable compaction triggered by seek.
With bloomfilter and fast storage, a miss on one level
is very cheap if the file handle is cached in table cache
(which is true if max_open_files is large).
| *Type:* ``bool``
| *Default:* ``True``
.. py:attribute:: delete_obsolete_files_period_micros .. py:attribute:: delete_obsolete_files_period_micros
The periodicity when obsolete files get deleted. The default The periodicity when obsolete files get deleted. The default
@ -454,15 +387,6 @@ Options object
| *Type:* ``int`` | *Type:* ``int``
| *Default:* ``(2**64) - 1`` | *Default:* ``(2**64) - 1``
.. py:attribute:: no_block_cache
Disable block cache. If this is set to true,
then no block cache should be used, and the block_cache should
point to ``None``
| *Type:* ``bool``
| *Default:* ``False``
.. py:attribute:: table_cache_numshardbits .. py:attribute:: table_cache_numshardbits
Number of shards used for table cache. Number of shards used for table cache.
@ -579,17 +503,6 @@ Options object
| *Type:* ``int`` | *Type:* ``int``
| *Default:* ``3600`` | *Default:* ``3600``
.. py:attribute:: block_size_deviation
This is used to close a block before it reaches the configured
'block_size'. If the percentage of free space in the current block is less
than this specified number and adding a new record to the block will
exceed the configured block size, then this block will be closed and the
new record will be written to the next block.
| *Type:* ``int``
| *Default:* ``10``
.. py:attribute:: advise_random_on_open .. py:attribute:: advise_random_on_open
If set true, will hint the underlying file system that the file If set true, will hint the underlying file system that the file
@ -797,15 +710,6 @@ Options object
*Default:* ``None`` *Default:* ``None``
.. py:attribute:: filter_policy
If not ``None`` use the specified filter policy to reduce disk reads.
A python filter policy must implement the
:py:class:`rocksdb.interfaces.FilterPolicy` interface.
Recommendes is a instance of :py:class:`rocksdb.BloomFilterPolicy`
*Default:* ``None``
.. py:attribute:: prefix_extractor .. py:attribute:: prefix_extractor
If not ``None``, use the specified function to determine the If not ``None``, use the specified function to determine the
@ -902,7 +806,8 @@ https://github.com/facebook/rocksdb/wiki/A-Tutorial-of-RocksDB-SST-formats
Wraps BlockBasedTableFactory of RocksDB. Wraps BlockBasedTableFactory of RocksDB.
.. py:method:: __init__(index_type='binary_search', hash_index_allow_collision=True, checksum='crc32') .. py:method:: __init__(index_type='binary_search', hash_index_allow_collision=True, checksum='crc32', block_cache, block_cache_compressed, filter_policy=None, no_block_cache=False, block_size=None, block_size_deviation=None, block_restart_interval=None, whole_key_filtering=None):
:param string index_type: :param string index_type:
* ``binary_search`` a space efficient index block that is optimized * ``binary_search`` a space efficient index block that is optimized
@ -922,6 +827,56 @@ https://github.com/facebook/rocksdb/wiki/A-Tutorial-of-RocksDB-SST-formats
even though they have different checksum type. even though they have different checksum type.
Can be either ``crc32`` or ``xxhash``. Can be either ``crc32`` or ``xxhash``.
:param block_cache:
Control over blocks (user data is stored in a set of blocks, and
a block is the unit of reading from disk).
If ``None``, rocksdb will automatically create and use an 8MB internal cache.
If not ``None`` use the specified cache for blocks. In that case it must
be an instance of :py:class:`rocksdb.LRUCache`
:param block_cache_compressed:
If ``None``, rocksdb will not use a compressed block cache.
If not ``None`` use the specified cache for compressed blocks. In that
case it must be an instance of :py:class:`rocksdb.LRUCache`
:param filter_policy:
If not ``None`` use the specified filter policy to reduce disk reads.
A python filter policy must implement the
:py:class:`rocksdb.interfaces.FilterPolicy` interface.
Recommended is a instance of :py:class:`rocksdb.BloomFilterPolicy`
:param bool no_block_cache:
Disable block cache. If this is set to true,
then no block cache should be used, and the block_cache should
point to ``None``
:param int block_size:
If set to ``None`` the rocksdb default of ``4096`` is used.
Approximate size of user data packed per block. Note that the
block size specified here corresponds to uncompressed data. The
actual size of the unit read from disk may be smaller if
compression is enabled. This parameter can be changed dynamically.
:param int block_size_deviation:
If set to ``None`` the rocksdb default of ``10`` is used.
This is used to close a block before it reaches the configured
'block_size'. If the percentage of free space in the current block is less
than this specified number and adding a new record to the block will
exceed the configured block size, then this block will be closed and the
new record will be written to the next block.
:param int block_restart_interval:
If set to ``None`` the rocksdb default of ``16`` is used.
Number of keys between restart points for delta encoding of keys.
This parameter can be changed dynamically. Most clients should
leave this parameter alone.
:param bool whole_key_filtering:
If set to ``None`` the rocksdb default of ``True`` is used.
If ``True``, place whole keys in the filter (not just prefixes).
This must generally be true for gets to be efficient.
.. py:class:: rocksdb.PlainTableFactory .. py:class:: rocksdb.PlainTableFactory
Plain Table with prefix-only seek. It wraps rocksdb PlainTableFactory. Plain Table with prefix-only seek. It wraps rocksdb PlainTableFactory.

View file

@ -35,7 +35,20 @@ For DB.iterkeys and DB.iteritems ``itertools.takewhile`` is a possible solution.
**SST Table Builders:** **SST Table Builders:**
* Removed ``NewTotalOrderPlainTableFactory``, because rocksdb drops it too. * Removed ``NewTotalOrderPlainTableFactory``, because rocksdb drops it too.
**Changed Options:**
In newer versions of rocksdb a bunch of options were moved or removed.
* Rename ``bloom_bits_per_prefix`` of :py:class:`rocksdb.PlainTableFactory` to ``bloom_bits_per_key`` * Rename ``bloom_bits_per_prefix`` of :py:class:`rocksdb.PlainTableFactory` to ``bloom_bits_per_key``
* Removed ``Options.db_stats_log_interval``.
* Removed ``Options.disable_seek_compaction``
* Moved ``Options.no_block_cache`` to ``BlockBasedTableFactory``
* Moved ``Options.block_size`` to ``BlockBasedTableFactory``
* Moved ``Options.block_size_deviation`` to ``BlockBasedTableFactory``
* Moved ``Options.block_restart_interval`` to ``BlockBasedTableFactory``
* Moved ``Options.whole_key_filtering`` to ``BlockBasedTableFactory``
New: New:
^^^^ ^^^^
@ -44,7 +57,6 @@ New:
* Add more option to :py:class:`rocksdb.PlainTableFactory` * Add more option to :py:class:`rocksdb.PlainTableFactory`
Version 0.2 Version 0.2
----------- -----------

View file

@ -20,9 +20,11 @@ A more production ready open can look like this ::
opts.write_buffer_size = 67108864 opts.write_buffer_size = 67108864
opts.max_write_buffer_number = 3 opts.max_write_buffer_number = 3
opts.target_file_size_base = 67108864 opts.target_file_size_base = 67108864
opts.filter_policy = rocksdb.BloomFilterPolicy(10)
opts.block_cache = rocksdb.LRUCache(2 * (1024 ** 3)) opts.table_factory = rocksdb.BlockBasedTableFactory(
opts.block_cache_compressed = rocksdb.LRUCache(500 * (1024 ** 2)) filter_policy=rocksdb.BloomFilterPolicy(10),
block_cache=rocksdb.LRUCache(2 * (1024 ** 3)),
block_cache_compressed=rocksdb.LRUCache(500 * (1024 ** 2)))
db = rocksdb.DB("test.db", opts) db = rocksdb.DB("test.db", opts)