ceph/src/rocksdb/docs/_posts/2018-08-01-rocksdb-tuning-advisor.markdown

   1 ---
   2 title: Rocksdb Tuning Advisor
   3 layout: post
   4 author: poojam23
   5 category: blog
   6 ---
   7
   8 The performance of Rocksdb is contingent on its tuning. However, because
   9 of the complexity of its underlying technology and a large number of
  10 configurable parameters, a good configuration is sometimes hard to obtain. The aim of
  11 the python command-line tool, Rocksdb Advisor, is to automate the process of
  12 suggesting improvements in the configuration based on advice from Rocksdb
  13 experts.
  14
  15 ### Overview
  16
  17 Experts share their wisdom as rules comprising of conditions and suggestions in the INI format (refer
  18 [rules.ini](https://github.com/facebook/rocksdb/blob/master/tools/advisor/advisor/rules.ini)).
  19 Users provide the Rocksdb configuration that they want to improve upon (as the
  20 familiar Rocksdb OPTIONS file —
  21 [example](https://github.com/facebook/rocksdb/blob/master/examples/rocksdb_option_file_example.ini))
  22 and the path of the file which contains Rocksdb logs and statistics.
  23 The [Advisor](https://github.com/facebook/rocksdb/blob/master/tools/advisor/advisor/rule_parser_example.py)
  24 creates appropriate DataSource objects (for Rocksdb
  25 [logs](https://github.com/facebook/rocksdb/blob/master/tools/advisor/advisor/db_log_parser.py),
  26 [options](https://github.com/facebook/rocksdb/blob/master/tools/advisor/advisor/db_options_parser.py),
  27 [statistics](https://github.com/facebook/rocksdb/blob/master/tools/advisor/advisor/db_stats_fetcher.py) etc.)
  28 and provides them to the [Rules Engine](https://github.com/facebook/rocksdb/blob/master/tools/advisor/advisor/rule_parser.py).
  29 The Rules uses rules from experts to parse data-sources and trigger appropriate rules.
  30 The Advisor's output gives information about which rules were triggered,
  31 why they were triggered and what each of them suggests. Each suggestion
  32 provided by a triggered rule advises some action on a Rocksdb
  33 configuration option, for example, increase CFOptions.write_buffer_size,
  34 set bloom_bits to 2 etc.
  35
  36 ### Usage
  37
  38 An example command to run the tool:
  39
  40 ```shell
  41 cd rocksdb/tools/advisor
  42 python3 -m advisor.rule_parser_example --rules_spec=advisor/rules.ini --rocksdb_options=test/input_files/OPTIONS-000005 --log_files_path_prefix=test/input_files/LOG-0 --stats_dump_period_sec=20
  43 ```
  44
  45 Sample output where a Rocksdb log-based rule has been triggered :
  46
  47 ```shell
  48 Rule: stall-too-many-memtables
  49 LogCondition: stall-too-many-memtables regex: Stopping writes because we have \d+ immutable memtables \(waiting for flush\), max_write_buffer_number is set to \d+
  50 Suggestion: inc-bg-flush option : DBOptions.max_background_flushes action : increase suggested_values : ['2']
  51 Suggestion: inc-write-buffer option : CFOptions.max_write_buffer_number action : increase
  52 scope: col_fam:
  53 {'default'}
  54 ```
  55
  56 ### Read more
  57
  58 For more information, refer to [advisor](https://github.com/facebook/rocksdb/tree/master/tools/advisor/README.md).