Skip to content

Latest commit

 

History

History
54 lines (36 loc) · 3.89 KB

Manual-Compaction.md

File metadata and controls

54 lines (36 loc) · 3.89 KB
Raw

Compaction can be triggered manually by calling the DB::CompactRange or DB::CompactFiles method. This is meant to be used by advanced users to implement custom compaction strategies, including but not limited to the following use cases -

  1. Optimize for read heavy workloads by compacting to lowest level after ingesting a large amount of data
  2. Force the data to go through the compaction filter in order to consolidate it
  3. Migrate to a new compaction configuration. For example, if changing number of levels, CompactRange can be called to compact to bottommost level and then move the files to a target level

The example code below shows how to use the APIs.

Options dbOptions;

DB* db;
Status s = DB::Open(dbOptions, "/tmp/rocksdb",  &db);

// Write some data
...
Slice begin("key1");
Slice end("key100");
CompactRangeOptions options;

s = db->CompactRange(options, &begin, &end);

Or

CompactionOptions options;
std::vector<std::string> input_file_names;
int output_level;
...
Status s = db->CompactFiles(options, input_file_names, output_level);

CompactRange

The begin and end arguments define the key range to be compacted. The behavior varies depending on the compaction style being used by the db. In case of universal and FIFO compaction styles, the begin and end arguments are ignored and all files are compacted. Also, files in each level are compacted and left in the same level. For leveled compaction style, all files containing keys in the given range are compacted to the last level containing files. If either begin or end are NULL, it is taken to mean the key before all keys in the db or the key after all keys respectively.

If more than one thread calls manual compaction, only one will actually schedule it while the other threads will simply wait for the scheduled manual compaction to complete. If CompactRangeOptions::exclusive_manual_compaction is set to true, the call will disable scheduling of automatic compaction jobs and wait for existing automatic compaction jobs to finish.

The CompactRangeOptions supports the following options -

  • CompactRangeOptions::exclusive_manual_compaction When set to true, no other compaction will run when this manual compaction is running. Default value is true
  • CompactRangeOptions::change_level,CompactRangeOptions::target_level Together, these options control the level where the compacted files will be placed. If target_level is -1, the compacted files will be moved to the minimum level whose computed max_bytes is still large enough to hold the files. Intermediate levels must be empty. For example, if the files were initially compacted to L5 and L2 is the minimum level large enough to hold the files, they will be placed in L2 if L3 and L4 are empty or in L4 if L3 is non-empty. If target_level is positive, the compacted files will be placed in that level provided intermediate levels are empty. If any any of the intermediate levels are not empty, the compacted files will be left where they are.
  • CompactRangeOptions::target_path_id Compaction outputs will be placed in options.db_paths[target_path_id] directory.
  • CompactRangeOptions::bottommost_level_compaction When set to BottommostLevelCompaction::kSkip, or when set to BottommostLevelCompaction::kIfHaveCompactionFilter and a compaction filter is defined for the column family, the bottommost level files are not compacted.

CompactFiles

This API compacts all the input files into a set of output files in the output_level. The number of output files is determined by the size of the data and the setting of CompactionOptions::output_file_size_limit. This API is not supported in ROCKSDB_LITE.