Compaction control

Compaction is the background process that merges SST files, drops tombstones and expired keys, and rebalances the LSM tree. Most of the time you just want it on. Occasionally you want to drive it from the application — bulk imports, end-of-day "tidy ups", or controlled benchmarking.

Upstream reference: Compaction wiki, Leveled Compaction, Universal Compaction.

Triggering a manual compaction

CompactRange compacts every SST file whose key range overlaps [start, limit]. Pass null for both to compact the entire keyspace.

// Compact the whole DB
db.CompactRange(start: null, limit: null);

// Compact a slice
db.CompactRange("user:1000", "user:2000");

// Per column family
db.CompactRange("a", "z", users);

A full compaction can be expensive — it rewrites every SST in the range — so reserve it for explicit maintenance windows.

Disabling auto-compactions

You can turn off background compactions on a column family. Useful when you're bulk-loading and want to consolidate at the end.

var cfOpts = new ColumnFamilyOptions()
    .SetDisableAutoCompactions(1);

…then re-enable via db.SetOptions after the load:

db.SetOptions(new Dictionary<string, string>
{
    ["disable_auto_compactions"] = "false",
});

db.CompactRange(null, null);   // consolidate

Flushing the MemTable

Flush pushes the active MemTable to an L0 SST file. Use it before:

• Backing up via a checkpoint (so all writes are on disk).
• Snapshotting WAL state for shipping to a follower.
• Running a manual compaction over recent data.

var flush = new FlushOptions().SetWaitForFlush(true);
db.Flush(flush);

Disable / enable file deletions

These two methods fence the database against background SST deletion. They're how you take a consistent "no files vanish from under me" view — used by Checkpoint under the covers and useful when building external backup tools.

db.DisableFileDeletions();
try
{
    // Copy SST files to a backup location…
}
finally
{
    db.EnableFileDeletions();
}

Observing compaction

db.GetProperty("rocksdb.estimate-pending-compaction-bytes");
db.GetProperty("rocksdb.num-running-compactions");
db.GetProperty("rocksdb.stats");                              // human-readable
db.GetProperty("rocksdb.cfstats", users);                     // CF-specific

EnableStatistics() on DbOptions turns on the full counter set:

var options = new DbOptions().EnableStatistics();
// …
string s = options.GetStatisticsString();

Upstream reference: Statistics wiki.

A typical maintenance routine

public static void NightlyMaintenance(RocksDb db)
{
    db.Flush(new FlushOptions().SetWaitForFlush(true));
    db.CompactRange(start: null, limit: null);

    Console.WriteLine(db.GetProperty("rocksdb.stats"));
}