[ceph.git] / ceph / doc / dev / mempool_accounting.rst

What is a mempool?
------------------
A memory pool (mempool) is a method for tracking memory consumption. Memory pools represent the memory consumption
of C++ classes and containers, and they are used to assess memory leaks and other insights around memory usage with
low overhead. Each memory pool tracks the number of bytes and items it contains. Mempools are statically declared,
and they serve the purpose of identifying memory-related checks in BlueStore.

Some common mempools that we can track
--------------------------------------

- ``bloom_filter``: tracks objects already in the cache pool in order to determine which objects in the pool are
  being accessed
- ``bluestore_alloc``: accounts for actual allocations done by an allocator
- ``bluestore_inline_bl``: measures encoded length of an inline buffer
- ``bluestore_fsck``: file system consistency check for BlueStore metadata; helps in tracking and debugging during OSD repairs
- ``bluestore_txc``: accounts for committed transactions counter
- ``bluestore_writing_deferred``: measures small writes which are deferred, i.e. first written into RocksDB WAL and later
  flushed to the disk
- ``bluestore_writing``: accounts for in-flight write buffers
- ``bluefs``:  file-system-like interface; provides just enough functionality to allow RocksDB to store its “files” and share
  the same raw device(s) with BlueStore
- ``buffer_anon``: stores arbitrary buffer data
- ``buffer_meta``: all the metadata associated with buffer anon buffers
- ``bluestore_cache_data``: mempool for writing and writing deferred
- ``bluestore_cache_onode``: object node (onode) metadata in the BlueStore cache
- ``bluestore_cache_meta``: key under PREFIX_OBJ where we are stored
- ``bluestore_cache_other``: right now accounts for:

  - ``map_t``: used to track raw extents on disk for SharedBlob and for the in-memory Blob with the blob namespace
  - ``coll_map``: collections_map
  - ``csum_data``: checksum data
- ``bluestore_cache_buffer``: accounts for buffer cache shards
- ``bluestore_extent``: a logical (as well as physical) extent, pointing to some portion of a blob
- ``bluestore_blob``: in-memory blob metadata associated cached buffers
- ``bluestore_shared_blob``: in-memory shared blob state; stores a reference to the set of collections it belongs to
  (includes cached buffers)
- ``bluefs_file_reader``: accounts for bluefs file reader buffer
- ``bluefs_file_writer``: accounts for bluefs file writer buffer

Check mempools usage
~~~~~~~~~~~~~~~~~~~~

Command to see BlueStore memory allocation in these mempools::

     $ ceph daemon osd.NNN dump_mempools


.. note:: see more:
    https://github.com/ceph/ceph/blob/main/src/include/mempool.h
Commit	Line	Data
1e59de90 TL	1	What is a mempool?
	2	------------------
	3	A memory pool (mempool) is a method for tracking memory consumption. Memory pools represent the memory consumption
	4	of C++ classes and containers, and they are used to assess memory leaks and other insights around memory usage with
	5	low overhead. Each memory pool tracks the number of bytes and items it contains. Mempools are statically declared,
	6	and they serve the purpose of identifying memory-related checks in BlueStore.
	7
	8	Some common mempools that we can track
	9	--------------------------------------
	10
	11	- ``bloom_filter``: tracks objects already in the cache pool in order to determine which objects in the pool are
	12	being accessed
	13	- ``bluestore_alloc``: accounts for actual allocations done by an allocator
	14	- ``bluestore_inline_bl``: measures encoded length of an inline buffer
	15	- ``bluestore_fsck``: file system consistency check for BlueStore metadata; helps in tracking and debugging during OSD repairs
	16	- ``bluestore_txc``: accounts for committed transactions counter
	17	- ``bluestore_writing_deferred``: measures small writes which are deferred, i.e. first written into RocksDB WAL and later
	18	flushed to the disk
	19	- ``bluestore_writing``: accounts for in-flight write buffers
	20	- ``bluefs``: file-system-like interface; provides just enough functionality to allow RocksDB to store its “files” and share
	21	the same raw device(s) with BlueStore
	22	- ``buffer_anon``: stores arbitrary buffer data
	23	- ``buffer_meta``: all the metadata associated with buffer anon buffers
	24	- ``bluestore_cache_data``: mempool for writing and writing deferred
	25	- ``bluestore_cache_onode``: object node (onode) metadata in the BlueStore cache
	26	- ``bluestore_cache_meta``: key under PREFIX_OBJ where we are stored
	27	- ``bluestore_cache_other``: right now accounts for:
	28
	29	- ``map_t``: used to track raw extents on disk for SharedBlob and for the in-memory Blob with the blob namespace
	30	- ``coll_map``: collections_map
	31	- ``csum_data``: checksum data
	32	- ``bluestore_cache_buffer``: accounts for buffer cache shards
	33	- ``bluestore_extent``: a logical (as well as physical) extent, pointing to some portion of a blob
	34	- ``bluestore_blob``: in-memory blob metadata associated cached buffers
	35	- ``bluestore_shared_blob``: in-memory shared blob state; stores a reference to the set of collections it belongs to
	36	(includes cached buffers)
	37	- ``bluefs_file_reader``: accounts for bluefs file reader buffer
	38	- ``bluefs_file_writer``: accounts for bluefs file writer buffer
	39
	40	Check mempools usage
	41	~~~~~~~~~~~~~~~~~~~~
	42
	43	Command to see BlueStore memory allocation in these mempools::
	44
	45	$ ceph daemon osd.NNN dump_mempools
	46
	47
	48	.. note:: see more:
	49	https://github.com/ceph/ceph/blob/main/src/include/mempool.h