[mirror_ubuntu-jammy-kernel.git] / Documentation / admin-guide / numastat.rst

===============================
Numa policy hit/miss statistics
===============================

/sys/devices/system/node/node*/numastat

All units are pages. Hugepages have separate counters.

The numa_hit, numa_miss and numa_foreign counters reflect how well processes
are able to allocate memory from nodes they prefer. If they succeed, numa_hit
is incremented on the preferred node, otherwise numa_foreign is incremented on
the preferred node and numa_miss on the node where allocation succeeded.

Usually preferred node is the one local to the CPU where the process executes,
but restrictions such as mempolicies can change that, so there are also two
counters based on CPU local node. local_node is similar to numa_hit and is
incremented on allocation from a node by CPU on the same node. other_node is
similar to numa_miss and is incremented on the node where allocation succeeds
from a CPU from a different node. Note there is no counter analogical to
numa_foreign.

In more detail:

=============== ============================================================
numa_hit	A process wanted to allocate memory from this node,
		and succeeded.

numa_miss	A process wanted to allocate memory from another node,
		but ended up with memory from this node.

numa_foreign	A process wanted to allocate on this node,
		but ended up with memory from another node.

local_node	A process ran on this node's CPU,
		and got memory from this node.

other_node	A process ran on a different node's CPU
		and got memory from this node.

interleave_hit 	Interleaving wanted to allocate from this node
		and succeeded.
=============== ============================================================

For easier reading you can use the numastat utility from the numactl package
(http://oss.sgi.com/projects/libnuma/). Note that it only works
well right now on machines with a small number of CPUs.

Note that on systems with memoryless nodes (where a node has CPUs but no
memory) the numa_hit, numa_miss and numa_foreign statistics can be skewed
heavily. In the current kernel implementation, if a process prefers a
memoryless node (i.e.  because it is running on one of its local CPU), the
implementation actually treats one of the nearest nodes with memory as the
preferred node. As a result, such allocation will not increase the numa_foreign
counter on the memoryless node, and will skew the numa_hit, numa_miss and
numa_foreign statistics of the nearest node.
Commit	Line	Data
e4a5c33e	1	===============================
1da177e4	2	Numa policy hit/miss statistics
e4a5c33e	3	===============================
1da177e4 LT	4
	5	/sys/devices/system/node/node*/numastat
	6
	7	All units are pages. Hugepages have separate counters.
	8
77691ee9 VB	9	The numa_hit, numa_miss and numa_foreign counters reflect how well processes
	10	are able to allocate memory from nodes they prefer. If they succeed, numa_hit
	11	is incremented on the preferred node, otherwise numa_foreign is incremented on
	12	the preferred node and numa_miss on the node where allocation succeeded.
	13
	14	Usually preferred node is the one local to the CPU where the process executes,
	15	but restrictions such as mempolicies can change that, so there are also two
	16	counters based on CPU local node. local_node is similar to numa_hit and is
	17	incremented on allocation from a node by CPU on the same node. other_node is
	18	similar to numa_miss and is incremented on the node where allocation succeeds
	19	from a CPU from a different node. Note there is no counter analogical to
	20	numa_foreign.
	21
	22	In more detail:
	23
e4a5c33e	24	=============== ============================================================
36519c9e GL	25	numa_hit A process wanted to allocate memory from this node,
	26	and succeeded.
	27
	28	numa_miss A process wanted to allocate memory from another node,
	29	but ended up with memory from this node.
	30
	31	numa_foreign A process wanted to allocate on this node,
77691ee9	32	but ended up with memory from another node.
36519c9e	33
77691ee9 VB	34	local_node A process ran on this node's CPU,
77691ee9 VB	35	and got memory from this node.
36519c9e	36
77691ee9 VB	37	other_node A process ran on a different node's CPU
77691ee9 VB	38	and got memory from this node.
36519c9e GL	39
	40	interleave_hit Interleaving wanted to allocate from this node
	41	and succeeded.
e4a5c33e	42	=============== ============================================================
1da177e4 LT	43
1da177e4 LT	44	For easier reading you can use the numastat utility from the numactl package
36519c9e	45	(http://oss.sgi.com/projects/libnuma/). Note that it only works
1da177e4 LT	46	well right now on machines with a small number of CPUs.
1da177e4 LT	47
77691ee9 VB	48	Note that on systems with memoryless nodes (where a node has CPUs but no
	49	memory) the numa_hit, numa_miss and numa_foreign statistics can be skewed
	50	heavily. In the current kernel implementation, if a process prefers a
	51	memoryless node (i.e. because it is running on one of its local CPU), the
	52	implementation actually treats one of the nearest nodes with memory as the
	53	preferred node. As a result, such allocation will not increase the numa_foreign
	54	counter on the memoryless node, and will skew the numa_hit, numa_miss and
	55	numa_foreign statistics of the nearest node.