[mirror_ubuntu-bionic-kernel.git] / Documentation / locking / locktorture.txt

Kernel Lock Torture Test Operation

CONFIG_LOCK_TORTURE_TEST

The CONFIG LOCK_TORTURE_TEST config option provides a kernel module
that runs torture tests on core kernel locking primitives. The kernel
module, 'locktorture', may be built after the fact on the running
kernel to be tested, if desired. The tests periodically output status
messages via printk(), which can be examined via the dmesg (perhaps
grepping for "torture").  The test is started when the module is loaded,
and stops when the module is unloaded. This program is based on how RCU
is tortured, via rcutorture.

This torture test consists of creating a number of kernel threads which
acquire the lock and hold it for specific amount of time, thus simulating
different critical region behaviors. The amount of contention on the lock
can be simulated by either enlarging this critical region hold time and/or
creating more kthreads.


MODULE PARAMETERS

This module has the following parameters:


	    ** Locktorture-specific **

nwriters_stress   Number of kernel threads that will stress exclusive lock
		  ownership (writers). The default value is twice the number
		  of online CPUs.

nreaders_stress   Number of kernel threads that will stress shared lock
		  ownership (readers). The default is the same amount of writer
		  locks. If the user did not specify nwriters_stress, then
		  both readers and writers be the amount of online CPUs.

torture_type	  Type of lock to torture. By default, only spinlocks will
		  be tortured. This module can torture the following locks,
		  with string values as follows:

		     o "lock_busted": Simulates a buggy lock implementation.

		     o "spin_lock": spin_lock() and spin_unlock() pairs.

		     o "spin_lock_irq": spin_lock_irq() and spin_unlock_irq()
					pairs.

		     o "rw_lock": read/write lock() and unlock() rwlock pairs.

		     o "rw_lock_irq": read/write lock_irq() and unlock_irq()
				      rwlock pairs.

		     o "mutex_lock": mutex_lock() and mutex_unlock() pairs.

		     o "rtmutex_lock": rtmutex_lock() and rtmutex_unlock()
				       pairs. Kernel must have CONFIG_RT_MUTEX=y.

		     o "rwsem_lock": read/write down() and up() semaphore pairs.

torture_runnable  Start locktorture at boot time in the case where the
		  module is built into the kernel, otherwise wait for
		  torture_runnable to be set via sysfs before starting.
		  By default it will begin once the module is loaded.


	    ** Torture-framework (RCU + locking) **

shutdown_secs	  The number of seconds to run the test before terminating
		  the test and powering off the system.  The default is
		  zero, which disables test termination and system shutdown.
		  This capability is useful for automated testing.

onoff_interval	  The number of seconds between each attempt to execute a
		  randomly selected CPU-hotplug operation.  Defaults
		  to zero, which disables CPU hotplugging.  In
		  CONFIG_HOTPLUG_CPU=n kernels, locktorture will silently
		  refuse to do any CPU-hotplug operations regardless of
		  what value is specified for onoff_interval.

onoff_holdoff	  The number of seconds to wait until starting CPU-hotplug
		  operations.  This would normally only be used when
		  locktorture was built into the kernel and started
		  automatically at boot time, in which case it is useful
		  in order to avoid confusing boot-time code with CPUs
		  coming and going. This parameter is only useful if
		  CONFIG_HOTPLUG_CPU is enabled.

stat_interval	  Number of seconds between statistics-related printk()s.
		  By default, locktorture will report stats every 60 seconds.
		  Setting the interval to zero causes the statistics to
		  be printed -only- when the module is unloaded, and this
		  is the default.

stutter		  The length of time to run the test before pausing for this
		  same period of time.  Defaults to "stutter=5", so as
		  to run and pause for (roughly) five-second intervals.
		  Specifying "stutter=0" causes the test to run continuously
		  without pausing, which is the old default behavior.

shuffle_interval  The number of seconds to keep the test threads affinitied
		  to a particular subset of the CPUs, defaults to 3 seconds.
		  Used in conjunction with test_no_idle_hz.

verbose		  Enable verbose debugging printing, via printk(). Enabled
		  by default. This extra information is mostly related to
		  high-level errors and reports from the main 'torture'
		  framework.


STATISTICS

Statistics are printed in the following format:

spin_lock-torture: Writes:  Total: 93746064  Max/Min: 0/0   Fail: 0
   (A)		    (B)		   (C)		  (D)	       (E)

(A): Lock type that is being tortured -- torture_type parameter.

(B): Number of writer lock acquisitions. If dealing with a read/write primitive
     a second "Reads" statistics line is printed.

(C): Number of times the lock was acquired.

(D): Min and max number of times threads failed to acquire the lock.

(E): true/false values if there were errors acquiring the lock. This should
     -only- be positive if there is a bug in the locking primitive's
     implementation. Otherwise a lock should never fail (i.e., spin_lock()).
     Of course, the same applies for (C), above. A dummy example of this is
     the "lock_busted" type.

USAGE

The following script may be used to torture locks:

	#!/bin/sh

	modprobe locktorture
	sleep 3600
	rmmod locktorture
	dmesg | grep torture:

The output can be manually inspected for the error flag of "!!!".
One could of course create a more elaborate script that automatically
checked for such errors.  The "rmmod" command forces a "SUCCESS",
"FAILURE", or "RCU_HOTPLUG" indication to be printk()ed.  The first
two are self-explanatory, while the last indicates that while there
were no locking failures, CPU-hotplug problems were detected.

Also see: Documentation/RCU/torture.txt
Commit	Line	Data
cdf26bb1 DB	1	Kernel Lock Torture Test Operation
	2
	3	CONFIG_LOCK_TORTURE_TEST
	4
	5	The CONFIG LOCK_TORTURE_TEST config option provides a kernel module
	6	that runs torture tests on core kernel locking primitives. The kernel
	7	module, 'locktorture', may be built after the fact on the running
	8	kernel to be tested, if desired. The tests periodically output status
	9	messages via printk(), which can be examined via the dmesg (perhaps
	10	grepping for "torture"). The test is started when the module is loaded,
	11	and stops when the module is unloaded. This program is based on how RCU
	12	is tortured, via rcutorture.
	13
	14	This torture test consists of creating a number of kernel threads which
	15	acquire the lock and hold it for specific amount of time, thus simulating
	16	different critical region behaviors. The amount of contention on the lock
	17	can be simulated by either enlarging this critical region hold time and/or
	18	creating more kthreads.
	19
	20
	21	MODULE PARAMETERS
	22
	23	This module has the following parameters:
	24
	25
	26	Locktorture-specific
	27
	28	nwriters_stress Number of kernel threads that will stress exclusive lock
	29	ownership (writers). The default value is twice the number
	30	of online CPUs.
	31
4f6332c1 DB	32	nreaders_stress Number of kernel threads that will stress shared lock
	33	ownership (readers). The default is the same amount of writer
	34	locks. If the user did not specify nwriters_stress, then
	35	both readers and writers be the amount of online CPUs.
	36
cdf26bb1 DB	37	torture_type Type of lock to torture. By default, only spinlocks will
	38	be tortured. This module can torture the following locks,
	39	with string values as follows:
	40
	41	o "lock_busted": Simulates a buggy lock implementation.
	42
	43	o "spin_lock": spin_lock() and spin_unlock() pairs.
	44
	45	o "spin_lock_irq": spin_lock_irq() and spin_unlock_irq()
	46	pairs.
	47
e34191fa DB	48	o "rw_lock": read/write lock() and unlock() rwlock pairs.
	49
	50	o "rw_lock_irq": read/write lock_irq() and unlock_irq()
	51	rwlock pairs.
	52
42ddc75d DB	53	o "mutex_lock": mutex_lock() and mutex_unlock() pairs.
42ddc75d DB	54
095777c4 DB	55	o "rtmutex_lock": rtmutex_lock() and rtmutex_unlock()
	56	pairs. Kernel must have CONFIG_RT_MUTEX=y.
	57
4a3b427f DB	58	o "rwsem_lock": read/write down() and up() semaphore pairs.
4a3b427f DB	59
cdf26bb1 DB	60	torture_runnable Start locktorture at boot time in the case where the
	61	module is built into the kernel, otherwise wait for
	62	torture_runnable to be set via sysfs before starting.
	63	By default it will begin once the module is loaded.
	64
	65
	66	Torture-framework (RCU + locking)
	67
	68	shutdown_secs The number of seconds to run the test before terminating
	69	the test and powering off the system. The default is
	70	zero, which disables test termination and system shutdown.
	71	This capability is useful for automated testing.
	72
	73	onoff_interval The number of seconds between each attempt to execute a
	74	randomly selected CPU-hotplug operation. Defaults
	75	to zero, which disables CPU hotplugging. In
	76	CONFIG_HOTPLUG_CPU=n kernels, locktorture will silently
	77	refuse to do any CPU-hotplug operations regardless of
	78	what value is specified for onoff_interval.
	79
	80	onoff_holdoff The number of seconds to wait until starting CPU-hotplug
	81	operations. This would normally only be used when
	82	locktorture was built into the kernel and started
	83	automatically at boot time, in which case it is useful
	84	in order to avoid confusing boot-time code with CPUs
	85	coming and going. This parameter is only useful if
	86	CONFIG_HOTPLUG_CPU is enabled.
	87
	88	stat_interval Number of seconds between statistics-related printk()s.
	89	By default, locktorture will report stats every 60 seconds.
	90	Setting the interval to zero causes the statistics to
	91	be printed -only- when the module is unloaded, and this
	92	is the default.
	93
	94	stutter The length of time to run the test before pausing for this
	95	same period of time. Defaults to "stutter=5", so as
	96	to run and pause for (roughly) five-second intervals.
	97	Specifying "stutter=0" causes the test to run continuously
	98	without pausing, which is the old default behavior.
	99
	100	shuffle_interval The number of seconds to keep the test threads affinitied
	101	to a particular subset of the CPUs, defaults to 3 seconds.
	102	Used in conjunction with test_no_idle_hz.
	103
	104	verbose Enable verbose debugging printing, via printk(). Enabled
	105	by default. This extra information is mostly related to
	106	high-level errors and reports from the main 'torture'
	107	framework.
	108
	109
	110	STATISTICS
	111
	112	Statistics are printed in the following format:
	113
	114	spin_lock-torture: Writes: Total: 93746064 Max/Min: 0/0 Fail: 0
4f6332c1	115	(A) (B) (C) (D) (E)
cdf26bb1 DB	116
	117	(A): Lock type that is being tortured -- torture_type parameter.
	118
4f6332c1 DB	119	(B): Number of writer lock acquisitions. If dealing with a read/write primitive
	120	a second "Reads" statistics line is printed.
	121
	122	(C): Number of times the lock was acquired.
cdf26bb1	123
4f6332c1	124	(D): Min and max number of times threads failed to acquire the lock.
cdf26bb1	125
4f6332c1	126	(E): true/false values if there were errors acquiring the lock. This should
cdf26bb1 DB	127	-only- be positive if there is a bug in the locking primitive's
	128	implementation. Otherwise a lock should never fail (i.e., spin_lock()).
	129	Of course, the same applies for (C), above. A dummy example of this is
	130	the "lock_busted" type.
	131
	132	USAGE
	133
	134	The following script may be used to torture locks:
	135
	136	#!/bin/sh
	137
	138	modprobe locktorture
	139	sleep 3600
	140	rmmod locktorture
	141	dmesg \| grep torture:
	142
	143	The output can be manually inspected for the error flag of "!!!".
	144	One could of course create a more elaborate script that automatically
	145	checked for such errors. The "rmmod" command forces a "SUCCESS",
	146	"FAILURE", or "RCU_HOTPLUG" indication to be printk()ed. The first
	147	two are self-explanatory, while the last indicates that while there
	148	were no locking failures, CPU-hotplug problems were detected.
	149
	150	Also see: Documentation/RCU/torture.txt