]>
Commit | Line | Data |
---|---|---|
1 | Kprobe-based Event Tracer | |
2 | ========================= | |
3 | ||
4 | Documentation is written by Masami Hiramatsu | |
5 | ||
6 | ||
7 | Overview | |
8 | -------- | |
9 | This tracer is similar to the events tracer which is based on Tracepoint | |
10 | infrastructure. Instead of Tracepoint, this tracer is based on kprobes(kprobe | |
11 | and kretprobe). It probes anywhere where kprobes can probe(this means, all | |
12 | functions body except for __kprobes functions). | |
13 | ||
14 | Unlike the function tracer, this tracer can probe instructions inside of | |
15 | kernel functions. It allows you to check which instruction has been executed. | |
16 | ||
17 | Unlike the Tracepoint based events tracer, this tracer can add and remove | |
18 | probe points on the fly. | |
19 | ||
20 | Similar to the events tracer, this tracer doesn't need to be activated via | |
21 | current_tracer, instead of that, just set probe points via | |
22 | /sys/kernel/debug/tracing/kprobe_events. And you can set filters on each | |
23 | probe events via /sys/kernel/debug/tracing/events/kprobes/<EVENT>/filter. | |
24 | ||
25 | ||
26 | Synopsis of kprobe_events | |
27 | ------------------------- | |
28 | p[:[GRP/]EVENT] SYMBOL[+offs]|MEMADDR [FETCHARGS] : Set a probe | |
29 | r[:[GRP/]EVENT] SYMBOL[+0] [FETCHARGS] : Set a return probe | |
30 | ||
31 | GRP : Group name. If omitted, use "kprobes" for it. | |
32 | EVENT : Event name. If omitted, the event name is generated | |
33 | based on SYMBOL+offs or MEMADDR. | |
34 | SYMBOL[+offs] : Symbol+offset where the probe is inserted. | |
35 | MEMADDR : Address where the probe is inserted. | |
36 | ||
37 | FETCHARGS : Arguments. Each probe can have up to 128 args. | |
38 | %REG : Fetch register REG | |
39 | @ADDR : Fetch memory at ADDR (ADDR should be in kernel) | |
40 | @SYM[+|-offs] : Fetch memory at SYM +|- offs (SYM should be a data symbol) | |
41 | $sN : Fetch Nth entry of stack (N >= 0) | |
42 | $sa : Fetch stack address. | |
43 | $aN : Fetch function argument. (N >= 0)(*) | |
44 | $rv : Fetch return value.(**) | |
45 | +|-offs(FETCHARG) : Fetch memory at FETCHARG +|- offs address.(***) | |
46 | NAME=FETCHARG: Set NAME as the argument name of FETCHARG. | |
47 | ||
48 | (*) aN may not correct on asmlinkaged functions and at the middle of | |
49 | function body. | |
50 | (**) only for return probe. | |
51 | (***) this is useful for fetching a field of data structures. | |
52 | ||
53 | ||
54 | Per-Probe Event Filtering | |
55 | ------------------------- | |
56 | Per-probe event filtering feature allows you to set different filter on each | |
57 | probe and gives you what arguments will be shown in trace buffer. If an event | |
58 | name is specified right after 'p:' or 'r:' in kprobe_events, the tracer adds | |
59 | an event under tracing/events/kprobes/<EVENT>, at the directory you can see | |
60 | 'id', 'enabled', 'format' and 'filter'. | |
61 | ||
62 | enabled: | |
63 | You can enable/disable the probe by writing 1 or 0 on it. | |
64 | ||
65 | format: | |
66 | This shows the format of this probe event. | |
67 | ||
68 | filter: | |
69 | You can write filtering rules of this event. | |
70 | ||
71 | id: | |
72 | This shows the id of this probe event. | |
73 | ||
74 | Event Profiling | |
75 | --------------- | |
76 | You can check the total number of probe hits and probe miss-hits via | |
77 | /sys/kernel/debug/tracing/kprobe_profile. | |
78 | The first column is event name, the second is the number of probe hits, | |
79 | the third is the number of probe miss-hits. | |
80 | ||
81 | ||
82 | Usage examples | |
83 | -------------- | |
84 | To add a probe as a new event, write a new definition to kprobe_events | |
85 | as below. | |
86 | ||
87 | echo p:myprobe do_sys_open dfd=$a0 filename=$a1 flags=$a2 mode=$a3 > /sys/kernel/debug/tracing/kprobe_events | |
88 | ||
89 | This sets a kprobe on the top of do_sys_open() function with recording | |
90 | 1st to 4th arguments as "myprobe" event. As this example shows, users can | |
91 | choose more familiar names for each arguments. | |
92 | ||
93 | echo r:myretprobe do_sys_open $rv >> /sys/kernel/debug/tracing/kprobe_events | |
94 | ||
95 | This sets a kretprobe on the return point of do_sys_open() function with | |
96 | recording return value as "myretprobe" event. | |
97 | You can see the format of these events via | |
98 | /sys/kernel/debug/tracing/events/kprobes/<EVENT>/format. | |
99 | ||
100 | cat /sys/kernel/debug/tracing/events/kprobes/myprobe/format | |
101 | name: myprobe | |
102 | ID: 75 | |
103 | format: | |
104 | field:unsigned short common_type; offset:0; size:2; | |
105 | field:unsigned char common_flags; offset:2; size:1; | |
106 | field:unsigned char common_preempt_count; offset:3; size:1; | |
107 | field:int common_pid; offset:4; size:4; | |
108 | field:int common_tgid; offset:8; size:4; | |
109 | ||
110 | field: unsigned long ip; offset:16;tsize:8; | |
111 | field: int nargs; offset:24;tsize:4; | |
112 | field: unsigned long dfd; offset:32;tsize:8; | |
113 | field: unsigned long filename; offset:40;tsize:8; | |
114 | field: unsigned long flags; offset:48;tsize:8; | |
115 | field: unsigned long mode; offset:56;tsize:8; | |
116 | ||
117 | print fmt: "(%lx) dfd=%lx filename=%lx flags=%lx mode=%lx", REC->ip, REC->dfd, REC->filename, REC->flags, REC->mode | |
118 | ||
119 | ||
120 | You can see that the event has 4 arguments as in the expressions you specified. | |
121 | ||
122 | echo > /sys/kernel/debug/tracing/kprobe_events | |
123 | ||
124 | This clears all probe points. | |
125 | ||
126 | Right after definition, each event is disabled by default. For tracing these | |
127 | events, you need to enable it. | |
128 | ||
129 | echo 1 > /sys/kernel/debug/tracing/events/kprobes/myprobe/enable | |
130 | echo 1 > /sys/kernel/debug/tracing/events/kprobes/myretprobe/enable | |
131 | ||
132 | And you can see the traced information via /sys/kernel/debug/tracing/trace. | |
133 | ||
134 | cat /sys/kernel/debug/tracing/trace | |
135 | # tracer: nop | |
136 | # | |
137 | # TASK-PID CPU# TIMESTAMP FUNCTION | |
138 | # | | | | | | |
139 | <...>-1447 [001] 1038282.286875: myprobe: (do_sys_open+0x0/0xd6) dfd=3 filename=7fffd1ec4440 flags=8000 mode=0 | |
140 | <...>-1447 [001] 1038282.286878: myretprobe: (sys_openat+0xc/0xe <- do_sys_open) $rv=fffffffffffffffe | |
141 | <...>-1447 [001] 1038282.286885: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=40413c flags=8000 mode=1b6 | |
142 | <...>-1447 [001] 1038282.286915: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 | |
143 | <...>-1447 [001] 1038282.286969: myprobe: (do_sys_open+0x0/0xd6) dfd=ffffff9c filename=4041c6 flags=98800 mode=10 | |
144 | <...>-1447 [001] 1038282.286976: myretprobe: (sys_open+0x1b/0x1d <- do_sys_open) $rv=3 | |
145 | ||
146 | ||
147 | Each line shows when the kernel hits an event, and <- SYMBOL means kernel | |
148 | returns from SYMBOL(e.g. "sys_open+0x1b/0x1d <- do_sys_open" means kernel | |
149 | returns from do_sys_open to sys_open+0x1b). | |
150 | ||
151 |