]>
Commit | Line | Data |
---|---|---|
53b2ba57 DM |
1 | All about co_lnotab, the line number table.\r |
2 | \r | |
3 | Code objects store a field named co_lnotab. This is an array of unsigned bytes\r | |
4 | disguised as a Python string. It is used to map bytecode offsets to source code\r | |
5 | line #s for tracebacks and to identify line number boundaries for line tracing.\r | |
6 | \r | |
7 | The array is conceptually a compressed list of\r | |
8 | (bytecode offset increment, line number increment)\r | |
9 | pairs. The details are important and delicate, best illustrated by example:\r | |
10 | \r | |
11 | byte code offset source code line number\r | |
12 | 0 1\r | |
13 | 6 2\r | |
14 | 50 7\r | |
15 | 350 307\r | |
16 | 361 308\r | |
17 | \r | |
18 | Instead of storing these numbers literally, we compress the list by storing only\r | |
19 | the increments from one row to the next. Conceptually, the stored list might\r | |
20 | look like:\r | |
21 | \r | |
22 | 0, 1, 6, 1, 44, 5, 300, 300, 11, 1\r | |
23 | \r | |
24 | The above doesn't really work, but it's a start. Note that an unsigned byte\r | |
25 | can't hold negative values, or values larger than 255, and the above example\r | |
26 | contains two such values. So we make two tweaks:\r | |
27 | \r | |
28 | (a) there's a deep assumption that byte code offsets and their corresponding\r | |
29 | line #s both increase monotonically, and\r | |
30 | (b) if at least one column jumps by more than 255 from one row to the next,\r | |
31 | more than one pair is written to the table. In case #b, there's no way to know\r | |
32 | from looking at the table later how many were written. That's the delicate\r | |
33 | part. A user of co_lnotab desiring to find the source line number\r | |
34 | corresponding to a bytecode address A should do something like this\r | |
35 | \r | |
36 | lineno = addr = 0\r | |
37 | for addr_incr, line_incr in co_lnotab:\r | |
38 | addr += addr_incr\r | |
39 | if addr > A:\r | |
40 | return lineno\r | |
41 | lineno += line_incr\r | |
42 | \r | |
43 | (In C, this is implemented by PyCode_Addr2Line().) In order for this to work,\r | |
44 | when the addr field increments by more than 255, the line # increment in each\r | |
45 | pair generated must be 0 until the remaining addr increment is < 256. So, in\r | |
46 | the example above, assemble_lnotab in compile.c should not (as was actually done\r | |
47 | until 2.2) expand 300, 300 to\r | |
48 | 255, 255, 45, 45,\r | |
49 | but to\r | |
50 | 255, 0, 45, 255, 0, 45.\r | |
51 | \r | |
52 | The above is sufficient to reconstruct line numbers for tracebacks, but not for\r | |
53 | line tracing. Tracing is handled by PyCode_CheckLineNumber() in codeobject.c\r | |
54 | and maybe_call_line_trace() in ceval.c.\r | |
55 | \r | |
56 | *** Tracing ***\r | |
57 | \r | |
58 | To a first approximation, we want to call the tracing function when the line\r | |
59 | number of the current instruction changes. Re-computing the current line for\r | |
60 | every instruction is a little slow, though, so each time we compute the line\r | |
61 | number we save the bytecode indices where it's valid:\r | |
62 | \r | |
63 | *instr_lb <= frame->f_lasti < *instr_ub\r | |
64 | \r | |
65 | is true so long as execution does not change lines. That is, *instr_lb holds\r | |
66 | the first bytecode index of the current line, and *instr_ub holds the first\r | |
67 | bytecode index of the next line. As long as the above expression is true,\r | |
68 | maybe_call_line_trace() does not need to call PyCode_CheckLineNumber(). Note\r | |
69 | that the same line may appear multiple times in the lnotab, either because the\r | |
70 | bytecode jumped more than 255 indices between line number changes or because\r | |
71 | the compiler inserted the same line twice. Even in that case, *instr_ub holds\r | |
72 | the first index of the next line.\r | |
73 | \r | |
74 | However, we don't *always* want to call the line trace function when the above\r | |
75 | test fails.\r | |
76 | \r | |
77 | Consider this code:\r | |
78 | \r | |
79 | 1: def f(a):\r | |
80 | 2: while a:\r | |
81 | 3: print 1,\r | |
82 | 4: break\r | |
83 | 5: else:\r | |
84 | 6: print 2,\r | |
85 | \r | |
86 | which compiles to this:\r | |
87 | \r | |
88 | 2 0 SETUP_LOOP 19 (to 22)\r | |
89 | >> 3 LOAD_FAST 0 (a)\r | |
90 | 6 POP_JUMP_IF_FALSE 17\r | |
91 | \r | |
92 | 3 9 LOAD_CONST 1 (1)\r | |
93 | 12 PRINT_ITEM \r | |
94 | \r | |
95 | 4 13 BREAK_LOOP \r | |
96 | 14 JUMP_ABSOLUTE 3\r | |
97 | >> 17 POP_BLOCK \r | |
98 | \r | |
99 | 6 18 LOAD_CONST 2 (2)\r | |
100 | 21 PRINT_ITEM \r | |
101 | >> 22 LOAD_CONST 0 (None)\r | |
102 | 25 RETURN_VALUE \r | |
103 | \r | |
104 | If 'a' is false, execution will jump to the POP_BLOCK instruction at offset 17\r | |
105 | and the co_lnotab will claim that execution has moved to line 4, which is wrong.\r | |
106 | In this case, we could instead associate the POP_BLOCK with line 5, but that\r | |
107 | would break jumps around loops without else clauses.\r | |
108 | \r | |
109 | We fix this by only calling the line trace function for a forward jump if the\r | |
110 | co_lnotab indicates we have jumped to the *start* of a line, i.e. if the current\r | |
111 | instruction offset matches the offset given for the start of a line by the\r | |
112 | co_lnotab. For backward jumps, however, we always call the line trace function,\r | |
113 | which lets a debugger stop on every evaluation of a loop guard (which usually\r | |
114 | won't be the first opcode in a line).\r | |
115 | \r | |
116 | Why do we set f_lineno when tracing, and only just before calling the trace\r | |
117 | function? Well, consider the code above when 'a' is true. If stepping through\r | |
118 | this with 'n' in pdb, you would stop at line 1 with a "call" type event, then\r | |
119 | line events on lines 2, 3, and 4, then a "return" type event -- but because the\r | |
120 | code for the return actually falls in the range of the "line 6" opcodes, you\r | |
121 | would be shown line 6 during this event. This is a change from the behaviour in\r | |
122 | 2.2 and before, and I've found it confusing in practice. By setting and using\r | |
123 | f_lineno when tracing, one can report a line number different from that\r | |
124 | suggested by f_lasti on this one occasion where it's desirable.\r |