+++ /dev/null
-#! /usr/bin/env python\r
-\r
-"""fixdiv - tool to fix division operators.\r
-\r
-To use this tool, first run `python -Qwarnall yourscript.py 2>warnings'.\r
-This runs the script `yourscript.py' while writing warning messages\r
-about all uses of the classic division operator to the file\r
-`warnings'. The warnings look like this:\r
-\r
- <file>:<line>: DeprecationWarning: classic <type> division\r
-\r
-The warnings are written to stderr, so you must use `2>' for the I/O\r
-redirect. I know of no way to redirect stderr on Windows in a DOS\r
-box, so you will have to modify the script to set sys.stderr to some\r
-kind of log file if you want to do this on Windows.\r
-\r
-The warnings are not limited to the script; modules imported by the\r
-script may also trigger warnings. In fact a useful technique is to\r
-write a test script specifically intended to exercise all code in a\r
-particular module or set of modules.\r
-\r
-Then run `python fixdiv.py warnings'. This first reads the warnings,\r
-looking for classic division warnings, and sorts them by file name and\r
-line number. Then, for each file that received at least one warning,\r
-it parses the file and tries to match the warnings up to the division\r
-operators found in the source code. If it is successful, it writes\r
-its findings to stdout, preceded by a line of dashes and a line of the\r
-form:\r
-\r
- Index: <file>\r
-\r
-If the only findings found are suggestions to change a / operator into\r
-a // operator, the output is acceptable input for the Unix 'patch'\r
-program.\r
-\r
-Here are the possible messages on stdout (N stands for a line number):\r
-\r
-- A plain-diff-style change ('NcN', a line marked by '<', a line\r
- containing '---', and a line marked by '>'):\r
-\r
- A / operator was found that should be changed to //. This is the\r
- recommendation when only int and/or long arguments were seen.\r
-\r
-- 'True division / operator at line N' and a line marked by '=':\r
-\r
- A / operator was found that can remain unchanged. This is the\r
- recommendation when only float and/or complex arguments were seen.\r
-\r
-- 'Ambiguous / operator (..., ...) at line N', line marked by '?':\r
-\r
- A / operator was found for which int or long as well as float or\r
- complex arguments were seen. This is highly unlikely; if it occurs,\r
- you may have to restructure the code to keep the classic semantics,\r
- or maybe you don't care about the classic semantics.\r
-\r
-- 'No conclusive evidence on line N', line marked by '*':\r
-\r
- A / operator was found for which no warnings were seen. This could\r
- be code that was never executed, or code that was only executed\r
- with user-defined objects as arguments. You will have to\r
- investigate further. Note that // can be overloaded separately from\r
- /, using __floordiv__. True division can also be separately\r
- overloaded, using __truediv__. Classic division should be the same\r
- as either of those. (XXX should I add a warning for division on\r
- user-defined objects, to disambiguate this case from code that was\r
- never executed?)\r
-\r
-- 'Phantom ... warnings for line N', line marked by '*':\r
-\r
- A warning was seen for a line not containing a / operator. The most\r
- likely cause is a warning about code executed by 'exec' or eval()\r
- (see note below), or an indirect invocation of the / operator, for\r
- example via the div() function in the operator module. It could\r
- also be caused by a change to the file between the time the test\r
- script was run to collect warnings and the time fixdiv was run.\r
-\r
-- 'More than one / operator in line N'; or\r
- 'More than one / operator per statement in lines N-N':\r
-\r
- The scanner found more than one / operator on a single line, or in a\r
- statement split across multiple lines. Because the warnings\r
- framework doesn't (and can't) show the offset within the line, and\r
- the code generator doesn't always give the correct line number for\r
- operations in a multi-line statement, we can't be sure whether all\r
- operators in the statement were executed. To be on the safe side,\r
- by default a warning is issued about this case. In practice, these\r
- cases are usually safe, and the -m option suppresses these warning.\r
-\r
-- 'Can't find the / operator in line N', line marked by '*':\r
-\r
- This really shouldn't happen. It means that the tokenize module\r
- reported a '/' operator but the line it returns didn't contain a '/'\r
- character at the indicated position.\r
-\r
-- 'Bad warning for line N: XYZ', line marked by '*':\r
-\r
- This really shouldn't happen. It means that a 'classic XYZ\r
- division' warning was read with XYZ being something other than\r
- 'int', 'long', 'float', or 'complex'.\r
-\r
-Notes:\r
-\r
-- The augmented assignment operator /= is handled the same way as the\r
- / operator.\r
-\r
-- This tool never looks at the // operator; no warnings are ever\r
- generated for use of this operator.\r
-\r
-- This tool never looks at the / operator when a future division\r
- statement is in effect; no warnings are generated in this case, and\r
- because the tool only looks at files for which at least one classic\r
- division warning was seen, it will never look at files containing a\r
- future division statement.\r
-\r
-- Warnings may be issued for code not read from a file, but executed\r
- using an exec statement or the eval() function. These may have\r
- <string> in the filename position, in which case the fixdiv script\r
- will attempt and fail to open a file named '<string>' and issue a\r
- warning about this failure; or these may be reported as 'Phantom'\r
- warnings (see above). You're on your own to deal with these. You\r
- could make all recommended changes and add a future division\r
- statement to all affected files, and then re-run the test script; it\r
- should not issue any warnings. If there are any, and you have a\r
- hard time tracking down where they are generated, you can use the\r
- -Werror option to force an error instead of a first warning,\r
- generating a traceback.\r
-\r
-- The tool should be run from the same directory as that from which\r
- the original script was run, otherwise it won't be able to open\r
- files given by relative pathnames.\r
-"""\r
-\r
-import sys\r
-import getopt\r
-import re\r
-import tokenize\r
-\r
-multi_ok = 0\r
-\r
-def main():\r
- try:\r
- opts, args = getopt.getopt(sys.argv[1:], "hm")\r
- except getopt.error, msg:\r
- usage(msg)\r
- return 2\r
- for o, a in opts:\r
- if o == "-h":\r
- print __doc__\r
- return\r
- if o == "-m":\r
- global multi_ok\r
- multi_ok = 1\r
- if not args:\r
- usage("at least one file argument is required")\r
- return 2\r
- if args[1:]:\r
- sys.stderr.write("%s: extra file arguments ignored\n", sys.argv[0])\r
- warnings = readwarnings(args[0])\r
- if warnings is None:\r
- return 1\r
- files = warnings.keys()\r
- if not files:\r
- print "No classic division warnings read from", args[0]\r
- return\r
- files.sort()\r
- exit = None\r
- for filename in files:\r
- x = process(filename, warnings[filename])\r
- exit = exit or x\r
- return exit\r
-\r
-def usage(msg):\r
- sys.stderr.write("%s: %s\n" % (sys.argv[0], msg))\r
- sys.stderr.write("Usage: %s [-m] warnings\n" % sys.argv[0])\r
- sys.stderr.write("Try `%s -h' for more information.\n" % sys.argv[0])\r
-\r
-PATTERN = ("^(.+?):(\d+): DeprecationWarning: "\r
- "classic (int|long|float|complex) division$")\r
-\r
-def readwarnings(warningsfile):\r
- prog = re.compile(PATTERN)\r
- try:\r
- f = open(warningsfile)\r
- except IOError, msg:\r
- sys.stderr.write("can't open: %s\n" % msg)\r
- return\r
- warnings = {}\r
- while 1:\r
- line = f.readline()\r
- if not line:\r
- break\r
- m = prog.match(line)\r
- if not m:\r
- if line.find("division") >= 0:\r
- sys.stderr.write("Warning: ignored input " + line)\r
- continue\r
- filename, lineno, what = m.groups()\r
- list = warnings.get(filename)\r
- if list is None:\r
- warnings[filename] = list = []\r
- list.append((int(lineno), intern(what)))\r
- f.close()\r
- return warnings\r
-\r
-def process(filename, list):\r
- print "-"*70\r
- assert list # if this fails, readwarnings() is broken\r
- try:\r
- fp = open(filename)\r
- except IOError, msg:\r
- sys.stderr.write("can't open: %s\n" % msg)\r
- return 1\r
- print "Index:", filename\r
- f = FileContext(fp)\r
- list.sort()\r
- index = 0 # list[:index] has been processed, list[index:] is still to do\r
- g = tokenize.generate_tokens(f.readline)\r
- while 1:\r
- startlineno, endlineno, slashes = lineinfo = scanline(g)\r
- if startlineno is None:\r
- break\r
- assert startlineno <= endlineno is not None\r
- orphans = []\r
- while index < len(list) and list[index][0] < startlineno:\r
- orphans.append(list[index])\r
- index += 1\r
- if orphans:\r
- reportphantomwarnings(orphans, f)\r
- warnings = []\r
- while index < len(list) and list[index][0] <= endlineno:\r
- warnings.append(list[index])\r
- index += 1\r
- if not slashes and not warnings:\r
- pass\r
- elif slashes and not warnings:\r
- report(slashes, "No conclusive evidence")\r
- elif warnings and not slashes:\r
- reportphantomwarnings(warnings, f)\r
- else:\r
- if len(slashes) > 1:\r
- if not multi_ok:\r
- rows = []\r
- lastrow = None\r
- for (row, col), line in slashes:\r
- if row == lastrow:\r
- continue\r
- rows.append(row)\r
- lastrow = row\r
- assert rows\r
- if len(rows) == 1:\r
- print "*** More than one / operator in line", rows[0]\r
- else:\r
- print "*** More than one / operator per statement",\r
- print "in lines %d-%d" % (rows[0], rows[-1])\r
- intlong = []\r
- floatcomplex = []\r
- bad = []\r
- for lineno, what in warnings:\r
- if what in ("int", "long"):\r
- intlong.append(what)\r
- elif what in ("float", "complex"):\r
- floatcomplex.append(what)\r
- else:\r
- bad.append(what)\r
- lastrow = None\r
- for (row, col), line in slashes:\r
- if row == lastrow:\r
- continue\r
- lastrow = row\r
- line = chop(line)\r
- if line[col:col+1] != "/":\r
- print "*** Can't find the / operator in line %d:" % row\r
- print "*", line\r
- continue\r
- if bad:\r
- print "*** Bad warning for line %d:" % row, bad\r
- print "*", line\r
- elif intlong and not floatcomplex:\r
- print "%dc%d" % (row, row)\r
- print "<", line\r
- print "---"\r
- print ">", line[:col] + "/" + line[col:]\r
- elif floatcomplex and not intlong:\r
- print "True division / operator at line %d:" % row\r
- print "=", line\r
- elif intlong and floatcomplex:\r
- print "*** Ambiguous / operator (%s, %s) at line %d:" % (\r
- "|".join(intlong), "|".join(floatcomplex), row)\r
- print "?", line\r
- fp.close()\r
-\r
-def reportphantomwarnings(warnings, f):\r
- blocks = []\r
- lastrow = None\r
- lastblock = None\r
- for row, what in warnings:\r
- if row != lastrow:\r
- lastblock = [row]\r
- blocks.append(lastblock)\r
- lastblock.append(what)\r
- for block in blocks:\r
- row = block[0]\r
- whats = "/".join(block[1:])\r
- print "*** Phantom %s warnings for line %d:" % (whats, row)\r
- f.report(row, mark="*")\r
-\r
-def report(slashes, message):\r
- lastrow = None\r
- for (row, col), line in slashes:\r
- if row != lastrow:\r
- print "*** %s on line %d:" % (message, row)\r
- print "*", chop(line)\r
- lastrow = row\r
-\r
-class FileContext:\r
- def __init__(self, fp, window=5, lineno=1):\r
- self.fp = fp\r
- self.window = 5\r
- self.lineno = 1\r
- self.eoflookahead = 0\r
- self.lookahead = []\r
- self.buffer = []\r
- def fill(self):\r
- while len(self.lookahead) < self.window and not self.eoflookahead:\r
- line = self.fp.readline()\r
- if not line:\r
- self.eoflookahead = 1\r
- break\r
- self.lookahead.append(line)\r
- def readline(self):\r
- self.fill()\r
- if not self.lookahead:\r
- return ""\r
- line = self.lookahead.pop(0)\r
- self.buffer.append(line)\r
- self.lineno += 1\r
- return line\r
- def truncate(self):\r
- del self.buffer[-window:]\r
- def __getitem__(self, index):\r
- self.fill()\r
- bufstart = self.lineno - len(self.buffer)\r
- lookend = self.lineno + len(self.lookahead)\r
- if bufstart <= index < self.lineno:\r
- return self.buffer[index - bufstart]\r
- if self.lineno <= index < lookend:\r
- return self.lookahead[index - self.lineno]\r
- raise KeyError\r
- def report(self, first, last=None, mark="*"):\r
- if last is None:\r
- last = first\r
- for i in range(first, last+1):\r
- try:\r
- line = self[first]\r
- except KeyError:\r
- line = "<missing line>"\r
- print mark, chop(line)\r
-\r
-def scanline(g):\r
- slashes = []\r
- startlineno = None\r
- endlineno = None\r
- for type, token, start, end, line in g:\r
- endlineno = end[0]\r
- if startlineno is None:\r
- startlineno = endlineno\r
- if token in ("/", "/="):\r
- slashes.append((start, line))\r
- if type == tokenize.NEWLINE:\r
- break\r
- return startlineno, endlineno, slashes\r
-\r
-def chop(line):\r
- if line.endswith("\n"):\r
- return line[:-1]\r
- else:\r
- return line\r
-\r
-if __name__ == "__main__":\r
- sys.exit(main())\r
projects / mirror_edk2.git / blobdiff