AppPkg/Applications/Python/Python-2.7.2/Lib/test/regrtest.py

   1 #! /usr/bin/env python
   2
   3 """
   4 Usage:
   5
   6 python -m test.regrtest [options] [test_name1 [test_name2 ...]]
   7 python path/to/Lib/test/regrtest.py [options] [test_name1 [test_name2 ...]]
   8
   9
  10 If no arguments or options are provided, finds all files matching
  11 the pattern "test_*" in the Lib/test subdirectory and runs
  12 them in alphabetical order (but see -M and -u, below, for exceptions).
  13
  14 For more rigorous testing, it is useful to use the following
  15 command line:
  16
  17 python -E -tt -Wd -3 -m test.regrtest [options] [test_name1 ...]
  18
  19
  20 Options:
  21
  22 -h/--help       -- print this text and exit
  23
  24 Verbosity
  25
  26 -v/--verbose    -- run tests in verbose mode with output to stdout
  27 -w/--verbose2   -- re-run failed tests in verbose mode
  28 -W/--verbose3   -- re-run failed tests in verbose mode immediately
  29 -q/--quiet      -- no output unless one or more tests fail
  30 -S/--slow       -- print the slowest 10 tests
  31    --header     -- print header with interpreter info
  32
  33 Selecting tests
  34
  35 -r/--random     -- randomize test execution order (see below)
  36    --randseed   -- pass a random seed to reproduce a previous random run
  37 -f/--fromfile   -- read names of tests to run from a file (see below)
  38 -x/--exclude    -- arguments are tests to *exclude*
  39 -s/--single     -- single step through a set of tests (see below)
  40 -u/--use RES1,RES2,...
  41                 -- specify which special resource intensive tests to run
  42 -M/--memlimit LIMIT
  43                 -- run very large memory-consuming tests
  44
  45 Special runs
  46
  47 -l/--findleaks  -- if GC is available detect tests that leak memory
  48 -L/--runleaks   -- run the leaks(1) command just before exit
  49 -R/--huntrleaks RUNCOUNTS
  50                 -- search for reference leaks (needs debug build, v. slow)
  51 -j/--multiprocess PROCESSES
  52                 -- run PROCESSES processes at once
  53 -T/--coverage   -- turn on code coverage tracing using the trace module
  54 -D/--coverdir DIRECTORY
  55                 -- Directory where coverage files are put
  56 -N/--nocoverdir -- Put coverage files alongside modules
  57 -t/--threshold THRESHOLD
  58                 -- call gc.set_threshold(THRESHOLD)
  59 -F/--forever    -- run the specified tests in a loop, until an error happens
  60
  61
  62 Additional Option Details:
  63
  64 -r randomizes test execution order. You can use --randseed=int to provide a
  65 int seed value for the randomizer; this is useful for reproducing troublesome
  66 test orders.
  67
  68 -s On the first invocation of regrtest using -s, the first test file found
  69 or the first test file given on the command line is run, and the name of
  70 the next test is recorded in a file named pynexttest.  If run from the
  71 Python build directory, pynexttest is located in the 'build' subdirectory,
  72 otherwise it is located in tempfile.gettempdir().  On subsequent runs,
  73 the test in pynexttest is run, and the next test is written to pynexttest.
  74 When the last test has been run, pynexttest is deleted.  In this way it
  75 is possible to single step through the test files.  This is useful when
  76 doing memory analysis on the Python interpreter, which process tends to
  77 consume too many resources to run the full regression test non-stop.
  78
  79 -f reads the names of tests from the file given as f's argument, one
  80 or more test names per line.  Whitespace is ignored.  Blank lines and
  81 lines beginning with '#' are ignored.  This is especially useful for
  82 whittling down failures involving interactions among tests.
  83
  84 -L causes the leaks(1) command to be run just before exit if it exists.
  85 leaks(1) is available on Mac OS X and presumably on some other
  86 FreeBSD-derived systems.
  87
  88 -R runs each test several times and examines sys.gettotalrefcount() to
  89 see if the test appears to be leaking references.  The argument should
  90 be of the form stab:run:fname where 'stab' is the number of times the
  91 test is run to let gettotalrefcount settle down, 'run' is the number
  92 of times further it is run and 'fname' is the name of the file the
  93 reports are written to.  These parameters all have defaults (5, 4 and
  94 "reflog.txt" respectively), and the minimal invocation is '-R :'.
  95
  96 -M runs tests that require an exorbitant amount of memory. These tests
  97 typically try to ascertain containers keep working when containing more than
  98 2 billion objects, which only works on 64-bit systems. There are also some
  99 tests that try to exhaust the address space of the process, which only makes
 100 sense on 32-bit systems with at least 2Gb of memory. The passed-in memlimit,
 101 which is a string in the form of '2.5Gb', determines howmuch memory the
 102 tests will limit themselves to (but they may go slightly over.) The number
 103 shouldn't be more memory than the machine has (including swap memory). You
 104 should also keep in mind that swap memory is generally much, much slower
 105 than RAM, and setting memlimit to all available RAM or higher will heavily
 106 tax the machine. On the other hand, it is no use running these tests with a
 107 limit of less than 2.5Gb, and many require more than 20Gb. Tests that expect
 108 to use more than memlimit memory will be skipped. The big-memory tests
 109 generally run very, very long.
 110
 111 -u is used to specify which special resource intensive tests to run,
 112 such as those requiring large file support or network connectivity.
 113 The argument is a comma-separated list of words indicating the
 114 resources to test.  Currently only the following are defined:
 115
 116     all -       Enable all special resources.
 117
 118     audio -     Tests that use the audio device.  (There are known
 119                 cases of broken audio drivers that can crash Python or
 120                 even the Linux kernel.)
 121
 122     curses -    Tests that use curses and will modify the terminal's
 123                 state and output modes.
 124
 125     largefile - It is okay to run some test that may create huge
 126                 files.  These tests can take a long time and may
 127                 consume >2GB of disk space temporarily.
 128
 129     network -   It is okay to run tests that use external network
 130                 resource, e.g. testing SSL support for sockets.
 131
 132     bsddb -     It is okay to run the bsddb testsuite, which takes
 133                 a long time to complete.
 134
 135     decimal -   Test the decimal module against a large suite that
 136                 verifies compliance with standards.
 137
 138     cpu -       Used for certain CPU-heavy tests.
 139
 140     subprocess  Run all tests for the subprocess module.
 141
 142     urlfetch -  It is okay to download files required on testing.
 143
 144     gui -       Run tests that require a running GUI.
 145
 146     xpickle -   Test pickle and cPickle against Python 2.4, 2.5 and 2.6 to
 147                 test backwards compatibility. These tests take a long time
 148                 to run.
 149
 150 To enable all resources except one, use '-uall,-<resource>'.  For
 151 example, to run all the tests except for the bsddb tests, give the
 152 option '-uall,-bsddb'.
 153 """
 154
 155 import StringIO
 156 import getopt
 157 import json
 158 import os
 159 import random
 160 import re
 161 import sys
 162 import time
 163 import traceback
 164 import warnings
 165 import unittest
 166 import tempfile
 167 import imp
 168 import platform
 169 import sysconfig
 170
 171
 172 # Some times __path__ and __file__ are not absolute (e.g. while running from
 173 # Lib/) and, if we change the CWD to run the tests in a temporary dir, some
 174 # imports might fail.  This affects only the modules imported before os.chdir().
 175 # These modules are searched first in sys.path[0] (so '' -- the CWD) and if
 176 # they are found in the CWD their __file__ and __path__ will be relative (this
 177 # happens before the chdir).  All the modules imported after the chdir, are
 178 # not found in the CWD, and since the other paths in sys.path[1:] are absolute
 179 # (site.py absolutize them), the __file__ and __path__ will be absolute too.
 180 # Therefore it is necessary to absolutize manually the __file__ and __path__ of
 181 # the packages to prevent later imports to fail when the CWD is different.
 182 for module in sys.modules.itervalues():
 183     if hasattr(module, '__path__'):
 184         module.__path__ = [os.path.abspath(path) for path in module.__path__]
 185     if hasattr(module, '__file__'):
 186         module.__file__ = os.path.abspath(module.__file__)
 187
 188
 189 # MacOSX (a.k.a. Darwin) has a default stack size that is too small
 190 # for deeply recursive regular expressions.  We see this as crashes in
 191 # the Python test suite when running test_re.py and test_sre.py.  The
 192 # fix is to set the stack limit to 2048.
 193 # This approach may also be useful for other Unixy platforms that
 194 # suffer from small default stack limits.
 195 if sys.platform == 'darwin':
 196     try:
 197         import resource
 198     except ImportError:
 199         pass
 200     else:
 201         soft, hard = resource.getrlimit(resource.RLIMIT_STACK)
 202         newsoft = min(hard, max(soft, 1024*2048))
 203         resource.setrlimit(resource.RLIMIT_STACK, (newsoft, hard))
 204
 205 # Test result constants.
 206 PASSED = 1
 207 FAILED = 0
 208 ENV_CHANGED = -1
 209 SKIPPED = -2
 210 RESOURCE_DENIED = -3
 211 INTERRUPTED = -4
 212
 213 from test import test_support
 214
 215 RESOURCE_NAMES = ('audio', 'curses', 'largefile', 'network', 'bsddb',
 216                   'decimal', 'cpu', 'subprocess', 'urlfetch', 'gui',
 217                   'xpickle')
 218
 219 TEMPDIR = os.path.abspath(tempfile.gettempdir())
 220
 221
 222 def usage(code, msg=''):
 223     print __doc__
 224     if msg: print msg
 225     sys.exit(code)
 226
 227
 228 def main(tests=None, testdir=None, verbose=0, quiet=False,
 229          exclude=False, single=False, randomize=False, fromfile=None,
 230          findleaks=False, use_resources=None, trace=False, coverdir='coverage',
 231          runleaks=False, huntrleaks=False, verbose2=False, print_slow=False,
 232          random_seed=None, use_mp=None, verbose3=False, forever=False,
 233          header=False):
 234     """Execute a test suite.
 235
 236     This also parses command-line options and modifies its behavior
 237     accordingly.
 238
 239     tests -- a list of strings containing test names (optional)
 240     testdir -- the directory in which to look for tests (optional)
 241
 242     Users other than the Python test suite will certainly want to
 243     specify testdir; if it's omitted, the directory containing the
 244     Python test suite is searched for.
 245
 246     If the tests argument is omitted, the tests listed on the
 247     command-line will be used.  If that's empty, too, then all *.py
 248     files beginning with test_ will be used.
 249
 250     The other default arguments (verbose, quiet, exclude,
 251     single, randomize, findleaks, use_resources, trace, coverdir,
 252     print_slow, and random_seed) allow programmers calling main()
 253     directly to set the values that would normally be set by flags
 254     on the command line.
 255     """
 256
 257     test_support.record_original_stdout(sys.stdout)
 258     try:
 259         opts, args = getopt.getopt(sys.argv[1:], 'hvqxsSrf:lu:t:TD:NLR:FwWM:j:',
 260             ['help', 'verbose', 'verbose2', 'verbose3', 'quiet',
 261              'exclude', 'single', 'slow', 'random', 'fromfile', 'findleaks',
 262              'use=', 'threshold=', 'trace', 'coverdir=', 'nocoverdir',
 263              'runleaks', 'huntrleaks=', 'memlimit=', 'randseed=',
 264              'multiprocess=', 'slaveargs=', 'forever', 'header'])
 265     except getopt.error, msg:
 266         usage(2, msg)
 267
 268     # Defaults
 269     if random_seed is None:
 270         random_seed = random.randrange(10000000)
 271     if use_resources is None:
 272         use_resources = []
 273     for o, a in opts:
 274         if o in ('-h', '--help'):
 275             usage(0)
 276         elif o in ('-v', '--verbose'):
 277             verbose += 1
 278         elif o in ('-w', '--verbose2'):
 279             verbose2 = True
 280         elif o in ('-W', '--verbose3'):
 281             verbose3 = True
 282         elif o in ('-q', '--quiet'):
 283             quiet = True;
 284             verbose = 0
 285         elif o in ('-x', '--exclude'):
 286             exclude = True
 287         elif o in ('-s', '--single'):
 288             single = True
 289         elif o in ('-S', '--slow'):
 290             print_slow = True
 291         elif o in ('-r', '--randomize'):
 292             randomize = True
 293         elif o == '--randseed':
 294             random_seed = int(a)
 295         elif o in ('-f', '--fromfile'):
 296             fromfile = a
 297         elif o in ('-l', '--findleaks'):
 298             findleaks = True
 299         elif o in ('-L', '--runleaks'):
 300             runleaks = True
 301         elif o in ('-t', '--threshold'):
 302             import gc
 303             gc.set_threshold(int(a))
 304         elif o in ('-T', '--coverage'):
 305             trace = True
 306         elif o in ('-D', '--coverdir'):
 307             coverdir = os.path.join(os.getcwd(), a)
 308         elif o in ('-N', '--nocoverdir'):
 309             coverdir = None
 310         elif o in ('-R', '--huntrleaks'):
 311             huntrleaks = a.split(':')
 312             if len(huntrleaks) not in (2, 3):
 313                 print a, huntrleaks
 314                 usage(2, '-R takes 2 or 3 colon-separated arguments')
 315             if not huntrleaks[0]:
 316                 huntrleaks[0] = 5
 317             else:
 318                 huntrleaks[0] = int(huntrleaks[0])
 319             if not huntrleaks[1]:
 320                 huntrleaks[1] = 4
 321             else:
 322                 huntrleaks[1] = int(huntrleaks[1])
 323             if len(huntrleaks) == 2 or not huntrleaks[2]:
 324                 huntrleaks[2:] = ["reflog.txt"]
 325         elif o in ('-M', '--memlimit'):
 326             test_support.set_memlimit(a)
 327         elif o in ('-u', '--use'):
 328             u = [x.lower() for x in a.split(',')]
 329             for r in u:
 330                 if r == 'all':
 331                     use_resources[:] = RESOURCE_NAMES
 332                     continue
 333                 remove = False
 334                 if r[0] == '-':
 335                     remove = True
 336                     r = r[1:]
 337                 if r not in RESOURCE_NAMES:
 338                     usage(1, 'Invalid -u/--use option: ' + a)
 339                 if remove:
 340                     if r in use_resources:
 341                         use_resources.remove(r)
 342                 elif r not in use_resources:
 343                     use_resources.append(r)
 344         elif o in ('-F', '--forever'):
 345             forever = True
 346         elif o in ('-j', '--multiprocess'):
 347             use_mp = int(a)
 348         elif o == '--header':
 349             header = True
 350         elif o == '--slaveargs':
 351             args, kwargs = json.loads(a)
 352             try:
 353                 result = runtest(*args, **kwargs)
 354             except BaseException, e:
 355                 result = INTERRUPTED, e.__class__.__name__
 356             print   # Force a newline (just in case)
 357             print json.dumps(result)
 358             sys.exit(0)
 359         else:
 360             print >>sys.stderr, ("No handler for option {}.  Please "
 361                 "report this as a bug at http://bugs.python.org.").format(o)
 362             sys.exit(1)
 363     if single and fromfile:
 364         usage(2, "-s and -f don't go together!")
 365     if use_mp and trace:
 366         usage(2, "-T and -j don't go together!")
 367     if use_mp and findleaks:
 368         usage(2, "-l and -j don't go together!")
 369
 370     good = []
 371     bad = []
 372     skipped = []
 373     resource_denieds = []
 374     environment_changed = []
 375     interrupted = False
 376
 377     if findleaks:
 378         try:
 379             import gc
 380         except ImportError:
 381             print 'No GC available, disabling findleaks.'
 382             findleaks = False
 383         else:
 384             # Uncomment the line below to report garbage that is not
 385             # freeable by reference counting alone.  By default only
 386             # garbage that is not collectable by the GC is reported.
 387             #gc.set_debug(gc.DEBUG_SAVEALL)
 388             found_garbage = []
 389
 390     if single:
 391         filename = os.path.join(TEMPDIR, 'pynexttest')
 392         try:
 393             fp = open(filename, 'r')
 394             next_test = fp.read().strip()
 395             tests = [next_test]
 396             fp.close()
 397         except IOError:
 398             pass
 399
 400     if fromfile:
 401         tests = []
 402         fp = open(os.path.join(test_support.SAVEDCWD, fromfile))
 403         for line in fp:
 404             guts = line.split() # assuming no test has whitespace in its name
 405             if guts and not guts[0].startswith('#'):
 406                 tests.extend(guts)
 407         fp.close()
 408
 409     # Strip .py extensions.
 410     removepy(args)
 411     removepy(tests)
 412
 413     stdtests = STDTESTS[:]
 414     nottests = NOTTESTS.copy()
 415     if exclude:
 416         for arg in args:
 417             if arg in stdtests:
 418                 stdtests.remove(arg)
 419             nottests.add(arg)
 420         args = []
 421
 422     # For a partial run, we do not need to clutter the output.
 423     if verbose or header or not (quiet or single or tests or args):
 424         # Print basic platform information
 425         print "==", platform.python_implementation(), \
 426                     " ".join(sys.version.split())
 427         print "==  ", platform.platform(aliased=True), \
 428                       "%s-endian" % sys.byteorder
 429         print "==  ", os.getcwd()
 430         print "Testing with flags:", sys.flags
 431
 432     alltests = findtests(testdir, stdtests, nottests)
 433     selected = tests or args or alltests
 434     if single:
 435         selected = selected[:1]
 436         try:
 437             next_single_test = alltests[alltests.index(selected[0])+1]
 438         except IndexError:
 439             next_single_test = None
 440     if randomize:
 441         random.seed(random_seed)
 442         print "Using random seed", random_seed
 443         random.shuffle(selected)
 444     if trace:
 445         import trace
 446         tracer = trace.Trace(ignoredirs=[sys.prefix, sys.exec_prefix],
 447                              trace=False, count=True)
 448
 449     test_times = []
 450     test_support.use_resources = use_resources
 451     save_modules = sys.modules.keys()
 452
 453     def accumulate_result(test, result):
 454         ok, test_time = result
 455         test_times.append((test_time, test))
 456         if ok == PASSED:
 457             good.append(test)
 458         elif ok == FAILED:
 459             bad.append(test)
 460         elif ok == ENV_CHANGED:
 461             bad.append(test)
 462             environment_changed.append(test)
 463         elif ok == SKIPPED:
 464             skipped.append(test)
 465         elif ok == RESOURCE_DENIED:
 466             skipped.append(test)
 467             resource_denieds.append(test)
 468
 469     if forever:
 470         def test_forever(tests=list(selected)):
 471             while True:
 472                 for test in tests:
 473                     yield test
 474                     if bad:
 475                         return
 476         tests = test_forever()
 477     else:
 478         tests = iter(selected)
 479
 480     if use_mp:
 481         try:
 482             from threading import Thread
 483         except ImportError:
 484             print "Multiprocess option requires thread support"
 485             sys.exit(2)
 486         from Queue import Queue
 487         from subprocess import Popen, PIPE
 488         debug_output_pat = re.compile(r"\[\d+ refs\]$")
 489         output = Queue()
 490         def tests_and_args():
 491             for test in tests:
 492                 args_tuple = (
 493                     (test, verbose, quiet),
 494                     dict(huntrleaks=huntrleaks, use_resources=use_resources)
 495                 )
 496                 yield (test, args_tuple)
 497         pending = tests_and_args()
 498         opt_args = test_support.args_from_interpreter_flags()
 499         base_cmd = [sys.executable] + opt_args + ['-m', 'test.regrtest']
 500         def work():
 501             # A worker thread.
 502             try:
 503                 while True:
 504                     try:
 505                         test, args_tuple = next(pending)
 506                     except StopIteration:
 507                         output.put((None, None, None, None))
 508                         return
 509                     # -E is needed by some tests, e.g. test_import
 510                     popen = Popen(base_cmd + ['--slaveargs', json.dumps(args_tuple)],
 511                                    stdout=PIPE, stderr=PIPE,
 512                                    universal_newlines=True,
 513                                    close_fds=(os.name != 'nt'))
 514                     stdout, stderr = popen.communicate()
 515                     # Strip last refcount output line if it exists, since it
 516                     # comes from the shutdown of the interpreter in the subcommand.
 517                     stderr = debug_output_pat.sub("", stderr)
 518                     stdout, _, result = stdout.strip().rpartition("\n")
 519                     if not result:
 520                         output.put((None, None, None, None))
 521                         return
 522                     result = json.loads(result)
 523                     if not quiet:
 524                         stdout = test+'\n'+stdout
 525                     output.put((test, stdout.rstrip(), stderr.rstrip(), result))
 526             except BaseException:
 527                 output.put((None, None, None, None))
 528                 raise
 529         workers = [Thread(target=work) for i in range(use_mp)]
 530         for worker in workers:
 531             worker.start()
 532         finished = 0
 533         try:
 534             while finished < use_mp:
 535                 test, stdout, stderr, result = output.get()
 536                 if test is None:
 537                     finished += 1
 538                     continue
 539                 if stdout:
 540                     print stdout
 541                 if stderr:
 542                     print >>sys.stderr, stderr
 543                 if result[0] == INTERRUPTED:
 544                     assert result[1] == 'KeyboardInterrupt'
 545                     raise KeyboardInterrupt   # What else?
 546                 accumulate_result(test, result)
 547         except KeyboardInterrupt:
 548             interrupted = True
 549             pending.close()
 550         for worker in workers:
 551             worker.join()
 552     else:
 553         for test in tests:
 554             if not quiet:
 555                 print test
 556                 sys.stdout.flush()
 557             if trace:
 558                 # If we're tracing code coverage, then we don't exit with status
 559                 # if on a false return value from main.
 560                 tracer.runctx('runtest(test, verbose, quiet)',
 561                               globals=globals(), locals=vars())
 562             else:
 563                 try:
 564                     result = runtest(test, verbose, quiet, huntrleaks)
 565                     accumulate_result(test, result)
 566                     if verbose3 and result[0] == FAILED:
 567                         print "Re-running test %r in verbose mode" % test
 568                         runtest(test, True, quiet, huntrleaks)
 569                 except KeyboardInterrupt:
 570                     interrupted = True
 571                     break
 572                 except:
 573                     raise
 574             if findleaks:
 575                 gc.collect()
 576                 if gc.garbage:
 577                     print "Warning: test created", len(gc.garbage),
 578                     print "uncollectable object(s)."
 579                     # move the uncollectable objects somewhere so we don't see
 580                     # them again
 581                     found_garbage.extend(gc.garbage)
 582                     del gc.garbage[:]
 583             # Unload the newly imported modules (best effort finalization)
 584             for module in sys.modules.keys():
 585                 if module not in save_modules and module.startswith("test."):
 586                     test_support.unload(module)
 587
 588     if interrupted:
 589         # print a newline after ^C
 590         print
 591         print "Test suite interrupted by signal SIGINT."
 592         omitted = set(selected) - set(good) - set(bad) - set(skipped)
 593         print count(len(omitted), "test"), "omitted:"
 594         printlist(omitted)
 595     if good and not quiet:
 596         if not bad and not skipped and not interrupted and len(good) > 1:
 597             print "All",
 598         print count(len(good), "test"), "OK."
 599     if print_slow:
 600         test_times.sort(reverse=True)
 601         print "10 slowest tests:"
 602         for time, test in test_times[:10]:
 603             print "%s: %.1fs" % (test, time)
 604     if bad:
 605         bad = set(bad) - set(environment_changed)
 606         if bad:
 607             print count(len(bad), "test"), "failed:"
 608             printlist(bad)
 609         if environment_changed:
 610             print "{} altered the execution environment:".format(
 611                 count(len(environment_changed), "test"))
 612             printlist(environment_changed)
 613     if skipped and not quiet:
 614         print count(len(skipped), "test"), "skipped:"
 615         printlist(skipped)
 616
 617         e = _ExpectedSkips()
 618         plat = sys.platform
 619         if e.isvalid():
 620             surprise = set(skipped) - e.getexpected() - set(resource_denieds)
 621             if surprise:
 622                 print count(len(surprise), "skip"), \
 623                       "unexpected on", plat + ":"
 624                 printlist(surprise)
 625             else:
 626                 print "Those skips are all expected on", plat + "."
 627         else:
 628             print "Ask someone to teach regrtest.py about which tests are"
 629             print "expected to get skipped on", plat + "."
 630
 631     if verbose2 and bad:
 632         print "Re-running failed tests in verbose mode"
 633         for test in bad:
 634             print "Re-running test %r in verbose mode" % test
 635             sys.stdout.flush()
 636             try:
 637                 test_support.verbose = True
 638                 ok = runtest(test, True, quiet, huntrleaks)
 639             except KeyboardInterrupt:
 640                 # print a newline separate from the ^C
 641                 print
 642                 break
 643             except:
 644                 raise
 645
 646     if single:
 647         if next_single_test:
 648             with open(filename, 'w') as fp:
 649                 fp.write(next_single_test + '\n')
 650         else:
 651             os.unlink(filename)
 652
 653     if trace:
 654         r = tracer.results()
 655         r.write_results(show_missing=True, summary=True, coverdir=coverdir)
 656
 657     if runleaks:
 658         os.system("leaks %d" % os.getpid())
 659
 660     sys.exit(len(bad) > 0 or interrupted)
 661
 662
 663 STDTESTS = [
 664     'test_grammar',
 665     'test_opcodes',
 666     'test_dict',
 667     'test_builtin',
 668     'test_exceptions',
 669     'test_types',
 670     'test_unittest',
 671     'test_doctest',
 672     'test_doctest2',
 673 ]
 674
 675 NOTTESTS = {
 676     'test_support',
 677     'test_future1',
 678     'test_future2',
 679 }
 680
 681 def findtests(testdir=None, stdtests=STDTESTS, nottests=NOTTESTS):
 682     """Return a list of all applicable test modules."""
 683     testdir = findtestdir(testdir)
 684     names = os.listdir(testdir)
 685     tests = []
 686     others = set(stdtests) | nottests
 687     for name in names:
 688         modname, ext = os.path.splitext(name)
 689         if modname[:5] == "test_" and ext == ".py" and modname not in others:
 690             tests.append(modname)
 691     return stdtests + sorted(tests)
 692
 693 def runtest(test, verbose, quiet,
 694             huntrleaks=False, use_resources=None):
 695     """Run a single test.
 696
 697     test -- the name of the test
 698     verbose -- if true, print more messages
 699     quiet -- if true, don't print 'skipped' messages (probably redundant)
 700     test_times -- a list of (time, test_name) pairs
 701     huntrleaks -- run multiple times to test for leaks; requires a debug
 702                   build; a triple corresponding to -R's three arguments
 703     Returns one of the test result constants:
 704         INTERRUPTED      KeyboardInterrupt when run under -j
 705         RESOURCE_DENIED  test skipped because resource denied
 706         SKIPPED          test skipped for some other reason
 707         ENV_CHANGED      test failed because it changed the execution environment
 708         FAILED           test failed
 709         PASSED           test passed
 710     """
 711
 712     test_support.verbose = verbose  # Tell tests to be moderately quiet
 713     if use_resources is not None:
 714         test_support.use_resources = use_resources
 715     try:
 716         return runtest_inner(test, verbose, quiet, huntrleaks)
 717     finally:
 718         cleanup_test_droppings(test, verbose)
 719
 720
 721 # Unit tests are supposed to leave the execution environment unchanged
 722 # once they complete.  But sometimes tests have bugs, especially when
 723 # tests fail, and the changes to environment go on to mess up other
 724 # tests.  This can cause issues with buildbot stability, since tests
 725 # are run in random order and so problems may appear to come and go.
 726 # There are a few things we can save and restore to mitigate this, and
 727 # the following context manager handles this task.
 728
 729 class saved_test_environment:
 730     """Save bits of the test environment and restore them at block exit.
 731
 732         with saved_test_environment(testname, verbose, quiet):
 733             #stuff
 734
 735     Unless quiet is True, a warning is printed to stderr if any of
 736     the saved items was changed by the test.  The attribute 'changed'
 737     is initially False, but is set to True if a change is detected.
 738
 739     If verbose is more than 1, the before and after state of changed
 740     items is also printed.
 741     """
 742
 743     changed = False
 744
 745     def __init__(self, testname, verbose=0, quiet=False):
 746         self.testname = testname
 747         self.verbose = verbose
 748         self.quiet = quiet
 749
 750     # To add things to save and restore, add a name XXX to the resources list
 751     # and add corresponding get_XXX/restore_XXX functions.  get_XXX should
 752     # return the value to be saved and compared against a second call to the
 753     # get function when test execution completes.  restore_XXX should accept
 754     # the saved value and restore the resource using it.  It will be called if
 755     # and only if a change in the value is detected.
 756     #
 757     # Note: XXX will have any '.' replaced with '_' characters when determining
 758     # the corresponding method names.
 759
 760     resources = ('sys.argv', 'cwd', 'sys.stdin', 'sys.stdout', 'sys.stderr',
 761                  'os.environ', 'sys.path', 'asyncore.socket_map')
 762
 763     def get_sys_argv(self):
 764         return id(sys.argv), sys.argv, sys.argv[:]
 765     def restore_sys_argv(self, saved_argv):
 766         sys.argv = saved_argv[1]
 767         sys.argv[:] = saved_argv[2]
 768
 769     def get_cwd(self):
 770         return os.getcwd()
 771     def restore_cwd(self, saved_cwd):
 772         os.chdir(saved_cwd)
 773
 774     def get_sys_stdout(self):
 775         return sys.stdout
 776     def restore_sys_stdout(self, saved_stdout):
 777         sys.stdout = saved_stdout
 778
 779     def get_sys_stderr(self):
 780         return sys.stderr
 781     def restore_sys_stderr(self, saved_stderr):
 782         sys.stderr = saved_stderr
 783
 784     def get_sys_stdin(self):
 785         return sys.stdin
 786     def restore_sys_stdin(self, saved_stdin):
 787         sys.stdin = saved_stdin
 788
 789     def get_os_environ(self):
 790         return id(os.environ), os.environ, dict(os.environ)
 791     def restore_os_environ(self, saved_environ):
 792         os.environ = saved_environ[1]
 793         os.environ.clear()
 794         os.environ.update(saved_environ[2])
 795
 796     def get_sys_path(self):
 797         return id(sys.path), sys.path, sys.path[:]
 798     def restore_sys_path(self, saved_path):
 799         sys.path = saved_path[1]
 800         sys.path[:] = saved_path[2]
 801
 802     def get_asyncore_socket_map(self):
 803         asyncore = sys.modules.get('asyncore')
 804         # XXX Making a copy keeps objects alive until __exit__ gets called.
 805         return asyncore and asyncore.socket_map.copy() or {}
 806     def restore_asyncore_socket_map(self, saved_map):
 807         asyncore = sys.modules.get('asyncore')
 808         if asyncore is not None:
 809             asyncore.close_all(ignore_all=True)
 810             asyncore.socket_map.update(saved_map)
 811
 812     def resource_info(self):
 813         for name in self.resources:
 814             method_suffix = name.replace('.', '_')
 815             get_name = 'get_' + method_suffix
 816             restore_name = 'restore_' + method_suffix
 817             yield name, getattr(self, get_name), getattr(self, restore_name)
 818
 819     def __enter__(self):
 820         self.saved_values = dict((name, get()) for name, get, restore
 821                                                    in self.resource_info())
 822         return self
 823
 824     def __exit__(self, exc_type, exc_val, exc_tb):
 825         saved_values = self.saved_values
 826         del self.saved_values
 827         for name, get, restore in self.resource_info():
 828             current = get()
 829             original = saved_values.pop(name)
 830             # Check for changes to the resource's value
 831             if current != original:
 832                 self.changed = True
 833                 restore(original)
 834                 if not self.quiet:
 835                     print >>sys.stderr, (
 836                           "Warning -- {} was modified by {}".format(
 837                                                  name, self.testname))
 838                     if self.verbose > 1:
 839                         print >>sys.stderr, (
 840                               "  Before: {}\n  After:  {} ".format(
 841                                                   original, current))
 842             # XXX (ncoghlan): for most resources (e.g. sys.path) identity
 843             # matters at least as much as value. For others (e.g. cwd),
 844             # identity is irrelevant. Should we add a mechanism to check
 845             # for substitution in the cases where it matters?
 846         return False
 847
 848
 849 def runtest_inner(test, verbose, quiet, huntrleaks=False):
 850     test_support.unload(test)
 851     if verbose:
 852         capture_stdout = None
 853     else:
 854         capture_stdout = StringIO.StringIO()
 855
 856     test_time = 0.0
 857     refleak = False  # True if the test leaked references.
 858     try:
 859         save_stdout = sys.stdout
 860         try:
 861             if capture_stdout:
 862                 sys.stdout = capture_stdout
 863             if test.startswith('test.'):
 864                 abstest = test
 865             else:
 866                 # Always import it from the test package
 867                 abstest = 'test.' + test
 868             with saved_test_environment(test, verbose, quiet) as environment:
 869                 start_time = time.time()
 870                 the_package = __import__(abstest, globals(), locals(), [])
 871                 the_module = getattr(the_package, test)
 872                 # Old tests run to completion simply as a side-effect of
 873                 # being imported.  For tests based on unittest or doctest,
 874                 # explicitly invoke their test_main() function (if it exists).
 875                 indirect_test = getattr(the_module, "test_main", None)
 876                 if indirect_test is not None:
 877                     indirect_test()
 878                 if huntrleaks:
 879                     refleak = dash_R(the_module, test, indirect_test,
 880                         huntrleaks)
 881                 test_time = time.time() - start_time
 882         finally:
 883             sys.stdout = save_stdout
 884     except test_support.ResourceDenied, msg:
 885         if not quiet:
 886             print test, "skipped --", msg
 887             sys.stdout.flush()
 888         return RESOURCE_DENIED, test_time
 889     except unittest.SkipTest, msg:
 890         if not quiet:
 891             print test, "skipped --", msg
 892             sys.stdout.flush()
 893         return SKIPPED, test_time
 894     except KeyboardInterrupt:
 895         raise
 896     except test_support.TestFailed, msg:
 897         print >>sys.stderr, "test", test, "failed --", msg
 898         sys.stderr.flush()
 899         return FAILED, test_time
 900     except:
 901         type, value = sys.exc_info()[:2]
 902         print >>sys.stderr, "test", test, "crashed --", str(type) + ":", value
 903         sys.stderr.flush()
 904         if verbose:
 905             traceback.print_exc(file=sys.stderr)
 906             sys.stderr.flush()
 907         return FAILED, test_time
 908     else:
 909         if refleak:
 910             return FAILED, test_time
 911         if environment.changed:
 912             return ENV_CHANGED, test_time
 913         # Except in verbose mode, tests should not print anything
 914         if verbose or huntrleaks:
 915             return PASSED, test_time
 916         output = capture_stdout.getvalue()
 917         if not output:
 918             return PASSED, test_time
 919         print "test", test, "produced unexpected output:"
 920         print "*" * 70
 921         print output
 922         print "*" * 70
 923         sys.stdout.flush()
 924         return FAILED, test_time
 925
 926 def cleanup_test_droppings(testname, verbose):
 927     import shutil
 928     import stat
 929     import gc
 930
 931     # First kill any dangling references to open files etc.
 932     gc.collect()
 933
 934     # Try to clean up junk commonly left behind.  While tests shouldn't leave
 935     # any files or directories behind, when a test fails that can be tedious
 936     # for it to arrange.  The consequences can be especially nasty on Windows,
 937     # since if a test leaves a file open, it cannot be deleted by name (while
 938     # there's nothing we can do about that here either, we can display the
 939     # name of the offending test, which is a real help).
 940     for name in (test_support.TESTFN,
 941                  "db_home",
 942                 ):
 943         if not os.path.exists(name):
 944             continue
 945
 946         if os.path.isdir(name):
 947             kind, nuker = "directory", shutil.rmtree
 948         elif os.path.isfile(name):
 949             kind, nuker = "file", os.unlink
 950         else:
 951             raise SystemError("os.path says %r exists but is neither "
 952                               "directory nor file" % name)
 953
 954         if verbose:
 955             print "%r left behind %s %r" % (testname, kind, name)
 956         try:
 957             # if we have chmod, fix possible permissions problems
 958             # that might prevent cleanup
 959             if (hasattr(os, 'chmod')):
 960                 os.chmod(name, stat.S_IRWXU | stat.S_IRWXG | stat.S_IRWXO)
 961             nuker(name)
 962         except Exception, msg:
 963             print >> sys.stderr, ("%r left behind %s %r and it couldn't be "
 964                 "removed: %s" % (testname, kind, name, msg))
 965
 966 def dash_R(the_module, test, indirect_test, huntrleaks):
 967     """Run a test multiple times, looking for reference leaks.
 968
 969     Returns:
 970         False if the test didn't leak references; True if we detected refleaks.
 971     """
 972     # This code is hackish and inelegant, but it seems to do the job.
 973     import copy_reg, _abcoll, _pyio
 974
 975     if not hasattr(sys, 'gettotalrefcount'):
 976         raise Exception("Tracking reference leaks requires a debug build "
 977                         "of Python")
 978
 979     # Save current values for dash_R_cleanup() to restore.
 980     fs = warnings.filters[:]
 981     ps = copy_reg.dispatch_table.copy()
 982     pic = sys.path_importer_cache.copy()
 983     try:
 984         import zipimport
 985     except ImportError:
 986         zdc = None # Run unmodified on platforms without zipimport support
 987     else:
 988         zdc = zipimport._zip_directory_cache.copy()
 989     abcs = {}
 990     modules = _abcoll, _pyio
 991     for abc in [getattr(mod, a) for mod in modules for a in mod.__all__]:
 992         # XXX isinstance(abc, ABCMeta) leads to infinite recursion
 993         if not hasattr(abc, '_abc_registry'):
 994             continue
 995         for obj in abc.__subclasses__() + [abc]:
 996             abcs[obj] = obj._abc_registry.copy()
 997
 998     if indirect_test:
 999         def run_the_test():
1000             indirect_test()
1001     else:
1002         def run_the_test():
1003             imp.reload(the_module)
1004
1005     deltas = []
1006     nwarmup, ntracked, fname = huntrleaks
1007     fname = os.path.join(test_support.SAVEDCWD, fname)
1008     repcount = nwarmup + ntracked
1009     print >> sys.stderr, "beginning", repcount, "repetitions"
1010     print >> sys.stderr, ("1234567890"*(repcount//10 + 1))[:repcount]
1011     dash_R_cleanup(fs, ps, pic, zdc, abcs)
1012     for i in range(repcount):
1013         rc_before = sys.gettotalrefcount()
1014         run_the_test()
1015         sys.stderr.write('.')
1016         dash_R_cleanup(fs, ps, pic, zdc, abcs)
1017         rc_after = sys.gettotalrefcount()
1018         if i >= nwarmup:
1019             deltas.append(rc_after - rc_before)
1020     print >> sys.stderr
1021     if any(deltas):
1022         msg = '%s leaked %s references, sum=%s' % (test, deltas, sum(deltas))
1023         print >> sys.stderr, msg
1024         with open(fname, "a") as refrep:
1025             print >> refrep, msg
1026             refrep.flush()
1027         return True
1028     return False
1029
1030 def dash_R_cleanup(fs, ps, pic, zdc, abcs):
1031     import gc, copy_reg
1032     import _strptime, linecache
1033     dircache = test_support.import_module('dircache', deprecated=True)
1034     import urlparse, urllib, urllib2, mimetypes, doctest
1035     import struct, filecmp
1036     from distutils.dir_util import _path_created
1037
1038     # Clear the warnings registry, so they can be displayed again
1039     for mod in sys.modules.values():
1040         if hasattr(mod, '__warningregistry__'):
1041             del mod.__warningregistry__
1042
1043     # Restore some original values.
1044     warnings.filters[:] = fs
1045     copy_reg.dispatch_table.clear()
1046     copy_reg.dispatch_table.update(ps)
1047     sys.path_importer_cache.clear()
1048     sys.path_importer_cache.update(pic)
1049     try:
1050         import zipimport
1051     except ImportError:
1052         pass # Run unmodified on platforms without zipimport support
1053     else:
1054         zipimport._zip_directory_cache.clear()
1055         zipimport._zip_directory_cache.update(zdc)
1056
1057     # clear type cache
1058     sys._clear_type_cache()
1059
1060     # Clear ABC registries, restoring previously saved ABC registries.
1061     for abc, registry in abcs.items():
1062         abc._abc_registry = registry.copy()
1063         abc._abc_cache.clear()
1064         abc._abc_negative_cache.clear()
1065
1066     # Clear assorted module caches.
1067     _path_created.clear()
1068     re.purge()
1069     _strptime._regex_cache.clear()
1070     urlparse.clear_cache()
1071     urllib.urlcleanup()
1072     urllib2.install_opener(None)
1073     dircache.reset()
1074     linecache.clearcache()
1075     mimetypes._default_mime_types()
1076     filecmp._cache.clear()
1077     struct._clearcache()
1078     doctest.master = None
1079
1080     # Collect cyclic trash.
1081     gc.collect()
1082
1083 def findtestdir(path=None):
1084     return path or os.path.dirname(__file__) or os.curdir
1085
1086 def removepy(names):
1087     if not names:
1088         return
1089     for idx, name in enumerate(names):
1090         basename, ext = os.path.splitext(name)
1091         if ext == '.py':
1092             names[idx] = basename
1093
1094 def count(n, word):
1095     if n == 1:
1096         return "%d %s" % (n, word)
1097     else:
1098         return "%d %ss" % (n, word)
1099
1100 def printlist(x, width=70, indent=4):
1101     """Print the elements of iterable x to stdout.
1102
1103     Optional arg width (default 70) is the maximum line length.
1104     Optional arg indent (default 4) is the number of blanks with which to
1105     begin each line.
1106     """
1107
1108     from textwrap import fill
1109     blanks = ' ' * indent
1110     # Print the sorted list: 'x' may be a '--random' list or a set()
1111     print fill(' '.join(str(elt) for elt in sorted(x)), width,
1112                initial_indent=blanks, subsequent_indent=blanks)
1113
1114 # Map sys.platform to a string containing the basenames of tests
1115 # expected to be skipped on that platform.
1116 #
1117 # Special cases:
1118 #     test_pep277
1119 #         The _ExpectedSkips constructor adds this to the set of expected
1120 #         skips if not os.path.supports_unicode_filenames.
1121 #     test_timeout
1122 #         Controlled by test_timeout.skip_expected.  Requires the network
1123 #         resource and a socket module.
1124 #
1125 # Tests that are expected to be skipped everywhere except on one platform
1126 # are also handled separately.
1127
1128 _expectations = {
1129     'win32':
1130         """
1131         test__locale
1132         test_bsddb185
1133         test_bsddb3
1134         test_commands
1135         test_crypt
1136         test_curses
1137         test_dbm
1138         test_dl
1139         test_fcntl
1140         test_fork1
1141         test_epoll
1142         test_gdbm
1143         test_grp
1144         test_ioctl
1145         test_largefile
1146         test_kqueue
1147         test_mhlib
1148         test_openpty
1149         test_ossaudiodev
1150         test_pipes
1151         test_poll
1152         test_posix
1153         test_pty
1154         test_pwd
1155         test_resource
1156         test_signal
1157         test_threadsignals
1158         test_timing
1159         test_wait3
1160         test_wait4
1161         """,
1162     'linux2':
1163         """
1164         test_bsddb185
1165         test_curses
1166         test_dl
1167         test_largefile
1168         test_kqueue
1169         test_ossaudiodev
1170         """,
1171     'unixware7':
1172         """
1173         test_bsddb
1174         test_bsddb185
1175         test_dl
1176         test_epoll
1177         test_largefile
1178         test_kqueue
1179         test_minidom
1180         test_openpty
1181         test_pyexpat
1182         test_sax
1183         test_sundry
1184         """,
1185     'openunix8':
1186         """
1187         test_bsddb
1188         test_bsddb185
1189         test_dl
1190         test_epoll
1191         test_largefile
1192         test_kqueue
1193         test_minidom
1194         test_openpty
1195         test_pyexpat
1196         test_sax
1197         test_sundry
1198         """,
1199     'sco_sv3':
1200         """
1201         test_asynchat
1202         test_bsddb
1203         test_bsddb185
1204         test_dl
1205         test_fork1
1206         test_epoll
1207         test_gettext
1208         test_largefile
1209         test_locale
1210         test_kqueue
1211         test_minidom
1212         test_openpty
1213         test_pyexpat
1214         test_queue
1215         test_sax
1216         test_sundry
1217         test_thread
1218         test_threaded_import
1219         test_threadedtempfile
1220         test_threading
1221         """,
1222     'riscos':
1223         """
1224         test_asynchat
1225         test_atexit
1226         test_bsddb
1227         test_bsddb185
1228         test_bsddb3
1229         test_commands
1230         test_crypt
1231         test_dbm
1232         test_dl
1233         test_fcntl
1234         test_fork1
1235         test_epoll
1236         test_gdbm
1237         test_grp
1238         test_largefile
1239         test_locale
1240         test_kqueue
1241         test_mmap
1242         test_openpty
1243         test_poll
1244         test_popen2
1245         test_pty
1246         test_pwd
1247         test_strop
1248         test_sundry
1249         test_thread
1250         test_threaded_import
1251         test_threadedtempfile
1252         test_threading
1253         test_timing
1254         """,
1255     'darwin':
1256         """
1257         test__locale
1258         test_bsddb
1259         test_bsddb3
1260         test_curses
1261         test_epoll
1262         test_gdb
1263         test_gdbm
1264         test_largefile
1265         test_locale
1266         test_kqueue
1267         test_minidom
1268         test_ossaudiodev
1269         test_poll
1270         """,
1271     'sunos5':
1272         """
1273         test_bsddb
1274         test_bsddb185
1275         test_curses
1276         test_dbm
1277         test_epoll
1278         test_kqueue
1279         test_gdbm
1280         test_gzip
1281         test_openpty
1282         test_zipfile
1283         test_zlib
1284         """,
1285     'hp-ux11':
1286         """
1287         test_bsddb
1288         test_bsddb185
1289         test_curses
1290         test_dl
1291         test_epoll
1292         test_gdbm
1293         test_gzip
1294         test_largefile
1295         test_locale
1296         test_kqueue
1297         test_minidom
1298         test_openpty
1299         test_pyexpat
1300         test_sax
1301         test_zipfile
1302         test_zlib
1303         """,
1304     'atheos':
1305         """
1306         test_bsddb185
1307         test_curses
1308         test_dl
1309         test_gdbm
1310         test_epoll
1311         test_largefile
1312         test_locale
1313         test_kqueue
1314         test_mhlib
1315         test_mmap
1316         test_poll
1317         test_popen2
1318         test_resource
1319         """,
1320     'cygwin':
1321         """
1322         test_bsddb185
1323         test_bsddb3
1324         test_curses
1325         test_dbm
1326         test_epoll
1327         test_ioctl
1328         test_kqueue
1329         test_largefile
1330         test_locale
1331         test_ossaudiodev
1332         test_socketserver
1333         """,
1334     'os2emx':
1335         """
1336         test_audioop
1337         test_bsddb185
1338         test_bsddb3
1339         test_commands
1340         test_curses
1341         test_dl
1342         test_epoll
1343         test_kqueue
1344         test_largefile
1345         test_mhlib
1346         test_mmap
1347         test_openpty
1348         test_ossaudiodev
1349         test_pty
1350         test_resource
1351         test_signal
1352         """,
1353     'freebsd4':
1354         """
1355         test_bsddb
1356         test_bsddb3
1357         test_epoll
1358         test_gdbm
1359         test_locale
1360         test_ossaudiodev
1361         test_pep277
1362         test_pty
1363         test_socketserver
1364         test_tcl
1365         test_tk
1366         test_ttk_guionly
1367         test_ttk_textonly
1368         test_timeout
1369         test_urllibnet
1370         test_multiprocessing
1371         """,
1372     'aix5':
1373         """
1374         test_bsddb
1375         test_bsddb185
1376         test_bsddb3
1377         test_bz2
1378         test_dl
1379         test_epoll
1380         test_gdbm
1381         test_gzip
1382         test_kqueue
1383         test_ossaudiodev
1384         test_tcl
1385         test_tk
1386         test_ttk_guionly
1387         test_ttk_textonly
1388         test_zipimport
1389         test_zlib
1390         """,
1391     'openbsd3':
1392         """
1393         test_ascii_formatd
1394         test_bsddb
1395         test_bsddb3
1396         test_ctypes
1397         test_dl
1398         test_epoll
1399         test_gdbm
1400         test_locale
1401         test_normalization
1402         test_ossaudiodev
1403         test_pep277
1404         test_tcl
1405         test_tk
1406         test_ttk_guionly
1407         test_ttk_textonly
1408         test_multiprocessing
1409         """,
1410     'netbsd3':
1411         """
1412         test_ascii_formatd
1413         test_bsddb
1414         test_bsddb185
1415         test_bsddb3
1416         test_ctypes
1417         test_curses
1418         test_dl
1419         test_epoll
1420         test_gdbm
1421         test_locale
1422         test_ossaudiodev
1423         test_pep277
1424         test_tcl
1425         test_tk
1426         test_ttk_guionly
1427         test_ttk_textonly
1428         test_multiprocessing
1429         """,
1430 }
1431 _expectations['freebsd5'] = _expectations['freebsd4']
1432 _expectations['freebsd6'] = _expectations['freebsd4']
1433 _expectations['freebsd7'] = _expectations['freebsd4']
1434 _expectations['freebsd8'] = _expectations['freebsd4']
1435
1436 class _ExpectedSkips:
1437     def __init__(self):
1438         import os.path
1439         from test import test_timeout
1440
1441         self.valid = False
1442         if sys.platform in _expectations:
1443             s = _expectations[sys.platform]
1444             self.expected = set(s.split())
1445
1446             # expected to be skipped on every platform, even Linux
1447             self.expected.add('test_linuxaudiodev')
1448
1449             if not os.path.supports_unicode_filenames:
1450                 self.expected.add('test_pep277')
1451
1452             if test_timeout.skip_expected:
1453                 self.expected.add('test_timeout')
1454
1455             if sys.maxint == 9223372036854775807L:
1456                 self.expected.add('test_imageop')
1457
1458             if sys.platform != "darwin":
1459                 MAC_ONLY = ["test_macos", "test_macostools", "test_aepack",
1460                             "test_plistlib", "test_scriptpackages",
1461                             "test_applesingle"]
1462                 for skip in MAC_ONLY:
1463                     self.expected.add(skip)
1464             elif len(u'\0'.encode('unicode-internal')) == 4:
1465                 self.expected.add("test_macostools")
1466
1467
1468             if sys.platform != "win32":
1469                 # test_sqlite is only reliable on Windows where the library
1470                 # is distributed with Python
1471                 WIN_ONLY = ["test_unicode_file", "test_winreg",
1472                             "test_winsound", "test_startfile",
1473                             "test_sqlite", "test_msilib"]
1474                 for skip in WIN_ONLY:
1475                     self.expected.add(skip)
1476
1477             if sys.platform != 'irix':
1478                 IRIX_ONLY = ["test_imageop", "test_al", "test_cd", "test_cl",
1479                              "test_gl", "test_imgfile"]
1480                 for skip in IRIX_ONLY:
1481                     self.expected.add(skip)
1482
1483             if sys.platform != 'sunos5':
1484                 self.expected.add('test_sunaudiodev')
1485                 self.expected.add('test_nis')
1486
1487             if not sys.py3kwarning:
1488                 self.expected.add('test_py3kwarn')
1489
1490             self.valid = True
1491
1492     def isvalid(self):
1493         "Return true iff _ExpectedSkips knows about the current platform."
1494         return self.valid
1495
1496     def getexpected(self):
1497         """Return set of test names we expect to skip on current platform.
1498
1499         self.isvalid() must be true.
1500         """
1501
1502         assert self.isvalid()
1503         return self.expected
1504
1505 if __name__ == '__main__':
1506     # findtestdir() gets the dirname out of __file__, so we have to make it
1507     # absolute before changing the working directory.
1508     # For example __file__ may be relative when running trace or profile.
1509     # See issue #9323.
1510     __file__ = os.path.abspath(__file__)
1511
1512     # sanity check
1513     assert __file__ == os.path.abspath(sys.argv[0])
1514
1515     # When tests are run from the Python build directory, it is best practice
1516     # to keep the test files in a subfolder.  It eases the cleanup of leftover
1517     # files using command "make distclean".
1518     if sysconfig.is_python_build():
1519         TEMPDIR = os.path.join(sysconfig.get_config_var('srcdir'), 'build')
1520         TEMPDIR = os.path.abspath(TEMPDIR)
1521         if not os.path.exists(TEMPDIR):
1522             os.mkdir(TEMPDIR)
1523
1524     # Define a writable temp dir that will be used as cwd while running
1525     # the tests. The name of the dir includes the pid to allow parallel
1526     # testing (see the -j option).
1527     TESTCWD = 'test_python_{}'.format(os.getpid())
1528
1529     TESTCWD = os.path.join(TEMPDIR, TESTCWD)
1530
1531     # Run the tests in a context manager that temporary changes the CWD to a
1532     # temporary and writable directory. If it's not possible to create or
1533     # change the CWD, the original CWD will be used. The original CWD is
1534     # available from test_support.SAVEDCWD.
1535     with test_support.temp_cwd(TESTCWD, quiet=True):
1536         main()