+++ /dev/null
-<HTML>\r
-\r
-<HEAD>\r
-<TITLE>Metaclasses in Python 1.5</TITLE>\r
-</HEAD>\r
-\r
-<BODY BGCOLOR="FFFFFF">\r
-\r
-<H1>Metaclasses in Python 1.5</H1>\r
-<H2>(A.k.a. The Killer Joke :-)</H2>\r
-\r
-<HR>\r
-\r
-(<i>Postscript:</i> reading this essay is probably not the best way to\r
-understand the metaclass hook described here. See a <A\r
-HREF="meta-vladimir.txt">message posted by Vladimir Marangozov</A>\r
-which may give a gentler introduction to the matter. You may also\r
-want to search Deja News for messages with "metaclass" in the subject\r
-posted to comp.lang.python in July and August 1998.)\r
-\r
-<HR>\r
-\r
-<P>In previous Python releases (and still in 1.5), there is something\r
-called the ``Don Beaudry hook'', after its inventor and champion.\r
-This allows C extensions to provide alternate class behavior, thereby\r
-allowing the Python class syntax to be used to define other class-like\r
-entities. Don Beaudry has used this in his infamous <A\r
-HREF="http://maigret.cog.brown.edu/pyutil/">MESS</A> package; Jim\r
-Fulton has used it in his <A\r
-HREF="http://www.digicool.com/releases/ExtensionClass/">Extension\r
-Classes</A> package. (It has also been referred to as the ``Don\r
-Beaudry <i>hack</i>,'' but that's a misnomer. There's nothing hackish\r
-about it -- in fact, it is rather elegant and deep, even though\r
-there's something dark to it.)\r
-\r
-<P>(On first reading, you may want to skip directly to the examples in\r
-the section "Writing Metaclasses in Python" below, unless you want\r
-your head to explode.)\r
-\r
-<P>\r
-\r
-<HR>\r
-\r
-<P>Documentation of the Don Beaudry hook has purposefully been kept\r
-minimal, since it is a feature of incredible power, and is easily\r
-abused. Basically, it checks whether the <b>type of the base\r
-class</b> is callable, and if so, it is called to create the new\r
-class.\r
-\r
-<P>Note the two indirection levels. Take a simple example:\r
-\r
-<PRE>\r
-class B:\r
- pass\r
-\r
-class C(B):\r
- pass\r
-</PRE>\r
-\r
-Take a look at the second class definition, and try to fathom ``the\r
-type of the base class is callable.''\r
-\r
-<P>(Types are not classes, by the way. See questions 4.2, 4.19 and in\r
-particular 6.22 in the <A\r
-HREF="http://www.python.org/cgi-bin/faqw.py" >Python FAQ</A>\r
-for more on this topic.)\r
-\r
-<P>\r
-\r
-<UL>\r
-\r
-<LI>The <b>base class</b> is B; this one's easy.<P>\r
-\r
-<LI>Since B is a class, its type is ``class''; so the <b>type of the\r
-base class</b> is the type ``class''. This is also known as\r
-types.ClassType, assuming the standard module <code>types</code> has\r
-been imported.<P>\r
-\r
-<LI>Now is the type ``class'' <b>callable</b>? No, because types (in\r
-core Python) are never callable. Classes are callable (calling a\r
-class creates a new instance) but types aren't.<P>\r
-\r
-</UL>\r
-\r
-<P>So our conclusion is that in our example, the type of the base\r
-class (of C) is not callable. So the Don Beaudry hook does not apply,\r
-and the default class creation mechanism is used (which is also used\r
-when there is no base class). In fact, the Don Beaudry hook never\r
-applies when using only core Python, since the type of a core object\r
-is never callable.\r
-\r
-<P>So what do Don and Jim do in order to use Don's hook? Write an\r
-extension that defines at least two new Python object types. The\r
-first would be the type for ``class-like'' objects usable as a base\r
-class, to trigger Don's hook. This type must be made callable.\r
-That's why we need a second type. Whether an object is callable\r
-depends on its type. So whether a type object is callable depends on\r
-<i>its</i> type, which is a <i>meta-type</i>. (In core Python there\r
-is only one meta-type, the type ``type'' (types.TypeType), which is\r
-the type of all type objects, even itself.) A new meta-type must\r
-be defined that makes the type of the class-like objects callable.\r
-(Normally, a third type would also be needed, the new ``instance''\r
-type, but this is not an absolute requirement -- the new class type\r
-could return an object of some existing type when invoked to create an\r
-instance.)\r
-\r
-<P>Still confused? Here's a simple device due to Don himself to\r
-explain metaclasses. Take a simple class definition; assume B is a\r
-special class that triggers Don's hook:\r
-\r
-<PRE>\r
-class C(B):\r
- a = 1\r
- b = 2\r
-</PRE>\r
-\r
-This can be though of as equivalent to:\r
-\r
-<PRE>\r
-C = type(B)('C', (B,), {'a': 1, 'b': 2})\r
-</PRE>\r
-\r
-If that's too dense for you, here's the same thing written out using\r
-temporary variables:\r
-\r
-<PRE>\r
-creator = type(B) # The type of the base class\r
-name = 'C' # The name of the new class\r
-bases = (B,) # A tuple containing the base class(es)\r
-namespace = {'a': 1, 'b': 2} # The namespace of the class statement\r
-C = creator(name, bases, namespace)\r
-</PRE>\r
-\r
-This is analogous to what happens without the Don Beaudry hook, except\r
-that in that case the creator function is set to the default class\r
-creator.\r
-\r
-<P>In either case, the creator is called with three arguments. The\r
-first one, <i>name</i>, is the name of the new class (as given at the\r
-top of the class statement). The <i>bases</i> argument is a tuple of\r
-base classes (a singleton tuple if there's only one base class, like\r
-the example). Finally, <i>namespace</i> is a dictionary containing\r
-the local variables collected during execution of the class statement.\r
-\r
-<P>Note that the contents of the namespace dictionary is simply\r
-whatever names were defined in the class statement. A little-known\r
-fact is that when Python executes a class statement, it enters a new\r
-local namespace, and all assignments and function definitions take\r
-place in this namespace. Thus, after executing the following class\r
-statement:\r
-\r
-<PRE>\r
-class C:\r
- a = 1\r
- def f(s): pass\r
-</PRE>\r
-\r
-the class namespace's contents would be {'a': 1, 'f': <function f\r
-...>}.\r
-\r
-<P>But enough already about writing Python metaclasses in C; read the\r
-documentation of <A\r
-HREF="http://maigret.cog.brown.edu/pyutil/">MESS</A> or <A\r
-HREF="http://www.digicool.com/papers/ExtensionClass.html" >Extension\r
-Classes</A> for more information.\r
-\r
-<P>\r
-\r
-<HR>\r
-\r
-<H2>Writing Metaclasses in Python</H2>\r
-\r
-<P>In Python 1.5, the requirement to write a C extension in order to\r
-write metaclasses has been dropped (though you can still do\r
-it, of course). In addition to the check ``is the type of the base\r
-class callable,'' there's a check ``does the base class have a\r
-__class__ attribute.'' If so, it is assumed that the __class__\r
-attribute refers to a class.\r
-\r
-<P>Let's repeat our simple example from above:\r
-\r
-<PRE>\r
-class C(B):\r
- a = 1\r
- b = 2\r
-</PRE>\r
-\r
-Assuming B has a __class__ attribute, this translates into:\r
-\r
-<PRE>\r
-C = B.__class__('C', (B,), {'a': 1, 'b': 2})\r
-</PRE>\r
-\r
-This is exactly the same as before except that instead of type(B),\r
-B.__class__ is invoked. If you have read <A HREF=\r
-"http://www.python.org/cgi-bin/faqw.py?req=show&file=faq06.022.htp"\r
->FAQ question 6.22</A> you will understand that while there is a big\r
-technical difference between type(B) and B.__class__, they play the\r
-same role at different abstraction levels. And perhaps at some point\r
-in the future they will really be the same thing (at which point you\r
-would be able to derive subclasses from built-in types).\r
-\r
-<P>At this point it may be worth mentioning that C.__class__ is the\r
-same object as B.__class__, i.e., C's metaclass is the same as B's\r
-metaclass. In other words, subclassing an existing class creates a\r
-new (meta)inststance of the base class's metaclass.\r
-\r
-<P>Going back to the example, the class B.__class__ is instantiated,\r
-passing its constructor the same three arguments that are passed to\r
-the default class constructor or to an extension's metaclass:\r
-<i>name</i>, <i>bases</i>, and <i>namespace</i>.\r
-\r
-<P>It is easy to be confused by what exactly happens when using a\r
-metaclass, because we lose the absolute distinction between classes\r
-and instances: a class is an instance of a metaclass (a\r
-``metainstance''), but technically (i.e. in the eyes of the python\r
-runtime system), the metaclass is just a class, and the metainstance\r
-is just an instance. At the end of the class statement, the metaclass\r
-whose metainstance is used as a base class is instantiated, yielding a\r
-second metainstance (of the same metaclass). This metainstance is\r
-then used as a (normal, non-meta) class; instantiation of the class\r
-means calling the metainstance, and this will return a real instance.\r
-And what class is that an instance of? Conceptually, it is of course\r
-an instance of our metainstance; but in most cases the Python runtime\r
-system will see it as an instance of a a helper class used by the\r
-metaclass to implement its (non-meta) instances...\r
-\r
-<P>Hopefully an example will make things clearer. Let's presume we\r
-have a metaclass MetaClass1. It's helper class (for non-meta\r
-instances) is callled HelperClass1. We now (manually) instantiate\r
-MetaClass1 once to get an empty special base class:\r
-\r
-<PRE>\r
-BaseClass1 = MetaClass1("BaseClass1", (), {})\r
-</PRE>\r
-\r
-We can now use BaseClass1 as a base class in a class statement:\r
-\r
-<PRE>\r
-class MySpecialClass(BaseClass1):\r
- i = 1\r
- def f(s): pass\r
-</PRE>\r
-\r
-At this point, MySpecialClass is defined; it is a metainstance of\r
-MetaClass1 just like BaseClass1, and in fact the expression\r
-``BaseClass1.__class__ == MySpecialClass.__class__ == MetaClass1''\r
-yields true.\r
-\r
-<P>We are now ready to create instances of MySpecialClass. Let's\r
-assume that no constructor arguments are required:\r
-\r
-<PRE>\r
-x = MySpecialClass()\r
-y = MySpecialClass()\r
-print x.__class__, y.__class__\r
-</PRE>\r
-\r
-The print statement shows that x and y are instances of HelperClass1.\r
-How did this happen? MySpecialClass is an instance of MetaClass1\r
-(``meta'' is irrelevant here); when an instance is called, its\r
-__call__ method is invoked, and presumably the __call__ method defined\r
-by MetaClass1 returns an instance of HelperClass1.\r
-\r
-<P>Now let's see how we could use metaclasses -- what can we do\r
-with metaclasses that we can't easily do without them? Here's one\r
-idea: a metaclass could automatically insert trace calls for all\r
-method calls. Let's first develop a simplified example, without\r
-support for inheritance or other ``advanced'' Python features (we'll\r
-add those later).\r
-\r
-<PRE>\r
-import types\r
-\r
-class Tracing:\r
- def __init__(self, name, bases, namespace):\r
- """Create a new class."""\r
- self.__name__ = name\r
- self.__bases__ = bases\r
- self.__namespace__ = namespace\r
- def __call__(self):\r
- """Create a new instance."""\r
- return Instance(self)\r
-\r
-class Instance:\r
- def __init__(self, klass):\r
- self.__klass__ = klass\r
- def __getattr__(self, name):\r
- try:\r
- value = self.__klass__.__namespace__[name]\r
- except KeyError:\r
- raise AttributeError, name\r
- if type(value) is not types.FunctionType:\r
- return value\r
- return BoundMethod(value, self)\r
-\r
-class BoundMethod:\r
- def __init__(self, function, instance):\r
- self.function = function\r
- self.instance = instance\r
- def __call__(self, *args):\r
- print "calling", self.function, "for", self.instance, "with", args\r
- return apply(self.function, (self.instance,) + args)\r
-\r
-Trace = Tracing('Trace', (), {})\r
-\r
-class MyTracedClass(Trace):\r
- def method1(self, a):\r
- self.a = a\r
- def method2(self):\r
- return self.a\r
-\r
-aninstance = MyTracedClass()\r
-\r
-aninstance.method1(10)\r
-\r
-print "the answer is %d" % aninstance.method2()\r
-</PRE>\r
-\r
-Confused already? The intention is to read this from top down. The\r
-Tracing class is the metaclass we're defining. Its structure is\r
-really simple.\r
-\r
-<P>\r
-\r
-<UL>\r
-\r
-<LI>The __init__ method is invoked when a new Tracing instance is\r
-created, e.g. the definition of class MyTracedClass later in the\r
-example. It simply saves the class name, base classes and namespace\r
-as instance variables.<P>\r
-\r
-<LI>The __call__ method is invoked when a Tracing instance is called,\r
-e.g. the creation of aninstance later in the example. It returns an\r
-instance of the class Instance, which is defined next.<P>\r
-\r
-</UL>\r
-\r
-<P>The class Instance is the class used for all instances of classes\r
-built using the Tracing metaclass, e.g. aninstance. It has two\r
-methods:\r
-\r
-<P>\r
-\r
-<UL>\r
-\r
-<LI>The __init__ method is invoked from the Tracing.__call__ method\r
-above to initialize a new instance. It saves the class reference as\r
-an instance variable. It uses a funny name because the user's\r
-instance variables (e.g. self.a later in the example) live in the same\r
-namespace.<P>\r
-\r
-<LI>The __getattr__ method is invoked whenever the user code\r
-references an attribute of the instance that is not an instance\r
-variable (nor a class variable; but except for __init__ and\r
-__getattr__ there are no class variables). It will be called, for\r
-example, when aninstance.method1 is referenced in the example, with\r
-self set to aninstance and name set to the string "method1".<P>\r
-\r
-</UL>\r
-\r
-<P>The __getattr__ method looks the name up in the __namespace__\r
-dictionary. If it isn't found, it raises an AttributeError exception.\r
-(In a more realistic example, it would first have to look through the\r
-base classes as well.) If it is found, there are two possibilities:\r
-it's either a function or it isn't. If it's not a function, it is\r
-assumed to be a class variable, and its value is returned. If it's a\r
-function, we have to ``wrap'' it in instance of yet another helper\r
-class, BoundMethod.\r
-\r
-<P>The BoundMethod class is needed to implement a familiar feature:\r
-when a method is defined, it has an initial argument, self, which is\r
-automatically bound to the relevant instance when it is called. For\r
-example, aninstance.method1(10) is equivalent to method1(aninstance,\r
-10). In the example if this call, first a temporary BoundMethod\r
-instance is created with the following constructor call: temp =\r
-BoundMethod(method1, aninstance); then this instance is called as\r
-temp(10). After the call, the temporary instance is discarded.\r
-\r
-<P>\r
-\r
-<UL>\r
-\r
-<LI>The __init__ method is invoked for the constructor call\r
-BoundMethod(method1, aninstance). It simply saves away its\r
-arguments.<P>\r
-\r
-<LI>The __call__ method is invoked when the bound method instance is\r
-called, as in temp(10). It needs to call method1(aninstance, 10).\r
-However, even though self.function is now method1 and self.instance is\r
-aninstance, it can't call self.function(self.instance, args) directly,\r
-because it should work regardless of the number of arguments passed.\r
-(For simplicity, support for keyword arguments has been omitted.)<P>\r
-\r
-</UL>\r
-\r
-<P>In order to be able to support arbitrary argument lists, the\r
-__call__ method first constructs a new argument tuple. Conveniently,\r
-because of the notation *args in __call__'s own argument list, the\r
-arguments to __call__ (except for self) are placed in the tuple args.\r
-To construct the desired argument list, we concatenate a singleton\r
-tuple containing the instance with the args tuple: (self.instance,) +\r
-args. (Note the trailing comma used to construct the singleton\r
-tuple.) In our example, the resulting argument tuple is (aninstance,\r
-10).\r
-\r
-<P>The intrinsic function apply() takes a function and an argument\r
-tuple and calls the function for it. In our example, we are calling\r
-apply(method1, (aninstance, 10)) which is equivalent to calling\r
-method(aninstance, 10).\r
-\r
-<P>From here on, things should come together quite easily. The output\r
-of the example code is something like this:\r
-\r
-<PRE>\r
-calling <function method1 at ae8d8> for <Instance instance at 95ab0> with (10,)\r
-calling <function method2 at ae900> for <Instance instance at 95ab0> with ()\r
-the answer is 10\r
-</PRE>\r
-\r
-<P>That was about the shortest meaningful example that I could come up\r
-with. A real tracing metaclass (for example, <A\r
-HREF="#Trace">Trace.py</A> discussed below) needs to be more\r
-complicated in two dimensions.\r
-\r
-<P>First, it needs to support more advanced Python features such as\r
-class variables, inheritance, __init__ methods, and keyword arguments.\r
-\r
-<P>Second, it needs to provide a more flexible way to handle the\r
-actual tracing information; perhaps it should be possible to write\r
-your own tracing function that gets called, perhaps it should be\r
-possible to enable and disable tracing on a per-class or per-instance\r
-basis, and perhaps a filter so that only interesting calls are traced;\r
-it should also be able to trace the return value of the call (or the\r
-exception it raised if an error occurs). Even the Trace.py example\r
-doesn't support all these features yet.\r
-\r
-<P>\r
-\r
-<HR>\r
-\r
-<H1>Real-life Examples</H1>\r
-\r
-<P>Have a look at some very preliminary examples that I coded up to\r
-teach myself how to write metaclasses:\r
-\r
-<DL>\r
-\r
-<DT><A HREF="Enum.py">Enum.py</A>\r
-\r
-<DD>This (ab)uses the class syntax as an elegant way to define\r
-enumerated types. The resulting classes are never instantiated --\r
-rather, their class attributes are the enumerated values. For\r
-example:\r
-\r
-<PRE>\r
-class Color(Enum):\r
- red = 1\r
- green = 2\r
- blue = 3\r
-print Color.red\r
-</PRE>\r
-\r
-will print the string ``Color.red'', while ``Color.red==1'' is true,\r
-and ``Color.red + 1'' raise a TypeError exception.\r
-\r
-<P>\r
-\r
-<DT><A NAME=Trace></A><A HREF="Trace.py">Trace.py</A>\r
-\r
-<DD>The resulting classes work much like standard\r
-classes, but by setting a special class or instance attribute\r
-__trace_output__ to point to a file, all calls to the class's methods\r
-are traced. It was a bit of a struggle to get this right. This\r
-should probably redone using the generic metaclass below.\r
-\r
-<P>\r
-\r
-<DT><A HREF="Meta.py">Meta.py</A>\r
-\r
-<DD>A generic metaclass. This is an attempt at finding out how much\r
-standard class behavior can be mimicked by a metaclass. The\r
-preliminary answer appears to be that everything's fine as long as the\r
-class (or its clients) don't look at the instance's __class__\r
-attribute, nor at the class's __dict__ attribute. The use of\r
-__getattr__ internally makes the classic implementation of __getattr__\r
-hooks tough; we provide a similar hook _getattr_ instead.\r
-(__setattr__ and __delattr__ are not affected.)\r
-(XXX Hm. Could detect presence of __getattr__ and rename it.)\r
-\r
-<P>\r
-\r
-<DT><A HREF="Eiffel.py">Eiffel.py</A>\r
-\r
-<DD>Uses the above generic metaclass to implement Eiffel style\r
-pre-conditions and post-conditions.\r
-\r
-<P>\r
-\r
-<DT><A HREF="Synch.py">Synch.py</A>\r
-\r
-<DD>Uses the above generic metaclass to implement synchronized\r
-methods.\r
-\r
-<P>\r
-\r
-<DT><A HREF="Simple.py">Simple.py</A>\r
-\r
-<DD>The example module used above.\r
-\r
-<P>\r
-\r
-</DL>\r
-\r
-<P>A pattern seems to be emerging: almost all these uses of\r
-metaclasses (except for Enum, which is probably more cute than useful)\r
-mostly work by placing wrappers around method calls. An obvious\r
-problem with that is that it's not easy to combine the features of\r
-different metaclasses, while this would actually be quite useful: for\r
-example, I wouldn't mind getting a trace from the test run of the\r
-Synch module, and it would be interesting to add preconditions to it\r
-as well. This needs more research. Perhaps a metaclass could be\r
-provided that allows stackable wrappers...\r
-\r
-<P>\r
-\r
-<HR>\r
-\r
-<H2>Things You Could Do With Metaclasses</H2>\r
-\r
-<P>There are lots of things you could do with metaclasses. Most of\r
-these can also be done with creative use of __getattr__, but\r
-metaclasses make it easier to modify the attribute lookup behavior of\r
-classes. Here's a partial list.\r
-\r
-<P>\r
-\r
-<UL>\r
-\r
-<LI>Enforce different inheritance semantics, e.g. automatically call\r
-base class methods when a derived class overrides<P>\r
-\r
-<LI>Implement class methods (e.g. if the first argument is not named\r
-'self')<P>\r
-\r
-<LI>Implement that each instance is initialized with <b>copies</b> of\r
-all class variables<P>\r
-\r
-<LI>Implement a different way to store instance variables (e.g. in a\r
-list kept outside the instance but indexed by the instance's id())<P>\r
-\r
-<LI>Automatically wrap or trap all or certain methods\r
-\r
-<UL>\r
-\r
-<LI>for tracing\r
-\r
-<LI>for precondition and postcondition checking\r
-\r
-<LI>for synchronized methods\r
-\r
-<LI>for automatic value caching\r
-\r
-</UL>\r
-<P>\r
-\r
-<LI>When an attribute is a parameterless function, call it on\r
-reference (to mimic it being an instance variable); same on assignment<P>\r
-\r
-<LI>Instrumentation: see how many times various attributes are used<P>\r
-\r
-<LI>Different semantics for __setattr__ and __getattr__ (e.g. disable\r
-them when they are being used recursively)<P>\r
-\r
-<LI>Abuse class syntax for other things<P>\r
-\r
-<LI>Experiment with automatic type checking<P>\r
-\r
-<LI>Delegation (or acquisition)<P>\r
-\r
-<LI>Dynamic inheritance patterns<P>\r
-\r
-<LI>Automatic caching of methods<P>\r
-\r
-</UL>\r
-\r
-<P>\r
-\r
-<HR>\r
-\r
-<H4>Credits</H4>\r
-\r
-<P>Many thanks to David Ascher and Donald Beaudry for their comments\r
-on earlier draft of this paper. Also thanks to Matt Conway and Tommy\r
-Burnette for putting a seed for the idea of metaclasses in my\r
-mind, nearly three years ago, even though at the time my response was\r
-``you can do that with __getattr__ hooks...'' :-)\r
-\r
-<P>\r
-\r
-<HR>\r
-\r
-</BODY>\r
-\r
-</HTML>\r
projects / mirror_edk2.git / blobdiff