[ceph.git] / ceph / src / boost / libs / python / doc / internals.html

<?xml version="1.0" encoding="utf-8" ?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<meta name="generator" content="Docutils 0.3.0: http://docutils.sourceforge.net/" />
<title>Boost.Python Internals Boost</title>
<link rel="stylesheet" href="../../../rst.css" type="text/css" />
</head>
<body>
<div class="document" id="boost-python-internals-logo">
<h1 class="title"><a class="reference" href="index.html">Boost.Python</a> Internals <a class="reference" href="../../../index.htm"><img alt="Boost" src="../../../boost.png" /></a></h1>
<div class="section" id="a-conversation-between-brett-calcott-and-david-abrahams">
<h1><a name="a-conversation-between-brett-calcott-and-david-abrahams">A conversation between Brett Calcott and David Abrahams</a></h1>
<table class="field-list" frame="void" rules="none">
<col class="field-name" />
<col class="field-body" />
<tbody valign="top">
<tr class="field"><th class="field-name">copyright:</th><td class="field-body">Copyright David Abrahams and Brett Calcott 2003. See
accompanying <a class="reference" href="../../../LICENSE_1_0.txt">license</a> for terms of use.</td>
</tr>
</tbody>
</table>
<p>In both of these cases, I'm quite capable of reading code - but the
thing I don't get from scanning the source is a sense of the
architecture, both structurally, and temporally (er, I mean in what
order things go on).</p>
<ol class="arabic">
<li><p class="first">What happens when you do the following:</p>
<pre class="literal-block">
struct boring {};
...etc...
class_&lt;boring&gt;(&quot;boring&quot;)
    ;
</pre>
</li>
</ol>
<p>There seems to be a fair bit going on.</p>
<blockquote>
<ul class="simple">
<li>Python needs a new ClassType to be registered.</li>
<li>We need to construct a new type that can hold our boring struct.</li>
<li>Inward and outward converters need to be registered for the type.</li>
</ul>
</blockquote>
<p>Can you gesture in the general direction where these things are done?</p>
<blockquote>
<p>I only have time for a &quot;off-the-top-of-my-head&quot; answer at the moment;
I suggest you step through the code with a debugger after reading this
to see how it works, fill in details, and make sure I didn't forget
anything.</p>
<blockquote>
<p>A new (Python) subclass of Boost.Python.Instance (see
libs/python/src/object/class.cpp) is created by invoking
Boost.Python.class, the metatype:</p>
<pre class="literal-block">
&gt;&gt;&gt; boring = Boost.Python.class(
...     'boring'
...   , bases_tuple       # in this case, just ()
...   , { 
...         '__module__' : module_name
...       , '__doc__' : doc_string # optional
...     }
... )
</pre>
<p>A handle to this object is stuck in the m_class_object field
of the registration associated with <tt class="literal"><span class="pre">typeid(boring)</span></tt>.  The
registry will keep that object alive forever, even if you
wipe out the 'boring' attribute of the extension module
(probably not a good thing).</p>
<p>Because you didn't specify <tt class="literal"><span class="pre">class&lt;boring,</span> <span class="pre">non_copyable,</span>
<span class="pre">...&gt;</span></tt>, a to-python converter for boring is registered which
copies its argument into a value_holder held by the the
Python boring object.</p>
<p>Because you didn't specify <tt class="literal"><span class="pre">class&lt;boring</span> <span class="pre">...&gt;(no_init)</span></tt>,
an <tt class="literal"><span class="pre">__init__</span></tt> function object is added to the class
dictionary which default-constructs a boring in a
value_holder (because you didn't specify some smart pointer
or derived wrapper class as a holder) held by the Python
boring object.</p>
<p><tt class="literal"><span class="pre">register_class_from_python</span></tt> is used to register a
from-python converter for <tt class="literal"><span class="pre">shared_ptr&lt;boring&gt;</span></tt>.
<tt class="literal"><span class="pre">boost::shared_ptr</span></tt>s are special among smart pointers
because their Deleter argument can be made to manage the
whole Python object, not just the C++ object it contains, no
matter how the C++ object is held.</p>
<p>If there were any <tt class="literal"><span class="pre">bases&lt;&gt;</span></tt>, we'd also be registering the
relationship between these base classes and boring in the
up/down cast graph (<tt class="literal"><span class="pre">inheritance.[hpp/cpp]</span></tt>).</p>
<p>In earlier versions of the code, we'd be registering lvalue
from-python converters for the class here, but now
from-python conversion for wrapped classes is handled as a
special case, before consulting the registry, if the source
Python object's metaclass is the Boost.Python metaclass.</p>
<p>Hmm, that from-python converter probably ought to be handled
the way class converters are, with no explicit conversions
registered.</p>
</blockquote>
</blockquote>
<ol class="arabic" start="2">
<li><p class="first">Can you give a brief overview of the data structures that are
present in the registry</p>
<blockquote>
<p>The registry is simple: it's just a map from typeid -&gt;
registration (see boost/python/converter/registrations.hpp).
<tt class="literal"><span class="pre">lvalue_chain</span></tt> and <tt class="literal"><span class="pre">rvalue_chain</span></tt> are simple endogenous
linked lists.</p>
<p>If you want to know more, just ask.</p>
<p>If you want to know about the cast graph, ask me something specific in
a separate message.</p>
</blockquote>
<p>and an overview of the process that happens as a type makes its
way from c++ to python and back again.</p>
</li>
</ol>
<blockquote>
<p>Big subject.  I suggest some background reading: look for relevant
info in the LLNL progress reports and the messages they link to.
Also,</p>
<blockquote>
<p><a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-May/001023.html">http://mail.python.org/pipermail/c++-sig/2002-May/001023.html</a></p>
<p><a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-December/003115.html">http://mail.python.org/pipermail/c++-sig/2002-December/003115.html</a></p>
<p><a class="reference" href="http://aspn.activestate.com/ASPN/Mail/Message/1280898">http://aspn.activestate.com/ASPN/Mail/Message/1280898</a></p>
<p><a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-July/001755.html">http://mail.python.org/pipermail/c++-sig/2002-July/001755.html</a></p>
</blockquote>
<p>from c++ to python:</p>
<blockquote>
<p>It depends on the type and the call policies in use or, for
<tt class="literal"><span class="pre">call&lt;&gt;(...)</span></tt>, <tt class="literal"><span class="pre">call_method&lt;&gt;(...)</span></tt>, or <tt class="literal"><span class="pre">object(...)</span></tt>, if
<tt class="literal"><span class="pre">ref</span></tt> or <tt class="literal"><span class="pre">ptr</span></tt> is used.  There are also two basic
categories to to-python conversion, &quot;return value&quot; conversion
(for Python-&gt;C++ calls) and &quot;argument&quot; conversion (for
C++-&gt;Python calls and explicit <tt class="literal"><span class="pre">object()</span></tt> conversions).  The
behavior of these two categories differs subtly in various ways
whose details I forget at the moment.  You can probably find
the answers in the above references, and certainly in the code.</p>
<p>The &quot;default&quot; case is by-value (copying) conversion, which uses
to_python_value as a to-python converter.</p>
<blockquote>
<p>Since there can sensibly be only one way to convert any type
to python (disregarding the idea of scoped registries for the
moment), it makes sense that to-python conversions can be
handled by specializing a template.  If the type is one of
the types handled by a built-in conversion
(builtin_converters.hpp), the corresponding template
specialization of to_python_value gets used.</p>
<p>Otherwise, to_python_value uses the <tt class="literal"><span class="pre">m_to_python</span></tt>
function in the registration for the C++ type.</p>
</blockquote>
<p>Other conversions, like by-reference conversions, are only
available for wrapped classes, and are requested explicitly by
using <tt class="literal"><span class="pre">ref(...)</span></tt>, <tt class="literal"><span class="pre">ptr(...)</span></tt>, or by specifying different
CallPolicies for a call, which can cause a different to-python
converter to be used.  These conversions are never registered
anywhere, though they do need to use the registration to find
the Python class corresponding to the C++ type being referred
to.  They just build a new Python instance and stick the
appropriate Holder instance in it.</p>
</blockquote>
<p>from python to C++:</p>
<blockquote>
<p>Once again I think there is a distinction between &quot;return value&quot;
and &quot;argument&quot; conversions, and I forget exactly what that is.</p>
<p>What happens depends on whether an lvalue conversion is needed
(see <a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-May/001023.html">http://mail.python.org/pipermail/c++-sig/2002-May/001023.html</a>)
All lvalue conversions are also registered in a type's rvalue
conversion chain, since when an rvalue will do, an lvalue is
certainly good enough.</p>
<p>An lvalue conversion can be done in one step (just get me the
pointer to the object - it can be <tt class="literal"><span class="pre">NULL</span></tt> if no conversion is
possible) while an rvalue conversion requires two steps to
support wrapped function overloading and multiple converters for
a given C++ target type: first tell me if a conversion is
possible, then construct the converted object as a second step.</p>
</blockquote>
</blockquote>
</div>
</div>
<hr class="footer"/>
<div class="footer">
<a class="reference" href="internals.rst">View document source</a>.
Generated on: 2003-09-12 14:51 UTC.
Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
</div>
</body>
</html>
Commit	Line	Data
7c673cae FG	1	<?xml version="1.0" encoding="utf-8" ?>
	2	<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
	3	<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
	4	<head>
	5	<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
	6	<meta name="generator" content="Docutils 0.3.0: http://docutils.sourceforge.net/" />
	7	<title>Boost.Python Internals Boost</title>
	8	<link rel="stylesheet" href="../../../rst.css" type="text/css" />
	9	</head>
	10	<body>
	11	<div class="document" id="boost-python-internals-logo">
	12	<h1 class="title"><a class="reference" href="index.html">Boost.Python</a> Internals <a class="reference" href="../../../index.htm"><img alt="Boost" src="../../../boost.png" /></a></h1>
	13	<div class="section" id="a-conversation-between-brett-calcott-and-david-abrahams">
	14	<h1><a name="a-conversation-between-brett-calcott-and-david-abrahams">A conversation between Brett Calcott and David Abrahams</a></h1>
	15	<table class="field-list" frame="void" rules="none">
	16	<col class="field-name" />
	17	<col class="field-body" />
	18	<tbody valign="top">
	19	<tr class="field"><th class="field-name">copyright:</th><td class="field-body">Copyright David Abrahams and Brett Calcott 2003. See
	20	accompanying <a class="reference" href="../../../LICENSE_1_0.txt">license</a> for terms of use.</td>
	21	</tr>
	22	</tbody>
	23	</table>
	24	<p>In both of these cases, I'm quite capable of reading code - but the
	25	thing I don't get from scanning the source is a sense of the
	26	architecture, both structurally, and temporally (er, I mean in what
	27	order things go on).</p>
	28	<ol class="arabic">
	29	<li><p class="first">What happens when you do the following:</p>
	30	<pre class="literal-block">
	31	struct boring {};
	32	...etc...
	33	class_<boring>("boring")
	34	;
	35	</pre>
	36	</li>
	37	</ol>
	38	<p>There seems to be a fair bit going on.</p>
	39	<blockquote>
	40	<ul class="simple">
	41	<li>Python needs a new ClassType to be registered.</li>
	42	<li>We need to construct a new type that can hold our boring struct.</li>
	43	<li>Inward and outward converters need to be registered for the type.</li>
	44	</ul>
	45	</blockquote>
	46	<p>Can you gesture in the general direction where these things are done?</p>
	47	<blockquote>
	48	<p>I only have time for a "off-the-top-of-my-head" answer at the moment;
	49	I suggest you step through the code with a debugger after reading this
	50	to see how it works, fill in details, and make sure I didn't forget
	51	anything.</p>
	52	<blockquote>
	53	<p>A new (Python) subclass of Boost.Python.Instance (see
	54	libs/python/src/object/class.cpp) is created by invoking
	55	Boost.Python.class, the metatype:</p>
	56	<pre class="literal-block">
	57	>>> boring = Boost.Python.class(
	58	... 'boring'
	59	... , bases_tuple # in this case, just ()
	60	... , {
	61	... '__module__' : module_name
	62	... , '__doc__' : doc_string # optional
	63	... }
	64	... )
65	</pre>
66	<p>A handle to this object is stuck in the m_class_object field
67	of the registration associated with <tt class="literal"><span class="pre">typeid(boring)</span></tt>. The
68	registry will keep that object alive forever, even if you
69	wipe out the 'boring' attribute of the extension module
70	(probably not a good thing).</p>
71	<p>Because you didn't specify <tt class="literal"><span class="pre">class<boring,</span> <span class="pre">non_copyable,</span>
72	<span class="pre">...></span></tt>, a to-python converter for boring is registered which
73	copies its argument into a value_holder held by the the
74	Python boring object.</p>
75	<p>Because you didn't specify <tt class="literal"><span class="pre">class<boring</span> <span class="pre">...>(no_init)</span></tt>,
76	an <tt class="literal"><span class="pre">__init__</span></tt> function object is added to the class
77	dictionary which default-constructs a boring in a
78	value_holder (because you didn't specify some smart pointer
79	or derived wrapper class as a holder) held by the Python
80	boring object.</p>
81	<p><tt class="literal"><span class="pre">register_class_from_python</span></tt> is used to register a
82	from-python converter for <tt class="literal"><span class="pre">shared_ptr<boring></span></tt>.
83	<tt class="literal"><span class="pre">boost::shared_ptr</span></tt>s are special among smart pointers
84	because their Deleter argument can be made to manage the
85	whole Python object, not just the C++ object it contains, no
86	matter how the C++ object is held.</p>
87	<p>If there were any <tt class="literal"><span class="pre">bases<></span></tt>, we'd also be registering the
88	relationship between these base classes and boring in the
89	up/down cast graph (<tt class="literal"><span class="pre">inheritance.[hpp/cpp]</span></tt>).</p>
90	<p>In earlier versions of the code, we'd be registering lvalue
91	from-python converters for the class here, but now
92	from-python conversion for wrapped classes is handled as a
93	special case, before consulting the registry, if the source
94	Python object's metaclass is the Boost.Python metaclass.</p>
95	<p>Hmm, that from-python converter probably ought to be handled
96	the way class converters are, with no explicit conversions
97	registered.</p>
98	</blockquote>
99	</blockquote>
100	<ol class="arabic" start="2">
101	<li><p class="first">Can you give a brief overview of the data structures that are
102	present in the registry</p>
103	<blockquote>
104	<p>The registry is simple: it's just a map from typeid ->
105	registration (see boost/python/converter/registrations.hpp).
106	<tt class="literal"><span class="pre">lvalue_chain</span></tt> and <tt class="literal"><span class="pre">rvalue_chain</span></tt> are simple endogenous
107	linked lists.</p>
108	<p>If you want to know more, just ask.</p>
109	<p>If you want to know about the cast graph, ask me something specific in
110	a separate message.</p>
111	</blockquote>
112	<p>and an overview of the process that happens as a type makes its
113	way from c++ to python and back again.</p>
114	</li>
115	</ol>
116	<blockquote>
117	<p>Big subject. I suggest some background reading: look for relevant
118	info in the LLNL progress reports and the messages they link to.
119	Also,</p>
120	<blockquote>
121	<p><a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-May/001023.html">http://mail.python.org/pipermail/c++-sig/2002-May/001023.html</a></p>
122	<p><a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-December/003115.html">http://mail.python.org/pipermail/c++-sig/2002-December/003115.html</a></p>
123	<p><a class="reference" href="http://aspn.activestate.com/ASPN/Mail/Message/1280898">http://aspn.activestate.com/ASPN/Mail/Message/1280898</a></p>
124	<p><a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-July/001755.html">http://mail.python.org/pipermail/c++-sig/2002-July/001755.html</a></p>
125	</blockquote>
126	<p>from c++ to python:</p>
127	<blockquote>
128	<p>It depends on the type and the call policies in use or, for
129	<tt class="literal"><span class="pre">call<>(...)</span></tt>, <tt class="literal"><span class="pre">call_method<>(...)</span></tt>, or <tt class="literal"><span class="pre">object(...)</span></tt>, if
130	<tt class="literal"><span class="pre">ref</span></tt> or <tt class="literal"><span class="pre">ptr</span></tt> is used. There are also two basic
131	categories to to-python conversion, "return value" conversion
132	(for Python->C++ calls) and "argument" conversion (for
133	C++->Python calls and explicit <tt class="literal"><span class="pre">object()</span></tt> conversions). The
134	behavior of these two categories differs subtly in various ways
135	whose details I forget at the moment. You can probably find
136	the answers in the above references, and certainly in the code.</p>
137	<p>The "default" case is by-value (copying) conversion, which uses
138	to_python_value as a to-python converter.</p>
139	<blockquote>
140	<p>Since there can sensibly be only one way to convert any type
141	to python (disregarding the idea of scoped registries for the
142	moment), it makes sense that to-python conversions can be
143	handled by specializing a template. If the type is one of
144	the types handled by a built-in conversion
145	(builtin_converters.hpp), the corresponding template
146	specialization of to_python_value gets used.</p>
147	<p>Otherwise, to_python_value uses the <tt class="literal"><span class="pre">m_to_python</span></tt>
148	function in the registration for the C++ type.</p>
149	</blockquote>
150	<p>Other conversions, like by-reference conversions, are only
151	available for wrapped classes, and are requested explicitly by
152	using <tt class="literal"><span class="pre">ref(...)</span></tt>, <tt class="literal"><span class="pre">ptr(...)</span></tt>, or by specifying different
153	CallPolicies for a call, which can cause a different to-python
154	converter to be used. These conversions are never registered
155	anywhere, though they do need to use the registration to find
156	the Python class corresponding to the C++ type being referred
157	to. They just build a new Python instance and stick the
158	appropriate Holder instance in it.</p>
159	</blockquote>
160	<p>from python to C++:</p>
161	<blockquote>
162	<p>Once again I think there is a distinction between "return value"
163	and "argument" conversions, and I forget exactly what that is.</p>
164	<p>What happens depends on whether an lvalue conversion is needed
165	(see <a class="reference" href="http://mail.python.org/pipermail/c++-sig/2002-May/001023.html">http://mail.python.org/pipermail/c++-sig/2002-May/001023.html</a>)
166	All lvalue conversions are also registered in a type's rvalue
167	conversion chain, since when an rvalue will do, an lvalue is
168	certainly good enough.</p>
169	<p>An lvalue conversion can be done in one step (just get me the
170	pointer to the object - it can be <tt class="literal"><span class="pre">NULL</span></tt> if no conversion is
171	possible) while an rvalue conversion requires two steps to
172	support wrapped function overloading and multiple converters for
173	a given C++ target type: first tell me if a conversion is
174	possible, then construct the converted object as a second step.</p>
175	</blockquote>
176	</blockquote>
177	</div>
178	</div>
179	<hr class="footer"/>
180	<div class="footer">
181	<a class="reference" href="internals.rst">View document source</a>.
182	Generated on: 2003-09-12 14:51 UTC.
183	Generated by <a class="reference" href="http://docutils.sourceforge.net/">Docutils</a> from <a class="reference" href="http://docutils.sourceforge.net/rst.html">reStructuredText</a> source.
184	</div>
185	</body>
186	</html>