docs/qapi-code-gen.txt

   1 = How to use the QAPI code generator =
   2
   3 * Note: as of this writing, QMP does not use QAPI. Eventually QMP
   4 commands will be converted to use QAPI internally. The following
   5 information describes QMP/QAPI as it will exist after the
   6 conversion.
   7
   8 QAPI is a native C API within QEMU which provides management-level
   9 functionality to internal/external users. For external
  10 users/processes, this interface is made available by a JSON-based
  11 QEMU Monitor protocol that is provided by the QMP server.
  12
  13 To map QMP-defined interfaces to the native C QAPI implementations,
  14 a JSON-based schema is used to define types and function
  15 signatures, and a set of scripts is used to generate types/signatures,
  16 and marshaling/dispatch code. The QEMU Guest Agent also uses these
  17 scripts, paired with a separate schema, to generate
  18 marshaling/dispatch code for the guest agent server running in the
  19 guest.
  20
  21 This document will describe how the schemas, scripts, and resulting
  22 code is used.
  23
  24
  25 == QMP/Guest agent schema ==
  26
  27 This file defines the types, commands, and events used by QMP.  It should
  28 fully describe the interface used by QMP.
  29
  30 This file is designed to be loosely based on JSON although it's technically
  31 executable Python.  While dictionaries are used, they are parsed as
  32 OrderedDicts so that ordering is preserved.
  33
  34 There are two basic syntaxes used, type definitions and command definitions.
  35
  36 The first syntax defines a type and is represented by a dictionary.  There are
  37 three kinds of user-defined types that are supported: complex types,
  38 enumeration types and union types.
  39
  40 Generally speaking, types definitions should always use CamelCase for the type
  41 names. Command names should be all lower case with words separated by a hyphen.
  42
  43
  44 === Includes ===
  45
  46 The QAPI schema definitions can be modularized using the 'include' directive:
  47
  48  { 'include': 'path/to/file.json'}
  49
  50 The directive is evaluated recursively, and include paths are relative to the
  51 file using the directive. Multiple includes of the same file are safe.
  52
  53
  54 === Complex types ===
  55
  56 A complex type is a dictionary containing a single key whose value is a
  57 dictionary.  This corresponds to a struct in C or an Object in JSON.  An
  58 example of a complex type is:
  59
  60  { 'type': 'MyType',
  61    'data': { 'member1': 'str', 'member2': 'int', '*member3': 'str' } }
  62
  63 The use of '*' as a prefix to the name means the member is optional.
  64
  65 The default initialization value of an optional argument should not be changed
  66 between versions of QEMU unless the new default maintains backward
  67 compatibility to the user-visible behavior of the old default.
  68
  69 With proper documentation, this policy still allows some flexibility; for
  70 example, documenting that a default of 0 picks an optimal buffer size allows
  71 one release to declare the optimal size at 512 while another release declares
  72 the optimal size at 4096 - the user-visible behavior is not the bytes used by
  73 the buffer, but the fact that the buffer was optimal size.
  74
  75 On input structures (only mentioned in the 'data' side of a command), changing
  76 from mandatory to optional is safe (older clients will supply the option, and
  77 newer clients can benefit from the default); changing from optional to
  78 mandatory is backwards incompatible (older clients may be omitting the option,
  79 and must continue to work).
  80
  81 On output structures (only mentioned in the 'returns' side of a command),
  82 changing from mandatory to optional is in general unsafe (older clients may be
  83 expecting the field, and could crash if it is missing), although it can be done
  84 if the only way that the optional argument will be omitted is when it is
  85 triggered by the presence of a new input flag to the command that older clients
  86 don't know to send.  Changing from optional to mandatory is safe.
  87
  88 A structure that is used in both input and output of various commands
  89 must consider the backwards compatibility constraints of both directions
  90 of use.
  91
  92 A complex type definition can specify another complex type as its base.
  93 In this case, the fields of the base type are included as top-level fields
  94 of the new complex type's dictionary in the QMP wire format. An example
  95 definition is:
  96
  97  { 'type': 'BlockdevOptionsGenericFormat', 'data': { 'file': 'str' } }
  98  { 'type': 'BlockdevOptionsGenericCOWFormat',
  99    'base': 'BlockdevOptionsGenericFormat',
 100    'data': { '*backing': 'str' } }
 101
 102 An example BlockdevOptionsGenericCOWFormat object on the wire could use
 103 both fields like this:
 104
 105  { "file": "/some/place/my-image",
 106    "backing": "/some/place/my-backing-file" }
 107
 108 === Enumeration types ===
 109
 110 An enumeration type is a dictionary containing a single key whose value is a
 111 list of strings.  An example enumeration is:
 112
 113  { 'enum': 'MyEnum', 'data': [ 'value1', 'value2', 'value3' ] }
 114
 115 === Union types ===
 116
 117 Union types are used to let the user choose between several different data
 118 types.  A union type is defined using a dictionary as explained in the
 119 following paragraphs.
 120
 121
 122 A simple union type defines a mapping from discriminator values to data types
 123 like in this example:
 124
 125  { 'type': 'FileOptions', 'data': { 'filename': 'str' } }
 126  { 'type': 'Qcow2Options',
 127    'data': { 'backing-file': 'str', 'lazy-refcounts': 'bool' } }
 128
 129  { 'union': 'BlockdevOptions',
 130    'data': { 'file': 'FileOptions',
 131              'qcow2': 'Qcow2Options' } }
 132
 133 In the QMP wire format, a simple union is represented by a dictionary that
 134 contains the 'type' field as a discriminator, and a 'data' field that is of the
 135 specified data type corresponding to the discriminator value:
 136
 137  { "type": "qcow2", "data" : { "backing-file": "/some/place/my-image",
 138                                "lazy-refcounts": true } }
 139
 140
 141 A union definition can specify a complex type as its base. In this case, the
 142 fields of the complex type are included as top-level fields of the union
 143 dictionary in the QMP wire format. An example definition is:
 144
 145  { 'type': 'BlockdevCommonOptions', 'data': { 'readonly': 'bool' } }
 146  { 'union': 'BlockdevOptions',
 147    'base': 'BlockdevCommonOptions',
 148    'data': { 'raw': 'RawOptions',
 149              'qcow2': 'Qcow2Options' } }
 150
 151 And it looks like this on the wire:
 152
 153  { "type": "qcow2",
 154    "readonly": false,
 155    "data" : { "backing-file": "/some/place/my-image",
 156               "lazy-refcounts": true } }
 157
 158
 159 Flat union types avoid the nesting on the wire. They are used whenever a
 160 specific field of the base type is declared as the discriminator ('type' is
 161 then no longer generated). The discriminator must be of enumeration type.
 162 The above example can then be modified as follows:
 163
 164  { 'enum': 'BlockdevDriver', 'data': [ 'raw', 'qcow2' ] }
 165  { 'type': 'BlockdevCommonOptions',
 166    'data': { 'driver': 'BlockdevDriver', 'readonly': 'bool' } }
 167  { 'union': 'BlockdevOptions',
 168    'base': 'BlockdevCommonOptions',
 169    'discriminator': 'driver',
 170    'data': { 'raw': 'RawOptions',
 171              'qcow2': 'Qcow2Options' } }
 172
 173 Resulting in this JSON object:
 174
 175  { "driver": "qcow2",
 176    "readonly": false,
 177    "backing-file": "/some/place/my-image",
 178    "lazy-refcounts": true }
 179
 180
 181 A special type of unions are anonymous unions. They don't form a dictionary in
 182 the wire format but allow the direct use of different types in their place. As
 183 they aren't structured, they don't have any explicit discriminator but use
 184 the (QObject) data type of their value as an implicit discriminator. This means
 185 that they are restricted to using only one discriminator value per QObject
 186 type. For example, you cannot have two different complex types in an anonymous
 187 union, or two different integer types.
 188
 189 Anonymous unions are declared using an empty dictionary as their discriminator.
 190 The discriminator values never appear on the wire, they are only used in the
 191 generated C code. Anonymous unions cannot have a base type.
 192
 193  { 'union': 'BlockRef',
 194    'discriminator': {},
 195    'data': { 'definition': 'BlockdevOptions',
 196              'reference': 'str' } }
 197
 198 This example allows using both of the following example objects:
 199
 200  { "file": "my_existing_block_device_id" }
 201  { "file": { "driver": "file",
 202              "readonly": false,
 203              "filename": "/tmp/mydisk.qcow2" } }
 204
 205
 206 === Commands ===
 207
 208 Commands are defined by using a list containing three members.  The first
 209 member is the command name, the second member is a dictionary containing
 210 arguments, and the third member is the return type.
 211
 212 An example command is:
 213
 214  { 'command': 'my-command',
 215    'data': { 'arg1': 'str', '*arg2': 'str' },
 216    'returns': 'str' }
 217
 218 === Events ===
 219
 220 Events are defined with the keyword 'event'.  When 'data' is also specified,
 221 additional info will be included in the event.  Finally there will be C API
 222 generated in qapi-event.h; when called by QEMU code, a message with timestamp
 223 will be emitted on the wire.  If timestamp is -1, it means failure to retrieve
 224 host time.
 225
 226 An example event is:
 227
 228 { 'event': 'EVENT_C',
 229   'data': { '*a': 'int', 'b': 'str' } }
 230
 231 Resulting in this JSON object:
 232
 233 { "event": "EVENT_C",
 234   "data": { "b": "test string" },
 235   "timestamp": { "seconds": 1267020223, "microseconds": 435656 } }
 236
 237 == Code generation ==
 238
 239 Schemas are fed into 3 scripts to generate all the code/files that, paired
 240 with the core QAPI libraries, comprise everything required to take JSON
 241 commands read in by a QMP/guest agent server, unmarshal the arguments into
 242 the underlying C types, call into the corresponding C function, and map the
 243 response back to a QMP/guest agent response to be returned to the user.
 244
 245 As an example, we'll use the following schema, which describes a single
 246 complex user-defined type (which will produce a C struct, along with a list
 247 node structure that can be used to chain together a list of such types in
 248 case we want to accept/return a list of this type with a command), and a
 249 command which takes that type as a parameter and returns the same type:
 250
 251     $ cat example-schema.json
 252     { 'type': 'UserDefOne',
 253       'data': { 'integer': 'int', 'string': 'str' } }
 254
 255     { 'command': 'my-command',
 256       'data':    {'arg1': 'UserDefOne'},
 257       'returns': 'UserDefOne' }
 258
 259 === scripts/qapi-types.py ===
 260
 261 Used to generate the C types defined by a schema. The following files are
 262 created:
 263
 264 $(prefix)qapi-types.h - C types corresponding to types defined in
 265                         the schema you pass in
 266 $(prefix)qapi-types.c - Cleanup functions for the above C types
 267
 268 The $(prefix) is an optional parameter used as a namespace to keep the
 269 generated code from one schema/code-generation separated from others so code
 270 can be generated/used from multiple schemas without clobbering previously
 271 created code.
 272
 273 Example:
 274
 275     $ python scripts/qapi-types.py --output-dir="qapi-generated" \
 276     --prefix="example-" --input-file=example-schema.json
 277     $ cat qapi-generated/example-qapi-types.c
 278 [Uninteresting stuff omitted...]
 279
 280     void qapi_free_UserDefOneList(UserDefOneList * obj)
 281     {
 282         QapiDeallocVisitor *md;
 283         Visitor *v;
 284
 285         if (!obj) {
 286             return;
 287         }
 288
 289         md = qapi_dealloc_visitor_new();
 290         v = qapi_dealloc_get_visitor(md);
 291         visit_type_UserDefOneList(v, &obj, NULL, NULL);
 292         qapi_dealloc_visitor_cleanup(md);
 293     }
 294
 295     void qapi_free_UserDefOne(UserDefOne * obj)
 296     {
 297         QapiDeallocVisitor *md;
 298         Visitor *v;
 299
 300         if (!obj) {
 301             return;
 302         }
 303
 304         md = qapi_dealloc_visitor_new();
 305         v = qapi_dealloc_get_visitor(md);
 306         visit_type_UserDefOne(v, &obj, NULL, NULL);
 307         qapi_dealloc_visitor_cleanup(md);
 308     }
 309
 310     $ cat qapi-generated/example-qapi-types.h
 311 [Uninteresting stuff omitted...]
 312
 313     #ifndef EXAMPLE_QAPI_TYPES_H
 314     #define EXAMPLE_QAPI_TYPES_H
 315
 316 [Builtin types omitted...]
 317
 318     typedef struct UserDefOne UserDefOne;
 319
 320     typedef struct UserDefOneList
 321     {
 322         union {
 323             UserDefOne *value;
 324             uint64_t padding;
 325         };
 326         struct UserDefOneList *next;
 327     } UserDefOneList;
 328
 329 [Functions on builtin types omitted...]
 330
 331     struct UserDefOne
 332     {
 333         int64_t integer;
 334         char * string;
 335     };
 336
 337     void qapi_free_UserDefOneList(UserDefOneList * obj);
 338     void qapi_free_UserDefOne(UserDefOne * obj);
 339
 340     #endif
 341
 342 === scripts/qapi-visit.py ===
 343
 344 Used to generate the visitor functions used to walk through and convert
 345 a QObject (as provided by QMP) to a native C data structure and
 346 vice-versa, as well as the visitor function used to dealloc a complex
 347 schema-defined C type.
 348
 349 The following files are generated:
 350
 351 $(prefix)qapi-visit.c: visitor function for a particular C type, used
 352                        to automagically convert QObjects into the
 353                        corresponding C type and vice-versa, as well
 354                        as for deallocating memory for an existing C
 355                        type
 356
 357 $(prefix)qapi-visit.h: declarations for previously mentioned visitor
 358                        functions
 359
 360 Example:
 361
 362     $ python scripts/qapi-visit.py --output-dir="qapi-generated"
 363     --prefix="example-" --input-file=example-schema.json
 364     $ cat qapi-generated/example-qapi-visit.c
 365 [Uninteresting stuff omitted...]
 366
 367     static void visit_type_UserDefOne_fields(Visitor *m, UserDefOne ** obj, Error **errp)
 368     {
 369         Error *err = NULL;
 370         visit_type_int(m, &(*obj)->integer, "integer", &err);
 371         if (err) {
 372             goto out;
 373         }
 374         visit_type_str(m, &(*obj)->string, "string", &err);
 375         if (err) {
 376             goto out;
 377         }
 378
 379     out:
 380         error_propagate(errp, err);
 381     }
 382
 383     void visit_type_UserDefOne(Visitor *m, UserDefOne ** obj, const char *name, Error **errp)
 384     {
 385         Error *err = NULL;
 386
 387         visit_start_struct(m, (void **)obj, "UserDefOne", name, sizeof(UserDefOne), &err);
 388         if (!err) {
 389             if (*obj) {
 390                 visit_type_UserDefOne_fields(m, obj, errp);
 391             }
 392             visit_end_struct(m, &err);
 393         }
 394         error_propagate(errp, err);
 395     }
 396
 397     void visit_type_UserDefOneList(Visitor *m, UserDefOneList ** obj, const char *name, Error **errp)
 398     {
 399         Error *err = NULL;
 400         GenericList *i, **prev;
 401
 402         visit_start_list(m, name, &err);
 403         if (err) {
 404             goto out;
 405         }
 406
 407         for (prev = (GenericList **)obj;
 408              !err && (i = visit_next_list(m, prev, &err)) != NULL;
 409              prev = &i) {
 410             UserDefOneList *native_i = (UserDefOneList *)i;
 411             visit_type_UserDefOne(m, &native_i->value, NULL, &err);
 412         }
 413
 414         error_propagate(errp, err);
 415         err = NULL;
 416         visit_end_list(m, &err);
 417     out:
 418         error_propagate(errp, err);
 419     }
 420     $ python scripts/qapi-commands.py --output-dir="qapi-generated" \
 421     --prefix="example-" --input-file=example-schema.json
 422     $ cat qapi-generated/example-qapi-visit.h
 423 [Uninteresting stuff omitted...]
 424
 425     #ifndef EXAMPLE_QAPI_VISIT_H
 426     #define EXAMPLE_QAPI_VISIT_H
 427
 428 [Visitors for builtin types omitted...]
 429
 430     void visit_type_UserDefOne(Visitor *m, UserDefOne ** obj, const char *name, Error **errp);
 431     void visit_type_UserDefOneList(Visitor *m, UserDefOneList ** obj, const char *name, Error **errp);
 432
 433     #endif
 434
 435 === scripts/qapi-commands.py ===
 436
 437 Used to generate the marshaling/dispatch functions for the commands defined
 438 in the schema. The following files are generated:
 439
 440 $(prefix)qmp-marshal.c: command marshal/dispatch functions for each
 441                         QMP command defined in the schema. Functions
 442                         generated by qapi-visit.py are used to
 443                         convert QObjects received from the wire into
 444                         function parameters, and uses the same
 445                         visitor functions to convert native C return
 446                         values to QObjects from transmission back
 447                         over the wire.
 448
 449 $(prefix)qmp-commands.h: Function prototypes for the QMP commands
 450                          specified in the schema.
 451
 452 Example:
 453
 454     $ cat qapi-generated/example-qmp-marshal.c
 455 [Uninteresting stuff omitted...]
 456
 457     static void qmp_marshal_output_my_command(UserDefOne * ret_in, QObject **ret_out, Error **errp)
 458     {
 459         Error *local_err = NULL;
 460         QmpOutputVisitor *mo = qmp_output_visitor_new();
 461         QapiDeallocVisitor *md;
 462         Visitor *v;
 463
 464         v = qmp_output_get_visitor(mo);
 465         visit_type_UserDefOne(v, &ret_in, "unused", &local_err);
 466         if (local_err) {
 467             goto out;
 468         }
 469         *ret_out = qmp_output_get_qobject(mo);
 470
 471     out:
 472         error_propagate(errp, local_err);
 473         qmp_output_visitor_cleanup(mo);
 474         md = qapi_dealloc_visitor_new();
 475         v = qapi_dealloc_get_visitor(md);
 476         visit_type_UserDefOne(v, &ret_in, "unused", NULL);
 477         qapi_dealloc_visitor_cleanup(md);
 478     }
 479
 480     static void qmp_marshal_input_my_command(QDict *args, QObject **ret, Error **errp)
 481     {
 482         Error *local_err = NULL;
 483         UserDefOne * retval = NULL;
 484         QmpInputVisitor *mi = qmp_input_visitor_new_strict(QOBJECT(args));
 485         QapiDeallocVisitor *md;
 486         Visitor *v;
 487         UserDefOne * arg1 = NULL;
 488
 489         v = qmp_input_get_visitor(mi);
 490         visit_type_UserDefOne(v, &arg1, "arg1", &local_err);
 491         if (local_err) {
 492             goto out;
 493         }
 494
 495         retval = qmp_my_command(arg1, &local_err);
 496         if (local_err) {
 497             goto out;
 498         }
 499
 500         qmp_marshal_output_my_command(retval, ret, &local_err);
 501
 502     out:
 503         error_propagate(errp, local_err);
 504         qmp_input_visitor_cleanup(mi);
 505         md = qapi_dealloc_visitor_new();
 506         v = qapi_dealloc_get_visitor(md);
 507         visit_type_UserDefOne(v, &arg1, "arg1", NULL);
 508         qapi_dealloc_visitor_cleanup(md);
 509         return;
 510     }
 511
 512     static void qmp_init_marshal(void)
 513     {
 514         qmp_register_command("my-command", qmp_marshal_input_my_command, QCO_NO_OPTIONS);
 515     }
 516
 517     qapi_init(qmp_init_marshal);
 518     $ cat qapi-generated/example-qmp-commands.h
 519 [Uninteresting stuff omitted...]
 520
 521     #ifndef EXAMPLE_QMP_COMMANDS_H
 522     #define EXAMPLE_QMP_COMMANDS_H
 523
 524     #include "example-qapi-types.h"
 525     #include "qapi/qmp/qdict.h"
 526     #include "qapi/error.h"
 527
 528     UserDefOne * qmp_my_command(UserDefOne * arg1, Error **errp);
 529
 530     #endif