ceph/src/jaegertracing/thrift/lib/cl/README.md

   1 Thrift Common Lisp Library
   2
   3 License
   4 =======
   5
   6 Licensed to the Apache Software Foundation (ASF) under one
   7 or more contributor license agreements. See the NOTICE file
   8 distributed with this work for additional information
   9 regarding copyright ownership. The ASF licenses this file
  10 to you under the Apache License, Version 2.0 (the
  11 "License"); you may not use this file except in compliance
  12 with the License. You may obtain a copy of the License at
  13
  14   http://www.apache.org/licenses/LICENSE-2.0
  15
  16 Unless required by applicable law or agreed to in writing,
  17 software distributed under the License is distributed on an
  18 "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
  19 KIND, either express or implied. See the License for the
  20 specific language governing permissions and limitations
  21 under the License.
  22
  23
  24
  25 Using Thrift with Common Lisp
  26 ============================
  27
  28  Thrift is a protocol and library for language-independent communication between cooperating
  29  processes. The communication takes the form of request and response messages, of which the forms
  30  are specified in advance throufh a shared interface definition. A Thrift definition file is translated
  31  into Lisp source files, which comprise several definitions:
  32
  33   * Three packages, one for the namespace of the implementation operators, and one each for request and
  34   response operators.
  35   * Various type definitions as implementations for Thrift typedef and enum definitions.
  36   * DEF-STRUCT and DEF-EXCEPTION forms for Thrift struct and exception definitions.
  37   * DEF-SERVICE forms for thrift service definitions.
  38
  39  Each service definition expands in a collection of generic function definitions. For each `op`
  40  in the service definition, two functions are defined
  41
  42   * `op`-request is defined for use by a client. It accepts an additional initial `protocol` argument,
  43     to act as the client proxy for the operation and mediate the interaction with a remote process
  44     through a Thrift-encoded transport stream.
  45   * `op`-response is defined for use by a server. It accepts a single `protocol` argument. A server
  46     uses it to decode the request message, invoke the base `op` function with the message arguments,
  47     encode and send the the result as a response, and handles exceptions.
  48
  49  The client interface is one operator
  50
  51   * `with-client (variable location) . body` : creates a connection in a dynamic context and closes it
  52     upon exit. The variable is bound to a client proxy stream/protocol instance, which wraps the
  53     base i/o stream - socket, file, etc, with an operators which implement the Thrift protocol
  54     and transport mechanisms.
  55
  56  The server interface combines server and service objects
  57
  58   * `serve (location service)` : accepts connections on the designated port and responds to
  59     requests of the service's operations.
  60
  61
  62 Building
  63 --------
  64
  65 The Thrift Common Lisp library is packaged as the ASDF[[1]] system `thrift`.
  66 It depends on the systems
  67
  68 * puri[[2]] : for the thrift uri class
  69 * closer-mop[[3]] : for class metadata
  70 * trivial-utf-8[[4]] : for string codecs
  71 * usocket[[5]] : for the socket transport
  72 * ieee-floats[[6]] : for conversion between ints and floats
  73 * trivial-gray-streams[[7]] : an abstraction layer for gray streams
  74 * alexandria[[8]] : handy utilities
  75
  76 The dependencies are bundled for local builds of tests and tutorial binaries -
  77 it is possible to use those bundles to load the library, too.
  78
  79 In order to build it, register those systems with ASDF and evaluate:
  80
  81     (asdf:load-system :thrift)
  82
  83 This will compile and load the Lisp compiler for Thrift definition files, the
  84 transport and protocol implementations, and the client and server interface
  85 functions. In order to use Thrift in an application, one must also author and/or
  86 load the interface definitions for the remote service.[[9]] If one is implementing a service,
  87 one must also define the actual functions to which Thrift is to act as the proxy
  88 interface. The remainder of this document follows the Thrift tutorial to illustrate how
  89 to perform the steps
  90
  91   * implement the service
  92   * translate the Thrift IDL
  93   * load the Lisp service interfaces
  94   * run a server for the service
  95   * use a client to access the service remotely
  96
  97 Note that, if one is to implement a new service, one will also need to author the
  98 IDL files, as there is no facility to generate them from a service implementation.
  99
 100
 101 Implement the Service
 102 ---------------------
 103
 104 The tutorial comprises serveral functions: `add`, `ping`, `zip`, and `calculate`.
 105 Each translated IDL file generates three packages for every service. In the case of
 106 the tutorial file, the relevant packages are:
 107
 108   * tutorial.calculator
 109   * tutorial.calculator-implementation
 110   * tutorial.calculator-response
 111
 112 This is to separate the request (generated), response (generated) and implementation
 113 (meant to be implemented by the programmer) functions for defined Thrift methods.
 114
 115 It is suggested to work in the `tutorial-implementation` package while implementing
 116 the services - it imports the `common-lisp` package, while the service-specific ones
 117 don't (to avoid conflicts between Thrift method names and function names in `common-lisp`).
 118
 119     ;; define the base operations
 120
 121     (in-package :tutorial-implementation)
 122
 123     (defun tutorial.calculator-implementation:add (num1 num2)
 124       (format t "~&Asked to add ~A and ~A." num1 num2)
 125       (+ num1 num2))
 126
 127     (defun tutorial.calculator-implementation:ping ()
 128       (print :ping))
 129
 130     (defun tutorial.calculator-implementation:zip ()
 131       (print :zip))
 132
 133     (defun tutorial.calculator-implementation:calculate (logid task)
 134       (calculate-op (work-op task) (work-num1 task) (work-num2 task)))
 135
 136     (defgeneric calculate-op (op arg1 arg2)
 137       (:method :around (op arg1 arg2)
 138         (let ((result (call-next-method)))
 139           (format t "~&Asked to calculate: ~d on  ~A and ~A = ~d." op arg1 arg2 result)
 140           result))
 141
 142       (:method ((op (eql operation.add)) arg1 arg2)
 143         (+ arg1 arg2))
 144       (:method ((op (eql operation.subtract)) arg1 arg2)
 145         (- arg1 arg2))
 146       (:method ((op (eql operation.multiply)) arg1 arg2)
 147         (* arg1 arg2))
 148       (:method ((op (eql operation.divide)) arg1 arg2)
 149         (/ arg1 arg2)))
 150
 151     (defun zip () (print 'zip))
 152
 153
 154 Translate the Thrift IDL
 155 ------------------------
 156
 157 IDL files employ the file extension `thrift`. In this case, there are two files to translate
 158   * `tutorial.thrift`
 159   * `shared.thrift`
 160 As the former includes the latter, one uses it to generate the interfaces:
 161
 162     $THRIFT/bin/thrift -r --gen cl $THRIFT/tutorial/tutorial.thrift
 163
 164 `-r` stands for recursion, while `--gen` lets one choose the language to translate to.
 165
 166
 167 Load the Lisp translated service interfaces
 168 -------------------------------------------
 169
 170 The translator generates three files for each IDL file. For example `tutorial-types.lisp`,
 171 `tutorial-vars.lisp` and an `.asd` file that can be used to load them both and pull in
 172 other includes (like `shared` within the tutorial) as dependencies.
 173
 174
 175 Run a Server for the Service
 176 ----------------------------
 177
 178 The actual service name, as specified in the `def-service` form in `tutorial.lisp`, is `calculator`.
 179 Each service definition defines a global variable with the service name and binds it to a
 180 service instance whch describes the operations.
 181
 182 In order to start a service, specify a location and the service instance.
 183
 184     (in-package :tutorial)
 185     (serve #u"thrift://127.0.0.1:9091" calculator)
 186
 187
 188 Use a Client to Access the Service Remotely
 189 -------------------------------------------
 190
 191
 192 [in some other process] run the client
 193
 194     (in-package :cl-user)
 195
 196     (macrolet ((show (form)
 197                  `(format *trace-output* "~%~s =>~{ ~s~}"
 198                           ',form
 199                           (multiple-value-list (ignore-errors ,form)))))
 200       (with-client (protocol #u"thrift://127.0.0.1:9091")
 201         (show (tutorial.calculator:ping protocol))
 202         (show (tutorial.calculator:add protocol 1 2))
 203         (show (tutorial.calculator:add protocol 1 4))
 204
 205         (let ((task (make-instance 'tutorial:work
 206                       :op operation.subtract :num1 15 :num2 10)))
 207           (show (tutorial.calculator:calculate protocol 1 task))
 208
 209           (setf (tutorial:work-op task) operation.divide
 210                 (tutorial:work-num1 task) 1
 211                 (tutorial:work-num2 task) 0)
 212           (show (tutorial.calculator:calculate protocol 1 task)))
 213
 214         (show (shared.shared-service:get-struct protocol 1))
 215
 216         (show (zip protocol))))
 217
 218 Issues
 219 ------
 220
 221 ### optional fields
 222  Where the IDL declares a field options, the def-struct form includes no
 223  initform for the slot and the encoding operator skips an unbound slot. This leave some ambiguity
 224  with bool fields.
 225
 226 ### instantiation protocol :
 227  struct classes are standard classes and exception classes are
 228  whatever the implementation prescribes. decoders apply make-struct to an initargs list.
 229  particularly at the service end, there are advantages to resourcing structs and decoding
 230  with direct side-effects on slot-values
 231
 232 ### maps:
 233  Maps are now represented as hash tables. As data through the call/reply interface is all statically
 234  typed, it is not necessary for the objects to themselves indicate the coding form. Association lists
 235  would be sufficient. As the key type is arbitrary, property lists offer no additional convenience:
 236  as `getf` operates with `eq` a new access interface would be necessary and they would not be
 237  available for function application.
 238
 239
 240  [1]: www.common-lisp.net/asdf
 241  [2]: http://github.com/lisp/com.b9.puri.ppcre
 242  [3]: www.common-lisp.net/closer-mop
 243  [4]: trivial-utf-8
 244  [5]: https://github.com/usocket/usocket
 245  [6]: https://github.com/marijnh/ieee-floats
 246  [7]: https://github.com/trivial-gray-streams/trivial-gray-streams
 247  [8]: https://gitlab.common-lisp.net/alexandria/alexandria
 248  [9]: http://wiki.apache.org/thrift/ThriftGeneration
 249
 250 * usocket[[5]] : for the socket transport
 251 * ieee-floats[[6]] : for conversion between ints and floats
 252 * trivial-gray-streams[[7]] : an abstraction layer for gray streams
 253 * alexandria[[8]] : handy utilities