]>
Commit | Line | Data |
---|---|---|
f67539c2 TL |
1 | Thrift Common Lisp Library |
2 | ||
3 | License | |
4 | ======= | |
5 | ||
6 | Licensed to the Apache Software Foundation (ASF) under one | |
7 | or more contributor license agreements. See the NOTICE file | |
8 | distributed with this work for additional information | |
9 | regarding copyright ownership. The ASF licenses this file | |
10 | to you under the Apache License, Version 2.0 (the | |
11 | "License"); you may not use this file except in compliance | |
12 | with the License. You may obtain a copy of the License at | |
13 | ||
14 | http://www.apache.org/licenses/LICENSE-2.0 | |
15 | ||
16 | Unless required by applicable law or agreed to in writing, | |
17 | software distributed under the License is distributed on an | |
18 | "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY | |
19 | KIND, either express or implied. See the License for the | |
20 | specific language governing permissions and limitations | |
21 | under the License. | |
22 | ||
23 | ||
24 | ||
25 | Using Thrift with Common Lisp | |
26 | ============================ | |
27 | ||
28 | Thrift is a protocol and library for language-independent communication between cooperating | |
29 | processes. The communication takes the form of request and response messages, of which the forms | |
30 | are specified in advance throufh a shared interface definition. A Thrift definition file is translated | |
31 | into Lisp source files, which comprise several definitions: | |
32 | ||
33 | * Three packages, one for the namespace of the implementation operators, and one each for request and | |
34 | response operators. | |
35 | * Various type definitions as implementations for Thrift typedef and enum definitions. | |
36 | * DEF-STRUCT and DEF-EXCEPTION forms for Thrift struct and exception definitions. | |
37 | * DEF-SERVICE forms for thrift service definitions. | |
38 | ||
39 | Each service definition expands in a collection of generic function definitions. For each `op` | |
40 | in the service definition, two functions are defined | |
41 | ||
42 | * `op`-request is defined for use by a client. It accepts an additional initial `protocol` argument, | |
43 | to act as the client proxy for the operation and mediate the interaction with a remote process | |
44 | through a Thrift-encoded transport stream. | |
45 | * `op`-response is defined for use by a server. It accepts a single `protocol` argument. A server | |
46 | uses it to decode the request message, invoke the base `op` function with the message arguments, | |
47 | encode and send the the result as a response, and handles exceptions. | |
48 | ||
49 | The client interface is one operator | |
50 | ||
51 | * `with-client (variable location) . body` : creates a connection in a dynamic context and closes it | |
52 | upon exit. The variable is bound to a client proxy stream/protocol instance, which wraps the | |
53 | base i/o stream - socket, file, etc, with an operators which implement the Thrift protocol | |
54 | and transport mechanisms. | |
55 | ||
56 | The server interface combines server and service objects | |
57 | ||
58 | * `serve (location service)` : accepts connections on the designated port and responds to | |
59 | requests of the service's operations. | |
60 | ||
61 | ||
62 | Building | |
63 | -------- | |
64 | ||
65 | The Thrift Common Lisp library is packaged as the ASDF[[1]] system `thrift`. | |
66 | It depends on the systems | |
67 | ||
68 | * puri[[2]] : for the thrift uri class | |
69 | * closer-mop[[3]] : for class metadata | |
70 | * trivial-utf-8[[4]] : for string codecs | |
71 | * usocket[[5]] : for the socket transport | |
72 | * ieee-floats[[6]] : for conversion between ints and floats | |
73 | * trivial-gray-streams[[7]] : an abstraction layer for gray streams | |
74 | * alexandria[[8]] : handy utilities | |
75 | ||
76 | The dependencies are bundled for local builds of tests and tutorial binaries - | |
77 | it is possible to use those bundles to load the library, too. | |
78 | ||
79 | In order to build it, register those systems with ASDF and evaluate: | |
80 | ||
81 | (asdf:load-system :thrift) | |
82 | ||
83 | This will compile and load the Lisp compiler for Thrift definition files, the | |
84 | transport and protocol implementations, and the client and server interface | |
85 | functions. In order to use Thrift in an application, one must also author and/or | |
86 | load the interface definitions for the remote service.[[9]] If one is implementing a service, | |
87 | one must also define the actual functions to which Thrift is to act as the proxy | |
88 | interface. The remainder of this document follows the Thrift tutorial to illustrate how | |
89 | to perform the steps | |
90 | ||
91 | * implement the service | |
92 | * translate the Thrift IDL | |
93 | * load the Lisp service interfaces | |
94 | * run a server for the service | |
95 | * use a client to access the service remotely | |
96 | ||
97 | Note that, if one is to implement a new service, one will also need to author the | |
98 | IDL files, as there is no facility to generate them from a service implementation. | |
99 | ||
100 | ||
101 | Implement the Service | |
102 | --------------------- | |
103 | ||
104 | The tutorial comprises serveral functions: `add`, `ping`, `zip`, and `calculate`. | |
105 | Each translated IDL file generates three packages for every service. In the case of | |
106 | the tutorial file, the relevant packages are: | |
107 | ||
108 | * tutorial.calculator | |
109 | * tutorial.calculator-implementation | |
110 | * tutorial.calculator-response | |
111 | ||
112 | This is to separate the request (generated), response (generated) and implementation | |
113 | (meant to be implemented by the programmer) functions for defined Thrift methods. | |
114 | ||
115 | It is suggested to work in the `tutorial-implementation` package while implementing | |
116 | the services - it imports the `common-lisp` package, while the service-specific ones | |
117 | don't (to avoid conflicts between Thrift method names and function names in `common-lisp`). | |
118 | ||
119 | ;; define the base operations | |
120 | ||
121 | (in-package :tutorial-implementation) | |
122 | ||
123 | (defun tutorial.calculator-implementation:add (num1 num2) | |
124 | (format t "~&Asked to add ~A and ~A." num1 num2) | |
125 | (+ num1 num2)) | |
126 | ||
127 | (defun tutorial.calculator-implementation:ping () | |
128 | (print :ping)) | |
129 | ||
130 | (defun tutorial.calculator-implementation:zip () | |
131 | (print :zip)) | |
132 | ||
133 | (defun tutorial.calculator-implementation:calculate (logid task) | |
134 | (calculate-op (work-op task) (work-num1 task) (work-num2 task))) | |
135 | ||
136 | (defgeneric calculate-op (op arg1 arg2) | |
137 | (:method :around (op arg1 arg2) | |
138 | (let ((result (call-next-method))) | |
139 | (format t "~&Asked to calculate: ~d on ~A and ~A = ~d." op arg1 arg2 result) | |
140 | result)) | |
141 | ||
142 | (:method ((op (eql operation.add)) arg1 arg2) | |
143 | (+ arg1 arg2)) | |
144 | (:method ((op (eql operation.subtract)) arg1 arg2) | |
145 | (- arg1 arg2)) | |
146 | (:method ((op (eql operation.multiply)) arg1 arg2) | |
147 | (* arg1 arg2)) | |
148 | (:method ((op (eql operation.divide)) arg1 arg2) | |
149 | (/ arg1 arg2))) | |
150 | ||
151 | (defun zip () (print 'zip)) | |
152 | ||
153 | ||
154 | Translate the Thrift IDL | |
155 | ------------------------ | |
156 | ||
157 | IDL files employ the file extension `thrift`. In this case, there are two files to translate | |
158 | * `tutorial.thrift` | |
159 | * `shared.thrift` | |
160 | As the former includes the latter, one uses it to generate the interfaces: | |
161 | ||
162 | $THRIFT/bin/thrift -r --gen cl $THRIFT/tutorial/tutorial.thrift | |
163 | ||
164 | `-r` stands for recursion, while `--gen` lets one choose the language to translate to. | |
165 | ||
166 | ||
167 | Load the Lisp translated service interfaces | |
168 | ------------------------------------------- | |
169 | ||
170 | The translator generates three files for each IDL file. For example `tutorial-types.lisp`, | |
171 | `tutorial-vars.lisp` and an `.asd` file that can be used to load them both and pull in | |
172 | other includes (like `shared` within the tutorial) as dependencies. | |
173 | ||
174 | ||
175 | Run a Server for the Service | |
176 | ---------------------------- | |
177 | ||
178 | The actual service name, as specified in the `def-service` form in `tutorial.lisp`, is `calculator`. | |
179 | Each service definition defines a global variable with the service name and binds it to a | |
180 | service instance whch describes the operations. | |
181 | ||
182 | In order to start a service, specify a location and the service instance. | |
183 | ||
184 | (in-package :tutorial) | |
185 | (serve #u"thrift://127.0.0.1:9091" calculator) | |
186 | ||
187 | ||
188 | Use a Client to Access the Service Remotely | |
189 | ------------------------------------------- | |
190 | ||
191 | ||
192 | [in some other process] run the client | |
193 | ||
194 | (in-package :cl-user) | |
195 | ||
196 | (macrolet ((show (form) | |
197 | `(format *trace-output* "~%~s =>~{ ~s~}" | |
198 | ',form | |
199 | (multiple-value-list (ignore-errors ,form))))) | |
200 | (with-client (protocol #u"thrift://127.0.0.1:9091") | |
201 | (show (tutorial.calculator:ping protocol)) | |
202 | (show (tutorial.calculator:add protocol 1 2)) | |
203 | (show (tutorial.calculator:add protocol 1 4)) | |
204 | ||
205 | (let ((task (make-instance 'tutorial:work | |
206 | :op operation.subtract :num1 15 :num2 10))) | |
207 | (show (tutorial.calculator:calculate protocol 1 task)) | |
208 | ||
209 | (setf (tutorial:work-op task) operation.divide | |
210 | (tutorial:work-num1 task) 1 | |
211 | (tutorial:work-num2 task) 0) | |
212 | (show (tutorial.calculator:calculate protocol 1 task))) | |
213 | ||
214 | (show (shared.shared-service:get-struct protocol 1)) | |
215 | ||
216 | (show (zip protocol)))) | |
217 | ||
218 | Issues | |
219 | ------ | |
220 | ||
221 | ### optional fields | |
222 | Where the IDL declares a field options, the def-struct form includes no | |
223 | initform for the slot and the encoding operator skips an unbound slot. This leave some ambiguity | |
224 | with bool fields. | |
225 | ||
226 | ### instantiation protocol : | |
227 | struct classes are standard classes and exception classes are | |
228 | whatever the implementation prescribes. decoders apply make-struct to an initargs list. | |
229 | particularly at the service end, there are advantages to resourcing structs and decoding | |
230 | with direct side-effects on slot-values | |
231 | ||
232 | ### maps: | |
233 | Maps are now represented as hash tables. As data through the call/reply interface is all statically | |
234 | typed, it is not necessary for the objects to themselves indicate the coding form. Association lists | |
235 | would be sufficient. As the key type is arbitrary, property lists offer no additional convenience: | |
236 | as `getf` operates with `eq` a new access interface would be necessary and they would not be | |
237 | available for function application. | |
238 | ||
239 | ||
240 | [1]: www.common-lisp.net/asdf | |
241 | [2]: http://github.com/lisp/com.b9.puri.ppcre | |
242 | [3]: www.common-lisp.net/closer-mop | |
243 | [4]: trivial-utf-8 | |
244 | [5]: https://github.com/usocket/usocket | |
245 | [6]: https://github.com/marijnh/ieee-floats | |
246 | [7]: https://github.com/trivial-gray-streams/trivial-gray-streams | |
247 | [8]: https://gitlab.common-lisp.net/alexandria/alexandria | |
248 | [9]: http://wiki.apache.org/thrift/ThriftGeneration | |
249 | ||
250 | * usocket[[5]] : for the socket transport | |
251 | * ieee-floats[[6]] : for conversion between ints and floats | |
252 | * trivial-gray-streams[[7]] : an abstraction layer for gray streams | |
253 | * alexandria[[8]] : handy utilities |