]> git.proxmox.com Git - ceph.git/blob - ceph/doc/radosgw/lua-scripting.rst
import quincy beta 17.1.0
[ceph.git] / ceph / doc / radosgw / lua-scripting.rst
1 =============
2 Lua Scripting
3 =============
4
5 .. versionadded:: Pacific
6
7 .. contents::
8
9 This feature allows users to upload Lua scripts to different context in the radosgw. The two supported context are "preRequest" that will execute a script before the
10 operation was taken, and "postRequest" that will execute after each operation is taken. Script may be uploaded to address requests for users of a specific tenant.
11 The script can access fields in the request and modify some fields. All Lua language features can be used in the script.
12
13 By default, all lua standard libraries are available in the script, however, in order to allow for other lua modules to be used in the script, we support adding packages to an allowlist:
14
15 - All packages in the allowlist are being re-installed using the luarocks package manager on radosgw restart. Therefore a restart is needed for adding or removing of packages to take effect
16 - To add a package that contains C source code that needs to be compiled, use the `--allow-compilation` flag. In this case a C compiler needs to be available on the host
17 - Lua packages are installed in, and used from, a directory local to the radosgw. Meaning that lua packages in the allowlist are separated from any lua packages available on the host.
18 By default, this directory would be `/tmp/luarocks/<entity name>`. Its prefix part (`/tmp/luarocks/`) could be set to a different location via the `rgw_luarocks_location` configuration parameter.
19 Note that this parameter should not be set to one of the default locations where luarocks install packages (e.g. `$HOME/.luarocks`, `/usr/lib64/lua`, `/usr/share/lua`)
20
21
22 .. toctree::
23 :maxdepth: 1
24
25
26 Script Management via CLI
27 -------------------------
28
29 To upload a script:
30
31 ::
32
33 # radosgw-admin script put --infile={lua-file} --context={preRequest|postRequest} [--tenant={tenant-name}]
34
35
36 To print the content of the script to standard output:
37
38 ::
39
40 # radosgw-admin script get --context={preRequest|postRequest} [--tenant={tenant-name}]
41
42
43 To remove the script:
44
45 ::
46
47 # radosgw-admin script rm --context={preRequest|postRequest} [--tenant={tenant-name}]
48
49
50 Package Management via CLI
51 --------------------------
52
53 To add a package to the allowlist:
54
55 ::
56
57 # radosgw-admin script-package add --package={package name} [--allow-compilation]
58
59
60 To add a specific version of a package to the allowlist:
61
62 ::
63
64 # radosgw-admin script-package add --package='{package name} {package version}' [--allow-compilation]
65
66
67 * When adding a different version of a package which already exists in the list, the newly
68 added version will override the existing one.
69
70 * When adding a package without a version specified, the latest version of the package
71 will be added.
72
73
74 To remove a package from the allowlist:
75
76 ::
77
78 # radosgw-admin script-package rm --package={package name}
79
80
81 To remove a specific version of a package from the allowlist:
82
83 ::
84
85 # radosgw-admin script-package rm --package='{package name} {package version}'
86
87
88 * When removing a package without a version specified, any existing versions of the
89 package will be removed.
90
91
92 To print the list of packages in the allowlist:
93
94 ::
95
96 # radosgw-admin script-package list
97
98
99 Context Free Functions
100 ----------------------
101 Debug Log
102 ~~~~~~~~~
103 The ``RGWDebugLog()`` function accepts a string and prints it to the debug log with priority 20.
104 Each log message is prefixed ``Lua INFO:``. This function has no return value.
105
106 Request Fields
107 -----------------
108
109 .. warning:: This feature is experimental. Fields may be removed or renamed in the future.
110
111 .. note::
112
113 - Although Lua is a case-sensitive language, field names provided by the radosgw are case-insensitive. Function names remain case-sensitive.
114 - Fields marked "optional" can have a nil value.
115 - Fields marked as "iterable" can be used by the pairs() function and with the # length operator.
116 - All table fields can be used with the bracket operator ``[]``.
117 - ``time`` fields are strings with the following format: ``%Y-%m-%d %H:%M:%S``.
118
119
120 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
121 | Field | Type | Description | Iterable | Writeable | Optional |
122 +====================================================+==========+==============================================================+==========+===========+==========+
123 | ``Request.RGWOp`` | string | radosgw operation | no | no | no |
124 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
125 | ``Request.DecodedURI`` | string | decoded URI | no | no | no |
126 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
127 | ``Request.ContentLength`` | integer | size of the request | no | no | no |
128 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
129 | ``Request.GenericAttributes`` | table | string to string generic attributes map | yes | no | no |
130 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
131 | ``Request.Response`` | table | response to the request | no | no | no |
132 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
133 | ``Request.Response.HTTPStatusCode`` | integer | HTTP status code | no | yes | no |
134 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
135 | ``Request.Response.HTTPStatus`` | string | HTTP status text | no | yes | no |
136 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
137 | ``Request.Response.RGWCode`` | integer | radosgw error code | no | yes | no |
138 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
139 | ``Request.Response.Message`` | string | response message | no | yes | no |
140 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
141 | ``Request.SwiftAccountName`` | string | swift account name | no | no | yes |
142 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
143 | ``Request.Bucket`` | table | info on the bucket | no | no | yes |
144 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
145 | ``Request.Bucket.Tenant`` | string | tenant of the bucket | no | no | no |
146 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
147 | ``Request.Bucket.Name`` | string | bucket name | no | no | no |
148 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
149 | ``Request.Bucket.Marker`` | string | bucket marker (initial id) | no | no | no |
150 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
151 | ``Request.Bucket.Id`` | string | bucket id | no | no | no |
152 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
153 | ``Request.Bucket.Count`` | integer | number of objects in the bucket | no | no | no |
154 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
155 | ``Request.Bucket.Size`` | integer | total size of objects in the bucket | no | no | no |
156 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
157 | ``Request.Bucket.ZoneGroupId`` | string | zone group of the bucket | no | no | no |
158 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
159 | ``Request.Bucket.CreationTime`` | time | creation time of the bucket | no | no | no |
160 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
161 | ``Request.Bucket.MTime`` | time | modification time of the bucket | no | no | no |
162 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
163 | ``Request.Bucket.Quota`` | table | bucket quota | no | no | no |
164 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
165 | ``Request.Bucket.Quota.MaxSize`` | integer | bucket quota max size | no | no | no |
166 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
167 | ``Request.Bucket.Quota.MaxObjects`` | integer | bucket quota max number of objects | no | no | no |
168 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
169 | ``Reques.Bucket.Quota.Enabled`` | boolean | bucket quota is enabled | no | no | no |
170 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
171 | ``Request.Bucket.Quota.Rounded`` | boolean | bucket quota is rounded to 4K | no | no | no |
172 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
173 | ``Request.Bucket.PlacementRule`` | table | bucket placement rule | no | no | no |
174 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
175 | ``Request.Bucket.PlacementRule.Name`` | string | bucket placement rule name | no | no | no |
176 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
177 | ``Request.Bucket.PlacementRule.StorageClass`` | string | bucket placement rule storage class | no | no | no |
178 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
179 | ``Request.Bucket.User`` | table | bucket owner | no | no | no |
180 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
181 | ``Request.Bucket.User.Tenant`` | string | bucket owner tenant | no | no | no |
182 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
183 | ``Request.Bucket.User.Id`` | string | bucket owner id | no | no | no |
184 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
185 | ``Request.Object`` | table | info on the object | no | no | yes |
186 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
187 | ``Request.Object.Name`` | string | object name | no | no | no |
188 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
189 | ``Request.Object.Instance`` | string | object version | no | no | no |
190 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
191 | ``Request.Object.Id`` | string | object id | no | no | no |
192 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
193 | ``Request.Object.Size`` | integer | object size | no | no | no |
194 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
195 | ``Request.Object.MTime`` | time | object mtime | no | no | no |
196 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
197 | ``Request.CopyFrom`` | table | information on copy operation | no | no | yes |
198 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
199 | ``Request.CopyFrom.Tenant`` | string | tenant of the object copied from | no | no | no |
200 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
201 | ``Request.CopyFrom.Bucket`` | string | bucket of the object copied from | no | no | no |
202 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
203 | ``Request.CopyFrom.Object`` | table | object copied from. See: ``Request.Object`` | no | no | yes |
204 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
205 | ``Request.ObjectOwner`` | table | object owner | no | no | no |
206 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
207 | ``Request.ObjectOwner.DisplayName`` | string | object owner display name | no | no | no |
208 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
209 | ``Request.ObjectOwner.User`` | table | object user. See: ``Request.Bucket.User`` | no | no | no |
210 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
211 | ``Request.ZoneGroup.Name`` | string | name of zone group | no | no | no |
212 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
213 | ``Request.ZoneGroup.Endpoint`` | string | endpoint of zone group | no | no | no |
214 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
215 | ``Request.UserAcl`` | table | user ACL | no | no | no |
216 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
217 | ``Request.UserAcl.Owner`` | table | user ACL owner. See: ``Request.ObjectOwner`` | no | no | no |
218 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
219 | ``Request.UserAcl.Grants`` | table | user ACL map of string to grant | yes | no | no |
220 | | | note: grants without an Id are not presented when iterated | | | |
221 | | | and only one of them can be accessed via brackets | | | |
222 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
223 | ``Request.UserAcl.Grants["<name>"]`` | table | user ACL grant | no | no | no |
224 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
225 | ``Request.UserAcl.Grants["<name>"].Type`` | integer | user ACL grant type | no | no | no |
226 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
227 | ``Request.UserAcl.Grants["<name>"].User`` | table | user ACL grant user | no | no | no |
228 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
229 | ``Request.UserAcl.Grants["<name>"].User.Tenant`` | table | user ACL grant user tenant | no | no | no |
230 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
231 | ``Request.UserAcl.Grants["<name>"].User.Id`` | table | user ACL grant user id | no | no | no |
232 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
233 | ``Request.UserAcl.Grants["<name>"].GroupType`` | integer | user ACL grant group type | no | no | no |
234 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
235 | ``Request.UserAcl.Grants["<name>"].Referer`` | string | user ACL grant referer | no | no | no |
236 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
237 | ``Request.BucketAcl`` | table | bucket ACL. See: ``Request.UserAcl`` | no | no | no |
238 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
239 | ``Request.ObjectAcl`` | table | object ACL. See: ``Request.UserAcl`` | no | no | no |
240 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
241 | ``Request.Environment`` | table | string to string environment map | yes | no | no |
242 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
243 | ``Request.Policy`` | table | policy | no | no | yes |
244 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
245 | ``Request.Policy.Text`` | string | policy text | no | no | no |
246 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
247 | ``Request.Policy.Id`` | string | policy Id | no | no | yes |
248 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
249 | ``Request.Policy.Statements`` | table | list of string statements | yes | no | no |
250 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
251 | ``Request.UserPolicies`` | table | list of user policies | yes | no | no |
252 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
253 | ``Request.UserPolicies[<index>]`` | table | user policy. See: ``Request.Policy`` | no | no | no |
254 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
255 | ``Request.RGWId`` | string | radosgw host id: ``<host>-<zone>-<zonegroup>`` | no | no | no |
256 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
257 | ``Request.HTTP`` | table | HTTP header | no | no | no |
258 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
259 | ``Request.HTTP.Parameters`` | table | string to string parameter map | yes | no | no |
260 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
261 | ``Request.HTTP.Resources`` | table | string to string resource map | yes | no | no |
262 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
263 | ``Request.HTTP.Metadata`` | table | string to string metadata map | yes | yes | no |
264 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
265 | ``Request.HTTP.StorageClass`` | string | storage class | no | yes | yes |
266 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
267 | ``Request.HTTP.Host`` | string | host name | no | no | no |
268 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
269 | ``Request.HTTP.Method`` | string | HTTP method | no | no | no |
270 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
271 | ``Request.HTTP.URI`` | string | URI | no | no | no |
272 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
273 | ``Request.HTTP.QueryString`` | string | HTTP query string | no | no | no |
274 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
275 | ``Request.HTTP.Domain`` | string | domain name | no | no | no |
276 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
277 | ``Request.Time`` | time | request time | no | no | no |
278 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
279 | ``Request.Dialect`` | string | "S3" or "Swift" | no | no | no |
280 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
281 | ``Request.Id`` | string | request Id | no | no | no |
282 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
283 | ``Request.TransactionId`` | string | transaction Id | no | no | no |
284 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
285 | ``Request.Tags`` | table | object tags map | yes | no | no |
286 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
287 | ``Request.User`` | table | user that triggered the request | no | no | no |
288 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
289 | ``Request.User.Tenant`` | string | triggering user tenant | no | no | no |
290 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
291 | ``Request.User.Id`` | string | triggering user id | no | no | no |
292 +----------------------------------------------------+----------+--------------------------------------------------------------+----------+-----------+----------+
293
294 Request Functions
295 --------------------
296 Operations Log
297 ~~~~~~~~~~~~~~
298 The ``Request.Log()`` function prints the requests into the operations log. This function has no parameters. It returns 0 for success and an error code if it fails.
299
300 Lua Code Samples
301 ----------------
302 - Print information on source and destination objects in case of copy:
303
304 .. code-block:: lua
305
306 function print_object(object)
307 RGWDebugLog(" Name: " .. object.Name)
308 RGWDebugLog(" Instance: " .. object.Instance)
309 RGWDebugLog(" Id: " .. object.Id)
310 RGWDebugLog(" Size: " .. object.Size)
311 RGWDebugLog(" MTime: " .. object.MTime)
312 end
313
314 if Request.CopyFrom and Request.Object and Request.CopyFrom.Object then
315 RGWDebugLog("copy from object:")
316 print_object(Request.CopyFrom.Object)
317 RGWDebugLog("to object:")
318 print_object(Request.Object)
319 end
320
321 - Print ACLs via a "generic function":
322
323 .. code-block:: lua
324
325 function print_owner(owner)
326 RGWDebugLog("Owner:")
327 RGWDebugLog(" Display Name: " .. owner.DisplayName)
328 RGWDebugLog(" Id: " .. owner.User.Id)
329 RGWDebugLog(" Tenant: " .. owner.User.Tenant)
330 end
331
332 function print_acl(acl_type)
333 index = acl_type .. "ACL"
334 acl = Request[index]
335 if acl then
336 RGWDebugLog(acl_type .. "ACL Owner")
337 print_owner(acl.Owner)
338 RGWDebugLog(" there are " .. #acl.Grants .. " grant for owner")
339 for k,v in pairs(acl.Grants) do
340 RGWDebugLog(" Grant Key: " .. k)
341 RGWDebugLog(" Grant Type: " .. v.Type)
342 RGWDebugLog(" Grant Group Type: " .. v.GroupType)
343 RGWDebugLog(" Grant Referer: " .. v.Referer)
344 RGWDebugLog(" Grant User Tenant: " .. v.User.Tenant)
345 RGWDebugLog(" Grant User Id: " .. v.User.Id)
346 end
347 else
348 RGWDebugLog("no " .. acl_type .. " ACL in request: " .. Request.Id)
349 end
350 end
351
352 print_acl("User")
353 print_acl("Bucket")
354 print_acl("Object")
355
356 - Use of operations log only in case of errors:
357
358 .. code-block:: lua
359
360 if Request.Response.HTTPStatusCode ~= 200 then
361 RGWDebugLog("request is bad, use ops log")
362 rc = Request.Log()
363 RGWDebugLog("ops log return code: " .. rc)
364 end
365
366 - Set values into the error message:
367
368 .. code-block:: lua
369
370 if Request.Response.HTTPStatusCode == 500 then
371 Request.Response.Message = "<Message> something bad happened :-( </Message>"
372 end
373
374 - Add metadata to objects that was not originally sent by the client:
375
376 In the `preRequest` context we should add:
377
378 .. code-block:: lua
379
380 if Request.RGWOp == 'put_obj' then
381 Request.HTTP.Metadata["x-amz-meta-mydata"] = "my value"
382 end
383
384 In the `postRequest` context we look at the metadata:
385
386 .. code-block:: lua
387
388 RGWDebugLog("number of metadata entries is: " .. #Request.HTTP.Metadata)
389 for k, v in pairs(Request.HTTP.Metadata) do
390 RGWDebugLog("key=" .. k .. ", " .. "value=" .. v)
391 end
392
393 - Use modules to create Unix socket based, JSON encoded, "access log":
394
395 First we should add the following packages to the allowlist:
396
397 ::
398
399 # radosgw-admin script-package add --package=luajson
400 # radosgw-admin script-package add --package=luasocket --allow-compilation
401
402
403 Then, do a restart for the radosgw and upload the following script to the `postRequest` context:
404
405 .. code-block:: lua
406
407 if Request.RGWOp == "get_obj" then
408 local json = require("json")
409 local socket = require("socket")
410 local unix = require("socket.unix")
411 local s = assert(unix())
412 E = {}
413
414 msg = {bucket = (Request.Bucket or (Request.CopyFrom or E).Bucket).Name,
415 time = Request.Time,
416 operation = Request.RGWOp,
417 http_status = Request.Response.HTTPStatusCode,
418 error_code = Request.Response.HTTPStatus,
419 object_size = Request.Object.Size,
420 trans_id = Request.TransactionId}
421
422 assert(s:connect("/tmp/socket"))
423 assert(s:send(json.encode(msg).."\n"))
424 assert(s:close())
425 end
426