]>
Commit | Line | Data |
---|---|---|
eafe8130 | 1 | ================== |
11fdf7f2 | 2 | PubSub Sync Module |
eafe8130 | 3 | ================== |
11fdf7f2 TL |
4 | |
5 | .. versionadded:: Nautilus | |
6 | ||
eafe8130 TL |
7 | .. contents:: |
8 | ||
11fdf7f2 | 9 | This sync module provides a publish and subscribe mechanism for the object store modification |
eafe8130 | 10 | events. Events are published into predefined topics. Topics can be subscribed to, and events |
11fdf7f2 | 11 | can be pulled from them. Events need to be acked. Also, events will expire and disappear |
eafe8130 TL |
12 | after a period of time. |
13 | ||
9f95a23c TL |
14 | A push notification mechanism exists too, currently supporting HTTP, |
15 | AMQP0.9.1 and Kafka endpoints. In this case, the events are pushed to an endpoint on top of storing them in Ceph. If events should only be pushed to an endpoint | |
eafe8130 | 16 | and do not need to be stored in Ceph, the `Bucket Notification`_ mechanism should be used instead of pubsub sync module. |
11fdf7f2 TL |
17 | |
18 | A user can create different topics. A topic entity is defined by its user and its name. A | |
19 | user can only manage its own topics, and can only subscribe to events published by buckets | |
20 | it owns. | |
21 | ||
eafe8130 TL |
22 | In order to publish events for specific bucket a notification entity needs to be created. A |
23 | notification can be created on a subset of event types, or for all event types (default). | |
24 | There can be multiple notifications for any specific topic, and the same topic could be used for multiple notifications. | |
11fdf7f2 TL |
25 | |
26 | A subscription to a topic can also be defined. There can be multiple subscriptions for any | |
27 | specific topic. | |
28 | ||
eafe8130 TL |
29 | REST API has been defined to provide configuration and control interfaces for the pubsub |
30 | mechanisms. This API has two flavors, one is S3-compatible and one is not. The two flavors can be used | |
31 | together, although it is recommended to use the S3-compatible one. | |
32 | The S3-compatible API is similar to the one used in the bucket notification mechanism. | |
11fdf7f2 | 33 | |
eafe8130 TL |
34 | Events are stored as RGW objects in a special bucket, under a special user. Events cannot |
35 | be accessed directly, but need to be pulled and acked using the new REST API. | |
11fdf7f2 | 36 | |
eafe8130 TL |
37 | .. toctree:: |
38 | :maxdepth: 1 | |
11fdf7f2 | 39 | |
eafe8130 TL |
40 | S3 Bucket Notification Compatibility <s3-notification-compatibility> |
41 | ||
42 | PubSub Zone Configuration | |
43 | ------------------------- | |
11fdf7f2 | 44 | |
eafe8130 | 45 | The pubsub sync module requires the creation of a new zone in a `Multisite`_ environment. |
9f95a23c TL |
46 | First, a master zone must exist (see: :ref:`master-zone-label`), |
47 | then a secondary zone should be created (see :ref:`secondary-zone-label`). | |
eafe8130 | 48 | In the creation of the secondary zone, its tier type must be set to ``pubsub``: |
11fdf7f2 TL |
49 | |
50 | :: | |
51 | ||
eafe8130 TL |
52 | # radosgw-admin zone create --rgw-zonegroup={zone-group-name} \ |
53 | --rgw-zone={zone-name} \ | |
54 | --endpoints={http://fqdn}[,{http://fqdn}] \ | |
55 | --sync-from-all=0 \ | |
56 | --sync-from={master-zone-name} \ | |
57 | --tier-type=pubsub | |
11fdf7f2 | 58 | |
11fdf7f2 | 59 | |
eafe8130 TL |
60 | PubSub Zone Configuration Parameters |
61 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
62 | ||
63 | :: | |
11fdf7f2 | 64 | |
eafe8130 TL |
65 | { |
66 | "tenant": <tenant>, # default: <empty> | |
67 | "uid": <uid>, # default: "pubsub" | |
68 | "data_bucket_prefix": <prefix> # default: "pubsub-" | |
69 | "data_oid_prefix": <prefix> # | |
70 | "events_retention_days": <days> # default: 7 | |
71 | } | |
11fdf7f2 TL |
72 | |
73 | * ``tenant`` (string) | |
74 | ||
75 | The tenant of the pubsub control user. | |
76 | ||
77 | * ``uid`` (string) | |
78 | ||
79 | The uid of the pubsub control user. | |
80 | ||
81 | * ``data_bucket_prefix`` (string) | |
82 | ||
83 | The prefix of the bucket name that will be created to store events for specific topic. | |
84 | ||
85 | * ``data_oid_prefix`` (string) | |
86 | ||
87 | The oid prefix for the stored events. | |
88 | ||
89 | * ``events_retention_days`` (integer) | |
90 | ||
91 | How many days to keep events that weren't acked. | |
92 | ||
eafe8130 TL |
93 | Configuring Parameters via CLI |
94 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
11fdf7f2 | 95 | |
eafe8130 | 96 | The tier configuration could be set using the following command: |
11fdf7f2 TL |
97 | |
98 | :: | |
99 | ||
eafe8130 | 100 | # radosgw-admin zone modify --rgw-zonegroup={zone-group-name} \ |
11fdf7f2 TL |
101 | --rgw-zone={zone-name} \ |
102 | --tier-config={key}={val}[,{key}={val}] | |
103 | ||
eafe8130 TL |
104 | Where the ``key`` in the configuration specifies the configuration variable that needs to be updated (from the list above), and |
105 | the ``val`` specifies its new value. For example, setting the pubsub control user ``uid`` to ``user_ps``: | |
11fdf7f2 TL |
106 | |
107 | :: | |
108 | ||
eafe8130 | 109 | # radosgw-admin zone modify --rgw-zonegroup={zone-group-name} \ |
11fdf7f2 TL |
110 | --rgw-zone={zone-name} \ |
111 | --tier-config=uid=pubsub | |
112 | ||
11fdf7f2 TL |
113 | A configuration field can be removed by using ``--tier-config-rm={key}``. |
114 | ||
115 | PubSub Performance Stats | |
116 | ------------------------- | |
eafe8130 TL |
117 | Same counters are shared between the pubsub sync module and the notification mechanism. |
118 | ||
119 | - ``pubsub_event_triggered``: running counter of events with at lease one topic associated with them | |
120 | - ``pubsub_event_lost``: running counter of events that had topics and subscriptions associated with them but that were not stored or pushed to any of the subscriptions | |
121 | - ``pubsub_store_ok``: running counter, for all subscriptions, of stored events | |
122 | - ``pubsub_store_fail``: running counter, for all subscriptions, of events failed to be stored | |
123 | - ``pubsub_push_ok``: running counter, for all subscriptions, of events successfully pushed to their endpoint | |
124 | - ``pubsub_push_fail``: running counter, for all subscriptions, of events failed to be pushed to their endpoint | |
125 | - ``pubsub_push_pending``: gauge value of events pushed to an endpoint but not acked or nacked yet | |
11fdf7f2 | 126 | |
eafe8130 TL |
127 | .. note:: |
128 | ||
129 | ``pubsub_event_triggered`` and ``pubsub_event_lost`` are incremented per event, while: | |
130 | ``pubsub_store_ok``, ``pubsub_store_fail``, ``pubsub_push_ok``, ``pubsub_push_fail``, are incremented per store/push action on each subscriptions. | |
11fdf7f2 TL |
131 | |
132 | PubSub REST API | |
eafe8130 | 133 | --------------- |
11fdf7f2 | 134 | |
eafe8130 | 135 | .. tip:: PubSub REST calls, and only them, should be sent to an RGW which belong to a PubSub zone |
11fdf7f2 TL |
136 | |
137 | Topics | |
138 | ~~~~~~ | |
eafe8130 | 139 | |
11fdf7f2 | 140 | Create a Topic |
eafe8130 TL |
141 | `````````````` |
142 | ||
143 | This will create a new topic. Topic creation is needed both for both flavors of the API. | |
144 | Optionally the topic could be provided with push endpoint parameters that would be used later | |
145 | when an S3-compatible notification is created. | |
146 | Upon successful request, the response will include the topic ARN that could be later used to reference this topic in an S3-compatible notification request. | |
147 | To update a topic, use the same command used for topic creation, with the topic name of an existing topic and different endpoint values. | |
11fdf7f2 | 148 | |
eafe8130 | 149 | .. tip:: Any S3-compatible notification already associated with the topic needs to be re-created for the topic update to take effect |
11fdf7f2 TL |
150 | |
151 | :: | |
152 | ||
e306af50 | 153 | PUT /topics/<topic-name>[?OpaqueData=<opaque data>][&push-endpoint=<endpoint>[&amqp-exchange=<exchange>][&amqp-ack-level=none|broker|routable][&verify-ssl=true|false][&kafka-ack-level=none|broker][&use-ssl=true|false][&ca-location=<file path>]] |
eafe8130 TL |
154 | |
155 | Request parameters: | |
156 | ||
9f95a23c TL |
157 | - push-endpoint: URI of an endpoint to send push notification to |
158 | - OpaqueData: opaque data is set in the topic configuration and added to all notifications triggered by the ropic | |
159 | ||
160 | The endpoint URI may include parameters depending with the type of endpoint: | |
161 | ||
162 | - HTTP endpoint | |
163 | ||
164 | - URI: ``http[s]://<fqdn>[:<port]`` | |
165 | - port defaults to: 80/443 for HTTP/S accordingly | |
166 | - verify-ssl: indicate whether the server certificate is validated by the client or not ("true" by default) | |
167 | ||
168 | - AMQP0.9.1 endpoint | |
eafe8130 | 169 | |
9f95a23c TL |
170 | - URI: ``amqp://[<user>:<password>@]<fqdn>[:<port>][/<vhost>]`` |
171 | - user/password defaults to: guest/guest | |
172 | - user/password may only be provided over HTTPS. Topic creation request will be rejected if not | |
173 | - port defaults to: 5672 | |
174 | - vhost defaults to: "/" | |
175 | - amqp-exchange: the exchanges must exist and be able to route messages based on topics (mandatory parameter for AMQP0.9.1) | |
e306af50 | 176 | - amqp-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Three ack methods exist: |
eafe8130 | 177 | |
9f95a23c TL |
178 | - "none": message is considered "delivered" if sent to broker |
179 | - "broker": message is considered "delivered" if acked by broker (default) | |
e306af50 | 180 | - "routable": message is considered "delivered" if broker can route to a consumer |
eafe8130 | 181 | |
9f95a23c TL |
182 | - Kafka endpoint |
183 | ||
184 | - URI: ``kafka://[<user>:<password>@]<fqdn>[:<port]`` | |
185 | - if ``use-ssl`` is set to "true", secure connection will be used for connecting with the broker ("false" by default) | |
186 | - if ``ca-location`` is provided, and secure connection is used, the specified CA will be used, instead of the default one, to authenticate the broker | |
187 | - user/password may only be provided over HTTPS. Topic creation request will be rejected if not | |
188 | - user/password may only be provided together with ``use-ssl``, connection to the broker would fail if not | |
189 | - port defaults to: 9092 | |
190 | - kafka-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Two ack methods exist: | |
191 | ||
192 | - "none": message is considered "delivered" if sent to broker | |
193 | - "broker": message is considered "delivered" if acked by broker (default) | |
eafe8130 TL |
194 | |
195 | The topic ARN in the response will have the following format: | |
11fdf7f2 | 196 | |
eafe8130 TL |
197 | :: |
198 | ||
199 | arn:aws:sns:<zone-group>:<tenant>:<topic> | |
11fdf7f2 TL |
200 | |
201 | Get Topic Information | |
eafe8130 | 202 | ````````````````````` |
11fdf7f2 | 203 | |
eafe8130 | 204 | Returns information about specific topic. This includes subscriptions to that topic, and push-endpoint information, if provided. |
11fdf7f2 TL |
205 | |
206 | :: | |
207 | ||
208 | GET /topics/<topic-name> | |
209 | ||
eafe8130 TL |
210 | Response will have the following format (JSON): |
211 | ||
212 | :: | |
11fdf7f2 | 213 | |
eafe8130 TL |
214 | { |
215 | "topic":{ | |
216 | "user":"", | |
217 | "name":"", | |
218 | "dest":{ | |
219 | "bucket_name":"", | |
220 | "oid_prefix":"", | |
221 | "push_endpoint":"", | |
9f95a23c TL |
222 | "push_endpoint_args":"", |
223 | "push_endpoint_topic":"" | |
eafe8130 TL |
224 | }, |
225 | "arn":"" | |
9f95a23c | 226 | "opaqueData":"" |
eafe8130 TL |
227 | }, |
228 | "subs":[] | |
229 | } | |
230 | ||
231 | - topic.user: name of the user that created the topic | |
232 | - name: name of the topic | |
233 | - dest.bucket_name: not used | |
234 | - dest.oid_prefix: not used | |
235 | - dest.push_endpoint: in case of S3-compliant notifications, this value will be used as the push-endpoint URL | |
9f95a23c | 236 | - if push-endpoint URL contain user/password information, request must be made over HTTPS. Topic get request will be rejected if not |
eafe8130 | 237 | - dest.push_endpoint_args: in case of S3-compliant notifications, this value will be used as the push-endpoint args |
9f95a23c | 238 | - dest.push_endpoint_topic: in case of S3-compliant notifications, this value will hold the topic name as sent to the endpoint (may be different than the internal topic name) |
eafe8130 TL |
239 | - topic.arn: topic ARN |
240 | - subs: list of subscriptions associated with this topic | |
11fdf7f2 TL |
241 | |
242 | Delete Topic | |
eafe8130 | 243 | ```````````` |
11fdf7f2 TL |
244 | |
245 | :: | |
246 | ||
247 | DELETE /topics/<topic-name> | |
248 | ||
249 | Delete the specified topic. | |
250 | ||
251 | List Topics | |
eafe8130 | 252 | ``````````` |
11fdf7f2 TL |
253 | |
254 | List all topics that user defined. | |
255 | ||
256 | :: | |
257 | ||
258 | GET /topics | |
eafe8130 | 259 | |
9f95a23c TL |
260 | - if push-endpoint URL contain user/password information, in any of the topic, request must be made over HTTPS. Topic list request will be rejected if not |
261 | ||
eafe8130 TL |
262 | S3-Compliant Notifications |
263 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
11fdf7f2 | 264 | |
eafe8130 | 265 | Detailed under: `Bucket Operations`_. |
11fdf7f2 | 266 | |
eafe8130 TL |
267 | .. note:: |
268 | ||
269 | - Notification creation will also create a subscription for pushing/pulling events | |
270 | - The generated subscription's name will have the same as the notification Id, and could be used later to fetch and ack events with the subscription API. | |
271 | - Notification deletion will deletes all generated subscriptions | |
272 | - In case that bucket deletion implicitly deletes the notification, | |
273 | the associated subscription will not be deleted automatically (any events of the deleted bucket could still be access), | |
274 | and will have to be deleted explicitly with the subscription deletion API | |
275 | - Filtering based on metadata (which is an extension to S3) is not supported, and such rules will be ignored | |
9f95a23c | 276 | - Filtering based on tags (which is an extension to S3) is not supported, and such rules will be ignored |
11fdf7f2 | 277 | |
eafe8130 TL |
278 | |
279 | Non S3-Compliant Notifications | |
280 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
11fdf7f2 TL |
281 | |
282 | Create a Notification | |
eafe8130 | 283 | ````````````````````` |
11fdf7f2 TL |
284 | |
285 | This will create a publisher for a specific bucket into a topic. | |
286 | ||
287 | :: | |
288 | ||
289 | PUT /notifications/bucket/<bucket>?topic=<topic-name>[&events=<event>[,<event>]] | |
290 | ||
eafe8130 | 291 | Request parameters: |
11fdf7f2 | 292 | |
eafe8130 TL |
293 | - topic-name: name of topic |
294 | - event: event type (string), one of: ``OBJECT_CREATE``, ``OBJECT_DELETE``, ``DELETE_MARKER_CREATE`` | |
295 | ||
11fdf7f2 | 296 | Delete Notification Information |
eafe8130 | 297 | ``````````````````````````````` |
11fdf7f2 TL |
298 | |
299 | Delete publisher from a specific bucket into a specific topic. | |
300 | ||
301 | :: | |
302 | ||
303 | DELETE /notifications/bucket/<bucket>?topic=<topic-name> | |
304 | ||
eafe8130 TL |
305 | Request parameters: |
306 | ||
307 | - topic-name: name of topic | |
308 | ||
309 | .. note:: When the bucket is deleted, any notification defined on it is also deleted | |
310 | ||
311 | List Notifications | |
312 | `````````````````` | |
313 | ||
314 | List all topics with associated events defined on a bucket. | |
315 | ||
316 | :: | |
317 | ||
318 | GET /notifications/bucket/<bucket> | |
319 | ||
320 | Response will have the following format (JSON): | |
321 | ||
322 | :: | |
11fdf7f2 | 323 | |
eafe8130 TL |
324 | {"topics":[ |
325 | { | |
326 | "topic":{ | |
327 | "user":"", | |
328 | "name":"", | |
329 | "dest":{ | |
330 | "bucket_name":"", | |
331 | "oid_prefix":"", | |
332 | "push_endpoint":"", | |
9f95a23c TL |
333 | "push_endpoint_args":"", |
334 | "push_endpoint_topic":"" | |
eafe8130 TL |
335 | } |
336 | "arn":"" | |
337 | }, | |
338 | "events":[] | |
339 | } | |
340 | ]} | |
11fdf7f2 | 341 | |
eafe8130 TL |
342 | Subscriptions |
343 | ~~~~~~~~~~~~~ | |
11fdf7f2 | 344 | |
eafe8130 TL |
345 | Create a Subscription |
346 | ````````````````````` | |
11fdf7f2 TL |
347 | |
348 | Creates a new subscription. | |
349 | ||
350 | :: | |
351 | ||
e306af50 | 352 | PUT /subscriptions/<sub-name>?topic=<topic-name>[?push-endpoint=<endpoint>[&amqp-exchange=<exchange>][&amqp-ack-level=none|broker|routable][&verify-ssl=true|false][&kafka-ack-level=none|broker][&ca-location=<file path>]] |
11fdf7f2 | 353 | |
eafe8130 | 354 | Request parameters: |
11fdf7f2 | 355 | |
eafe8130 TL |
356 | - topic-name: name of topic |
357 | - push-endpoint: URI of endpoint to send push notification to | |
11fdf7f2 | 358 | |
9f95a23c TL |
359 | The endpoint URI may include parameters depending with the type of endpoint: |
360 | ||
361 | - HTTP endpoint | |
11fdf7f2 | 362 | |
9f95a23c TL |
363 | - URI: ``http[s]://<fqdn>[:<port]`` |
364 | - port defaults to: 80/443 for HTTP/S accordingly | |
365 | - verify-ssl: indicate whether the server certificate is validated by the client or not ("true" by default) | |
366 | ||
367 | - AMQP0.9.1 endpoint | |
368 | ||
369 | - URI: ``amqp://[<user>:<password>@]<fqdn>[:<port>][/<vhost>]`` | |
370 | - user/password defaults to : guest/guest | |
371 | - port defaults to: 5672 | |
372 | - vhost defaults to: "/" | |
373 | - amqp-exchange: the exchanges must exist and be able to route messages based on topics (mandatory parameter for AMQP0.9.1) | |
e306af50 | 374 | - amqp-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Three ack methods exist: |
9f95a23c TL |
375 | |
376 | - "none": message is considered "delivered" if sent to broker | |
377 | - "broker": message is considered "delivered" if acked by broker (default) | |
e306af50 | 378 | - "routable": message is considered "delivered" if broker can route to a consumer |
9f95a23c TL |
379 | |
380 | - Kafka endpoint | |
381 | ||
382 | - URI: ``kafka://[<user>:<password>@]<fqdn>[:<port]`` | |
383 | - if ``ca-location`` is provided, secure connection will be used for connection with the broker | |
384 | - user/password may only be provided over HTTPS. Topic creation request will be rejected if not | |
385 | - user/password may only be provided together with ``ca-location``. Topic creation request will be rejected if not | |
386 | - port defaults to: 9092 | |
387 | - kafka-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Two ack methods exist: | |
388 | ||
389 | - "none": message is considered "delivered" if sent to broker | |
390 | - "broker": message is considered "delivered" if acked by broker (default) | |
11fdf7f2 | 391 | |
11fdf7f2 | 392 | |
eafe8130 TL |
393 | Get Subscription Information |
394 | ```````````````````````````` | |
395 | ||
396 | Returns information about specific subscription. | |
11fdf7f2 TL |
397 | |
398 | :: | |
399 | ||
400 | GET /subscriptions/<sub-name> | |
401 | ||
eafe8130 TL |
402 | Response will have the following format (JSON): |
403 | ||
404 | :: | |
405 | ||
406 | { | |
407 | "user":"", | |
408 | "name":"", | |
409 | "topic":"", | |
410 | "dest":{ | |
411 | "bucket_name":"", | |
412 | "oid_prefix":"", | |
413 | "push_endpoint":"", | |
9f95a23c TL |
414 | "push_endpoint_args":"", |
415 | "push_endpoint_topic":"" | |
eafe8130 TL |
416 | } |
417 | "s3_id":"" | |
418 | } | |
419 | ||
420 | - user: name of the user that created the subscription | |
421 | - name: name of the subscription | |
422 | - topic: name of the topic the subscription is associated with | |
9f95a23c TL |
423 | - dest.bucket_name: name of the bucket storing the events |
424 | - dest.oid_prefix: oid prefix for the events stored in the bucket | |
425 | - dest.push_endpoint: in case of S3-compliant notifications, this value will be used as the push-endpoint URL | |
426 | - if push-endpoint URL contain user/password information, request must be made over HTTPS. Topic get request will be rejected if not | |
427 | - dest.push_endpoint_args: in case of S3-compliant notifications, this value will be used as the push-endpoint args | |
428 | - dest.push_endpoint_topic: in case of S3-compliant notifications, this value will hold the topic name as sent to the endpoint (may be different than the internal topic name) | |
429 | - s3_id: in case of S3-compliant notifications, this will hold the notification name that created the subscription | |
11fdf7f2 TL |
430 | |
431 | Delete Subscription | |
eafe8130 | 432 | ``````````````````` |
11fdf7f2 | 433 | |
eafe8130 | 434 | Removes a subscription. |
11fdf7f2 TL |
435 | |
436 | :: | |
437 | ||
438 | DELETE /subscriptions/<sub-name> | |
439 | ||
11fdf7f2 TL |
440 | Events |
441 | ~~~~~~ | |
442 | ||
443 | Pull Events | |
eafe8130 | 444 | ``````````` |
11fdf7f2 | 445 | |
eafe8130 | 446 | Pull events sent to a specific subscription. |
11fdf7f2 TL |
447 | |
448 | :: | |
449 | ||
450 | GET /subscriptions/<sub-name>?events[&max-entries=<max-entries>][&marker=<marker>] | |
451 | ||
eafe8130 TL |
452 | Request parameters: |
453 | ||
454 | - marker: pagination marker for list of events, if not specified will start from the oldest | |
455 | - max-entries: max number of events to return | |
456 | ||
457 | The response will hold information on the current marker and whether there are more events not fetched: | |
458 | ||
459 | :: | |
460 | ||
461 | {"next_marker":"","is_truncated":"",...} | |
462 | ||
463 | ||
464 | The actual content of the response is depended with how the subscription was created. | |
465 | In case that the subscription was created via an S3-compatible notification, | |
466 | the events will have an S3-compatible record format (JSON): | |
467 | ||
468 | :: | |
469 | ||
470 | {"Records":[ | |
471 | { | |
472 | "eventVersion":"2.1" | |
473 | "eventSource":"aws:s3", | |
474 | "awsRegion":"", | |
475 | "eventTime":"", | |
476 | "eventName":"", | |
477 | "userIdentity":{ | |
478 | "principalId":"" | |
479 | }, | |
480 | "requestParameters":{ | |
481 | "sourceIPAddress":"" | |
482 | }, | |
483 | "responseElements":{ | |
484 | "x-amz-request-id":"", | |
485 | "x-amz-id-2":"" | |
486 | }, | |
487 | "s3":{ | |
488 | "s3SchemaVersion":"1.0", | |
489 | "configurationId":"", | |
490 | "bucket":{ | |
491 | "name":"", | |
492 | "ownerIdentity":{ | |
493 | "principalId":"" | |
494 | }, | |
495 | "arn":"", | |
496 | "id":"" | |
497 | }, | |
498 | "object":{ | |
499 | "key":"", | |
500 | "size":"0", | |
501 | "eTag":"", | |
502 | "versionId":"", | |
503 | "sequencer":"", | |
9f95a23c TL |
504 | "metadata":[], |
505 | "tags":[] | |
eafe8130 TL |
506 | } |
507 | }, | |
508 | "eventId":"", | |
9f95a23c | 509 | "opaqueData":"", |
eafe8130 TL |
510 | } |
511 | ]} | |
512 | ||
513 | - awsRegion: zonegroup | |
514 | - eventTime: timestamp indicating when the event was triggered | |
515 | - eventName: either ``s3:ObjectCreated:``, or ``s3:ObjectRemoved:`` | |
516 | - userIdentity: not supported | |
517 | - requestParameters: not supported | |
518 | - responseElements: not supported | |
519 | - s3.configurationId: notification ID that created the subscription for the event | |
eafe8130 TL |
520 | - s3.bucket.name: name of the bucket |
521 | - s3.bucket.ownerIdentity.principalId: owner of the bucket | |
522 | - s3.bucket.arn: ARN of the bucket | |
523 | - s3.bucket.id: Id of the bucket (an extension to the S3 notification API) | |
524 | - s3.object.key: object key | |
525 | - s3.object.size: not supported | |
526 | - s3.object.eTag: object etag | |
527 | - s3.object.version: object version in case of versioned bucket | |
528 | - s3.object.sequencer: monotonically increasing identifier of the change per object (hexadecimal format) | |
529 | - s3.object.metadata: not supported (an extension to the S3 notification API) | |
9f95a23c | 530 | - s3.object.tags: not supported (an extension to the S3 notification API) |
eafe8130 | 531 | - s3.eventId: unique ID of the event, that could be used for acking (an extension to the S3 notification API) |
9f95a23c | 532 | - s3.opaqueData: opaque data is set in the topic configuration and added to all notifications triggered by the ropic (an extension to the S3 notification API) |
eafe8130 TL |
533 | |
534 | In case that the subscription was not created via a non S3-compatible notification, | |
535 | the events will have the following event format (JSON): | |
536 | ||
537 | :: | |
11fdf7f2 | 538 | |
eafe8130 TL |
539 | {"events":[ |
540 | { | |
541 | "id":"", | |
542 | "event":"", | |
543 | "timestamp":"", | |
544 | "info":{ | |
545 | "attrs":{ | |
546 | "mtime":"" | |
547 | }, | |
548 | "bucket":{ | |
549 | "bucket_id":"", | |
550 | "name":"", | |
551 | "tenant":"" | |
552 | }, | |
553 | "key":{ | |
554 | "instance":"", | |
555 | "name":"" | |
556 | } | |
557 | } | |
558 | } | |
559 | ]} | |
560 | ||
561 | - id: unique ID of the event, that could be used for acking | |
562 | - event: one of: ``OBJECT_CREATE``, ``OBJECT_DELETE``, ``DELETE_MARKER_CREATE`` | |
563 | - timestamp: timestamp indicating when the event was sent | |
564 | - info.attrs.mtime: timestamp indicating when the event was triggered | |
565 | - info.bucket.bucket_id: id of the bucket | |
566 | - info.bucket.name: name of the bucket | |
567 | - info.bucket.tenant: tenant the bucket belongs to | |
568 | - info.key.instance: object version in case of versioned bucket | |
569 | - info.key.name: object key | |
11fdf7f2 TL |
570 | |
571 | Ack Event | |
eafe8130 | 572 | ````````` |
11fdf7f2 TL |
573 | |
574 | Ack event so that it can be removed from the subscription history. | |
575 | ||
576 | :: | |
577 | ||
578 | POST /subscriptions/<sub-name>?ack&event-id=<event-id> | |
579 | ||
eafe8130 | 580 | Request parameters: |
11fdf7f2 | 581 | |
eafe8130 | 582 | - event-id: id of event to be acked |
11fdf7f2 | 583 | |
eafe8130 TL |
584 | .. _Multisite : ../multisite |
585 | .. _Bucket Notification : ../notifications | |
586 | .. _Bucket Operations: ../s3/bucketops |