]>
Commit | Line | Data |
---|---|---|
eafe8130 | 1 | ================== |
11fdf7f2 | 2 | PubSub Sync Module |
eafe8130 | 3 | ================== |
11fdf7f2 TL |
4 | |
5 | .. versionadded:: Nautilus | |
6 | ||
eafe8130 TL |
7 | .. contents:: |
8 | ||
11fdf7f2 | 9 | This sync module provides a publish and subscribe mechanism for the object store modification |
eafe8130 | 10 | events. Events are published into predefined topics. Topics can be subscribed to, and events |
11fdf7f2 | 11 | can be pulled from them. Events need to be acked. Also, events will expire and disappear |
eafe8130 TL |
12 | after a period of time. |
13 | ||
9f95a23c TL |
14 | A push notification mechanism exists too, currently supporting HTTP, |
15 | AMQP0.9.1 and Kafka endpoints. In this case, the events are pushed to an endpoint on top of storing them in Ceph. If events should only be pushed to an endpoint | |
eafe8130 | 16 | and do not need to be stored in Ceph, the `Bucket Notification`_ mechanism should be used instead of pubsub sync module. |
11fdf7f2 | 17 | |
7f7e6c64 TL |
18 | A user can create different topics. A topic entity is defined by its name and is per tenant. A |
19 | user can only associate its topics (via notification configuration) with buckets it owns. | |
11fdf7f2 | 20 | |
eafe8130 TL |
21 | In order to publish events for specific bucket a notification entity needs to be created. A |
22 | notification can be created on a subset of event types, or for all event types (default). | |
23 | There can be multiple notifications for any specific topic, and the same topic could be used for multiple notifications. | |
11fdf7f2 TL |
24 | |
25 | A subscription to a topic can also be defined. There can be multiple subscriptions for any | |
26 | specific topic. | |
27 | ||
eafe8130 TL |
28 | REST API has been defined to provide configuration and control interfaces for the pubsub |
29 | mechanisms. This API has two flavors, one is S3-compatible and one is not. The two flavors can be used | |
30 | together, although it is recommended to use the S3-compatible one. | |
31 | The S3-compatible API is similar to the one used in the bucket notification mechanism. | |
11fdf7f2 | 32 | |
7f7e6c64 | 33 | Events are stored as RGW objects in a special bucket, under a special user (pubsub control user). Events cannot |
eafe8130 | 34 | be accessed directly, but need to be pulled and acked using the new REST API. |
11fdf7f2 | 35 | |
eafe8130 TL |
36 | .. toctree:: |
37 | :maxdepth: 1 | |
11fdf7f2 | 38 | |
eafe8130 TL |
39 | S3 Bucket Notification Compatibility <s3-notification-compatibility> |
40 | ||
41 | PubSub Zone Configuration | |
42 | ------------------------- | |
11fdf7f2 | 43 | |
7f7e6c64 | 44 | The pubsub sync module requires the creation of a new zone in a :ref:`multisite` environment... |
9f95a23c TL |
45 | First, a master zone must exist (see: :ref:`master-zone-label`), |
46 | then a secondary zone should be created (see :ref:`secondary-zone-label`). | |
eafe8130 | 47 | In the creation of the secondary zone, its tier type must be set to ``pubsub``: |
11fdf7f2 TL |
48 | |
49 | :: | |
50 | ||
eafe8130 TL |
51 | # radosgw-admin zone create --rgw-zonegroup={zone-group-name} \ |
52 | --rgw-zone={zone-name} \ | |
53 | --endpoints={http://fqdn}[,{http://fqdn}] \ | |
54 | --sync-from-all=0 \ | |
55 | --sync-from={master-zone-name} \ | |
56 | --tier-type=pubsub | |
11fdf7f2 | 57 | |
11fdf7f2 | 58 | |
eafe8130 TL |
59 | PubSub Zone Configuration Parameters |
60 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
61 | ||
62 | :: | |
11fdf7f2 | 63 | |
eafe8130 TL |
64 | { |
65 | "tenant": <tenant>, # default: <empty> | |
66 | "uid": <uid>, # default: "pubsub" | |
67 | "data_bucket_prefix": <prefix> # default: "pubsub-" | |
68 | "data_oid_prefix": <prefix> # | |
69 | "events_retention_days": <days> # default: 7 | |
70 | } | |
11fdf7f2 TL |
71 | |
72 | * ``tenant`` (string) | |
73 | ||
74 | The tenant of the pubsub control user. | |
75 | ||
76 | * ``uid`` (string) | |
77 | ||
78 | The uid of the pubsub control user. | |
79 | ||
80 | * ``data_bucket_prefix`` (string) | |
81 | ||
82 | The prefix of the bucket name that will be created to store events for specific topic. | |
83 | ||
84 | * ``data_oid_prefix`` (string) | |
85 | ||
86 | The oid prefix for the stored events. | |
87 | ||
88 | * ``events_retention_days`` (integer) | |
89 | ||
90 | How many days to keep events that weren't acked. | |
91 | ||
eafe8130 TL |
92 | Configuring Parameters via CLI |
93 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
11fdf7f2 | 94 | |
eafe8130 | 95 | The tier configuration could be set using the following command: |
11fdf7f2 TL |
96 | |
97 | :: | |
98 | ||
eafe8130 | 99 | # radosgw-admin zone modify --rgw-zonegroup={zone-group-name} \ |
11fdf7f2 TL |
100 | --rgw-zone={zone-name} \ |
101 | --tier-config={key}={val}[,{key}={val}] | |
102 | ||
eafe8130 TL |
103 | Where the ``key`` in the configuration specifies the configuration variable that needs to be updated (from the list above), and |
104 | the ``val`` specifies its new value. For example, setting the pubsub control user ``uid`` to ``user_ps``: | |
11fdf7f2 TL |
105 | |
106 | :: | |
107 | ||
eafe8130 | 108 | # radosgw-admin zone modify --rgw-zonegroup={zone-group-name} \ |
11fdf7f2 TL |
109 | --rgw-zone={zone-name} \ |
110 | --tier-config=uid=pubsub | |
111 | ||
11fdf7f2 TL |
112 | A configuration field can be removed by using ``--tier-config-rm={key}``. |
113 | ||
114 | PubSub Performance Stats | |
115 | ------------------------- | |
eafe8130 TL |
116 | Same counters are shared between the pubsub sync module and the notification mechanism. |
117 | ||
118 | - ``pubsub_event_triggered``: running counter of events with at lease one topic associated with them | |
119 | - ``pubsub_event_lost``: running counter of events that had topics and subscriptions associated with them but that were not stored or pushed to any of the subscriptions | |
120 | - ``pubsub_store_ok``: running counter, for all subscriptions, of stored events | |
121 | - ``pubsub_store_fail``: running counter, for all subscriptions, of events failed to be stored | |
122 | - ``pubsub_push_ok``: running counter, for all subscriptions, of events successfully pushed to their endpoint | |
123 | - ``pubsub_push_fail``: running counter, for all subscriptions, of events failed to be pushed to their endpoint | |
124 | - ``pubsub_push_pending``: gauge value of events pushed to an endpoint but not acked or nacked yet | |
11fdf7f2 | 125 | |
eafe8130 TL |
126 | .. note:: |
127 | ||
128 | ``pubsub_event_triggered`` and ``pubsub_event_lost`` are incremented per event, while: | |
129 | ``pubsub_store_ok``, ``pubsub_store_fail``, ``pubsub_push_ok``, ``pubsub_push_fail``, are incremented per store/push action on each subscriptions. | |
11fdf7f2 TL |
130 | |
131 | PubSub REST API | |
eafe8130 | 132 | --------------- |
11fdf7f2 | 133 | |
eafe8130 | 134 | .. tip:: PubSub REST calls, and only them, should be sent to an RGW which belong to a PubSub zone |
11fdf7f2 TL |
135 | |
136 | Topics | |
137 | ~~~~~~ | |
eafe8130 | 138 | |
11fdf7f2 | 139 | Create a Topic |
eafe8130 TL |
140 | `````````````` |
141 | ||
142 | This will create a new topic. Topic creation is needed both for both flavors of the API. | |
143 | Optionally the topic could be provided with push endpoint parameters that would be used later | |
144 | when an S3-compatible notification is created. | |
145 | Upon successful request, the response will include the topic ARN that could be later used to reference this topic in an S3-compatible notification request. | |
146 | To update a topic, use the same command used for topic creation, with the topic name of an existing topic and different endpoint values. | |
11fdf7f2 | 147 | |
eafe8130 | 148 | .. tip:: Any S3-compatible notification already associated with the topic needs to be re-created for the topic update to take effect |
11fdf7f2 TL |
149 | |
150 | :: | |
151 | ||
e306af50 | 152 | PUT /topics/<topic-name>[?OpaqueData=<opaque data>][&push-endpoint=<endpoint>[&amqp-exchange=<exchange>][&amqp-ack-level=none|broker|routable][&verify-ssl=true|false][&kafka-ack-level=none|broker][&use-ssl=true|false][&ca-location=<file path>]] |
eafe8130 TL |
153 | |
154 | Request parameters: | |
155 | ||
9f95a23c TL |
156 | - push-endpoint: URI of an endpoint to send push notification to |
157 | - OpaqueData: opaque data is set in the topic configuration and added to all notifications triggered by the ropic | |
158 | ||
159 | The endpoint URI may include parameters depending with the type of endpoint: | |
160 | ||
161 | - HTTP endpoint | |
162 | ||
163 | - URI: ``http[s]://<fqdn>[:<port]`` | |
164 | - port defaults to: 80/443 for HTTP/S accordingly | |
165 | - verify-ssl: indicate whether the server certificate is validated by the client or not ("true" by default) | |
166 | ||
167 | - AMQP0.9.1 endpoint | |
eafe8130 | 168 | |
9f95a23c TL |
169 | - URI: ``amqp://[<user>:<password>@]<fqdn>[:<port>][/<vhost>]`` |
170 | - user/password defaults to: guest/guest | |
171 | - user/password may only be provided over HTTPS. Topic creation request will be rejected if not | |
172 | - port defaults to: 5672 | |
173 | - vhost defaults to: "/" | |
7f7e6c64 | 174 | - amqp-exchange: the exchanges must exist and be able to route messages based on topics (mandatory parameter for AMQP0.9.1). Different topics pointing to the same endpoint must use the same exchange |
e306af50 | 175 | - amqp-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Three ack methods exist: |
eafe8130 | 176 | |
9f95a23c TL |
177 | - "none": message is considered "delivered" if sent to broker |
178 | - "broker": message is considered "delivered" if acked by broker (default) | |
e306af50 | 179 | - "routable": message is considered "delivered" if broker can route to a consumer |
eafe8130 | 180 | |
7f7e6c64 TL |
181 | .. tip:: The topic-name is used for the AMQP topic ("routing key" for a topic exchange) |
182 | ||
9f95a23c TL |
183 | - Kafka endpoint |
184 | ||
185 | - URI: ``kafka://[<user>:<password>@]<fqdn>[:<port]`` | |
186 | - if ``use-ssl`` is set to "true", secure connection will be used for connecting with the broker ("false" by default) | |
187 | - if ``ca-location`` is provided, and secure connection is used, the specified CA will be used, instead of the default one, to authenticate the broker | |
188 | - user/password may only be provided over HTTPS. Topic creation request will be rejected if not | |
189 | - user/password may only be provided together with ``use-ssl``, connection to the broker would fail if not | |
190 | - port defaults to: 9092 | |
191 | - kafka-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Two ack methods exist: | |
192 | ||
193 | - "none": message is considered "delivered" if sent to broker | |
194 | - "broker": message is considered "delivered" if acked by broker (default) | |
eafe8130 TL |
195 | |
196 | The topic ARN in the response will have the following format: | |
11fdf7f2 | 197 | |
eafe8130 TL |
198 | :: |
199 | ||
200 | arn:aws:sns:<zone-group>:<tenant>:<topic> | |
11fdf7f2 TL |
201 | |
202 | Get Topic Information | |
eafe8130 | 203 | ````````````````````` |
11fdf7f2 | 204 | |
eafe8130 | 205 | Returns information about specific topic. This includes subscriptions to that topic, and push-endpoint information, if provided. |
11fdf7f2 TL |
206 | |
207 | :: | |
208 | ||
209 | GET /topics/<topic-name> | |
210 | ||
eafe8130 TL |
211 | Response will have the following format (JSON): |
212 | ||
213 | :: | |
11fdf7f2 | 214 | |
eafe8130 TL |
215 | { |
216 | "topic":{ | |
217 | "user":"", | |
218 | "name":"", | |
219 | "dest":{ | |
220 | "bucket_name":"", | |
221 | "oid_prefix":"", | |
222 | "push_endpoint":"", | |
9f95a23c | 223 | "push_endpoint_args":"", |
7f7e6c64 TL |
224 | "push_endpoint_topic":"", |
225 | "stored_secret":"", | |
eafe8130 TL |
226 | }, |
227 | "arn":"" | |
9f95a23c | 228 | "opaqueData":"" |
eafe8130 TL |
229 | }, |
230 | "subs":[] | |
231 | } | |
232 | ||
233 | - topic.user: name of the user that created the topic | |
234 | - name: name of the topic | |
235 | - dest.bucket_name: not used | |
236 | - dest.oid_prefix: not used | |
237 | - dest.push_endpoint: in case of S3-compliant notifications, this value will be used as the push-endpoint URL | |
9f95a23c | 238 | - if push-endpoint URL contain user/password information, request must be made over HTTPS. Topic get request will be rejected if not |
eafe8130 | 239 | - dest.push_endpoint_args: in case of S3-compliant notifications, this value will be used as the push-endpoint args |
9f95a23c | 240 | - dest.push_endpoint_topic: in case of S3-compliant notifications, this value will hold the topic name as sent to the endpoint (may be different than the internal topic name) |
eafe8130 TL |
241 | - topic.arn: topic ARN |
242 | - subs: list of subscriptions associated with this topic | |
11fdf7f2 TL |
243 | |
244 | Delete Topic | |
eafe8130 | 245 | ```````````` |
11fdf7f2 TL |
246 | |
247 | :: | |
248 | ||
249 | DELETE /topics/<topic-name> | |
250 | ||
251 | Delete the specified topic. | |
252 | ||
253 | List Topics | |
eafe8130 | 254 | ``````````` |
11fdf7f2 | 255 | |
7f7e6c64 | 256 | List all topics associated with a tenant. |
11fdf7f2 TL |
257 | |
258 | :: | |
259 | ||
260 | GET /topics | |
eafe8130 | 261 | |
9f95a23c TL |
262 | - if push-endpoint URL contain user/password information, in any of the topic, request must be made over HTTPS. Topic list request will be rejected if not |
263 | ||
eafe8130 TL |
264 | S3-Compliant Notifications |
265 | ~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
11fdf7f2 | 266 | |
eafe8130 | 267 | Detailed under: `Bucket Operations`_. |
11fdf7f2 | 268 | |
eafe8130 TL |
269 | .. note:: |
270 | ||
271 | - Notification creation will also create a subscription for pushing/pulling events | |
272 | - The generated subscription's name will have the same as the notification Id, and could be used later to fetch and ack events with the subscription API. | |
273 | - Notification deletion will deletes all generated subscriptions | |
274 | - In case that bucket deletion implicitly deletes the notification, | |
275 | the associated subscription will not be deleted automatically (any events of the deleted bucket could still be access), | |
276 | and will have to be deleted explicitly with the subscription deletion API | |
277 | - Filtering based on metadata (which is an extension to S3) is not supported, and such rules will be ignored | |
9f95a23c | 278 | - Filtering based on tags (which is an extension to S3) is not supported, and such rules will be ignored |
11fdf7f2 | 279 | |
eafe8130 TL |
280 | |
281 | Non S3-Compliant Notifications | |
282 | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ | |
11fdf7f2 TL |
283 | |
284 | Create a Notification | |
eafe8130 | 285 | ````````````````````` |
11fdf7f2 TL |
286 | |
287 | This will create a publisher for a specific bucket into a topic. | |
288 | ||
289 | :: | |
290 | ||
291 | PUT /notifications/bucket/<bucket>?topic=<topic-name>[&events=<event>[,<event>]] | |
292 | ||
eafe8130 | 293 | Request parameters: |
11fdf7f2 | 294 | |
eafe8130 TL |
295 | - topic-name: name of topic |
296 | - event: event type (string), one of: ``OBJECT_CREATE``, ``OBJECT_DELETE``, ``DELETE_MARKER_CREATE`` | |
297 | ||
11fdf7f2 | 298 | Delete Notification Information |
eafe8130 | 299 | ``````````````````````````````` |
11fdf7f2 TL |
300 | |
301 | Delete publisher from a specific bucket into a specific topic. | |
302 | ||
303 | :: | |
304 | ||
305 | DELETE /notifications/bucket/<bucket>?topic=<topic-name> | |
306 | ||
eafe8130 TL |
307 | Request parameters: |
308 | ||
309 | - topic-name: name of topic | |
310 | ||
311 | .. note:: When the bucket is deleted, any notification defined on it is also deleted | |
312 | ||
313 | List Notifications | |
314 | `````````````````` | |
315 | ||
316 | List all topics with associated events defined on a bucket. | |
317 | ||
318 | :: | |
319 | ||
320 | GET /notifications/bucket/<bucket> | |
321 | ||
322 | Response will have the following format (JSON): | |
323 | ||
324 | :: | |
11fdf7f2 | 325 | |
eafe8130 TL |
326 | {"topics":[ |
327 | { | |
328 | "topic":{ | |
329 | "user":"", | |
330 | "name":"", | |
331 | "dest":{ | |
332 | "bucket_name":"", | |
333 | "oid_prefix":"", | |
334 | "push_endpoint":"", | |
9f95a23c TL |
335 | "push_endpoint_args":"", |
336 | "push_endpoint_topic":"" | |
eafe8130 TL |
337 | } |
338 | "arn":"" | |
339 | }, | |
340 | "events":[] | |
341 | } | |
342 | ]} | |
11fdf7f2 | 343 | |
eafe8130 TL |
344 | Subscriptions |
345 | ~~~~~~~~~~~~~ | |
11fdf7f2 | 346 | |
eafe8130 TL |
347 | Create a Subscription |
348 | ````````````````````` | |
11fdf7f2 TL |
349 | |
350 | Creates a new subscription. | |
351 | ||
352 | :: | |
353 | ||
e306af50 | 354 | PUT /subscriptions/<sub-name>?topic=<topic-name>[?push-endpoint=<endpoint>[&amqp-exchange=<exchange>][&amqp-ack-level=none|broker|routable][&verify-ssl=true|false][&kafka-ack-level=none|broker][&ca-location=<file path>]] |
11fdf7f2 | 355 | |
eafe8130 | 356 | Request parameters: |
11fdf7f2 | 357 | |
eafe8130 TL |
358 | - topic-name: name of topic |
359 | - push-endpoint: URI of endpoint to send push notification to | |
11fdf7f2 | 360 | |
9f95a23c TL |
361 | The endpoint URI may include parameters depending with the type of endpoint: |
362 | ||
363 | - HTTP endpoint | |
11fdf7f2 | 364 | |
9f95a23c TL |
365 | - URI: ``http[s]://<fqdn>[:<port]`` |
366 | - port defaults to: 80/443 for HTTP/S accordingly | |
367 | - verify-ssl: indicate whether the server certificate is validated by the client or not ("true" by default) | |
368 | ||
369 | - AMQP0.9.1 endpoint | |
370 | ||
371 | - URI: ``amqp://[<user>:<password>@]<fqdn>[:<port>][/<vhost>]`` | |
372 | - user/password defaults to : guest/guest | |
373 | - port defaults to: 5672 | |
374 | - vhost defaults to: "/" | |
375 | - amqp-exchange: the exchanges must exist and be able to route messages based on topics (mandatory parameter for AMQP0.9.1) | |
e306af50 | 376 | - amqp-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Three ack methods exist: |
9f95a23c TL |
377 | |
378 | - "none": message is considered "delivered" if sent to broker | |
379 | - "broker": message is considered "delivered" if acked by broker (default) | |
e306af50 | 380 | - "routable": message is considered "delivered" if broker can route to a consumer |
9f95a23c TL |
381 | |
382 | - Kafka endpoint | |
383 | ||
384 | - URI: ``kafka://[<user>:<password>@]<fqdn>[:<port]`` | |
385 | - if ``ca-location`` is provided, secure connection will be used for connection with the broker | |
386 | - user/password may only be provided over HTTPS. Topic creation request will be rejected if not | |
387 | - user/password may only be provided together with ``ca-location``. Topic creation request will be rejected if not | |
388 | - port defaults to: 9092 | |
389 | - kafka-ack-level: no end2end acking is required, as messages may persist in the broker before delivered into their final destination. Two ack methods exist: | |
390 | ||
391 | - "none": message is considered "delivered" if sent to broker | |
392 | - "broker": message is considered "delivered" if acked by broker (default) | |
11fdf7f2 | 393 | |
11fdf7f2 | 394 | |
eafe8130 TL |
395 | Get Subscription Information |
396 | ```````````````````````````` | |
397 | ||
398 | Returns information about specific subscription. | |
11fdf7f2 TL |
399 | |
400 | :: | |
401 | ||
402 | GET /subscriptions/<sub-name> | |
403 | ||
eafe8130 TL |
404 | Response will have the following format (JSON): |
405 | ||
406 | :: | |
407 | ||
408 | { | |
409 | "user":"", | |
410 | "name":"", | |
411 | "topic":"", | |
412 | "dest":{ | |
413 | "bucket_name":"", | |
414 | "oid_prefix":"", | |
415 | "push_endpoint":"", | |
9f95a23c TL |
416 | "push_endpoint_args":"", |
417 | "push_endpoint_topic":"" | |
eafe8130 TL |
418 | } |
419 | "s3_id":"" | |
420 | } | |
421 | ||
422 | - user: name of the user that created the subscription | |
423 | - name: name of the subscription | |
424 | - topic: name of the topic the subscription is associated with | |
9f95a23c TL |
425 | - dest.bucket_name: name of the bucket storing the events |
426 | - dest.oid_prefix: oid prefix for the events stored in the bucket | |
427 | - dest.push_endpoint: in case of S3-compliant notifications, this value will be used as the push-endpoint URL | |
428 | - if push-endpoint URL contain user/password information, request must be made over HTTPS. Topic get request will be rejected if not | |
429 | - dest.push_endpoint_args: in case of S3-compliant notifications, this value will be used as the push-endpoint args | |
430 | - dest.push_endpoint_topic: in case of S3-compliant notifications, this value will hold the topic name as sent to the endpoint (may be different than the internal topic name) | |
431 | - s3_id: in case of S3-compliant notifications, this will hold the notification name that created the subscription | |
11fdf7f2 TL |
432 | |
433 | Delete Subscription | |
eafe8130 | 434 | ``````````````````` |
11fdf7f2 | 435 | |
eafe8130 | 436 | Removes a subscription. |
11fdf7f2 TL |
437 | |
438 | :: | |
439 | ||
440 | DELETE /subscriptions/<sub-name> | |
441 | ||
11fdf7f2 TL |
442 | Events |
443 | ~~~~~~ | |
444 | ||
445 | Pull Events | |
eafe8130 | 446 | ``````````` |
11fdf7f2 | 447 | |
eafe8130 | 448 | Pull events sent to a specific subscription. |
11fdf7f2 TL |
449 | |
450 | :: | |
451 | ||
452 | GET /subscriptions/<sub-name>?events[&max-entries=<max-entries>][&marker=<marker>] | |
453 | ||
eafe8130 TL |
454 | Request parameters: |
455 | ||
456 | - marker: pagination marker for list of events, if not specified will start from the oldest | |
457 | - max-entries: max number of events to return | |
458 | ||
459 | The response will hold information on the current marker and whether there are more events not fetched: | |
460 | ||
461 | :: | |
462 | ||
463 | {"next_marker":"","is_truncated":"",...} | |
464 | ||
465 | ||
466 | The actual content of the response is depended with how the subscription was created. | |
467 | In case that the subscription was created via an S3-compatible notification, | |
468 | the events will have an S3-compatible record format (JSON): | |
469 | ||
470 | :: | |
471 | ||
472 | {"Records":[ | |
473 | { | |
474 | "eventVersion":"2.1" | |
475 | "eventSource":"aws:s3", | |
476 | "awsRegion":"", | |
477 | "eventTime":"", | |
478 | "eventName":"", | |
479 | "userIdentity":{ | |
480 | "principalId":"" | |
481 | }, | |
482 | "requestParameters":{ | |
483 | "sourceIPAddress":"" | |
484 | }, | |
485 | "responseElements":{ | |
486 | "x-amz-request-id":"", | |
487 | "x-amz-id-2":"" | |
488 | }, | |
489 | "s3":{ | |
490 | "s3SchemaVersion":"1.0", | |
491 | "configurationId":"", | |
492 | "bucket":{ | |
493 | "name":"", | |
494 | "ownerIdentity":{ | |
495 | "principalId":"" | |
496 | }, | |
497 | "arn":"", | |
498 | "id":"" | |
499 | }, | |
500 | "object":{ | |
501 | "key":"", | |
502 | "size":"0", | |
503 | "eTag":"", | |
504 | "versionId":"", | |
505 | "sequencer":"", | |
9f95a23c TL |
506 | "metadata":[], |
507 | "tags":[] | |
eafe8130 TL |
508 | } |
509 | }, | |
510 | "eventId":"", | |
9f95a23c | 511 | "opaqueData":"", |
eafe8130 TL |
512 | } |
513 | ]} | |
514 | ||
515 | - awsRegion: zonegroup | |
516 | - eventTime: timestamp indicating when the event was triggered | |
517 | - eventName: either ``s3:ObjectCreated:``, or ``s3:ObjectRemoved:`` | |
518 | - userIdentity: not supported | |
519 | - requestParameters: not supported | |
520 | - responseElements: not supported | |
521 | - s3.configurationId: notification ID that created the subscription for the event | |
eafe8130 TL |
522 | - s3.bucket.name: name of the bucket |
523 | - s3.bucket.ownerIdentity.principalId: owner of the bucket | |
524 | - s3.bucket.arn: ARN of the bucket | |
525 | - s3.bucket.id: Id of the bucket (an extension to the S3 notification API) | |
526 | - s3.object.key: object key | |
527 | - s3.object.size: not supported | |
528 | - s3.object.eTag: object etag | |
529 | - s3.object.version: object version in case of versioned bucket | |
530 | - s3.object.sequencer: monotonically increasing identifier of the change per object (hexadecimal format) | |
531 | - s3.object.metadata: not supported (an extension to the S3 notification API) | |
9f95a23c | 532 | - s3.object.tags: not supported (an extension to the S3 notification API) |
eafe8130 | 533 | - s3.eventId: unique ID of the event, that could be used for acking (an extension to the S3 notification API) |
9f95a23c | 534 | - s3.opaqueData: opaque data is set in the topic configuration and added to all notifications triggered by the ropic (an extension to the S3 notification API) |
eafe8130 TL |
535 | |
536 | In case that the subscription was not created via a non S3-compatible notification, | |
537 | the events will have the following event format (JSON): | |
538 | ||
539 | :: | |
11fdf7f2 | 540 | |
eafe8130 TL |
541 | {"events":[ |
542 | { | |
543 | "id":"", | |
544 | "event":"", | |
545 | "timestamp":"", | |
546 | "info":{ | |
547 | "attrs":{ | |
548 | "mtime":"" | |
549 | }, | |
550 | "bucket":{ | |
551 | "bucket_id":"", | |
552 | "name":"", | |
553 | "tenant":"" | |
554 | }, | |
555 | "key":{ | |
556 | "instance":"", | |
557 | "name":"" | |
558 | } | |
559 | } | |
560 | } | |
561 | ]} | |
562 | ||
563 | - id: unique ID of the event, that could be used for acking | |
564 | - event: one of: ``OBJECT_CREATE``, ``OBJECT_DELETE``, ``DELETE_MARKER_CREATE`` | |
565 | - timestamp: timestamp indicating when the event was sent | |
566 | - info.attrs.mtime: timestamp indicating when the event was triggered | |
567 | - info.bucket.bucket_id: id of the bucket | |
568 | - info.bucket.name: name of the bucket | |
569 | - info.bucket.tenant: tenant the bucket belongs to | |
570 | - info.key.instance: object version in case of versioned bucket | |
571 | - info.key.name: object key | |
11fdf7f2 TL |
572 | |
573 | Ack Event | |
eafe8130 | 574 | ````````` |
11fdf7f2 TL |
575 | |
576 | Ack event so that it can be removed from the subscription history. | |
577 | ||
578 | :: | |
579 | ||
580 | POST /subscriptions/<sub-name>?ack&event-id=<event-id> | |
581 | ||
eafe8130 | 582 | Request parameters: |
11fdf7f2 | 583 | |
eafe8130 | 584 | - event-id: id of event to be acked |
11fdf7f2 | 585 | |
eafe8130 TL |
586 | .. _Bucket Notification : ../notifications |
587 | .. _Bucket Operations: ../s3/bucketops |