ceph/doc/radosgw/s3-notification-compatibility.rst

   1 =====================================
   2 S3 Bucket Notifications Compatibility
   3 =====================================
   4
   5 Ceph's `Bucket Notifications`_ and `PubSub Module`_ APIs follow `AWS S3 Bucket Notifications API`_. However, some differences exist, as listed below.
   6
   7
   8 .. note::
   9
  10     Compatibility is different depending on which of the above mechanism is used
  11
  12 Supported Destination
  13 ---------------------
  14
  15 AWS supports: **SNS**, **SQS** and **Lambda** as possible destinations (AWS internal destinations).
  16 Currently, we support: **HTTP/S** and **AMQP**. And also support pulling and acking of events stored in Ceph (as an intenal destination).
  17
  18 We are using the **SNS** ARNs to represent the **HTTP/S** and **AMQP** destinations.
  19
  20 Notification Configuration XML
  21 ------------------------------
  22
  23 Following tags (and the tags inside them) are not supported:
  24
  25 +-----------------------------------+----------------------------------------------+
  26 | Tag                               | Remaks                                       |
  27 +===================================+==============================================+
  28 | ``<QueueConfiguration>``          | not needed, we treat all destinations as SNS |
  29 +-----------------------------------+----------------------------------------------+
  30 | ``<CloudFunctionConfiguration>``  | not needed, we treat all destinations as SNS |
  31 +-----------------------------------+----------------------------------------------+
  32
  33 REST API Extension
  34 ------------------
  35
  36 Ceph's bucket notification API has the following extensions:
  37
  38 - Deletion of a specific notification, or all notifications on a bucket, using the ``DELETE`` verb
  39
  40  - In S3, all notifications are deleted when the bucket is deleted, or when an empty notification is set on the bucket
  41
  42 - Getting the information on a specific notification (when more than one exists on a bucket)
  43
  44   - In S3, it is only possible to fetch all notifications on a bucket
  45
  46 - In addition to filtering based on prefix/suffix of object keys we support:
  47
  48   - Filtering based on regular expression matching
  49
  50   - Filtering based on metadata attributes attached to the object
  51
  52 - Filtering overlapping is allowed, so that same event could be sent as different notification
  53
  54
  55 Unsupported Fields in the Event Record
  56 --------------------------------------
  57
  58 The records sent for bucket notification follow format described in: `Event Message Structure`_.
  59 However, the following fields may be sent empty, under the different deployment options (Notification/PubSub):
  60
  61 +----------------------------------------+--------------+---------------+------------------------------------------------------------+
  62 | Field                                  | Notification | PubSub        | Description                                                |
  63 +========================================+==============+===============+============================================================+
  64 | ``userIdentity.principalId``           | Supported    | Not Supported | The identity of the user that triggered the event          |
  65 +----------------------------------------+--------------+---------------+------------------------------------------------------------+
  66 | ``requestParameters.sourceIPAddress``  |         Not Supported        | The IP address of the client that triggered the event      |
  67 +----------------------------------------+--------------+---------------+------------------------------------------------------------+
  68 | ``requestParameters.x-amz-request-id`` | Supported    | Not Supported | The request id that triggered the event                    |
  69 +----------------------------------------+--------------+---------------+------------------------------------------------------------+
  70 | ``requestParameters.x-amz-id-2``       | Supported    | Not Supported | The IP address of the RGW on which the event was triggered |
  71 +----------------------------------------+--------------+---------------+------------------------------------------------------------+
  72 | ``s3.object.size``                     | Supported    | Not Supported | The size of the object                                     |
  73 +----------------------------------------+--------------+---------------+------------------------------------------------------------+
  74
  75 Event Types
  76 -----------
  77
  78 +----------------------------------------------+-----------------+-------------------------------------------+
  79 | Event                                        | Notification    | PubSub                                    |
  80 +==============================================+=================+===========================================+
  81 | ``s3:ObjectCreated:*``                       | Supported                                                   |
  82 +----------------------------------------------+-----------------+-------------------------------------------+
  83 | ``s3:ObjectCreated:Put``                     | Supported       | Supported at ``s3:ObjectCreated:*`` level |
  84 +----------------------------------------------+-----------------+-------------------------------------------+
  85 | ``s3:ObjectCreated:Post``                    | Supported       | Not Supported                             |
  86 +----------------------------------------------+-----------------+-------------------------------------------+
  87 | ``s3:ObjectCreated:Copy``                    | Supported       | Supported at ``s3:ObjectCreated:*`` level |
  88 +----------------------------------------------+-----------------+-------------------------------------------+
  89 | ``s3:ObjectCreated:CompleteMultipartUpload`` | Supported       | Supported at ``s3:ObjectCreated:*`` level |
  90 +----------------------------------------------+-----------------+-------------------------------------------+
  91 | ``s3:ObjectRemoved:*``                       | Supported       | Supported only the specific events below  |
  92 +----------------------------------------------+-----------------+-------------------------------------------+
  93 | ``s3:ObjectRemoved:Delete``                  | Supported                                                   |
  94 +----------------------------------------------+-----------------+-------------------------------------------+
  95 | ``s3:ObjectRemoved:DeleteMarkerCreated``     | Supported                                                   |
  96 +----------------------------------------------+-----------------+-------------------------------------------+
  97 | ``s3:ObjectRestore:Post``                    | Not applicable to Ceph                                      |
  98 +----------------------------------------------+-----------------+-------------------------------------------+
  99 | ``s3:ObjectRestore:Complete``                | Not applicable to Ceph                                      |
 100 +----------------------------------------------+-----------------+-------------------------------------------+
 101 | ``s3:ReducedRedundancyLostObject``           | Not applicable to Ceph                                      |
 102 +----------------------------------------------+-----------------+-------------------------------------------+
 103
 104 Topic Configuration
 105 -------------------
 106 In the case of bucket notifications, the topics management API will be derived from `AWS Simple Notification Service API`_.
 107 Note that most of the API is not applicable to Ceph, and only the following actions are implemented:
 108
 109  - ``CreateTopic``
 110  - ``DeleteTopic``
 111  - ``ListTopics``
 112
 113 We also extend it by:
 114
 115  - ``GetTopic`` - allowing for fetching a specific topic, instead of all user topics
 116  - In ``CreateTopic`` we allow setting endpoint attributes
 117
 118 .. _AWS Simple Notification Service API: https://docs.aws.amazon.com/sns/latest/api/API_Operations.html
 119 .. _AWS S3 Bucket Notifications API: https://docs.aws.amazon.com/AmazonS3/latest/dev/NotificationHowTo.html
 120 .. _Event Message Structure: https://docs.aws.amazon.com/AmazonS3/latest/dev/notification-content-structure.html
 121 .. _`PubSub Module`: ../pubsub-module
 122 .. _`Bucket Notifications`: ../notifications