AWS S3 as a Backup Target #

Minimal IAM Permissions #

Using AWS S3 or S3-compatible storage as the backup target requires configuring IAM polices for the target bucket. The scope of the access key must be limited to the appropriate IAM permissions as well. [1]

Important
The basic functionality of uploading and verifying backups requires only the actions listed in this section. Allowing addtional actions described in other sections is not necessary and not recommended unless the backup service strictly requires them.

General (no per-object ACLs) #

We can use AWS S3 as a target storage in Rclone configs. Following is the set of permission required to allow rclone CLI to read and write files:

s3:ListBucket
s3:DeleteObject
s3:GetObject
s3:PutObject

If a the backup path within the bucket is under a certain prefix, we can further refine the permissions to allow listing only objects below the given prefix. For example, let’s say we want the backup server to only use the subdir backups/dataset-1/. In this case, we can write the permissions in the following way:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::[iam-arn]"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::[bucker-arn]",
            "Condition": {
                "StringEquals": {
                    "s3:prefix": [
                        "",
                        "backups/",
                        "backups/dataset-1/"
                    ],
                    "s3:delimiter": "/"
                }
            }
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::[iam-arn]"
            },
            "Action": "s3:ListBucket",
            "Resource": "arn:aws:s3:::[bucker-arn]",
            "Condition": {
                "StringLike": {
                    "s3:prefix": "backups/dataset-1/*"
                }
            }
        },
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::[iam-arn]"
            },
            "Action": [
                "s3:DeleteObject",
                "s3:GetObject",
                "s3:PutObject"
            ],
            "Resource": "arn:aws:s3:::[bucker-arn]/backups/dataset-1/*"
        }
    ]
}

Notice that the first clause uses the StringEquals condition, while the second one uses StringLike. This ensures that the rclone CLI can list only the contents of its own prefix and everything nested within the prefix.

With per-object ACLs #

Normal backups usually work with the four actions listed above. But if the backup service is required to set per-object ACLs, we should also allow the s3:PutObjectAcl action. [2]

In some online guides and docs this action is often allowed together with s3:PutObject and other object-level actions. [3]

Support Multi-Part Uploads #

For files larger than 5 MB, backup services often switch to multi-part uploads. For the service to be able to handle parts of interrupted uploads, we should also include the following permissions. [4]

Bucket level:

s3:ListBucketMultipartUploads

Object / Prefix level:

s3:AbortMultipartUpload
s3:ListMultipartUploadParts

Additional Permissions #

List All Buckets #

While not necessary for the backup process itself, some backup tools may take advantage of additonal permissions for convenience, such as listing all available buckets and the target bucket selection in the UI. [5] [6]

Such services may use the following actions.

s3:GetBucketLocation
s3:ListAllMyBuckets

While the policy example in the Rclone docs includes s3:ListAllMyBuckets and s3:PutObjectAcl described above, these permissions are not strictly necessary for it to work. So, it is best to not allow these actions.

Create Buckets #

To enable the backup service to create buckets, we can allow the s3:CreateBucket action.