Earlier this year, AWS made a critical change to S3 bucket default behaviour regarding S3 object ownership and disabled S3 ACL’s by default on all new S3 buckets. The change involved making the bucket owner the default owner of written objects. Previously, there was an ability to have full ownership of a bucket but have no rights to any objects in the bucket due to ACL’s. This was fraught with challenges in maintaining access to objects in an S3 bucket.
S3 Object Ownership is an Amazon S3 bucket-level setting that you can use to control ownership of objects uploaded to your bucket and to disable or enable access control lists (ACLs). By default, Object Ownership is set to the Bucket owner enforced setting and all ACLs are disabled. When ACLs are disabled, the bucket owner owns all the objects in the bucket and manages access to data exclusively using access management policies. In modern use cases, it is recommended to disable ACL’s unless there is a specific requirement to control access over specific objects.
Object Ownership has three settings that you can use to control ownership of objects uploaded to your bucket and to disable or enable ACLs:
ACLs disabled
Bucket owner enforced (default) – ACLs are disabled, and the bucket owner automatically owns and has full control over every object in the bucket. ACLs no longer affect permissions to data in the S3 bucket. The bucket uses policies to define access control.
ACLs enabled
Bucket owner preferred – The bucket owner owns and has full control over new objects that other accounts write to the bucket with the bucket-owner-full-control canned ACL.
Object writer – The AWS account that uploads an object owns the object, has full control over it, and can grant other users access to it through ACLs.
Background to Challenge
When working with new buckets, having object ownership set to the bucket owner by default is generally ok. Where it gets complicated is when you want to disable ACL’s on an existing bucket with existing objects. A common scenario for this are log buckets for centralised logging. There is a lot of apprehension when changing such settings when there is a need to ensure existing objects are still accessible for audit and compliance purposes. In this particular use case, we had this exact challenge.
The Changes
Changing the object ownership to “bucket owner enforced” applies to all new and existing objects. In the case of an existing bucket that already has objects in it, after you disable ACLs, the object and bucket ACLs are no longer part of an access evaluation, and access is granted or denied on the basis of policies. For existing buckets, you can re-enable ACLs at any time after you disable them, and your preexisting bucket and object ACLs are restored.
Instead of using bucket ACL’s to control access to objects – you apply a bucket policy. The bucket policy determines who has access to objects and/or prefixes in your bucket. Seems simple enough then, right? Well, sort of as it depends on your application/service context and which option you choose (ENFORCED or PREFERRED).
Bucket-Owner-Enforced
After you disable ACLs, your bucket accepts only PUT requests that do not specify an ACL or PUT requests with bucket owner full control ACLs, such as the bucket-owner-full-control canned ACL or equivalent forms of this ACL expressed in XML. Existing applications that support bucket owner full control ACLs see no impact. PUT requests that contain other ACLs (for example, custom grants to certain AWS accounts) fail and return a 400 error with the error code AccessControlListNotSupported.
Bucket-Owner-Preferred
With the “bucket-owner-preferred” setting, the bucket continues to accept and honour bucket and object ACLs. With this setting, new objects that are written with the bucket-owner-full-control canned ACL are automatically owned by the bucket owner rather than the object writer. All other ACL behaviours remain in place. To require all Amazon S3 PUT operations to include the bucket-owner-full-control canned ACL, you can add a bucket policy that allows only object uploads using this ACL.
Visually – the way permissions work with these settings is as per the below:
Important Notes
- If you use S3 Versioning, the bucket owner owns and has full control over all object versions in your bucket. Applying the Bucket owner enforced setting does not add a new version of an object.
- New objects can be uploaded to your bucket only if they use bucket owner full control ACLs or don’t specify an ACL. Object uploads fail if they specify any other ACL.
- You can re-enable ACLs by changing from the Bucket owner enforced setting to another Object Ownership setting at any time.
- If you used object ACLs for permissions management before you applied the Bucket owner enforced setting and you didn’t migrate these object ACL permissions to your bucket policy, after you re-enable ACLs, these permissions are restored.
- Objects written to the bucket while the bucket owner enforced setting was applied are still owned by the bucket owner
- Edge Cases: If you want the object writer to maintain full control of the object that they upload, “object-writer” is the best Object Ownership setting for your use case. If you want to control access at the individual object level, bucket-owner-preferred is the best choice.
- When you use S3 replication and the source and destination buckets are owned by different AWS accounts, you can disable ACLs (with the bucket-owner-enforced setting) to change replica ownership to the AWS account that owns the destination bucket. This setting mimics the existing owner override behaviour without the need of the s3:ObjectOwnerOverrideToBucketOwner permission.
- All objects that are replicated to the destination bucket with the bucket-owner-enforced setting are owned by the destination bucket owner.
Challenge
Any ACL’s that were provided on logging buckets for log ownership were set using “accessControl: s3.BucketAccessControl.LOG_DELIVERY_WRITE” which permitted “delivery.logs.amazonaws.com” to write logs such as VPC flowlogs and CloudWatch logs to the S3 bucket. The change to the object owner (which is now effectively “objectOwnership: s3.ObjectOwnership.BUCKET_OWNER_ENFORCED”) in combination with disabling of ACL’s results in “delivery.logs.amazonaws.com” being unable to write logs to a logging bucket.
Incorrect References
Reference: https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-server-access-logging.html
The S3 user guide has a sample policy to apply to a bucket policy that permits the logging service to write logs to a logging bucket – however the reference to “logging.amazonaws.com” is invalid. The service that is responsible for writing logs is actually “delivery.logs.amazonaws.com”.
Reference: https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/AWS-logs-and-resource-policy.html
To add to the confusion is that the policy is missing the “s3:x-amz-acl”: “bucket-owner-full-control” condition. This condition is described in the below policy (and they are completely conflicting policies between both pages). In our example use case, without this condition, we had no new logs coming through until it was added but also using the defined policy in the above reference also doesn’t work.
Solution
The working policy (described in CDK typescript) is the following (where defaultConstants.DELIVERY_LOGS_SERVICENAME resolves to “delivery.logs.amazonaws.com”):
Relating this back to the example policy in https://docs.aws.amazon.com/AmazonS3/latest/userguide/enable-server-access-logging.html – the example policy should show:
Conclusion
Ensuring your bucket policy provides the equivalent permissions as the previous ACL, disabling ACL’s by enforcing “bucket-owner-enforced” is completely safe. When configuring policies for log buckets – set the principal to “delivery.logs.amazonaws.com”.
