How do S3 permissions work?

Amazon S3 is a great (and cheap) content storage and delivery service, used by millions of websites and applications around the world. However, their permissions system can be opaque to new users, and difficult to understand, due to the variety of ways you can set permissions, and inconsistent terminology in different UIs. This post aims to clear up the confusion.

First of all, it’s important to understand some AWS/S3 terminology:

  • Buckets are like a top level folder in S3. You typically create a bucket for each individual requirement you may have (e.g. an uploads bucket, a backup bucket, maybe a cdn/asset bucket)
  • Objects are files inside of a bucket. Each object can have it’s own permissions
  • IAM stands for identity and access management, and is the system used to create sub-accounts with limited permissions for your main AWS account (which you should almost never be using, kind of like the root user).

Also worth noting, while it may appear that S3 has folders, it really doesn’t, it just has file names that look like folders (e.g. path/to/some/file.jpg isn’t an object named file.jpg in a folder, its an object named path/to/some/file.jpg that’s in a specific bucket).

Getting Started

When dealing with S3, you have two distinct permission systems. Access Control Policies (ACPs) are a simplified permission system primarily used by the web UI, that basically just wraps the other permission system in a layer of abstraction. The other system are IAM access policies (broken down into user and bucket policies depending on what you apply them to), and are JSON objects that define very fine grained permissions.

The other thing to keep in mind is that permissions can apply either to a bucket or an object. Bucket permissions are different than object permissions, and are tracked differently. If you want someone to be able to view the list of files in a bucket, and actually view/download those files, you must grant them permission on the bucket itself, as well as each object.

ACPs/ACLs

ACPs (access control policies) or ACLs (access control lists) are a very simplistic permission system offered by S3. When using the web UI, the “Permissions” tab of an Object’s properties represents the ACP. Objects and Buckets can each have an ACL, and offer similar permissions.

Bucket ACLs affect bucket operations, but not operations on the contents of the bucket. For example, you may have the read permission on a bucket, so you can get a list of objects in the bucket, but if you don’t have the read permission on an individual object, you won’t be able to view it.

ACLs have 4 available permissions:

  • read (The web UI displays this as “List” for buckets and “Read/Download” for objects)
    • When applied to a bucket: authorized users can list the file names, their sizes, and last modified dates from a bucket
    • When applied to an object: authorized users can download the file
  • write (The web UI displays this as “Upload/Delete” for buckets, and does not display the option for objects)
    • When applied to a bucket: authorized users can upload new files to the bucket. They can also delete objects (even if they don’t have permissions on THAT object).
    • When applied to an object: authorized users can replace the file or delete it
  • read-acp (the web UI displays this as “View Permissions”)
    • When applied to a bucket: authorized users can view the ACL of the bucket
    • When applied to an object: authorized users can view the ACL of the object
  • write-acp (the web UI displays this as “Edit Permissions”)
    • When applied to a bucket: authorized users can modify the ACL of the bucket
    • When applied to an object: authorized users can modify the ACL of the object

Aside from the permissions, you must also specify the “Grantee”. You can have up to 20 policies per object, each having some combination of the above 4 permissions, applied to a specific Grantee. The grantee can be the email address of an AWS account to grant permissions on (it will include all IAM accounts), or one of the special predefined groups that AWS provides.

You can not apply an ACL to an individual IAM account.

The special groups available as grantees are:

  • aws – Represents your own AWS account (including all IAM users)
  • AllUsers (displayed as “Everyone” in the web UI) – Allow anyone to access the resource. Allows authenticated or anonymous requests. Essential the “public” group
  • AuthenticatedUsers (displayed as “Authenticated Users” in the web UI) – Allow any AWS account to access the resource. Requests must be signed.
  • LogDelivery (displayed as “Log Delivery” in the web UI) – Allows AWS to write server access logs to the bucket

ACL Permissions and their corresponding S3 operations

ACL permissions are just a combination of more fine grained AWS policy permissions. If you want to know exactly which operations you are permitting, refer to the following:

  • read
    • Buckets: s3:ListBucket, s3:ListBucketVersions, and s3:ListBucketMultipartUploads
    • Objects: s3:GetObject, s3:GetObjectVersion, and s3:GetObjectTorrent
  • write:
    • Buckets: s3:PutObject and s3:DeleteObject
    • Objects: Not applicable
  • read-acp:
    • Buckets: s3:GetBucketAcl
    • Objects: s3:GetObjectAcl and s3:GetObjectVersionAcl
  • write-acp:
    • Buckets: s3:PutBucketAcl
    • Objects: s3:PutObjectAcl and s3:PutObjectVersionAcl

Bucket Policies

Bucket policies are AWS Access Policies that apply to a specific S3 bucket, and are a great way to apply more fine grained access controls to an entire bucket, or to apply the same permissions to a large number of objects without the need to manually change them all to adjust the policy. A bucket policy can apply to the bucket itself, and all objects in the bucket, although you can easily specific specific resources or patterns (e.g. you could specify a resource of “movies/*” to apply the permissions to all objects in the movies folder).

You can add a policy to your S3 bucket using the web ui. The action is under the Permissions tab of the bucket properties:

For information on creating access policies, keep reading.

IAM User Policies

User policies are AWS Access Policies that apply to a specific S3 IAM user account. They are a great way to apply very limited permissions to an IAM role.

Here are a few common use cases:

  • A role used for database backups should only be able to create objects, and only in a specific bucket, not view/delete them (s3:PutObject)
  • A role used for a dashboard or some other sort of display should have read only permissions on the bucket/objects

Since ACL permissions can’t be a applied to a specific IAM account, IAM user policies are the answer. It can be difficult to decide if you should use an IAM or bucket policy in some cases. If you want to give a specific user permissions across various buckets, an IAM policy is probably best. Also, if you have a large number of users each needing different sets of permissions, IAM policies may be more suitable that a bucket policy, as bucket policies are limited to 20kb.

You can use inline user policies, group policies, or managed user policies.

Unlike bucket policies, you do not specify the principal for a user policy, as it always applies to whichever user is performing the operation.

Access Policies

An access policy is made up of one or more statements. Each statement specify principals which define who the statement apply to, resources define what the statement applies to, and then various allowed or disallowed operations.

Here is an example bucket policy (you’ll note that it’s a JSON object):

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "ExampleStatement1",
         "Effect": "Allow",
         "Principal": {
            "AWS": "arn:aws:iam::Account-ID:user/brandon"
         },
         "Action": [
            "s3:GetBucketLocation",
            "s3:ListBucket",
            "s3:GetObject"
         ],
         "Resource": [
            "arn:aws:s3:::examplebucket"
         ]
      }
   ]
}

As a note, the version must be one of the defined policy language versions. At the time of this post, 2012-10-17 is the latest version.

The example statement given uses the following elements:

  • Sid (statement id) – An optional identifier for a policy statement
  • Effect – A required element that specifies whether the statement will result in an allow or a deny. Can be one of “Allow” or “Deny”.
  • Principal – A required element that specifies the user (IAM user, federated user, or assumed-role user), AWS account, AWS service, or other entity that the statement applies to. For user policies, the principal is omitted as the policy always applies to the current user performing an operation
  • Action – Describes which specific actions will be allowed or denied (based on the specified Effect). Each AWS service has it’s own set of actions that describe tasks you can perform with that service. You can use the wildcard character (*)  to grant broad permissions (e.g. s3:* will allow a user to perform any valid s3 operation on a resource). Please note that there is also a “NotAction” element that lets you specify an exception to the list of actions.
  • Resource – Describes which objects the statement applies to, using ARN (Amazon Resource Name) format. There is also a “NotResource” element that lets you specify exceptions to the list of resources specified,

For a full list of available elements, their values, and what they do, refer to the AWS documentation on Access Policy Language Elements.

Here is an example of an IAM user policy that allows the user to upload files to a specific folder in a specific S3 bucket, but explicitly denies all other operations (regardless of other policies that may grant permissions on it). Perfect for a backup user:

{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Sid": "BackupUserDenyEverything",
         "Effect": "Deny",
         "Action": "*",
         "NotAction": "s3:PutObject",
         "Resource": "*"
      },
      {
         "Sid": "BackupUserAllowUploads",
         "Effect": "Allow",
         "Action": [
            "s3:PutObject"
         ],
         "Resource": [
            "arn:aws:s3:::mybackups/mysql_backups/*"
         ]
      }
   ]
}

You’ll note that we use a combination of two statements to get the desired effect. The first statement prevents the user from performing any AWS operation except for s3:PutObject. However, it doesn’t actually grant the user permission to use the s3:PutObject operation so we must create a second statement to do that.

Evaluation Permissions

The last piece of the puzzle is how to determine if a request will be allowed or denied, which can be confusing if you don’t know the rules. However, once you know the rules, it’s very straight forward.

AccessPolicyLanguage_Evaluation_Flow.diagramAs you can see in the above diagram, it’s actually quite simple. If any of your policies explicitly deny an operation (e.g. our example user policy), the end result is a Deny. If any of your policies allows an operation, and there are no explicit denies, then the operation is allowed. If none of your policies explicitly allows or denies an operation, the operation is denied.

Conclusion

Hopefully after reading this post you have a basic understanding of S3 permissions and how to use them securely. I strongly recommend reading the AWS documentation for more information.

 

2 Comment

  1. Matt says: Reply

    Hi Brandon,

    are you sure about this bullet from the ACL write section:
    “When applied to a bucket: authorized users can upload new files to the bucket. They can also delete objects (even if they don’t have permissions on THAT object).”

    From my experiments, I don’t think having this permission at the bucket level will override Deny statements on objects written into the bucket policy.

    Maybe the account owner can do it.

  2. Vince says: Reply

    Hey Brandon, nice article.

    A question which you may have an answer to – how do you determine whether an IAM user can read, update or delete an object when their permissions are applied via user policies?

    Is there any API calls that can retrieve IAM user specific permissions like there is with Object ACL calls?

    Thanks
    Vince

Leave a Reply