When all your S3 buckets are in the same account, you usually don’t have issues with access. But when you have several accounts (like a lot of companies have these days), you will run into issues. This is mostly because of the way S3 has evolved.
Let me break that down. S3 is a gigantic repository of name-value pairs. The blobs of data that make up the ‘value’ part of the name-value pair can be very very tiny or very very large. Each of these blobs is called an object. These objects are placed in buckets. By default each S3 account is allowed 100 buckets.
Each bucket has a globally unique bucket name. If you create a bucket, no one else can create a bucket with the same name. Don’t worry though, if you delete the bucket, then they can create one with that name. Also note that the buckets exist in regions although the console says it is a global resource.
The bucket is owned by the account in which the bucket resides. By ‘account’ I mean the ‘root’ user of the account. Each object in the bucket is owned by the account that put the object in the bucket. If you have only one AWS account, this is straightforward. Nothing to worry about. But if you have multiple accounts and put the object to the bucket from another account using the CLI, this can soon turn messy. Especially if you have thousands of files.
You can find explanations of the permission model in AWS documents here and here. Essentially this flowchart is as complex as it gets.
The ultimate authority on permission to the object is the object itself. If you decide to give permissions to thousands of objects in a bucket, you have to do it on each individual object. This will take forever. So the best practice is to always create a role for each application that will need access to objects in a bucket, and give that role permissions on the object when it is placed there. Thereafter, have the user assume that role to access the objects.
