Amazon Web Services, or AWS, is a widely known collection of cloud services created by Amazon. It is a big suite of cloud services that can be used to accomplish a lot of different tasks, all of them based on the cloud, of course, so you can access these services from any location at any time you want.
One of the services provided through AWS is called S3, and today we are going to talk about this service and its cp command, so if you want to know what is the AWS S3 cp command then stay with us and keep reading.
Contents
First off, what is S3? As we said, S3 is one of the services available in Amazon Web Services, its full name is Amazon Simple Storage Service, and as you can guess it is a storage service. It is similar to other storage services like, for example, Google Drive, Dropbox, and Microsoft OneDrive, though it has some differences and a few functions that make it a bit more advanced.
The S3 service is based on the concept of buckets. Buckets are, to put it simply, the “containers” of different files (called objects) that you are going to place in them while using this service.
S3 is a fast, secure, and scalable storage service that can be deployed all over the Amazon Web Services, which consists of (for now) 54 locations across the world, including different locations in North America, Europe, Asia, Africa, Oceania, and South America.
To manage the different buckets in Amazon S3 and their contents is possible to use different commands through the AWS CLI, which a Command Line Interface provided by Amazon to manage their different cloud services based in AWS. One of the many commands that can be used in this command-line interface is cp, so keep reading because we are going to tell you a lot about this tool.
AWS S3 cp command: what is it?
So, what is this cp command exactly? Actually, the cp command is almost the same as the Unix cp command. In Unix and Linux systems this command is used to copy files and folders, and its functions is basically the same in the case of AWS S3, but there is a big and very important difference: it can be used to copy local files but also S3 objects.
Is possible to use S3 to copy files or objects both locally and also to other S3 buckets. The command has a lot of options, so let’s check a few of the more used ones:
–dryrun: this is a very important option that a lot of users use, even more, those who are starting with S3. Like in most software tools, a dry run is basically a “simulation” of the results expected from running a certain command or task. You can use this option to make sure that what you are copying is correct and to verify that you will get the expected result. If you use this option no real changes will be made, you will simply get an output so you can verify if everything would go according to your plans.
–exclude: the exclude option is used to exclude specific files or folders that match a certain given pattern. For example, if you want to copy an entire folder to another location but you want to exclude the .jpeg files included in that folder, then you will have to use this option.
–source-region: this one is a very important option when we copy files or objects from one bucket to another because we have to specify the origin region of the source bucket.
–region: works the same way as –source-region, but this one is used to specify the region of the destination bucket.
–recursive: as you can guess this one is to make the cp command recursive, which means that all the files and folders under the directory that we are copying will be copied too.
AWS S3 cp examples: how does it work?
Let’s see some quick example of how the S3 cp command works:
In the next example we will copy a file called “myphoto.jpg” from our local system to the bucket “myshinybucket”:
aws s3 cp myphoto.jpg s3://myshinybucket/myphoto.jpg
Let’s see another one, in this case, let’s copy the file mydocument.txt from the bucket “oldbucket” to the other one called “newbucket”:
aws s3 cp s3://oldbucket/mydocument.txt s3://newbucket/mydocument.txt
And now for another example let’s copy an entire folder (called “myfolder”) recursively from our local system to a bucket (called “jpgbucket”), but excluding all .png files:
aws s3 cp myfolder s3://jpgbucket/ --recursive --exclude "*.png"
As we can see, using this command is actually fairly simple, and there is a lot more examples that we could include, though this should be enough to cover the basics of the S3 cp command.
S3 CP Synopsis
Here’s the full list of arguments and options for the AWS S3 cp command:
cp <LocalPath> <S3Uri> or <S3Uri> <LocalPath> or <S3Uri> <S3Uri> [--dryrun] [--quiet] [--include <value>] [--exclude <value>] [--acl <value>] [--follow-symlinks | --no-follow-symlinks] [--no-guess-mime-type] [--sse <value>] [--sse-c <value>] [--sse-c-key <value>] [--sse-kms-key-id <value>] [--sse-c-copy-source <value>] [--sse-c-copy-source-key <value>] [--storage-class <value>] [--grants <value> [<value>...]] [--website-redirect <value>] [--content-type <value>] [--cache-control <value>] [--content-disposition <value>] [--content-encoding <value>] [--content-language <value>] [--expires <value>] [--source-region <value>] [--only-show-errors] [--no-progress] [--page-size <value>] [--ignore-glacier-warnings] [--force-glacier-transfer] [--request-payer <value>] [--metadata <value>] [--metadata-directive <value>] [--expected-size <value>] [--recursive]
Summary
Today we have learned about AWS and the S3 service, which is a storage service based on Amazon’s cloud platform. One of the different ways to manage this service is the AWS CLI, a command-line interface. In this CLI there are a lot of commands available, one of which is cp.
The cp command is very similar to its Unix counterpart, being used to copy files, folders, and objects. It can be used to copy content from a local system to an S3 bucket, from bucket to bucket or even from a bucket to our local system, and we can use different options to accomplish different tasks with this command, for example copying a folder recursively.
This time we have barely scratched the surface of what we can do with the AWS command-line interface, though we have covered the basics and some advanced functions of the AWS S3 cp command, so it should be more than enough if you are just looking for information about it.
However, if you want to dig deeper into the AWS CLI and Amazon Web Services we suggest you check its official documentation, which is the most up-to-date place to get the information you are looking for.