How to Replicate an S3 Bucket Across Regions

s3 logo

If you want to store multiple copies of objects in your S3 buckets in different regions, S3 can be configured to automatically replicate objects from a source bucket to replica buckets across the world, increasing performance and access latency. of your applications.

It should be noted that if the objects you are replicating are publicly available on the Internet, such as a media bucket for downloaded content, you can simply use AWS CloudFront CDN to cache content in edge locations everywhere.

How does replication work?

Replication basically runs every time you download or update an object in a bucket. You create a replication rule, which defines the source and destination bucket. Objects uploaded to the source bucket will be uploaded to the destination bucket. For replication to work, you must enable versioning. This will slightly increase the storage costs, depending on how much you overwrite the data.

There are some limitations. AWS does not replicate deleted objects, so you will need to delete each copy manually. And, while replication usually occurs fairly quickly, it can take up to 15 minutes or more if you are replicating large objects.

When an object is replicated, a metadata tag is defined in the source object. All replicas will have the REPLICA tag, and the source object will be tagged COMPLETED. This means that you can have two-way replication – two compartments with replication rules pointing to each other. S3 will not replicate objects imported with the replica tag, otherwise you will be stuck in an infinite replication loop.

This is called multi-master replication, but there is no system in place to ensure that objects downloaded later are replicated later. In this configuration, it is possible for data to get out of sync if you update two objects simultaneously, and it is possible to mine stale data when synchronizing from one master to another.

Replication configuration

Of S3 management console, navigate to the source bucket. You will need to enable versioning under “Properties”:

activate version management

Once this option is enabled, you can add a new replication rule under Management> Replication> Add rule.

add a replication rule

By default it will replicate all objects, but you can set a filter if you want.

source compartment parameters

Under Destination, select the destination bucket. You can also configure the storage class for replica objects, as well as set the owner if you are transferring between accounts. There is also an option for “S3 Replication Time Control”, which ensures that objects will be replicated within 15 minutes, backed up by a service level agreement.

    Select the destination bucket

On the next screen, you will need to select or create an IAM role for replication. Permissions are handled automatically, just enter a name.

Select or create an IAM role for replication

Click “Next”, then click “Save.” The rule must be active immediately; you can test uploading an object, and you should see it replicated in the destination bucket, then you will see the replication status tag change to COMPLETED.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.