Skip to main content

Amazon S3

Overview

Amazon Simple Storage Service (S3) Replication is an elastic, fully managed, low cost feature that replicates objects between buckets. S3 Replication offers the most flexibility and functionality in cloud storage, giving you the controls you need to meet your data sovereignty and other business needs.

With Amazon S3 Replication, you can configure Amazon S3 to automatically replicate S3 objects across different AWS Regions by using S3 Cross-Region Replication (CRR) or between buckets in the same AWS Region by using S3 Same-Region Replication (SRR). S3 Replication offers the flexibility of replicating to multiple destination buckets in the same, or different AWS Regions. S3 Replication supports two-way replication between two or more buckets in the same or different AWS Regions. While live replication like CRR and SRR automatically replicates newly uploaded objects as they are written to your bucket, S3 Batch Replication allows you to replicate existing objects. You can use S3 Batch Replication to backfill a newly created bucket with existing objects, retry objects that were previously unable to replicate, migrate data across accounts, or add new buckets to your data lake. Customers needing a predictable replication time backed by a Service Level Agreement (SLA) can use Replication Time Control (RTC) to replicate objects in less than 15 minutes.

Amazon S3 Replication also provides detailed metrics and notifications to monitor the status of object replication between buckets. You can monitor replication progress by tracking bytes pending, operations pending, replication latency, and operations failed replication using the S3 console or Amazon CloudWatch. You can also set up S3 Event Notifications to receive replication failure notifications to quickly diagnose and correct configuration issues. S3 Replication metrics and notifications helps you closely monitor replication progress. To learn more, visit Monitoring progress with replication metrics and Amazon S3 event notifications.

When to use S3 Replication

If you need to maintain multiple copies of your data in the same, or different AWS Regions, with different encryption types, or across different accounts. S3 Replication powers your global content distribution needs, compliant storage needs, and data sharing across accounts.

If you need to ensure your replica copies are identical to the source data, you can use S3 Replication to make copies of your objects that retain all metadata, such as the original object creation time, object access control lists (ACLs), and version IDs.

You can use S3 Replication to put objects into S3 Glacier, S3 Glacier Deep Archive, or another storage class in the destination buckets. You can also replicate your data to the same storage class and then use S3 Lifecyle policies to move your objects to a more cost-effective storage.

Regardless of who owns the source object, you can tell Amazon S3 to change replica ownership to the AWS account that owns the destination bucket to restrict access to object replicas.

You can use Amazon S3 Replication Time Control (S3 RTC) to replicate your data in a predictable time frame. S3 RTC replicates 99.99 percent of new objects stored in Amazon S3 within 15 minutes of upload and is backed by a Service Level Agreement (SLA).

Amazon S3 Versioning, Object Lock, & Replication

More info

Amazon S3 Cross-Region Replication (CRR)

With S3 Cross-Region Replication (CRR), you can replicate objects (and their respective metadata and object tags) into other AWS Regions for reduced latency, compliance, security, disaster recovery, and other use cases. S3 CRR can be configured from a single source S3 bucket to replicate objects into one or more destination buckets in another AWS Region.

Amazon S3 CRR automatically replicates data between buckets across different AWS Regions. With CRR, you can set up replication at a bucket level, a shared prefix level, or an object level using S3 object tags. You can use CRR to provide lower-latency data access in different geographic regions. CRR can also help if you have a compliance requirement to store copies of data hundreds of miles apart. You can use CRR to change account ownership for the replicated objects to protect data from accidental deletion. To learn more about CRR, visit the S3 Replication documentation.

Use cases

Compliance — Amazon S3 stores your data across multiple geographically distant Availability Zones by default, but compliance requirements might dictate that you store data at even greater distances. CRR enables you to replicate data between distant AWS Regions to satisfy these requirements.

Latency performance — If your customers or end-users are distributed across one or more geographic locations, you can minimize latency for data access by maintaining multiple object copies in AWS Regions that are geographically closer to your customers.

Regional efficiency — If you have compute clusters in two or more AWS Regions that analyze the same set of objects, you might choose to maintain object copies in all of those AWS Regions.

Amazon S3 Same-Region Replication (SRR)

Amazon S3 SRR is an S3 feature that automatically replicates data between buckets within the same AWS Region. With SRR, you can set up replication at a bucket level, a shared prefix level, or an object level using S3 object tags. You can use SRR to make one or more copies of your data in the same AWS Region. SRR helps you address data sovereignty and compliance requirements by keeping a copy of your data in a separate AWS account in the same region as the original. You can use SRR to change account ownership for the replicated objects to protect data from accidental deletion. You can also use SRR to easily aggregate logs from different S3 buckets for in-region processing, or to configure live replication between test and development environment. 

Use cases

Aggregate logs into a single bucket — If you store logs in multiple buckets or across multiple accounts, you can easily replicate logs into a single, in-region bucket. This allows for simpler processing of logs in a single location.

Replication between developer and test accounts — If you or your customers have developer and test accounts that use the same data, you can replicate objects between those multiple accounts, while maintaining object metadata, by implementing SRR rules.

Abide by data sovereignty laws — Often customers are required to store data in separate AWS accounts while being barred from letting the data leave a certain region. Same-Region replication can help you back up critical data when compliance regulations don't allow the data to leave your country.

Amazon S3 Batch Replication

While live replication like CRR and SRR automatically replicates newly uploaded objects as they are written to your bucket, S3 Batch Replication allows you to replicate existing objects. You can use S3 Batch Replication to backfill a newly created bucket with existing objects, retry objects that were previously unable to replicate, migrate data across accounts, or add new buckets to your data lake. S3 Batch Replication works on any amount of data, giving you a fully managed way to meet your data sovereignty and compliance, disaster recovery, and performance optimization needs. You can get started with S3 Batch Replication with just a few clicks in the S3 console or a single API request.

S3 Batch Replication complements Same-Region Replication (SRR) and Cross-Region Replication (CRR). S3 Batch Replication replicates existing objects, while SRR and CRR monitor new object uploads and replicate them between buckets.

Use cases

Backfill newly created buckets — If you have a new multi-region storage initiative that requires you to set up new buckets and backfill them with existing objects from another bucket, you can use Batch Replication to replicate these objects.

Retry replication — If you need to retry replication for a variety of reasons - including when objects failed to replicate initially, when objects have previously been successfully replicated to one destination but now need to be replicated to another destination, or when replicating replica objects from another source - you can use Batch Replication to retry replication.

Migration — If you need to migrate existing objects across buckets, whether they are in a different Region or a different account, you can use Batch Replication for the migration to preserve metadata and version ID.

Amazon S3 Replication Time Control

Amazon S3 replication time control helps you meet compliance "or business requirements" for data replication and provides visibility into Amazon S3 replication activity. Replication time control replicates most objects "that you upload" to Amazon S3 in seconds, and 99.99 percent of those objects within 15 minutes. S3 Replication Time Control, by default, includes S3 replication metrics and S3 event notifications, with which you can monitor the total number of S3 API operations that are pending replication, the total size of objects pending replication, and the maximum replication time.

S3 Replication Time Control is backed by a Service Level Agreement (SLA) on the replication of 99.9% of objects within 15 minutes during any billing month.

To learn more about S3 Replication Time Control, visit the S3 Replication documentation page or the S3 Replication FAQs.

Getting started with S3 Replication

Amazon S3 Replication (CRR, SRR) and S3 Replication Time Control can be configured at the S3 bucket level, a shared prefix level, or an object level using S3 object tags. Additionally, S3 Replication Time Control can be enabled for one or more region pairs.

To get started with S3 Replication, access the S3 Replication tutorial, access the S3 Batch Replication tutorial, and view the S3 Replication FAQs and the S3 Replication documentation. For for S3 Replication pricing, view the S3 pricing page.