Forum

Introducing S3: Mul...
 
Notifications
Clear all

Introducing S3: Multi-region Storage Backup with Cross-Region Replication  


taichi
Posts: 83
Moderator
(@taichi)
Member
Joined: 8 months ago

S3: Multi-region Storage Backup with Cross-Region Replication

This lab walks you through the process of enabling Cross-Region Replication on an S3 bucket. You will create source and destination buckets, enable versioning, then create varius replication policies to demonstrate different methods of replicating objects.

 

 

SPL-205 - Version 1.0.3

 

Lab overview

Amazon S3 is object storage built to store and retrieve any amount of data from anywhere on the Internet. It’s a simple storage service that offers an extremely durable, highly available, and infinitely scalable data storage infrastructure at very low costs.

Cross-region replication (CRR) enables automatic, asynchronous copying of objects across buckets in different AWS Regions. Cross-Region Replication can help you do the following:

  • Comply with compliance requirements — Although Amazon S3 stores your data across multiple geographically distant Availability Zones by default, compliance requirements might dictate that you store data at even greater distances. Cross-region replication allows you to replicate data between distant AWS Regions to satisfy these requirements.
  • Minimize latency — If your customers are in two geographic locations, you can minimize latency in accessing objects by maintaining object copies in AWS Regions that are geographically closer to your users.
  • Increase operational efficiency — If you have compute clusters in two different AWS Regions that analyze the same set of objects, you might choose to maintain object copies in those Regions.
  • Maintain object copies under different ownership — Regardless of who owns the source object you can tell Amazon S3 to change replica ownership to the AWS account that owns the destination bucket. This is referred to as the owner override option. You might use this option restrict access to object replicas.

This lab demonstrates the process of configuring Cross-Region Replication (CRR) between two S3 buckets in separate regions.

Topics covered

By the end of this lab, you will be able to:

  • Create a source and destination S3 bucket.
  • Create a Cross-Region Replication policy.
  • Enable replication for an entire bucket, specific folder, or specific tag.
  • Identify the conditions necessary for replicating objects.
  • Delete replicated files and understand how deletions are replicated.

Technical knowledge prerequisites

To successfully complete this lab, you should be familiar with basic navigation of the AWS Management Console and Amazon S3 buckets.

  •  

Task 1: Create and configure source and destination buckets

Before Cross-Region Replication (CRR) can be enabled, you must first create the source and destination buckets. Versioning must be enabled for both buckets in order to configure CRR. Any objects that reside in the bucket before versioning is enabled will not be replicated.

In this task, you will:

  • Create the source and destination buckets.
  • Enable versioning on each bucket.
  1. At the top-left of the AWS Management Console, click the Services menu, and then click S3.

Note: You can also search for  at the top of the services menu.

  1. Click Create bucket and then configure:
  • Bucket name: 
  • Replace NUMBER with a random four digit number
  • Note: Bucket names cannot contain uppercase letters or spaces
  • Region: US East (N. Virginia)
  1. Click Create bucket to accept the default options and create the bucket.

  2. Click the name of the bucket you created in the previous step, and then click the Properties tab.

  3. Click the Versioning card, then:

  • Select  Enable versioning
  • Click Save

  • Now that you have created the source bucket and enabled versioning on it, you will create the destination bucket to replicate to. The destination bucket must have versioning enabled as well, but this time you will enable it using the bucket creation wizard.

  1. Click the Amazon S3 link at the top-left to return to the S3 console main page

  2. Click Create bucket then configure:

  • Bucket name: 
  • Replace NUMBER with a random four digit number
  • Region: US West (Oregon)
  1. Click Create bucket

  2. Click the name of your my-destination bucket.

  3. Click the Versioning card, then:

  • Select  Enable versioning
  • Click Save
  1. Click the Amazon S3 link at the top-left to return to the S3 console main page

Task 2: Enable Cross-Region Replication on a bucket

Now that the source and destination buckets have been created and configured, replication can be enabled. Cross-Region Replication policies are used to determine which objects in a bucket are replicated. You can replicate an entire bucket, a specific folder within a bucket, or any objects with a specified tab. However, objects that already exist in the bucket before replication is enabled will NOT be replicated.

In this task, you will:

  • Create files to test replication with.
  • Create a replication policy to enable replication of an entire bucket.
  • Validate the object replicated properly.
  1. On your local system, create four text files to use throughout this task:
  1. On the S3 console main page, click the name of your source bucket.

  2. Click  Upload

  3. Click Add files and then browse to and select the pre-crr.txt file you created in step 1 of this task.

  4. Click Upload

  5. Verify the pre-crr.txt file is listed in the bucket.

  6. Click the Management tab, and then click Replication

  7. Click  Add rule and then configure:

  • Set source: Entire bucket
  • Click Next
  • Destination bucket:
    •  Buckets in this account
    • Click your my-destination bucket - You may have to scroll down to see the bucket
  • Click Next
  1. On the Configure rule options screen, configure:
  • IAM role: S3-CRR-Role
  • Rule name: 

Note: In the search box, type  to find the IAM role.

  1. Click Next

  2. Review your settings, and then click Save

 If you receive the following warning message after you create the replication rule, you can safely ignore it.

Warning

Now that replication is enabled on the bucket, verify whether or not the file in the bucket has been replicated.

  1. Click the Amazon S3 link at the top-left to return to the S3 console main page.

  2. Click the name of your destination bucket to open it.

 Notice that the destination bucket is empty, even though replication is enabled and the source bucket contains a file. This is because only new files uploaded to the source bucket after replication is enabled will be replicated to the destination bucket.

  1. Click the Amazon S3 link at the top-left to return to the S3 console main page.

  2. Click the name of your source bucket.

  3. Upload the crr-bucket.txt file to your bucket.

  4. Click the Amazon S3 link at the top-left to return to the S3 console main page, and then click the name of your destination bucket to open it.

  5. Notice that the destination bucket now contains the crr-bucket.txt file you uploaded to the source bucket.

If no files are listed, wait a few seconds, and then click the  refresh button at the top-right.

 Congratulations! You've successfully configured Cross-Region Replication for an entire S3 bucket.

Task 3: Configure replication of a single folder

In Amazon S3, folders are considered prefixes. For example, a folder in your S3 bucket named Source would be a prefix notated as Source/. A file inside that folder would be notated as Source/File.

Knowing this allows you to create a replication policy based on a prefix, which will include all objects in the folder. Choosing a folder to replicate allows you to replicate a specific set of objects easily, rather than an entire bucket.

In this task, you will:

  • Create a replication policy to enable replication of the contents of a folder (prefix).
  • Create the folder to replicate and upload a file into it.
  • Validate the object replicated properly.
  1. If you are not already on the S3 console main page, click the Amazon S3 link at the top-left to navigate to it.

  2. Click the name of your source bucket.

  3. Click  Create folder and then configure:

  • Folder name: 
  • Click Save
  1. Click the Management tab, and then click Replication

  2. Select the  crr-full-bucket policy you created previously.

  3. Click the Actions  menu, and then click Disable rule.

  4. Click Confirm to disable the rule.

  5. Click  Add rule

  6. Under Set source, click  Prefix or tags, and then:

  • Enter 
  • Click prefix crr-test
  • Click Next
  1. On the Set destination screen, verify your destination bucket is displayed, and then click Next

  2. On the Configure rule options screen, configure:

  • Rule name: 
  • Click Next
  1. On the Review screen, verify your settings, and then click Save

  2. Click the Overview tab to return to the bucket file list.

  3. Open the crr-test folder.

  4. In the crr-test folder, upload your crr-folder.txt file.

  5. Navigate to your my-destination bucket.

  6. You should see two objects listed in the destination folder:

  • crr-test (folder)
  • crr-bucket.txt (file)
  1. Open the crr-test folder. You should see the crr-folder.txt file you uploaded to the source bucket.

  2. Click the Amazon S3 link at the top-left to return to the S3 console main page

 Congratulations! You've successfully configured Cross-Region Replication for a single folder within an S3 bucket.

Task 4: Configure replication using tags

Tags can be used to identify specific objects to replicate, rather than replicating the entire bucket or folder.

In this task, you will:

  • Create a replication policy to replicate any object with a specific tag.
  • Upload a file and add the tag to it.
  • Validate the object replicated properly.

 Much like versioning, objects with tags must be uploaded to the source bucket after the replication policy using tags has been created and enabled. Objects that are uploaded and tagged prior to the policy being created will not replicate.

  1. If you are not already on the S3 console main page, click the Amazon S3 link at the top-left to navigate there.

  2. On the S3 console main page, click the name of your source bucket.

  3. Click the Management tab, and then click Replication

  4. Click  Add rule and then configure:

  • Set source:

    •  Prefix or tags
    • Enter 
    • Click tag replicate
    • Enter 
    • Press Enter
  •  Tag keys and values are case sensitive. Refer to the additional resources section at the end of the lab for more information.

  1. Click Next

  2. On the Set destination screen, verify your destination bucket is displayed, and then click Next

  3. On the Configure rule options screen, configure:

  • Rule name: 
  • Click Next
  1. On the Review screen, verify your settings, and then click Save

  2. Click the Overview tab.

  3. Click  Upload

  4. Click Add files

  5. Select the crr-tag.txt file you created previously, and then click Next

  6. On the Set permissions screen, accept the defaults, and then click Next

  7. On the Set properties screen, scroll down to the Tag section.

  8. In the Tag section, configure:

  • Key: 
  • Value: 

 Tags are case-sensitive.

  1. Click Save and then click Next

  2. On the Review screen, review the settings, and then click Upload

  3. Verify the crr-tag.txt file was uploaded successfully.

  4. Navigate to your destination bucket.

  5. You should notice that the destination bucket now contains the crr-tag.txt file you just uploaded to the source bucket.

If no files are listed, wait a few seconds, then click the  refresh button at the top-right.

  1. Click the Amazon S3 link at the top-left to return to the S3 console main page.

 Congratulations! You've successfully configured Cross-Region Replication using a tag.

Task 5: Deleting Replicated Files

To protect against malicious intent and accidental deletion, object deletions that occur in a source bucket are not replicated to the destination bucket.

In this task, you will:

  • Delete a file that has been replicated, then observe the results.
  1. If you are not already on the S3 console main page, click the Amazon S3 link at the top-left to navigate there.

  2. Navigate to your source bucket.

  3. In your source bucket:

  • Select  crr-tag.txt
  • Click the Actions menu, and then click Delete.
  1. In the confirmation window that opens, click Delete

  2. Verify the crr-tag.txt file has been deleted from the source bucket.

  3. Navigate to your destination bucket.

  4. You should notice that the crr-tag.txt file still exists in the destination bucket.

 Congratulations! You've discovered that deleting an object from a source bucket does not delete it from the destination bucket.

Conclusion

 Congratulations! You now have successfully:

  • Configured S3 buckets for versioning.
  • Created S3 Cross-Region Replication policies.
  • Replicated objects with policies for full buckets, folders, and tags.
  • Observed how the replication of deletions are handled.

End Lab

Follow these steps to close the console, end your lab, and evaluate the experience.

  1. Return to the AWS Management Console.

  2. On the navigation bar, click [email protected]<AccountNumber>, and then click Sign Out.

  3. Click End Lab

  4. Click OK

  5. (Optional):

  • Select the applicable number of stars 
  • Type a comment
  • Click Submit

    • 1 star = Very dissatisfied
    • 2 stars = Dissatisfied
    • 3 stars = Neutral
    • 4 stars = Satisfied
    • 5 stars = Very satisfied

You may close the dialog if you don't want to provide feedback.

Additional resources

For feedback, suggestions, or corrections, please email us at [email protected].

 

 

Share: