Code360 powered by Coding Ninjas X Naukri.com. Code360 powered by Coding Ninjas X Naukri.com
Table of contents
1.
Introduction
2.
Amazon Data Lifecycle Manager work
2.1.
Snapshots
2.2.
EBS-backed AMIs
2.3.
Target resource tags
2.4.
Amazon Data Lifecycle Manager tags
2.5.
Lifecycle Policies
2.6.
Policy Schedules
3.
Quotas
4.
Automate Snapshot Lifecycles
5.
Automate Cross-account Snapshot Copies
5.1.
Source Account 
5.2.
Target Account 
6.
Create cross-account snapshot copy policies
6.1.
Specify snapshot description filters
6.2.
Considerations for cross-account snapshot copy policies
7.
View, Modify AND Delete Lifecycle Policies
7.1.
View lifecycle policies
7.2.
Modify lifecycle policies
7.3.
Delete lifecycle policies
8.
AWS Identity and Access Management
9.
Monitor The Lifecycle of Snapshots And AMIs
9.1.
Console and AWS CLI
9.2.
AWS CloudTrail
9.3.
Monitor policies using CloudWatch Events
9.4.
Monitor policies using Amazon CloudWatch
10.
Frequently Asked Questions
10.1.
What is the use of target tags?
10.2.
What is a target account?
10.3.
Give examples of those characters which cannot be used while naming tag keys?
11.
Conclusion
Last Updated: Mar 27, 2024

Amazon Data Lifecycle Manager

Master Python: Predicting weather forecasts
Speaker
Ashwin Goyal
Product Manager @

Introduction

Amazon DLM, which stands for Amazon data lifecycle manager, is an automated procedure that allows the user to back up the data stored on their Amazon EBS volumes. Furthermore, it can be used to automate the creation, retention, and deletion of EBS snapshots as well as EBS-backed AMIs. The user gets various benefits by automating the snapshot and AMI management, which are as follows:

  • Users can protect their valuable data by enforcing a regular backup schedule.
  • Enables the user to create standardized AMIs that can be refreshed regularly. 
  • The user gets the option of deleting outdated backups, which further helps reduce storage costs.
  • In case of a disaster, users can set up recovery backup policies for isolated accounts.


It provides a complete backup solution for the Amazon EC2 instances at no additional cost when it is combined with the monitoring features of Amazon CloudWatch Events and AWS CloudTrail.

Recommended Topic: Amazon Hirepro

Amazon Data Lifecycle Manager work

The various elements of Amazon data lifecycle manager are:

Snapshots

These are the primary means to backup data from EBS volumes, and successive snapshots are incremented to save storage costs as they contain only that volume of data that has changed. When a snapshot is deleted from a series of snapshots for a volume, then only the unique data related to that snapshot gets removed while the rest of the captured history of the volume remains preserved.

EBS-backed AMIs

The information which is required to launch an instance is provided by the AMI. Multiple instances can be launched using the same single AMI. Amazon data lifecycle manager supports only EBS-backed AMIs, they contain a snapshot for each of the EBS c=volume that's attached to the source instance.

Target resource tags

It uses resource tags in order to identify the resources to back up. Tags are metadata that are customizable and can be assigned to AWS resources. Multiple tags can be assigned to an instance if the user wants to run multiple policies on it. Tag keys are case-sensitive, and characters such as '\' or '=' can't be used in a tag key.

Amazon Data Lifecycle Manager tags

The amazon data lifecycle manager applies some tags to all of the snapshots and AMIs created by a policy in order to distinguish them from other snapshots and AMIs created using other means. Those tags are

aws:dlm:lifecycle-policy-id
aws:dlm:lifecycle-schedule-name
aws:dlm:expirationTime
dlm:managed


The user has the ability to specify custom tags. The target tags which are used to associate a certain volume with a snapshot policy can optionally be applied to snapshots created by the policy. The same can be done for tags associated with AMI policies.

Lifecycle Policies

There are three core settings of a lifecycle policy which are:

  • Policy type: it is used to define the type of resources that the policy can manage. Some of the types of lifecycle policies that the amazon data lifecycle manager supports are:

    • Snapshot Lifecycle Policy - it can target individual EBS volumes and is used to automate the lifecycle of EBS snapshots.
    • EBS-backed AMI lifecycle policy - it can only target instances and is used to automate the lifecycle of EBS-backed AMIs.
    • Cross-account copy event policy - it is used to automate the snapshot copies across accounts.
       
  • Resource type: it is used to define the types of resources that are targeted by the policy. A snapshot lifecycle policy can target either volume or instances. Volume is used to create snapshots of individual volumes, whereas instance is used to create multi-volume snapshots of all of the volumes that are attached to an instance. 
     
  • Target tags: it is used to specify those tags that must be assigned to an EBS volume or EC2 instance for it to be targeted by the policy.
     
  • Policy Schedules: it is used to define when snapshots and AMIs are to be created and for how long to retain them.

Policy Schedules

It is used to define when a snapshot or AMI is created by the policy. Each policy can have up to four schedules- one compulsory and up to three optional.

Adding multiple schedules to a single policy allows the user to create snapshots or AMIs at different frequencies using the same policy. For each schedule, the user can define the frequency, fast snapshot restores settings and tags. Each schedule gets initiated separately based on its frequency. In case multiple schedules get initiated at the same time, then only one snapshot or AMI is created, and the retention settings of that schedule are applied, which has the highest retention period.

Get the tech career you deserve, faster!
Connect with our expert counsellors to understand how to hack your way to success
User rating 4.7/5
1:1 doubt support
95% placement record
Akash Pal
Senior Software Engineer
326% Hike After Job Bootcamp
Himanshu Gusain
Programmer Analyst
32 LPA After Job Bootcamp
After Job
Bootcamp

Quotas

A user's account has the following quotas

Automate Snapshot Lifecycles

Creating a snapshot lifecycle policy

  • Open the Amazon EC2 console.
  • Choose the Elastic Block Store, Lifecycle Manager from the navigation pane. And then choose create lifecycle policy.
  • Select EBS snapshot policy and click on next on the Select policy type window.
  • Follow the below steps in the Target resources section. 
    • For choosing the Target resource types, choose volume if you wish to create a snapshot of individual volumes or choose instance if you want to create multi-volume snapshots from the attached volumes.
    • Target resource location can be used to specify where the source resources are to be located, that is, if the source resources are located within the AWS region, then choose AWS region, and if they are located on an outpost, then choose AWS outpost.
  • Provide a brief description of the policy in the Description
  • Select an IAM role that has permission to manage snapshots and describe volumes and instances. In order to use the default role, choose the default role or if you wish to choose a custom IAM role, then choose another role.
  • Policy tags are tags that are used to identify and categorize policies. You can add tags to apply the lifecycle policy.
  • Select enable policy to start the policy run or select the disable policy to prevent the policy from running for Policy status after creation.
  • Click on next.
  • Since a policy can have up to 4 schedules, configure the policy schedules on the Configure schedule screen. For each policy schedule that is added, do the following:
    • In the schedule details section:
      • Mention a name for the schedule under the schedule name
      • Configure the interval between the policy runs using the frequency. It can be configured to daily, weekly, monthly, or yearly. 
      • Specify the start time of the policy run under the Starting at section.
      • Use the retention type to specify the retention policy for AMIs created by the schedule.
    • In the Tagging section:
      • If you wish to copy all of the user-defined tags from the source instance to the AMIs created, use Copy tags from the source.
      • To prevent automatic tagging when AMIs are created, remove the instance-id:$(instance-id) tile.
      • Use the add tags if you wish to add additional tags.
    • Choose Enable AMI deprecation if you wish to deprecate AMIs when they should no longer be used. In case the schedule uses count-based AMI retention, specify the number of oldest AMIs to deprecate. The count must be lesser than or equal to the schedule's AMI retention count. And if the schedule uses age-based AMI retention, then you must specify the period after which AMI is to be deprecated. The count must be lesser than or equal to the schedule's AMI retention period.
    • Users can copy AMIs up to three additional regions in their accounts. To copy AMIs created by the schedule to different regions, choose the enable cross-region copy under the cross-region copy section. For each destination region, you can specify the below:
      • A retention policy for the AMI copy.
      • Encryption status for the AMI copy.
      • A deprecation rule for the AMI copy.
    • You can add additional schedules using the Add another schedule.
  • After reviewing the policy summary, Click on create policy.

Automate Cross-account Snapshot Copies

It allows the user to copy their Amazon EBS snapshots to a specific region in an isolated account and further encrypt those snapshots with an encryption key. It helps the user to protect their data from any loss in case their account gets compromised.

Automating cross-account snapshot copies requires two accounts:

Source Account 

It is the account that creates and shares the snapshots with the target account. It's mandatory to create an EBS snapshot policy for this account that creates snapshots at set intervals and then shares them with other AWS accounts.

Target Account 

It is the account that creates copies of the shared snapshots. It is the account with which snapshots are shared. It's mandatory to create a cross-account copy event policy for this account that automatically copies the shared snapshots.

Create cross-account snapshot copy policies

In order to create and prepare source as well as target accounts for cross-account snapshot copying, perform the following:

Step-1: Create an EBS snapshot policy (Source account)

Step-2: Share the customer-managed key (Source account)

Step-3: Create a cross-account copy event policy (target account)

Step-4: Allow the IAM role to use the required KMS keys (target account)

Specify snapshot description filters

When creating a snapshot copy policy for the target account, we must specify a snapshot description filter. It enables the user to specify an additional level of filtering, which enables the user to control which snapshots gets copied by the policy. The snapshot filter description is specified by a regular expression. Examples of a regular expressions that can be used are :

  • . * - This filter will match all the snapshot descriptions. If this is used, then all the snapshots will be copied, which are shared by one of the source accounts.
  • . *production.* - This filter will match any snapshot which has the word production in its Description. If this is used, then all the snapshots will be copied, which are shared by one of the source accounts that have the specified text in their Description.

Considerations for cross-account snapshot copy policies

  • The user can copy only those snapshots that are either unencrypted or encrypted using a customer-managed key.
  • The user can create a cross-account copy event policy in order to copy snapshots that are shared outside of amazon dlm.
  • If the user wishes to encrypt snapshots in the target account, then the IAM role selected must have permission to use the KMS key, which is required.

View, Modify AND Delete Lifecycle Policies

View lifecycle policies

To view a lifecycle policy

  • Open the Amazon EC2 console
  • Select the Elastic block store and lifecycle manager from the navigation pane
  • Choose a lifecycle policy. Information about the policy is displayed in the Details tab.

Modify lifecycle policies

To modify a lifecycle policy

  • To view a lifecycle policy
  • Open the Amazon EC2 console
  • Select the Elastic block store and lifecycle manager from the navigation pane.
  • Choose a lifecycle policy.
  • Click on Actions, Modify the lifecycle policy.
  • Perform the modifications required in the policy settings
  • Click on the Update policy

Delete lifecycle policies

To delete a lifecycle policy

  • To view a lifecycle policy
  • Open the Amazon EC2 console
  • Select the Elastic block store and lifecycle manager from the navigation pane.
  • Choose a lifecycle policy.
  • Click on Actions, and Delete lifecycle policy.
  • Click on delete lifecycle policy when prompted for confirmation.

AWS Identity and Access Management

In order to access the amazon data lifecycle manager, credentials are required. The credentials must have the necessary permissions to access the AWS resources like instances, volumes, snapshots, and AMIs. 

Monitor The Lifecycle of Snapshots And AMIs

The following features can be used to monitor the lifecycle of the snapshots and AMIs:

Console and AWS CLI

The user can view the lifecycle policies by using the Amazon EC2 console or the AWS CLI. There is a timestamp and policy-related tags associated with each snapshot and AMI created by policy. The user can filter snapshots and AMIs using these tags, which can be used to verify whether backups are being created as intended.

AWS CloudTrail

With this, the user can track user activity and API usage in order to demonstrate compliance with internal policies and regulations standards.

Monitor policies using CloudWatch Events

It is used to handle event notifications programmatically. All the events are emitted on a best effort basis.

Monitor policies using Amazon CloudWatch

It collects raw data, which is processed into readable near real-time metrics which can be used to see how many Amazon EBS snapshots and EBS-backed AMIs are created, deleted, and copied by the user's policies over the time. It even allows the user to set up an alarm for certain thresholds.

Frequently Asked Questions

What is the use of target tags?

Target tags are used to specify those tags that must be assigned to an EBS volume or EC2 instance for it to be targeted by the policy.


What is a target account?

A target account is an account that creates copies of the shared snapshots. It is the account with which snapshots are to be shared.


Give examples of those characters which cannot be used while naming tag keys?

Tag keys are case-sensitive, and characters such as '\' or '=' can't be used in a tag key.

Conclusion

In this article, we have extensively discussed the Amazon data lifecycle manager.

After reading about Amazon's data lifecycle manager, are you not feeling excited to read/explore more articles on AWS? Don't worry; Coding Ninjas has you covered. To learn about the difference between GCP and AWSwhy to get certified by AWS, and how to prepare for AWS certification.

Check out the Amazon Interview Experience to learn about Amazon’s hiring process.

If you wish to enhance your skills in Data Structures and AlgorithmsCompetitive ProgrammingJavaScript, etc., you should check out our Guided path column at Coding Ninjas Studio. We at Coding Ninjas Studio organize many contests in which you can participate. You can also prepare for the contests and test your coding skills by giving the mock test series available. In case you have just started the learning process, and your dream is to crack major tech giants like Amazon, Microsoft, etc., then you should check out the most frequently asked problems and the interview experiences of your seniors that will surely help you in landing a job in your dream company. 

Do upvote if you find the blogs helpful.

Happy Learning!

Previous article
Application Services in AWS
Next article
AWS DMS Endpoints
Live masterclass