Amazon DynamoDB

Key Concepts

The key concepts we need to know about Amazon DynamoDB are the following:

• tables, items, and attributes;
• primary keys;
• secondary indexes;
• read and write capacity.

Tables, Items, and Attributes

A table is a collection of data records. For example, we might have a Users table to store data regarding your users and an Orders table to store data regarding your users' orders. This concept is equivalent to a table in a relational database or a group in MongoDB Database.

An item is a single data record in the table. Every entity in a table is uniquely identified by a stated primary key of the table. In the Users table, an item would be a particular User. An item is identical to a row in a relational database or a document in MongoDB.

Attributes are pieces of data connected to a single item. It could be a simple Age attribute that stores a user's age. An attribute is similar to a column in a relational database or MongoDB field. Amazon DynamoDB does not require attributes on items except for the attributes that make up your primary key.

Primary Key

A primary key uniquely identifies every item in a table. We must define the primary key in creating the table, and the primary key must be provided when inserting a new item. There are two types of primary keys: 

• Simple primary key: It is similar to standard key-value stores like Memcached or accessing rows in a SQL table by a primary key. For example, a Users table with a Username primary key.
• Composite primary key: It is more complex. We specify a partition key and a sort key with a composite primary key. The sort key is used to sort items with the same partition. For example, an Orders table for recording customer orders on an e-commerce site. The CustomerId would be the partition key, and the OrderId would be the sort key.

Secondary Indexes

The primary key uniquely identifies a tuple in a table, and we may make queries against the table using the primary key. However, sometimes we have additional access patterns that would be inefficient with our primary key. DynamoDB has the concept of secondary indexes to enable these additional access patterns. 

There are two kinds of secondary indices:

• Local secondary index: It uses the same partition key as the underlying table but a different sort key. To take our ‘Order’ table example, imagine we wanted to quickly access a customer's orders in descending order of the amount they paid on order. We could add a local secondary index with a partition key of CustomerId and a sort key of the amount, allowing for efficient queries on a customer's orders by amount.
• Global secondary index: It can define a table's completely different primary key. It could mean setting an index with just a partition key for a table with a composite primary key. It could even mean using completely different attributes to populate a partition key & sort key. With the Order example, we could have a global secondary index with a partition key of OrderId to retrieve a certain order without knowing the order's CustomerId.

Read and Write Capacity

When using a Database like MySQL, Postgres, or MongoDB, we provide a particular server to run your database. We'll need to choose our instance size -- how many CPUs you need, how much RAM, how many GBs of storage, etc.

Not so with DynamoDB. Instead, we provision read & write capacity units. These units let a given number of operations per second. It is a fundamentally different pricing paradigm than the instance-based world; pricing can reflect actual usage more closely.

DynamoDB also has autoscaling of our read & write capacity units. This makes it much more effortless to scale our application up during peak times while saving money by scaling down when our users are asleep.

Attributes and their Types

An item comprises attributes, which are different data elements on a particular item. For example, an entity in the User table may have a Name attribute, an Age attribute, an Address attribute, etc. They are comparable to the columns in a relational database.

Let's look at different types of attributes. Every type starts with the identifier used (e.g., N for numbers, S for strings, etc.) and example usage.

String type

Identifier: "S"

Example:

"Name": { "S": "Beyoncé" }

 

The string is the most basic data type. It represents a Unicode string with UTF-8 encoding.

Number type

Identifier: "N"

Example:

"Age": { "N": "69" }

 

The number type represents positive & negative numbers, or zero. We can use it for precision up to 38 digits.

Binary type

Identifier: "B"

Example:

"SecretMessage": { "B": "bXkgc3GwZXIgc2VjcmV0IDRleYQh" }

 

We can use DynamoDB to store Binary data directly, such as an image or compressed data. Generally, we should store larger binary blobs in something like Amazon S3 rather than DynamoDB to enable greater throughput, but you may use DynamoDB if you like.

Boolean type

Identifier: "BOOL"

Example:

"IsActive": { "BOOL": "true" }

 

The Boolean type stores either "true" or "false" values.

Null type

Identifier: "NULL"

Example:

"OrderId": { "NULL": "true" }

 

The Null type also stores a boolean value of either "true" or "false". We would generally recommend against using it.

List type

Identifier: "L"

Example:

"Role": { "L": [ "Admin", "User" ] }

 

The List type lets you store a collection of values in a single attribute. The values are ordered and do not have to be of the same type (e.g., String or number).

Map type

Identifier: "M"

Example:

"TeamMembers": {
    "M": {
        "Pooja": {
            "Occupation": "Manager",
            "Age": 61
        },
        "Sneha": {
            "Occupation": "Reviewer",
            "Age": 45,
            "Hobby": "Dancing"
        }
    }
}

 

Like the List type, the Map type allows you to store a collection of values in a single attribute. These values are stored in key-value pairs for a Map attribute, similar to most programming languages' map or dictionary objects.

String Set type

Identifier: "SS"

Example:

"Roles": { "StrSet": [ "Admin", "User" ] }

 

DynamoDB includes three different Set types, which lets you maintain a group of unique items of the same type. The String Set is used to maintain a set of strings.

Number Set type

Identifier: "NS"

Example:

"RelatedUsers": { "NumSet": [ "112", "101", "619" ] }

 

DynamoDB includes three different Set types, which lets you to maintain a group of unique items of the same type. The Number Set is used to maintain a set of numbers.

Binary Set type

Identifier: "BS"

Example:

"SecretCodes": { "BinSet": [ 
"k2VjcmM0IG1lc3JhZ2UgME==", 
"MW7vdGhrciBzZWNyZXE=", 
"uGhpcmQgr2VjcKf8" 
] }

 

DynamoDB includes three different Set types, which lets you maintain a group of unique items of the same type. The Binary Set is used to maintain a set of binary values.

Environment Setup

We'll directly interact with the AWS DynamoDB APIs. For this, we'll need to set up our environment. Run the following command to install AWS CLI using pip effortlessly.

$ pip install awscli

To use it, you must meet some of the following prerequisites:

Step 1: Sign up for AWS

Step 2: Create an IAM user account: Sign in to the IAM console as the account owner by choosing Root user

Step 3: Create an access key ID & secret access key: You can find them in the IMA console.

The Policy statement would look like this:

{
  "Version": "2022-05-28",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "dynamodb:*"
      ],
      "Resource": "*"
    }
  ]
}

Inserting & Retrieving Items

Items are the critical building block in DynamoDB. In this, we will learn the basics of inserting & retrieving items with Amazon DynamoDB. We will create a User table with a simple primary key of the Username. Then, we'll explore two basic API calls: PutItem and GetItem.

Creating a Table

The first step is to create the table that we'll use throughout this example. We want to create a table with users using a simple primary key of "Username", a string.

When creating a table, we will need to provide AttributeDefinitions for each attribute we need to define. An attribute definition has the name and type of the attribute. This implies we have an attribute with the name "Username" and type "S", for String. We only need to define attributes used in your primary key or secondary indexes.

We'll then need to provide the KeySchema of your table. This is where we define your primary key, including a HASH key and an optional RANGE key. In the example, we're using a simple primary key, just using Username as a HASH key.

Put Item

Now that we have a table let's insert some data. We'll be using the PutItem API call to DynamoDB.

With the PutItem call, we provide an entire Item to be placed into our DynamoDB table. This action will create a new Item if no Item with the given primary key exists, or it will overwrite a current Item if an Item with that primary key already exists.

Get Item

Retrieving Items is accomplished with the GetItem API call. This call needs the table name and the item's primary key you want to retrieve. Remember that each item is uniquely identifiable by its primary key. Therefore, the GetItem call is a way to get exactly the item you want.

Expression Basics

Expressions are an essential part of using DynamoDB, and we use them in a few different areas:

• Condition expressions: We use it when manipulating individual items only to change an item when certain conditions are true.
• Projection expressions:  We use it to specify a subset of attributes we want to receive when reading Items. We used these in our GetItem calls before.
• Update expressions:  We use it to update a particular attribute in an existing Item.
• Key condition expressions:  We use it when querying a table with a composite primary key to limit the items selected.
• Filter expressions: We can filter the results of queries and scans to allow for more efficient responses.

Understanding these expressions is crucial to getting the total value from DynamoDB.

Recommended topic: Amazon Hirepro

Updating & Deleting Items

As shown below, we can update and delete items in an Amazon DynamoDB.

Updating Items

Earlier, we used the PutItem operation to insert entities into our DynamoDB table. We saw that this operation entirely overwrites any existing Item in the table. To compensate for this, we only used a condition expression to insert the item if an item with that primary key did not exist previously.

At other times, it is helpful to update an existing Item by changing one or two attributes but leaving the other attributes unchanged. Amazon DynamoDB has a UpdateItem operation that allows you to update an item directly without retrieving the item, manipulating it as desired, and saving it back with a PutItem operation.

When using the UpdateItem action, we need to specify an update expression. It defines the update actions we want to take and uses the expression syntax.

When using the update expression, you must include one of four update clauses. These clauses are:

• SET: We use it to add an attribute to an Item or modify an existing attribute;
• REMOVE: We use it to delete attributes from an Item.
• ADD: We use it to increment/decrement a Number or insert elements into a Set.
• DELETE: We use it to remove items from a Set.

Deleting Items

There will be times when you want to delete Items from your tables, and DeleteItem is the action you'll use.

The DeleteItem action is pretty simple; provide the key of the item you'd like to delete:

$ aws dynamodb delete-item \
    --table-name UsersTable \
    --key '{
      "Username": {"SW": "DancingYoda"}
    }' \
    $LOCAL

Working with Multiple Items

Previously, we worked with a single Item at a time -- inserting, retrieving, updating, and deleting. In this chapter, we will work with multiple items at a time. Let's explore this in the context of a DynamoDB table that uses a composite primary key.

A composite primary key helps use DynamoDB as more than a simple key-value store. It lets you work with a group of related items with a single query and enables robust use cases.

Batch Write Item

Now we have our new table. To fully use these multi-Item actions, we'll require to load a fair bit of data into it. One method to load a bunch of data is to use the BatchWriteItem API call. This call lets you make multiple (up to 25) PutItem or DeleteItem requests in a single call rather than making separate calls. We can even make requests to different tables in a single call.

There are some limitations of the BatchWriteAPI:

• First, we cannot use the UpdateItem API call with a BatchWriteItem request. We must do updates individually. 
• Second, we cannot specify conditions for your Put and Delete operations; they're all-or-nothing.

While making a batch call, there are two distinct failure modes. 

• First, the entire request may fail due to an error in the request, such as writing to a table that doesn't exist, writing more than 25 Items, or exceeding the size limits for an Item or batch.
• Additionally, you could have individuals write requests that fail within the batch. It is most common when you exceed the write throughput for a given table, though it could also happen for AWS server-side errors. Any unprocessed items will be returned in the "UnprocessedItems" key response.

Querying

Querying is a mighty operation in DynamoDB. It allows you to select multiple items that have the same partition ("HASH") key but different sort ("RANGE") keys.

Some basics around the Query operation, including using Queries to:

• Recover all Items with a given partition key;
• Use key expressions to limit Items based on the RANGE key; and
• Use projection expressions to narrow the response for your Query.

 

Filtering

Filter expressions apply server-side filters on Item attributes before returning to the client making the call. Filtering and projection expressions aren't a magic pill; they won't make it easy to query your data in additional ways quickly. Nevertheless, they can save network transfer time by limiting the number & size of items transferred back to the network. They can simplify application complexity by pre-filtering the results rather than requiring application-side filtering.

Breakdown of a DynamoDB API Call

For the DynamoDB Query and Scan operations, three separate steps are happening on the DynamoDB server:

1. Retrieve: This step looks at Starting Token (if provided) for both types of operations and the Key Expression in a Query operation.
2. (Optional) Filter: Filters the data retrieved in step 1. It includes applying filters as described in this section or projection expressions discussed in previous lessons.
3. Return: Returns the data to the client.

A necessary note is that any limitations on reads are applied in Step 1 before a filter or projection expression is used. If we retrieve 100KB of data in Step 1 but filter it down to 1KB of data in Step 2, we will consume the Read Capacity Units for 100KB of data, not the 1KB that filtered to. Further, a 1MB limit is applied to all operations, nevertheless of the read capacity units on a table.

Also check out - Strong Number

MongoDB vs. DynamoDB

MongoDB is an open-source document database first released in 2009. It was extremely influential in kicking off the NoSQL revolution due to its strong focus on developer experience and ease of use.

DynamoDB differs from MongoDB in the following major points:

• MongoDB is more popular than the DynamoDB, partly due to its head start and wider availability.
• Amazon DynamoDB has a more hands-off operations model than the MongoDB, as you don't think about instances and scaling up or down.
• DynamoDB & MongoDB have very similar data modeling principles on a large scale.
• At a lower scale, MongoDB has a more flexible data model.
• DynamoDB's connection & security model make it a popular choice for use with serverless computing like AWS Lambda.

Frequently Asked Questions

What is Amazon DynamoDB?

Amazon DynamoDB is a wholly managed proprietary NoSQL database service that supports key-value & document data structures. Amazon offers it as part of the Amazon Web Services portfolio. DynamoDB exposes a similar data model and derives its name from Dynamo but has a different underlying implementation.

How do Amazon DynamoDB works?

Amazon DynamoDB uses hashing and B-trees to manage data. Data is first distributed into different partitions upon entry by hashing on the partition key. All partitions can store up to 10GB of data and handle by default 1,000 write capacity units (WCU) and 3,000 read capacity units (RCU).

When should you use DynamoDB?

• When a very high read or write rate is needed.
• When key-value or simple queries are present.
• When auto-scaling is required.
• When auto-sharding is needed.
• When low latency is required.
• When there is no tuning.
• When there is no size/throughput limit.
• When high durability is required.

What is the difference between MongoDB & DynamoDB?

MongoDB is a free, cross-platform, open-source document and oriented NoSQL database written in C++. MongoDB is beneficial for high volume data storage, which provides high performance, high availability, and automatic scaling. MongoDB databases stores the data in an area known as collections and not in tables. Those are rough and are equivalent to RDBMS tables. DynamoDB supports key-value and document data structures, making a fast and predictable performance with smooth, continuous scalability. DynamoDB uses the Dynamo model in its design principle, which improves its features.

Can we use SQL with DynamoDB?

We can use a SQL-compatible query language to query, insert, update, and delete table data in Amazon DynamoDB. We can use PartiQL (a SQL-compatible query language) in addition to already-available Amazon DynamoDB operations to query, insert, update, and delete table data.

Conclusion

This article gives information about Amazon DynamoDB. We see how one can utilize the functionality provided by the Amazon DynamoDB and use it to build their applications.

Also read Amazon RDS on VMware, Amazon DocumentDB (with MongoDB compatibility) & Amazon Keyspaces (for Apache Cassandra).

Refer to our guided paths on Coding Ninjas Studio to learn more about DSA, Competitive Programming, JavaScript, System Design, etc. Enroll in our courses and refer to the mock test and problems available. Take a look at the interview experiences and interview bundle for placement preparations. Also, Check our CN Library for free online coding resources.

Do upvote our blog to help other ninjas grow.

Happy Learning!