Dynamo Architecture
In this tutorial, we are going to discuss about the Dynamo Architecture. Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability.
At a high level, Dynamo is a Distributed Hash Table (DHT) that is replicated across the cluster for high availability and fault tolerance.
Here’s an overview of Dynamo architecture:
1. Tables, Items, and Attributes
- Table: A collection of items.
- Item: A collection of attributes. Each item is uniquely identifiable within a table by a primary key.
- Attributes: Fundamental data elements, similar to fields or columns in other database systems.
2. Primary Keys
- Partition Key: A single attribute that uniquely identifies an item. DynamoDB uses the partition key’s value as an input to an internal hash function, which determines the partition (physical storage location) in which the item is stored.
- Composite Primary Key: Consists of a partition key and a sort key. The combination of these two attributes must be unique within a table.
3. Partitions
- Partition: The fundamental unit of scalability in DynamoDB. Data is automatically distributed across partitions based on the partition key.
- Provisioned Throughput: Each partition can be allocated read and write capacity units, which control the read and write throughput.
4. Indexes
- Global Secondary Index (GSI): An index with a partition key and optional sort key that can be different from those on the base table.
- Local Secondary Index (LSI): An index that uses the same partition key as the base table but allows for a different sort key.
5. Data Replication
- Automatic Replication: DynamoDB automatically replicates data across multiple Availability Zones within an AWS Region to provide high availability and durability.
6. Streams
- DynamoDB Streams: Capture changes to items in a DynamoDB table. Streams can be used to trigger AWS Lambda functions for real-time processing.
7. Data Types
- Scalar Types: Number, String, Binary, Boolean, Null.
- Document Types: List, Map.
- Set Types: String Set, Number Set, Binary Set.
8. Consistency Models
- Eventually Consistent Reads: By default, DynamoDB provides eventual consistency for reads, which can result in stale data but offers higher performance and availability.
- Strongly Consistent Reads: Optional feature that ensures a read returns the most recent write for a given item.
9. Provisioned vs. On-Demand Capacity
- Provisioned Capacity: You specify the number of reads and writes per second you expect your application to require.
- On-Demand Capacity: DynamoDB automatically adjusts the read and write throughput as your application traffic fluctuates.
10. Security
- Authentication and Access Control: AWS Identity and Access Management (IAM) policies can be used to control access to DynamoDB tables.
- Encryption: DynamoDB supports encryption at rest and in transit, ensuring data is secure.
11. Integration with Other AWS Services
- AWS Lambda: Trigger functions in response to table updates.
- Amazon Kinesis: Real-time data processing.
- AWS Data Pipeline: Data workflow orchestration.
- Amazon EMR: Big data processing.
12. Data distribution
- Dynamo uses Consistent Hashing to distribute its data among nodes. Consistent hashing also makes it easy to add or remove nodes from a Dynamo cluster.
13. Handling temporary failures
- To handle temporary failures, Dynamo replicates data to a sloppy quorum of other nodes in the system instead of a strict majority quorum.
14. Inter-node communication and failure detection
- Dynamo’s nodes use gossip protocol to keep track of the cluster state.
15. High availability
- Dynamo makes the system “always writeable” (or highly available) by using hinted handoff.
This Dynamo architecture allows to provide low-latency responses and handle high request rates, making it suitable for many applications ranging from simple web apps to complex, large-scale distributed systems.
That’s all about the Dynamo architecture. If you have any queries or feedback, please write us email at contact@waytoeasylearn.com. Enjoy learning, Enjoy system design..!!