EBS - Amazon block storage
Instance stores :
Block-level storage volumes behave like physical hard drives.
An instance store provides temporary block-level storage for an Amazon EC2 instance. An instance store is disk storage that is physically attached to the host computer for an EC2 instance, and therefore has the same lifespan as the instance. When the instance is terminated, you lose any data in the instance store.
Amazon elastic blockstore EBS : -
EBS is a service that provides block-level storage volumes that you can use with Amazon EC2 instances. If you stop or terminate an Amazon EC2 instance, all the data on the attached EBS volume remains available.
To create an EBS volume, you define the configuration (such as volume size and type) and provision it. After you create an EBS volume, it can attach to an Amazon EC2 instance.
Because EBS volumes are for data that needs to persist, it’s important to back up the data. You can take incremental backups of EBS volumes by creating Amazon EBS snapshots.
An EBS snapshot is an incremental backup. This means that the first backup taken of a volume copies all the data. For subsequent backups, only the blocks of data that have changed since the most recent snapshot are saved.
Incremental backups are different from full backups, in which all the data in a storage volume copies each time a backup occurs. The full backup includes data that has not changed since the most recent backup.
Object storage vs block storage
When you modify a file in block storage, only the pieces that are changed are updated.
When a file in object storage is modified, the entire object is updated.
Object storage
In object storage, each object consists of data, metadata, and a key.
The data might be an image, video, text document, or any other type of file. Metadata contains information about what the data is, how it is used, the object size, and so on. An object’s key is its unique identifier.
EBS - Amazon block storage
An instance store provides temporary block-level storage for an Amazon EC2 instance. An instance store is disk storage that is physically attached to the host computer for an EC2 instance, and therefore has the same lifespan as the instance. When the instance is terminated, you lose any data in the instance store.
Amazon elastic blockstore EBS : -
EBS is a service that provides block-level storage volumes that you can use with Amazon EC2 instances. If you stop or terminate an Amazon EC2 instance, all the data on the attached EBS volume remains available.
To create an EBS volume, you define the configuration (such as volume size and type) and provision it. After you create an EBS volume, it can attach to an Amazon EC2 instance.
Because EBS volumes are for data that needs to persist, it’s important to back up the data. You can take incremental backups of EBS volumes by creating Amazon EBS snapshots.
An EBS snapshot is an incremental backup. This means that the first backup taken of a volume copies all the data. For subsequent backups, only the blocks of data that have changed since the most recent snapshot are saved.
Incremental backups are different from full backups, in which all the data in a storage volume copies each time a backup occurs. The full backup includes data that has not changed since the most recent backup.
Object storage vs block storage
When you modify a file in block storage, only the pieces that are changed are updated.
When a file in object storage is modified, the entire object is updated.
Object storage
In object storage, each object consists of data, metadata, and a key.
The data might be an image, video, text document, or any other type of file. Metadata contains information about what the data is, how it is used, the object size, and so on. An object’s key is its unique identifier.
Amazon Simple Storage Service (AmazonS3):
- Object level storage
- stores data as objects in buckets.
- upload any type of file such as image, text
- maximum file size for an object in Amazon S3 is 5 TB.
- Can set permissions to control visibility and access to it.
- Set Amazon S3 versioning feature to track changes to your objects over time.
Amazon Simple Storage Service (AmazonS3):
- Object level storage
- stores data as objects in buckets.
- upload any type of file such as image, text
- maximum file size for an object in Amazon S3 is 5 TB.
- Can set permissions to control visibility and access to it.
- Set Amazon S3 versioning feature to track changes to your objects over time.
Amazon S3 storage classes :
With Amazon S3, you pay only for what you use. You can choose from storage classes to select a fit for your business and cost needs. When selecting an Amazon S3 storage class, consider these two factors:
- How often you plan to retrieve your data
- How available you need your data to be
1. S3 Standard:
- Used for frequently used data
- high avaibility and durability and low latency
- available across multiple availbilty zone
- use cases, including cloud applications, dynamic websites, content distribution, mobile and gaming applications, and big data analytics
2. S3 standard infrequent access -(S3standard-IA)
-S3 Standard-IA is for data that is accessed less frequently, but requires rapid access when needed.
- available across multiple availbilty zone
- with a low per GB storage price and per GB retrieval charge.- high durability and low latency
3. S3 one zone infrequent access S3 one zone-IA)
- Standard-IA is for data that is accessed less frequently, but requires rapid access when needed.
-Stores in one availabilty zone, has lower storage price than S3 Infrequented access
-You want to save costs on storage.
-It’s a good choice for storing secondary backup copies of on-premises data or easily re-creatable data.
4. S3 intelligent tiering
-Ideal for data with unknown or changing access patterns
-Requires a small monthly monitoring and automation fee per object
- Monitors objects’ access patterns. If you haven’t accessed an object for 30 consecutive days, Amazon S3 automatically moves it to S3 Standard-IA. (infrequent access)
-If you access an object in the infrequent access tier, Amazon S3 automatically moves it to S3 Standard.
5. S3 Glacier
Low-cost storage designed for data-archiving.
Able to retrieve objects withing few minutes to hours.
S3 Glacier is low-cost storage class that is ideal for data archiving. For ex you might use this storage class to store archived customer records or older photos and video files.
6. S3 Glacier Deep Archive
*Lowest-cost object storage class ideal for archiving
*Able to retrieve objects within 12 hours
When deciding between Amazon S3 Glacier and Amazon S3 Glacier Deep Archive, consider how quickly you need to retrieve archived objects. You can retrieve objects stored in the S3 Glacier storage class within a few minutes to a few hours. By comparison, you can retrieve objects stored in the S3 Glacier Deep Archive storage class within 12 hours.
File storage : EFS
In file storage, multiple clients can access the data that is stored in shared file folders. In this approach, storage server uses block storage with local file system to organize files.
Compared to block storage and object storage, file storage is ideal for use cases in which a large number of services and resources need to access the same data at the same time.
Amazon file system is a scalable file system used with AWS Cloud services and on-premises resources. As you add and remove files, Amazon EFS grows and shrinks automatically. It can scale on demand to petabytes without disrupting applications.
EBS VS EFS
EBS: An Amazon EBS stores data in single availabilty zone. To attach EBS volume to E2 instance, both should reside in same availablity zone.
EFS : EBS is a regional service, It stores data in multiple availabilty zone. Duplicate storage enables you to access data concurrently from all availabilty zone in the region where file is located. Additionaly on-premises server can access Amazon EFS using Aws direct connect.
Relational Databases:
In a relational database, data is stored in a way that relates it to other pieces of data.
Relational databases use structured query language (SQL) to store and query data.
Amazon Relational Database Service: (Amazon RDS)
Its a services which enables you to run Relational database in AWS Cloud. Amazon RDS is a managed service that automates tasks such as hardware provisioning, database setup, patching, and backups.
You can integrate Amazon RDS with other services to fulfill your business and operational needs, such as using AWS Lambda to query your database from a serverless application.
Amazon RDS provides a number of different security options. Many Amazon RDS database engines offer encryption at rest (protecting data while it is stored) and encryption in transit (protecting data while it is being sent and received).
Amazon RDS database engines:
Amazon RDS is available on six database engines, which optimize for memory, performance, or input/output (I/O). Supported database engines include: Amazon Aurora ,PostgreSQL , MySQL , MariaDB ,Oracle Database , Microsoft SQL Server
Amazon Aurora :
Its a enterprise-class relational database. Its compatible with mysql and post-gre sql relational databases. Its 5 time faster than standard mysql and 3 times faster than standard post gre sql.
Amazon aurora helps to reduce database cost, by reducing uncessarry input/output operations , while ensuring database resources remain reliable and available.
Consider Amazon Aurora if your workloads require high availability. It replicates six copies of your data across three Availability Zones and continuously backs up your data to Amazon S3.
Non relational DB :
In non relational database you create Tables. Table is a place where you can store and query data.
Non relational database are sometimes called 'NoSql database' because it use structure other than row and columns to organzie data. One type of structural approach for non relational data base is key-value pair. With key-value pair data is organized into item(key) and item has attributes(Value) You can think of attributes as being different features of data.
In a key-value database, you can add or remove attributes from items in the table at any time. Additionally, not every item in the table has to have the same attributes.
Amazon DynamoDB :
Its a key-value database service. It offers single digit-millisecond performance.
Dynamo DB is serverless, which means no need to install, maintain or operate software. And no need to provision, patch or manage servers.
As the size of the database shrinks or grows Dynamo db automatically scales to adjust for changes in capacity while maintaining consistent performances. ex: Amazon prime day
Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. You can use Amazon DynamoDB to create a database table that can store and retrieve any amount of data, and serve any level of request traffic. Amazon DynamoDB automatically spreads the data and traffic for the table over a sufficient number of servers to handle the request capacity specified by the customer and the amount of data stored, while maintaining consistent and fast performance.
AWS Database migration service
Its a cloud service to migrate data stores like relational database, non relational database, data warehouse .
Amazon RDS VS Dynamo DB :
RDS: Uses relational database. Automatic high availabilty recovery provided. Customer ownership of data, schema and network.
Dynamo Db: uses non-relational db, key-value pair. Its serverless and auto scaling.
Amazon redshift:
Efficiently analyze all your data using exisiting BI Tools. Amazon Redshift is a data warehousing service that you can use for big data analytics. It offers the ability to collect data from many sources and helps you to understand relationships and trends across your data.
AWS Database Migration Service (AWS DMS)
Amazon S3 storage classes :
With Amazon S3, you pay only for what you use. You can choose from storage classes to select a fit for your business and cost needs. When selecting an Amazon S3 storage class, consider these two factors:
- How often you plan to retrieve your data
- How available you need your data to be
1. S3 Standard:
- Used for frequently used data
- high avaibility and durability and low latency
- available across multiple availbilty zone
- use cases, including cloud applications, dynamic websites, content distribution, mobile and gaming applications, and big data analytics
2. S3 standard infrequent access -(S3standard-IA)
-S3 Standard-IA is for data that is accessed less frequently, but requires rapid access when needed.
- available across multiple availbilty zone
- high durability and low latency
3. S3 one zone infrequent access S3 one zone-IA)
- Standard-IA is for data that is accessed less frequently, but requires rapid access when needed.
4. S3 intelligent tiering
-Requires a small monthly monitoring and automation fee per object
- Monitors objects’ access patterns. If you haven’t accessed an object for 30 consecutive days, Amazon S3 automatically moves it to S3 Standard-IA. (infrequent access)
-If you access an object in the infrequent access tier, Amazon S3 automatically moves it to S3 Standard.
5. S3 Glacier
Low-cost storage designed for data-archiving.
Able to retrieve objects withing few minutes to hours.
6. S3 Glacier Deep Archive
*Able to retrieve objects within 12 hours
When deciding between Amazon S3 Glacier and Amazon S3 Glacier Deep Archive, consider how quickly you need to retrieve archived objects. You can retrieve objects stored in the S3 Glacier storage class within a few minutes to a few hours. By comparison, you can retrieve objects stored in the S3 Glacier Deep Archive storage class within 12 hours.
File storage : EFS
In file storage, multiple clients can access the data that is stored in shared file folders. In this approach, storage server uses block storage with local file system to organize files.
Compared to block storage and object storage, file storage is ideal for use cases in which a large number of services and resources need to access the same data at the same time.
Amazon file system is a scalable file system used with AWS Cloud services and on-premises resources. As you add and remove files, Amazon EFS grows and shrinks automatically. It can scale on demand to petabytes without disrupting applications.
EBS VS EFS
EBS: An Amazon EBS stores data in single availabilty zone. To attach EBS volume to E2 instance, both should reside in same availablity zone.
EFS : EBS is a regional service, It stores data in multiple availabilty zone. Duplicate storage enables you to access data concurrently from all availabilty zone in the region where file is located. Additionaly on-premises server can access Amazon EFS using Aws direct connect.
Relational Databases:
In a relational database, data is stored in a way that relates it to other pieces of data.
Relational databases use structured query language (SQL) to store and query data.
Amazon Relational Database Service: (Amazon RDS)
Its a services which enables you to run Relational database in AWS Cloud. Amazon RDS is a managed service that automates tasks such as hardware provisioning, database setup, patching, and backups.
You can integrate Amazon RDS with other services to fulfill your business and operational needs, such as using AWS Lambda to query your database from a serverless application.
Amazon RDS provides a number of different security options. Many Amazon RDS database engines offer encryption at rest (protecting data while it is stored) and encryption in transit (protecting data while it is being sent and received).
Amazon RDS database engines:
Amazon RDS is available on six database engines, which optimize for memory, performance, or input/output (I/O). Supported database engines include: Amazon Aurora ,PostgreSQL , MySQL , MariaDB ,Oracle Database , Microsoft SQL Server
Amazon Aurora :
Its a enterprise-class relational database. Its compatible with mysql and post-gre sql relational databases. Its 5 time faster than standard mysql and 3 times faster than standard post gre sql.
Amazon aurora helps to reduce database cost, by reducing uncessarry input/output operations , while ensuring database resources remain reliable and available.
Consider Amazon Aurora if your workloads require high availability. It replicates six copies of your data across three Availability Zones and continuously backs up your data to Amazon S3.
Non relational DB :
In non relational database you create Tables. Table is a place where you can store and query data.
Non relational database are sometimes called 'NoSql database' because it use structure other than row and columns to organzie data. One type of structural approach for non relational data base is key-value pair. With key-value pair data is organized into item(key) and item has attributes(Value) You can think of attributes as being different features of data.
In a key-value database, you can add or remove attributes from items in the table at any time. Additionally, not every item in the table has to have the same attributes.
Amazon DynamoDB :
Its a key-value database service. It offers single digit-millisecond performance.
Dynamo DB is serverless, which means no need to install, maintain or operate software. And no need to provision, patch or manage servers.
As the size of the database shrinks or grows Dynamo db automatically scales to adjust for changes in capacity while maintaining consistent performances. ex: Amazon prime day
Amazon DynamoDB is a fully managed NoSQL database service that provides fast and predictable performance with seamless scalability. You can use Amazon DynamoDB to create a database table that can store and retrieve any amount of data, and serve any level of request traffic. Amazon DynamoDB automatically spreads the data and traffic for the table over a sufficient number of servers to handle the request capacity specified by the customer and the amount of data stored, while maintaining consistent and fast performance.
AWS Database migration service
Its a cloud service to migrate data stores like relational database, non relational database, data warehouse .
Amazon RDS VS Dynamo DB :
RDS: Uses relational database. Automatic high availabilty recovery provided. Customer ownership of data, schema and network.
Dynamo Db: uses non-relational db, key-value pair. Its serverless and auto scaling.
Amazon redshift:
Efficiently analyze all your data using exisiting BI Tools. Amazon Redshift is a data warehousing service that you can use for big data analytics. It offers the ability to collect data from many sources and helps you to understand relationships and trends across your data.
AWS Database Migration Service (AWS DMS)
Aws DMS enables you to migrate relational databases, nonrelational databases, and other types of data stores.
With AWS DMS you can move data from source database and a target database. Source database and target database can be same type or different type.
During migration , the source database remains operational, reducing the downtime for any application relying on the database.
For example, suppose that you have a MySQL database that is stored on premises in an Amazon EC2 instance or in Amazon RDS. Consider the MySQL database to be your source database. Using AWS DMS, you could migrate your data to a target database, such as an Amazon Aurora database.
Use cases for AWS DMS:
Development and database migrations: Enabling developers to test applications against production data without affecting production users.
Database consolidation: Combing several database into single database
Continous replication: Sending ongoing copies of your data to other target sources instead of doing a one-time migration.
Other database Services:
1. Amazon document db:
Its a document database service, it supports MongoDB workloads
2. Amazon Neptune :
Its a graph database service.You can use Amazon Neptune to build and run applications that work with highly connected datasets, such as recommendation engines, fraud detection, and knowledge graphs.
3. Amazon quantum ledger database (Amazon QLDB) :
Its a ledger database service. You can use Amazon QLDB to review complete history of all the changes that have been made to your application data.
4. Amazon managed blockchain :
Its used to create and manage block chain networks with open-source framework. block chain is a distrubuted ledger system that lets multiple parties run transactions and share data without central authority.
5. Amazon elastic cache :
Its a service that adds caching layers on top of your databases to help improve the read times of common requests. It supports two types of data stores: Redis and Memcached.
6. Amazon DynamoDB Accelerator :
is an in-memory cache for DynamoDB. It helps improve response times from single-digit milliseconds to microseconds.
Aws DMS enables you to migrate relational databases, nonrelational databases, and other types of data stores.
With AWS DMS you can move data from source database and a target database. Source database and target database can be same type or different type.
During migration , the source database remains operational, reducing the downtime for any application relying on the database.
For example, suppose that you have a MySQL database that is stored on premises in an Amazon EC2 instance or in Amazon RDS. Consider the MySQL database to be your source database. Using AWS DMS, you could migrate your data to a target database, such as an Amazon Aurora database.
Use cases for AWS DMS:
Development and database migrations: Enabling developers to test applications against production data without affecting production users.
Database consolidation: Combing several database into single database
Continous replication: Sending ongoing copies of your data to other target sources instead of doing a one-time migration.
Other database Services:
1. Amazon document db:
Its a document database service, it supports MongoDB workloads
2. Amazon Neptune :
Its a graph database service.You can use Amazon Neptune to build and run applications that work with highly connected datasets, such as recommendation engines, fraud detection, and knowledge graphs.
3. Amazon quantum ledger database (Amazon QLDB) :
Its a ledger database service. You can use Amazon QLDB to review complete history of all the changes that have been made to your application data.
4. Amazon managed blockchain :
Its used to create and manage block chain networks with open-source framework. block chain is a distrubuted ledger system that lets multiple parties run transactions and share data without central authority.
5. Amazon elastic cache :
Its a service that adds caching layers on top of your databases to help improve the read times of common requests. It supports two types of data stores: Redis and Memcached.
6. Amazon DynamoDB Accelerator :
is an in-memory cache for DynamoDB. It helps improve response times from single-digit milliseconds to microseconds.
No comments:
Post a Comment