• views: 297
• date: 30 September 2022
• posted by: mitsumi

What you'll learn
How to build a Data Lake using Firehose API for .NET, S3, AWS Glue and Athena
Requirements
You must have an active AWS Account to utilized the included code.
Prior knowledge of AWS Web Services would be helpful, but not needed.
Description
The purpose of this class is to demonstrate a proof of concept using a series of lab exercise's (in the AWS Console using AWS Kinesis Data Firehose, AWS Glue, S3, Athena and the AWS SDK, with C# code using the AWS SDK) of building a Data Lake in the AWS ecosystem.In this class, we will be sending data from a local SQL Server database to AWS RDS securely, and automatically.I am utilizing the .NET SDK for AWS, however, this could easily be migrated to your language of choice once you understand the concepts that IВ am teaching. I will be providing the full working source code of this proof of concept.I will also provide a working example of creating Parquet files and sending to S3 (without using Firehose).
Overview
Section 1: Introduction
Lecture 1 Introduction
Section 2: Overview of AWS Web Services in a Data Lake
Lecture 2 AWS Serverless pieces to our Data Lake jigsaw puzzle
Section 3: AWS RDS and S3
Lecture 3 S3 Bucket creation
Lecture 4 RDS creation
Section 4: AWS Kinesis Firehose, C# and the AWS SDK
Lecture 5 Creating Kinesis Data Firehose delivery streams
Lecture 6 Console .NET core C# entity-framework project
Lecture 7 Kinesis Firehose C# API class
Lecture 8 Kinesis Firehose C# API send data example
Section 5: AWS Glue, Athena, ETL jobs, Triggers, and using Hangfire
Lecture 9 Creating Glue Crawlers
Lecture 10 Query Data in Athena using SQL
Lecture 11 Creating AWS ETL jobs
Lecture 12 Hangfire - Automation
Section 6: AWS Lake Formation, Cognito and Parquet .NET
Lecture 13 AWS Lake Formation
Lecture 14 AWS Cognito
Lecture 15 Parquet .NET - Alternative to using Firehose
Section 7: Final thoughts and advice
Lecture 16 Conclusion
AWS developers interested in learning about building a data lake using AWS services and the AWS SDK.



Download link

rapidgator.net:


uploadgig.com:


nitroflare.com:


1dl.net:


   
   
   

Forex Trading

Crypto Trading. 20 USD Bonus after KYC

We need your support!
Make a donation to help us stay online
        
Bitcoin (BTC)
bc1q08g9d22cxkawsjlf8etuek2pc9n2a3hs4cdrld
	
Bitcoin Cash (BCH)
qqvwexzhvgauxq2apgc4j0ewvcak6hh6lsnzmvtkem

Ethereum (ETH)
0xb55513D2c91A6e3c497621644ec99e206CDaf239

Litecoin (LTC)
ltc1qt6g2trfv9tjs4qj68sqc4uf0ukvc9jpnsyt59u

USDT (ERC20)
0xb55513D2c91A6e3c497621644ec99e206CDaf239

USDT (TRC20)
TYdPNrz7v1P9riWBWZ317oBgJueheGjATm

0 1 2 3 4 5

| Print version |

	Information
		Users of GUESTS are not allowed to comment this publication.