Azure Synapse Analytics | Data Ingestion Patterns, Polybase, and Copy Command

Having right data ingestion framework and patterns are important considerations to have consistent data ready for analytics in timely manner, especially when you are dealing with large data volume and expect your data to grow every day. In this video, you will learn about different data ingestion patterns and options available with Azure Synapse Analytics - dedicated SQL pool. 00:00:00 Introduction 00:00:28 ELT (Extract Transform and Load) vs ELT (Extract Load and Transform) 00:01:57 Single Gated Bulk Data Load 00:05:06 Parallel Data Load 00:07:01 Parallel Data Load - Polybase 00:09:42 Data Import - with CTAS 00:11:52 Data Export - with CETAS 00:14:58 Parallel Data Load - Copy Command 00:19:25 Demo - Bulk Data Load with bcp 00:26:55 Demo - Bulk Data Load with Polybase 00:53:07 Demo - Data Import (CTAS) vs. Data Export (CETAS) 00:57:32 Demo - Bulk Data Load with Copy Command 01:11:49 Demo - Incremental Data Refresh 01:17:29 Demo - Monitor Data Load Operations Thank you for watching, in my next video, I am going to talk in detail about multi-layer security architecture available with Azure Synapse and how you can build modern data warehouse with confidence in Azure Synapse. Stay tuned. GitHub Repo to download deck and script used in the video: https://github.com/AasTrailblazers/Az... Data loading strategies for dedicated SQL pool in Azure Synapse Analytics https://docs.microsoft.com/en-us/azur... Design a PolyBase data loading strategy for dedicated SQL pool in Azure Synapse Analytics https://docs.microsoft.com/en-us/azur... CREATE EXTERNAL TABLE https://docs.microsoft.com/en-us/sql/... CETAS with Synapse SQL https://docs.microsoft.com/en-us/azur... Bulk load data using the COPY statement https://docs.microsoft.com/en-us/sql/... What is the file splitting guidance for the COPY command loading CSV files? https://docs.microsoft.com/en-us/sql/... How to maximize COPY load throughput with file splits https://techcommunity.microsoft.com/t... bcp Utility https://docs.microsoft.com/en-us/sql/...