Data Lake, Lakehouse & Warehouse

DPTH02 Populate Your Data Warehouse with a Metadata-Driven, Pattern-Based Approach

11/21/2024

8:00am - 9:15am

Level: Introductory to Intermediate

Meagan Longoria

Data & Analytics Consultant

Denny Cherry & Associates Consulting

This demo-filled session will show how to populate a data warehouse (SQL Server, Azure SQL, or Fabric warehouse) by using a meta-data driven, parameterized approach with Data Factory. We'll discuss what metadata needs to be captured to achieve goals such as increased reusability and restartability and minimized deployment and maintenance effort. And we'll look at common Data Factory orchestration and execution patterns.

You will learn:

  • Demonstrate how to build metadata-driven pipelines
  • Discuss the benefits of metadata-driven pipelines
  • Demonstrate use of control tables to capture and serve metadata