Delta Lake is an open-source storage layer that brings reliability to Data Lakes, by providing data warehouse-like features, on top of Data Lake. It has a big ecosystem, and there are various tools and architectures based on that - Delta Lake Storage, Delta Engine, Delta Architecture, Delta Live Tables,
How Does Delta Lake Work?, ACID Guarantees on Delta Lake, Understanding Delta Ecosystem, Setting up Azure Databricks Environment
3.35
Horas
Horas dedicadas al curso.
8
Clases
Clases totales
12
Estudio
Horas dedicadas de estudio
5
Nivel
Intermedio
Getting Started with Delta Lake
- How Does Delta Lake Work?
- ACID Guarantees on Delta Lake
- Understanding Delta Ecosystem
- Setting up Azure Databricks Environment
Working with Delta Lake Storage
- Storing Data in Delta Format
- Creating Delta Table
- Inserting Data to Delta Table
- Performing DML Operations: Update / Delete / Merge
- Schema Enforcement and Evolution
- Applying Table Constraints
- Accessing Data with Time Travel
Handling Streaming Data on Delta Lake
- Understanding Streaming on Delta Lake
- Setting up Streaming Environment
- Building Streaming Pipeline with Delta Lake
- Updating Delta Table with Streaming Data
- Transaction Log Checkpointing
- Handling Concurrency
Optimizing with Delta Engine in Databricks
- Understanding Delta Engine
- Improving Performance with Statistics and Data Skipping
- Optimizing & Z-ordering Delta Table
- Cleaning Files with Vacuum
- Working with Photon Execution Engine
- Enabling Auto Optimization on Delta Table
- Using Delta Cache
Building a Lakehouse Architecture
- Understanding Lakehouse Architecture
- Inside Lakehouse: Understanding Delta Architecture
- Using Change Data Feed
- Delta Architecture: With Batch Data & Change Data Feed
Building ETL Pipelines with Delta Live Tables
- Understanding Delta Live Tables
- Building Simple ETL Pipeline with Delta Live Tables
- Building Complex & Incremental ETL Pipeline
Implementing Common Use Cases
- Querying Delta Tables with Azure Synapse Analytics
- Consuming Delta Tables in Power BI
Satisfacción
¿Qué aprendí?
- Almacenamiento de datos en formato Delta
- Creación de tabla Delta
- Insertar datos en la tabla Delta
- Realización de operaciones DML: Actualizar/Eliminar/Fusionar
- Aplicación y evolución del esquema
- Aplicación de restricciones de tabla
- Acceso a datos con viajes en el tiempo
