Prioritising Data Quality with dbt-expectations: A Practical Approach to Building Reliable Data Pipelines
Discover how dbt-expectations enhances data quality checks within dbt pipelines, ensuring reliable analytics and streamlined workflows.
Discover how dbt-expectations enhances data quality checks within dbt pipelines, ensuring reliable analytics and streamlined workflows.
Discover streamlined ETL pipeline deployment in AWS using Azure DevOps, ensuring reliability and efficiency for data-driven decision-making.
Cevo Consultant and AWS Ambassador, Matt Gillard, covers off some of the announcements and themes from Day 2 at AWS re:Invent 2023.
We’ll discuss how to implement data warehousing ETL using Iceberg for data storage/management and Spark for data processing. A Pyspark ETL app will be used for demonstration in an EMR local environment. Finally the ETL results will be queried by Athena for verification.