Overview

Modern Databases & Data Pipelines: Hands-On Data Engineering with Kafka, Airflow, and dbtBuild, automate, and scale end-to-end data pipelines with the tools defining modern data engineering in 2025. Databases are no longer static repositories-they are the beating heart of every digital system. In today's data-driven world, engineers must design ecosystems that move, transform, and validate data continuously across cloud, hybrid, and on-prem environments. This book takes you beyond theory into the hands-on reality of building, deploying, and operating modern, production-grade data systems. What You'll Learn Master the modern database trio: PostgreSQL for OLTP, DuckDB for fast local analytics, and ClickHouse for distributed OLAP queries. Stream data in real time with Apache Kafka - implement change data capture (CDC), manage offsets, and build fault-tolerant ingestion layers. Orchestrate with Apache Airflow - create production DAGs, manage scheduling, automate recovery, and integrate transformations seamlessly. Transform and model data with dbt - apply modular SQL development, Jinja macros, version control, and automated testing. Implement data quality & observability - integrate Great Expectations, Prometheus, and Grafana for continuous validation and live monitoring. Automate deployment using GitHub Actions, Terraform, Docker, and Helm - build reproducible environments and scale effortlessly. Secure and govern your pipeline - enforce encryption, RBAC, audit logs, and compliance-ready lineage tracking. Hands-On Practice LabsEvery chapter includes a fully tested lab with real-world configurations, SQL scripts, and reproducible Docker-based environments. You'll deploy working pipelines that move data from PostgreSQL → Kafka → Airflow → dbt → ClickHouse → Grafana, complete with monitoring and automated recovery. Who This Book Is For Data Engineers modernizing legacy ETL into ELT and real-time pipelines Developers & Analysts looking to understand orchestration, automation, and transformation tools DevOps & Platform Engineers expanding into DataOps, CI/CD, and infrastructure automation Students and Career Switchers building a portfolio-ready full-stack data project Architects & Technical Leads designing end-to-end, resilient data ecosystems Why This Book Stands OutUnlike generic tutorials, this book is purely practical and up-to-date - written for 2025 and beyond. You won't just read about data pipelines - you'll build them, monitor them, secure them, and automate them using real tools trusted by global data teams. No screenshots. No fluff. Just code, flowcharts, and structured logic - engineered for professionals who build systems that move data at scale. By the End of This BookYou'll have deployed a complete, production-style data platform, capable of streaming, transforming, validating, and visualizing data in real time - from raw ingestion to actionable analytics. Turn your data into a living, automated system. Build the pipelines that power tomorrow's intelligence - today.

Full Product Details

9798273653337

Table of Contents

Reviews

Author Information

Tab Content 6

Countries Available