Data Pipeline Architectures

A modern open lakehouse with medallion architecture — bronze, silver, and gold layers backed by Iceberg on S3, queried via Trino.

A fully managed, GCP-native pipeline from Pub/Sub event ingestion through Dataflow processing to BigQuery analytics.

A real-time streaming architecture with Kafka as the event backbone and Flink for stateful aggregations, CEP, and stream-to-table writes.

A SaaS-first ELT pattern — Fivetran handles zero-code ingestion into Snowflake, dbt handles modular tested transformations.