Comment by matthewhelm

I wholeheartedly agree. When I worked at Shopify, we had to hand-roll our incremental data models using Spark, and the complexity of managing deep DAGs made tasks like backfilling and refactoring a huge pain. Tools like dbt and SQLMesh face similar challenges.

The chaos of existing approaches was a large part of what drove me to join Materialize. With Materialize, you can use dbt on “easy-mode”, while Materialize handles incremental logic, removing the usual headaches around processing time and keeping everything up to date within a second or two.

I recently gave a talk at Data Council about this unlock, it’s total magic: https://youtu.be/pLb5sFZ7nWw

For anyone interested, my colleague Seth also discussed this in a recent blog post: https://materialize.com/blog/migrating-postgres-materialize/