Tagged with

4 articles found

SQL Isn’t Dying, It’s Mutating, and Your Job Description Is Outdated

AI agents are writing more data pipeline code than ever. Yet SQL isn’t disappearing, it’s becoming the specification language your entire stack speaks. Here’s what actually changed.

#AI Coding Agents#data pipelines

apache spark

Your Spark Pipelines Are Still Writing Code Like It’s 2015. SDP Is Here to Fix That.

Spark Declarative Pipelines are becoming native to Apache Spark 4.1. Here’s what changes for data engineers, and why the open-source community is finally paying attention.

#apache spark#data pipelines#Delta Live Tables

Data Engineering

The Python Trap: Why Your SQL-First Team Will Hate DLT (and Why That’s Okay)

A real-world comparison of open-source DLT and managed Fivetran when your team lives in SQL, not Python. Includes costs, AI-assisted coding as a band-aid, and whether dlt is ready to kill Prefect.

#Data Engineering#data pipelines#dbt...

Data Engineering

Your ‘Terabyte-Scale’ Data Pipeline Probably Processes 15 GB a Day

Data engineers on Reddit reveal the massive gap between resume inflation and reality, most incremental loads are 1-15 GB, not terabytes. Here’s what real-world volumes look like across industries and why domain matters more than raw size.

#Data Engineering#data pipelines#etl...