Category:

Showing page 2 of 17

The Post-Acquisition Shuffle: What Lean Data Teams Should Actually Do About Orchestration

Prefect just bought Dagster. Your orchestration roadmap just got rewritten. Here’s what Airflow, Kestra, Mage, and the alternatives actually deliver for small teams.

#airflow#dagster#Data Orchestration...

Data Engineering

The Medallion Architecture Civil War: Silver vs. Gold for Fact and Dimension Tables

A Reddit debate exposes the dirty secret of medallion architecture: nobody agrees where fact and dimension tables actually belong. Here’s what the BigBasket case study reveals.

#Data Engineering#Data Modeling#dimension tables...

Apache Ossie

Snowflake Gave Up Control: Why Apache Ossie Is the Semantic Standard We Actually Needed

The Open Semantic Interchange (OSI) has moved from Snowflake to the ASF as Apache Ossie. This isn’t just a rebrand, it’s the structural fix that makes a semantic standard actually trustworthy.

#Apache Ossie#Data Engineering#data governance...

Google Research

Google’s TabFM Just Made XGBoost Feel Like a Horse-Drawn Carriage

Google Research’s TabFM brings zero-shot in-context learning to tabular data, eliminating hyperparameter tuning and feature engineering. Here’s how it works and why it matters.

#Google Research#in-context learning#TabFM...

geospatial

Stop Calculating Distances Wrong: The Architecture of Real-Time Proximity Discovery

Why your naive Haversine loop will kill your app at scale, and how Uber, Redis, and PostGIS actually solve proximity-based user discovery.

#geospatial#real-time#Scalability...

data-stack

PostgreSQL as Your Entire Data Stack: The Multi-Tool Orthodoxy Is Crumbling

One database to rule them all? A deep dive into pushing PostgreSQL to serve as a full data-centric stack, challenging the multi-tool orthodoxy with real benchmarks and production-ready code.

#data-stack#OLAP#oltp...

database performance

30 Indexes on One Table? You’re Not Scaling, You’re Bleeding

How to diagnose index maintenance blocking, safely remove dead indexes, and whether you should ditch RDBMS for a key-value store.

#database performance#indexing#RDBMS...

database history

The Database Gods You’ve Never Heard Of: Why Patricia Selinger Matters More Than Codd

How forgotten pioneers like Patricia Selinger and Raymond Boyce built the relational foundation you take for granted, and why erasing their legacy is hurting modern system design.

#database history#Patricia Selinger#relational model...

backend architecture

Orders API

#backend architecture#CRUD generation#database design...

Data Engineering

The Python Trap: Why Your SQL-First Team Will Hate DLT (and Why That’s Okay)

A real-world comparison of open-source DLT and managed Fivetran when your team lives in SQL, not Python. Includes costs, AI-assisted coding as a band-aid, and whether dlt is ready to kill Prefect.

#Data Engineering#data pipelines#dbt...

AI Agents

Databricks LTAP: The 40-Year OLTP/OLAP Wall Just Got a Sledgehammer

Databricks claims LTAP collapses the four-decade divide between transactional and analytical databases into a single copy of data. No CDC, no ETL, no copies. Here’s why it matters and where it could fail.

#AI Agents#data architecture#databricks...

apache spark

You Migrated from Spark to dbt. Your Incremental Models Got Slower. Here’s Why.

A real-world account of migrating from Spark-based ETL to dbt ELT on Redshift, highlighting the surprising performance drawbacks of Redshift Serverless for incremental models compared to provisioned Redshift.

#apache spark#Data Engineering#dbt...