/ THE FEED /
Deep, recent reads ranked by depth and freshness.
01PlanetScale · architecture
Designing a sharded Postgres architecture requires grouping tables around shared query keys rather than starting from table structures alone. The post details how to configure shard groups, reference tables, SQL expressions in shard indexes, and global secondary indexes using Neki. This setup enables efficient single-shard queries and transactions while maintaining full Postgres semantics.
DEPTH ■■■■■8 min read2 days ago
02GitHub · architecture
GitHub faced infrastructure capacity limits and a 25ms cross-region network round trip when migrating repositories to Azure. The team changed the storage quorum from repository-wide locks to per-reference serialization and introduced ephemeral virtual replicas backed by object storage. This raised write throughput, prevented slow replicas from blocking traffic, and decoupled read scaling from the write quorum.
DEPTH ■■■■■8 min read4 days ago
03Meta · architecture
Meta built Prometheus, a gigawatt-scale AI supercluster designed to scale up to one million GPUs using multi-generation NVIDIA hardware and specialized backend Ethernet networks. The team achieved high goodput and significantly reduced system interruptions by implementing an in-rack sparing strategy, parallelism-aware fault tolerance, and close co-engineering with vendors. These infrastructure advancements allowed the team to successfully train frontier foundational models like Muse Wonder.
DEPTH ■■■■■17 min read6 days ago
04PlanetScale · practice
Postgres ON DELETE CASCADE can cause production outages, replication failures, and performance degradation during large deletions or schema migrations due to long-running transactions holding locks and generating massive WAL. The author recommends using soft deletes with batched background cleanup instead to keep transactions short and bounded. This approach avoids table locks and simplifies data recovery.
DEPTH ■■■■■5 min read4 days ago
05Cloudflare · scaling
Cloudflare tackled exponential traffic growth against linear hardware growth by continuously evolving their data pipeline architecture over five years. Key optimizations included breaking up monolithic schemas, switching to zstd compression, decoupling pipeline transit from database replication, decentralizing sampling, and moving aggregations earlier in the pipeline. These strategies significantly reduced backbone bandwidth usage, network congestion, and storage requirements while preserving data quality.
DEPTH ■■■■■12 min read12 days ago
06GitHub · architecture
GitHub is redesigning its Git infrastructure to handle the massive surge in concurrent read and write operations driven by agentic software development and high-volume workloads. The team is decoupling storage from compute by leveraging Azure Blob Storage for durability and lightweight compute workers for requests, which separates scaling dimensions. In internal benchmarks, this approach achieves up to 35 times higher write throughput and independent read scalability.
DEPTH ■■■■■9 min read2 weeks ago
07Dropbox · architecture
Dropbox engineers evolved their calendar assistant, Reclaim, to support natural-language AI requests by building a custom agent platform that integrates with existing scheduling logic. They utilized a shared action system and a Preview Mode backed by Redis to ensure AI suggestions are consistent and reviewable before affecting real calendars. This approach allows the product to adopt AI capabilities without fragmenting its core architecture or duplicating functionality.
DEPTH ■■■■■7 min read3 weeks ago
08PlanetScale · architecture
This post examines how PostgreSQL uses TOAST to manage oversized rows and variable-length columns through compression and off-row storage. It compares the pglz and LZ4 compression algorithms across different workloads, highlighting trade-offs in search windows, file sizes, and CPU overhead. The analysis demonstrates how these mechanisms impact storage efficiency and read performance for large text and JSON data.
DEPTH ■■■■■12 min read13 days ago
09PlanetScale · performance
Following competitive benchmarks that highlighted opportunities for optimization, the developers of the TIN full-text search index for Postgres released versions 1.0.4 and 1.0.6. These updates added Snowball stemming for 18 languages, optimized top-k sorting queries with deterministic tiebreakers, and improved overall query performance. As a result, benchmark tests show that TIN outperforms competing extensions across conjunction, disjunction, and phrase query workloads.
DEPTH ■■■■■8 min read2 weeks ago
