/ THE FEED /
Deep, recent reads ranked by depth and freshness.
01PlanetScale · architecture
Designing a sharded Postgres architecture requires grouping tables around shared query keys rather than starting from table structures alone. The post details how to configure shard groups, reference tables, SQL expressions in shard indexes, and global secondary indexes using Neki. This setup enables efficient single-shard queries and transactions while maintaining full Postgres semantics.
DEPTH ■■■■■8 min read3 days ago
02The Pragmatic Engineer · architecture
Anthropic experienced a massive increase in code merges due to the heavy use of AI coding agents, which quickly overwhelmed standard Git scaling limits and CI checkouts. To solve this, a small engineering team built a custom centralized version control system called Scion backed by Google Spanner and Cloud Storage, along with a code review tool called Antique. This move successfully migrated the entire monorepo off Git and eliminated bottlenecks, though the architecture drew controversy over similarities to Meta's Mononoke.
DEPTH ■■■■■12 min read4 days ago
03GitHub · architecture
GitHub faced abuse and performance issues because tools were scraping user identities to reconstruct repository star history. The engineering team collaborated with maintainers to build a purpose-built aggregate endpoint that returns daily star counts without identities and filters by time before aggregation. This reduced request volume from thousands to a few dozen and eliminated heavy database scans.
DEPTH ■■■■■8 min read11 days ago
04PlanetScale · practice
Postgres ON DELETE CASCADE can cause production outages, replication failures, and performance degradation during large deletions or schema migrations due to long-running transactions holding locks and generating massive WAL. The author recommends using soft deletes with batched background cleanup instead to keep transactions short and bounded. This approach avoids table locks and simplifies data recovery.
DEPTH ■■■■■5 min read5 days ago
05Zalando · architecture
Zalando replaced manual schema mapping for diverse supplier data with an LLM-powered Retrieval-Augmented Generation (RAG) flywheel. The system combines Amazon Titan embeddings stored in PostgreSQL via pgvector with tool-constrained generative models running at zero temperature. Every LLM suggestion passes through strict API validation checks before being reviewed by operations specialists, creating a continuous learning loop that improves future predictions.
DEPTH ■■■■■14 min read2 weeks ago
06Zalando · postmortem
An engineering team investigated why a client-side circuit breaker repeatedly failed to close on its own after an outage. By exporting OpenTelemetry traces into a custom visualization tool, they discovered that low probe traffic caused the autoscaler to scale down the service to minimum capacity, triggering a thundering herd of requests and immediate timeouts the moment the circuit breaker closed. Adjusting the system to account for capacity and probe representativeness prevents this oscillation.
DEPTH ■■■■■10 min read2 weeks ago
07PlanetScale · architecture
This post examines how PostgreSQL uses TOAST to manage oversized rows and variable-length columns through compression and off-row storage. It compares the pglz and LZ4 compression algorithms across different workloads, highlighting trade-offs in search windows, file sizes, and CPU overhead. The analysis demonstrates how these mechanisms impact storage efficiency and read performance for large text and JSON data.
DEPTH ■■■■■12 min read2 weeks ago
08Stack Overflow · practice
Standard regression tests usually only verify developer intent rather than the actual, unwritten behaviors that systems accumulate in production over time. By capturing real production traffic and network-level interactions, teams can build characterization test suites to automatically detect unintended behavioral shifts. This approach prevents accidental regressions during legacy refactoring, rewrites, and major framework upgrades.
DEPTH ■■■■■6 min read3 days ago
09PlanetScale · performance
Following competitive benchmarks that highlighted opportunities for optimization, the developers of the TIN full-text search index for Postgres released versions 1.0.4 and 1.0.6. These updates added Snowball stemming for 18 languages, optimized top-k sorting queries with deterministic tiebreakers, and improved overall query performance. As a result, benchmark tests show that TIN outperforms competing extensions across conjunction, disjunction, and phrase query workloads.
DEPTH ■■■■■8 min read2 weeks ago
10Stack Overflow · practice
Researchers analyzed thousands of job postings to understand how artificial intelligence is changing technical skill requirements. The findings indicate that instead of creating a workforce of pure generalists, the market is developing distinct new specialized skill bundles and hybrid roles. Consequently, organizations are facing talent debt as the half-life of technical skills rapidly shrinks.
DEPTH ■■■■■15 min read11 days ago
11Grab · architecture
Existing user personalization systems fail to systematically account for residential property values, leading to one-dimensional profiles and regional blind spots. A conceptual four-step method is proposed that combines public property data, large language models, building-level price estimation, and hierarchical clustering to classify user affluence. The approach remains unimplemented, with future viability depending on technical validation, data quality, and privacy approvals.
DEPTH ■■■■■4 min read2 weeks ago
12Grab · architecture
The organization proposes a conceptual data pipeline to analyze aggregated GrabMart transaction data for symptom-related product demand, aiming to supplement clinical surveillance for disease outbreaks. The design relies on data aggregation, statistical baseline comparisons, and privacy safeguards like differential privacy without involving individual health inference. The system remains in an early planning phase requiring rigorous validation, privacy review, and public health oversight.
DEPTH ■■■■■5 min read13 days ago
13PlanetScale · scaling
Growing applications often encounter hot shards when certain tenants or data subsets outgrow initial sharding strategies based on workspace IDs. To mitigate this, teams can scale up resources, isolate large tenants into specific key ranges, or fundamentally change the shard key for problematic tables based on access patterns. Neki facilitates these adjustments by allowing declarative data topologies and online resharding without downtime.
DEPTH ■■■■■7 min read4 weeks ago
