/ THE FEED /

Read what ships.

Deep, recent reads ranked by depth and freshness.

Machine Learning Redis Infrastructure Clear all

01Cloudflare · architecture
Origin Range Requests: Serving large files without range-request tuning
Cloudflare replaced a legacy, fixed-size range caching design that suffered from bandwidth waste and memory pressure with a Pingora-based chunk coordinator. The new implementation dynamically combines cached chunks and aligned origin fetches to serve range requests accurately without manual tuning. This approach cut p99 time to first byte in half for R2 storage and reduced tiered cache traffic by six percent.
DEPTH ■■■■■8 min read2 days ago
Databases Performance Redis
Sign in to save and rate
02GitHub · architecture
The bridge to agent-scale Git infrastructure
GitHub faced infrastructure capacity limits and a 25ms cross-region network round trip when migrating repositories to Azure. The team changed the storage quorum from repository-wide locks to per-reference serialization and introduced ephemeral virtual replicas backed by object storage. This raised write throughput, prevented slow replicas from blocking traffic, and decoupled read scaling from the write quorum.
DEPTH ■■■■■8 min read4 days ago
Infrastructure
Sign in to save and rate
03Meta · architecture
The Thin Orchestrator: How We Made Our AI Agents More Reliable by Giving the Router Less to Do
Meta engineers reduced hallucinations and improved reliability in their data pipeline agents by replacing a single heavyweight LLM orchestrator with a thin deterministic router and fourteen specialized subagents. The thin orchestrator handles routing and reliability checks through a fixed protocol, while isolated subagents execute domain reasoning. This architecture reduced configuration preparation time from a day to under an hour.
DEPTH ■■■■■7 min read6 days ago
Machine Learning
Sign in to save and rate
04Stack Overflow · architecture
MCP standardized tools. Now we need the infrastructure around them.
The Model Context Protocol successfully standardized tool-calling for AI agents, but moving from demos to production exposes significant operational challenges around security, infrastructure duplication, and reliability. Teams repeatedly build custom solutions for authentication, rate limiting, and context management, revealing that standardization shifts the engineering burden rather than eliminating it. Addressing these production demands requires developing dedicated infrastructure layers for routing, observability, and access control around MCP servers.
DEPTH ■■■■■8 min read9 days ago
Machine Learning
Sign in to save and rate
05Cloudflare · performance
Cutting our inference bill by 68% by being obsessive about optimization
An engineering team investigated a steady rise in AI code review costs and found that cache misses and redundant model calls were driving up expenses. By restructuring system prompts to enable cross-repository prompt caching, eliminating wasted tool calls, and switching to more efficient models, the team cut inference costs by 68% while also significantly improving review latency.
DEPTH ■■■■■15 min read6 days ago
Machine Learning
Sign in to save and rate
06Meta · architecture
Inside Prometheus: Meta’s Journey to a Gigawatt-Scale Supercluster
Meta built Prometheus, a gigawatt-scale AI supercluster designed to scale up to one million GPUs using multi-generation NVIDIA hardware and specialized backend Ethernet networks. The team achieved high goodput and significantly reduced system interruptions by implementing an in-rack sparing strategy, parallelism-aware fault tolerance, and close co-engineering with vendors. These infrastructure advancements allowed the team to successfully train frontier foundational models like Muse Wonder.
DEPTH ■■■■■17 min read6 days ago
Infrastructure Machine Learning
Sign in to save and rate
07Grab · architecture
Powering AI-led research through simulation
The engineering team built a self-contained simulation platform called sim-rs to model marketplace dispatch dynamics and accelerate experimentation. By connecting automated software agents to this simulator with strict safety safeguards, the team successfully optimized strategy policies and simulator performance. This closed-loop lab environment allows agents to test ideas rapidly while ensuring that only valid, production-ready improvements are produced.
DEPTH ■■■■■10 min read2 weeks ago
Machine Learning
Sign in to save and rate
08Stack Overflow · architecture
Part 5: Operating an LLM system: observability, cost, routing, and the platform underneath
This post provides a concrete specification for the operability layer of production LLM systems, covering observability signals, structured logging, cost controls, routing, and failover mechanisms. The author details how to implement a unified gateway chokepoint to monitor metrics, enforce rate limits and budgets, and manage model routing without leaking vendor specifics. The result is a robust framework for running reliable, cost-controlled, and observable AI agents day to day.
DEPTH ■■■■■19 min read2 weeks ago
Machine Learning
Sign in to save and rate
09Meta · architecture
Meta’s Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI
Meta developed GEM, a unified recommendation foundation model using an LLM-style architecture called Kunlun to handle heterogeneous ads data across multiple surfaces. The team implemented block attention, custom low-precision kernels, and post-training intelligence transfer techniques like LoopFM and Solaris to scale training efficiency and bypass real-time serving latency bottlenecks. The resulting system delivered an 18.9% conversion improvement and a 12% click-through rate increase across Meta ads surfaces.
DEPTH ■■■■■15 min read12 days ago
Machine Learning
Sign in to save and rate
10Cloudflare · scaling
Rewinding the exponential: How we keep hardware usage down
Cloudflare tackled exponential traffic growth against linear hardware growth by continuously evolving their data pipeline architecture over five years. Key optimizations included breaking up monolithic schemas, switching to zstd compression, decoupling pipeline transit from database replication, decentralizing sampling, and moving aggregations earlier in the pipeline. These strategies significantly reduced backbone bandwidth usage, network congestion, and storage requirements while preserving data quality.
DEPTH ■■■■■12 min read12 days ago
Infrastructure
Sign in to save and rate
11GitHub · architecture
Building Git infrastructure for agent-scale development
GitHub is redesigning its Git infrastructure to handle the massive surge in concurrent read and write operations driven by agentic software development and high-volume workloads. The team is decoupling storage from compute by leveraging Azure Blob Storage for durability and lightweight compute workers for requests, which separates scaling dimensions. In internal benchmarks, this approach achieves up to 35 times higher write throughput and independent read scalability.
DEPTH ■■■■■9 min read2 weeks ago
Infrastructure
Sign in to save and rate
12Dropbox · architecture
Evolving our calendar assistant Reclaim to be AI-native without starting over
Dropbox engineers evolved their calendar assistant, Reclaim, to support natural-language AI requests by building a custom agent platform that integrates with existing scheduling logic. They utilized a shared action system and a Preview Mode backed by Redis to ensure AI suggestions are consistent and reviewable before affecting real calendars. This approach allows the product to adopt AI capabilities without fragmenting its core architecture or duplicating functionality.
DEPTH ■■■■■7 min read3 weeks ago
Infrastructure Machine Learning
Sign in to save and rate
13Stack Overflow · architecture
Part 4: Safety and governance for LLM systems: guardrails, PII, audit, and memory
This post details safety and governance practices for LLM systems, focusing on layered guardrails, boundary-level PII handling, and immutable audit trails. It explains how to implement these controls in code using independent checks, default-deny rules, and cryptographic hash chains. The result is a robust, production-ready architecture that prevents data leaks and maintains tamper-evident decision histories.
DEPTH ■■■■■19 min read2 weeks ago
Machine Learning
Sign in to save and rate
14Meta · architecture
OCP Summit 2026: Powering AI with Open Innovations in Scale-Up, Scale-Out, and Scale-Across Networks
Meta shared its next-generation open networking infrastructure for AI at the OCP Summit 2026, including liquid-cooled 102.4T switch platforms, multiplane scale-out fabrics, and Ethernet-based scale-up solutions. The team also highlighted collaborative industry standards like ESUN and OSPA to unify hardware and transport layers. These open innovations aim to support massive gigawatt-scale AI clusters across diverse data center environments.
DEPTH ■■■■■12 min read12 days ago
Machine Learning
Sign in to save and rate
15Cloudflare · architecture
How Commerce.com and Cloudflare are getting storefronts ready for AI agents
Commerce.com and Cloudflare collaborated to enable WebMCP at the edge for nearly 100,000 storefronts, allowing AI agents to interact reliably with e-commerce sites using structured tools rather than fragile HTML scraping. The team implemented edge-level configuration rules using custom hostname metadata to provide individual merchants with a seamless opt-out mechanism and added support for custom MCP paths. This integration allows agents to execute shopping tasks directly within the active browser session while ensuring fallback safety and zero extra latency for opted-out stores.
DEPTH ■■■■■6 min read4 days ago
Machine Learning
Sign in to save and rate
16Stack Overflow · architecture
Part 3: Knowing when your agent doesn’t know: the confidence layer
This post outlines a maturity model for LLM systems centered around a confidence layer, which combines independent signals like verification, historical accuracy, and adversarial model judges to determine when an agent can act autonomously. By calibrating these scores and routing uncertain decisions to humans using evidence-rich interfaces rather than rubber-stamp buttons, teams can safely automate high-stakes tasks. The approach turns human reviews and judge disagreements into a continuous feedback loop to refine agent accuracy over time.
DEPTH ■■■■■12 min read2 weeks ago
Machine Learning
Sign in to save and rate
17Stack Overflow · architecture
Part 2: Evals as a deployment gate — and how to know when they drift
This post explains how to implement a robust LLM evaluation framework by combining CI deployment gates with continuous drift detection and shadow evaluation. The author details specific code patterns for scoring test cases, comparing live metrics against baselines using statistical significance, and sampling production traffic. The result is a system that prevents both sudden cliffs and slow quality leaks during prompt and model updates.
DEPTH ■■■■■8 min read2 weeks ago
Machine Learning
Sign in to save and rate
18Meta · architecture
GenRecs Tokenization: Transforming Ads Recommendation at Meta with Stable Model Vocabulary
Meta implemented GenRecs Tokenization to replace dedicated item IDs with a stable semantic vocabulary for ads recommendation. Using a three-stage encoding and quantization pipeline, the system enables language models to reason about ad content and user preferences simultaneously. This approach resulted in a +0.35% lift in conversions and improved cold-start performance for new entities.
DEPTH ■■■■■6 min read6 days ago
Machine Learning
Sign in to save and rate
19Stack Overflow · architecture
Part 1: Make your AI agents boring: the determinism layer
This post outlines a maturity model for running LLM agents in production by containing the model to a single node within a deterministic, fixed-graph system. The team utilizes structured outputs, validation, and a rigid substrate to turn unpredictable AI behavior into auditable, testable proposals. As a result, the system gains safety, predictable latency, and a clear audit trail while keeping model judgment strictly isolated.
DEPTH ■■■■■13 min read2 weeks ago
Machine Learning
Sign in to save and rate
20Meta · scaling
2026 World Cup: Breaking Records, Spiky Charts and Reliability at Scale
Meta's infrastructure handled record-breaking traffic during the 2026 World Cup, including 30 million messages per second on WhatsApp, by relying on automated load balancing, capacity buffers, and targeted pre-builds. Traffic patterns revealed that surges occurred primarily during breaks rather than gameplay, requiring region-specific forecasting rather than cross-region load shifting. The systems maintained full reliability and allowed normal continuous deployment schedules to continue without manual intervention or system freezes.
DEPTH ■■■■■5 min read10 days ago
Infrastructure
Sign in to save and rate