Read what ships.

01Cloudflare · architecture

Origin Range Requests: Serving large files without range-request tuning

Cloudflare replaced a legacy, fixed-size range caching design that suffered from bandwidth waste and memory pressure with a Pingora-based chunk coordinator. The new implementation dynamically combines cached chunks and aligned origin fetches to serve range requests accurately without manual tuning. This approach cut p99 time to first byte in half for R2 storage and reduced tiered cache traffic by six percent.

DEPTH ■■■■■8 min read2 days ago

DatabasesPerformanceRedis

Sign in to save and rate

02GitHub · architecture

The bridge to agent-scale Git infrastructure

GitHub faced infrastructure capacity limits and a 25ms cross-region network round trip when migrating repositories to Azure. The team changed the storage quorum from repository-wide locks to per-reference serialization and introduced ephemeral virtual replicas backed by object storage. This raised write throughput, prevented slow replicas from blocking traffic, and decoupled read scaling from the write quorum.

DEPTH ■■■■■8 min read4 days ago

Infrastructure

Sign in to save and rate

03Datadog · architecture

Beyond container image signing: How we enforce software supply chain policies at Datadog

Container image signing alone cannot detect upstream security compromises occurring before the image is signed. To address this, the engineering team built an internal system called Turnstile that generates cryptographically signed attestations across an artifact's life cycle and evaluates them against environment-specific policies at deployment time. This approach enables the organization to enforce strict security and compliance requirements at scale without disrupting developer workflows.

DEPTH ■■■■■15 min read5 days ago

Sign in to save and rate

04Cloudflare · architecture

Bringing post-quantum cryptography to geographically fenced key management

Cloudflare introduced a post-quantum geographically restricted key manager in closed beta to enforce data sovereignty and protect customers against harvest-now-decrypt-later attacks. The team implemented a hybrid ML-KEM-768 and X25519 cryptographic design that separates globally distributed wrapped private keys from regionally restricted root keys. This ensures that decryption keys are only accessible within authorized geographical boundaries and fails closed if validation fails.

DEPTH ■■■■■12 min read2 days ago

Security

Sign in to save and rate

05Stack Overflow · architecture

MCP standardized tools. Now we need the infrastructure around them.

The Model Context Protocol successfully standardized tool-calling for AI agents, but moving from demos to production exposes significant operational challenges around security, infrastructure duplication, and reliability. Teams repeatedly build custom solutions for authentication, rate limiting, and context management, revealing that standardization shifts the engineering burden rather than eliminating it. Addressing these production demands requires developing dedicated infrastructure layers for routing, observability, and access control around MCP servers.

DEPTH ■■■■■8 min read10 days ago

Machine Learning

Sign in to save and rate

06GitHub · architecture

From 2,500 requests to 24: Rebuilding star history without the stargazers

GitHub faced abuse and performance issues because tools were scraping user identities to reconstruct repository star history. The engineering team collaborated with maintainers to build a purpose-built aggregate endpoint that returns daily star counts without identities and filters by time before aggregation. This reduced request volume from thousands to a few dozen and eliminated heavy database scans.

DEPTH ■■■■■8 min read11 days ago

Microservices

Sign in to save and rate

07Cloudflare · performance

Cutting our inference bill by 68% by being obsessive about optimization

An engineering team investigated a steady rise in AI code review costs and found that cache misses and redundant model calls were driving up expenses. By restructuring system prompts to enable cross-repository prompt caching, eliminating wasted tool calls, and switching to more efficient models, the team cut inference costs by 68% while also significantly improving review latency.

DEPTH ■■■■■15 min read6 days ago

Machine Learning

Sign in to save and rate

08Stack Overflow · architecture

Part 5: Operating an LLM system: observability, cost, routing, and the platform underneath

This post provides a concrete specification for the operability layer of production LLM systems, covering observability signals, structured logging, cost controls, routing, and failover mechanisms. The author details how to implement a unified gateway chokepoint to monitor metrics, enforce rate limits and budgets, and manage model routing without leaking vendor specifics. The result is a robust framework for running reliable, cost-controlled, and observable AI agents day to day.

DEPTH ■■■■■19 min read2 weeks ago

Machine Learning

Sign in to save and rate

09Dropbox · architecture

Evolving our calendar assistant Reclaim to be AI-native without starting over

Dropbox engineers evolved their calendar assistant, Reclaim, to support natural-language AI requests by building a custom agent platform that integrates with existing scheduling logic. They utilized a shared action system and a Preview Mode backed by Redis to ensure AI suggestions are consistent and reviewable before affecting real calendars. This approach allows the product to adopt AI capabilities without fragmenting its core architecture or duplicating functionality.

DEPTH ■■■■■7 min read3 weeks ago

InfrastructureMachine Learning

Sign in to save and rate

10Cloudflare · performance

Yandex outage: Data shows that instead of going dark, malicious bots went on the move

Following physical strikes on Yandex data centers, overall HTTP traffic from the network dropped significantly, leading to speculation that malicious bot farms had been disabled. However, traffic analysis revealed that search crawling simply decreased while malicious automation quickly migrated to other hosting providers with identical fingerprints. Consequently, overall malicious bot traffic remained flat, demonstrating that network disruptions do not stop portable automated workloads.

DEPTH ■■■■■8 min read11 days ago

Incidents & Postmortems

Sign in to save and rate

11Stack Overflow · architecture

Part 4: Safety and governance for LLM systems: guardrails, PII, audit, and memory

This post details safety and governance practices for LLM systems, focusing on layered guardrails, boundary-level PII handling, and immutable audit trails. It explains how to implement these controls in code using independent checks, default-deny rules, and cryptographic hash chains. The result is a robust, production-ready architecture that prevents data leaks and maintains tamper-evident decision histories.

DEPTH ■■■■■19 min read2 weeks ago

Machine Learning

Sign in to save and rate

12Datadog · architecture

How we extended Apache DataFusion to execute one query across many machines

Datadog needed to execute interactive queries across large datasets that exceeded single-node limits while maintaining low latency. To solve this, the team built Distributed DataFusion, an open-source framework extending Apache DataFusion across multiple machines using zero-copy streaming via Apache Arrow and Substrait. As a result, heavy queries complete up to 10 times faster without imposing distribution overhead on lightweight queries.

DEPTH ■■■■■12 min read3 weeks ago

Sign in to save and rate

13GitHub · architecture

Building Git infrastructure for agent-scale development

GitHub is redesigning its Git infrastructure to handle the massive surge in concurrent read and write operations driven by agentic software development and high-volume workloads. The team is decoupling storage from compute by leveraging Azure Blob Storage for durability and lightweight compute workers for requests, which separates scaling dimensions. In internal benchmarks, this approach achieves up to 35 times higher write throughput and independent read scalability.

DEPTH ■■■■■9 min read2 weeks ago

Infrastructure

Sign in to save and rate

14Stack Overflow · architecture

Part 3: Knowing when your agent doesn’t know: the confidence layer

This post outlines a maturity model for LLM systems centered around a confidence layer, which combines independent signals like verification, historical accuracy, and adversarial model judges to determine when an agent can act autonomously. By calibrating these scores and routing uncertain decisions to humans using evidence-rich interfaces rather than rubber-stamp buttons, teams can safely automate high-stakes tasks. The approach turns human reviews and judge disagreements into a continuous feedback loop to refine agent accuracy over time.

DEPTH ■■■■■12 min read2 weeks ago

Machine Learning

Sign in to save and rate

15Cloudflare · postmortem

Inside Hyperdrive: How we solved a rare Hyperdrive MySQL caching bug

During the beta of MySQL support for Hyperdrive, engineers investigated rare empty query responses and connection hangs occurring in 0.001% of requests. They discovered that sharing the data center cache between client and endpoint nodes caused protocol capability mismatches when translating MySQL wire messages. The team resolved the issue by standardizing the cache to always store responses encoded with canonical Hyperdrive capabilities.

DEPTH ■■■■■9 min read11 days ago

MySQL

Sign in to save and rate

16Datadog · architecture

How we built an async-aware Python profiler

Traditional Python profilers treat asyncio tasks as independent stacks, obscuring logical execution paths and relationships. The Datadog team built an async-aware profiling model that tracks task dependencies to reconstruct unified execution paths ("stacked stacks") while reducing profiling overhead by over 60 percent. This approach provides clearer performance visibility for asynchronous applications without excessive production latency.

DEPTH ■■■■■19 min read3 weeks ago

Sign in to save and rate

17GitHub · architecture

Staying Evergreen: How we automated secure access and repo hygiene in a shared org

Manual repository housekeeping and access reviews for outside collaborators become difficult to maintain as organizations scale. The team built Evergreen, a GitHub App that automates quarterly access reviews and stale repository archiving using native platform features and IssueOps. This automation successfully enforces secure defaults by revoking unanswered access while maintaining a full audit trail in git history.

DEPTH ■■■■■8 min read10 days ago

Sign in to save and rate

18Stack Overflow · architecture

Part 2: Evals as a deployment gate — and how to know when they drift

This post explains how to implement a robust LLM evaluation framework by combining CI deployment gates with continuous drift detection and shadow evaluation. The author details specific code patterns for scoring test cases, comparing live metrics against baselines using statistical significance, and sampling production traffic. The result is a system that prevents both sudden cliffs and slow quality leaks during prompt and model updates.

DEPTH ■■■■■8 min read2 weeks ago

Machine Learning

Sign in to save and rate

19Cloudflare · architecture

Under the hood, off the hot path: How D1 scales to millions of databases

Cloudflare scaled its serverless SQLite database, D1, to support millions of databases by decoupling the control plane from the data plane. The team used distributed metadata caching via Workers KV, optimized database connections through Hyperdrive, and implemented a self-correcting heartbeat mechanism. This architecture prevents centralized bottlenecks and allows D1 to handle rapid on-demand database creation efficiently.

DEPTH ■■■■■22 min read12 days ago

Databases

Sign in to save and rate

20Stack Overflow · architecture

Part 1: Make your AI agents boring: the determinism layer

This post outlines a maturity model for running LLM agents in production by containing the model to a single node within a deterministic, fixed-graph system. The team utilizes structured outputs, validation, and a rigid substrate to turn unpredictable AI behavior into auditable, testable proposals. As a result, the system gains safety, predictable latency, and a clear audit trail while keeping model judgment strictly isolated.

DEPTH ■■■■■13 min read2 weeks ago

Machine Learning

Sign in to save and rate