Read what ships.

01Cloudflare · architecture

Origin Range Requests: Serving large files without range-request tuning

Cloudflare replaced a legacy, fixed-size range caching design that suffered from bandwidth waste and memory pressure with a Pingora-based chunk coordinator. The new implementation dynamically combines cached chunks and aligned origin fetches to serve range requests accurately without manual tuning. This approach cut p99 time to first byte in half for R2 storage and reduced tiered cache traffic by six percent.

DEPTH ■■■■■8 min read2 days ago

DatabasesPerformanceRedis

Sign in to save and rate

02Amazon Web Services · architecture

Automate medical coding with Step Functions and Amazon Bedrock AgentCore

Automating medical coding requires a hybrid approach that combines deterministic orchestration with bounded agentic reasoning to ensure safety, auditability, and human oversight. The team built a workflow using AWS Step Functions, AWS Lambda, and Amazon Bedrock AgentCore that evaluates terms cheapest-first via block-lists, exact matching, and finally an LLM agent for ambiguous cases. This design successfully autocodes confident cases while routing ambiguous ones to human reviewers with full audit trails and decision explanations.

DEPTH ■■■■■15 min read3 days ago

AWS

Sign in to save and rate

03Cloudflare · architecture

Bringing post-quantum cryptography to geographically fenced key management

Cloudflare introduced a post-quantum geographically restricted key manager in closed beta to enforce data sovereignty and protect customers against harvest-now-decrypt-later attacks. The team implemented a hybrid ML-KEM-768 and X25519 cryptographic design that separates globally distributed wrapped private keys from regionally restricted root keys. This ensures that decryption keys are only accessible within authorized geographical boundaries and fails closed if validation fails.

DEPTH ■■■■■12 min read2 days ago

Security

Sign in to save and rate

04Stack Overflow · architecture

MCP standardized tools. Now we need the infrastructure around them.

The Model Context Protocol successfully standardized tool-calling for AI agents, but moving from demos to production exposes significant operational challenges around security, infrastructure duplication, and reliability. Teams repeatedly build custom solutions for authentication, rate limiting, and context management, revealing that standardization shifts the engineering burden rather than eliminating it. Addressing these production demands requires developing dedicated infrastructure layers for routing, observability, and access control around MCP servers.

DEPTH ■■■■■8 min read10 days ago

Machine Learning

Sign in to save and rate

05Amazon Web Services · architecture

Multi-region disaster recovery for Amazon API Gateway WebSocket APIs

Real-time applications using WebSocket connections face disruptions during regional AWS outages because connections are tied to specific regions. The team implemented an active-passive multi-region disaster recovery architecture using Route 53 DNS failover, DynamoDB Global Tables, an automated canary health probe, and client-side exponential backoff with jitter. This design automatically detects failures, redirects traffic, and restores client state without manual intervention.

DEPTH ■■■■■10 min read11 days ago

Sign in to save and rate

06Cloudflare · performance

Cutting our inference bill by 68% by being obsessive about optimization

An engineering team investigated a steady rise in AI code review costs and found that cache misses and redundant model calls were driving up expenses. By restructuring system prompts to enable cross-repository prompt caching, eliminating wasted tool calls, and switching to more efficient models, the team cut inference costs by 68% while also significantly improving review latency.

DEPTH ■■■■■15 min read6 days ago

Machine Learning

Sign in to save and rate

07Stack Overflow · architecture

Part 5: Operating an LLM system: observability, cost, routing, and the platform underneath

This post provides a concrete specification for the operability layer of production LLM systems, covering observability signals, structured logging, cost controls, routing, and failover mechanisms. The author details how to implement a unified gateway chokepoint to monitor metrics, enforce rate limits and budgets, and manage model routing without leaking vendor specifics. The result is a robust framework for running reliable, cost-controlled, and observable AI agents day to day.

DEPTH ■■■■■19 min read2 weeks ago

Machine Learning

Sign in to save and rate

08Amazon Web Services · architecture

How UTS established a modern curriculum data services foundation on AWS

University of Technology Sydney rearchitected its curriculum data integration using serverless AWS services to decouple systems of record from downstream consumers. By implementing domain-driven design with an event-driven architecture and purpose-built persistence layers, the team achieved zero production incidents, low latency, and reduced operational overhead while managed by a single engineer.

DEPTH ■■■■■9 min read11 days ago

Sign in to save and rate

09Cloudflare · performance

Yandex outage: Data shows that instead of going dark, malicious bots went on the move

Following physical strikes on Yandex data centers, overall HTTP traffic from the network dropped significantly, leading to speculation that malicious bot farms had been disabled. However, traffic analysis revealed that search crawling simply decreased while malicious automation quickly migrated to other hosting providers with identical fingerprints. Consequently, overall malicious bot traffic remained flat, demonstrating that network disruptions do not stop portable automated workloads.

DEPTH ■■■■■8 min read11 days ago

Incidents & Postmortems

Sign in to save and rate

10Stack Overflow · architecture

Part 4: Safety and governance for LLM systems: guardrails, PII, audit, and memory

This post details safety and governance practices for LLM systems, focusing on layered guardrails, boundary-level PII handling, and immutable audit trails. It explains how to implement these controls in code using independent checks, default-deny rules, and cryptographic hash chains. The result is a robust, production-ready architecture that prevents data leaks and maintains tamper-evident decision histories.

DEPTH ■■■■■19 min read2 weeks ago

Machine Learning

Sign in to save and rate

11Amazon Web Services · tooling

Scope workloads and share resilience policies with AWS Resilience Hub

Engineering teams often struggle to scope workloads and maintain up-to-date resilience policies in shared Kubernetes clusters after migrations or deployments. AWS Resilience Hub introduced features enabling Kubernetes label-based resource scoping, generative AI-powered dependency insights, and cross-account resilience policy sharing via AWS Organizations. These capabilities allow teams to isolate specific workloads, prioritize dependency investigations, and enforce centralized recovery standards.

DEPTH ■■■■■14 min read11 days ago

Sign in to save and rate

12Stack Overflow · architecture

Part 3: Knowing when your agent doesn’t know: the confidence layer

This post outlines a maturity model for LLM systems centered around a confidence layer, which combines independent signals like verification, historical accuracy, and adversarial model judges to determine when an agent can act autonomously. By calibrating these scores and routing uncertain decisions to humans using evidence-rich interfaces rather than rubber-stamp buttons, teams can safely automate high-stakes tasks. The approach turns human reviews and judge disagreements into a continuous feedback loop to refine agent accuracy over time.

DEPTH ■■■■■12 min read2 weeks ago

Machine Learning

Sign in to save and rate

13Cloudflare · postmortem

Inside Hyperdrive: How we solved a rare Hyperdrive MySQL caching bug

During the beta of MySQL support for Hyperdrive, engineers investigated rare empty query responses and connection hangs occurring in 0.001% of requests. They discovered that sharing the data center cache between client and endpoint nodes caused protocol capability mismatches when translating MySQL wire messages. The team resolved the issue by standardizing the cache to always store responses encoded with canonical Hyperdrive capabilities.

DEPTH ■■■■■9 min read11 days ago

MySQL

Sign in to save and rate

14Amazon Web Services · architecture

Stopping business email compromise with AI-driven email security: How VIPRE protects modern enterprises on AWS

VIPRE developed an AI-driven, cloud-native email security solution on AWS to protect enterprises against business email compromise and advanced phishing attacks. The system uses layered machine learning, semantic analysis, and dynamic sandboxing orchestrated via serverless containers and message queues to evaluate over half a million messages daily. In production testing, the architecture successfully identified thousands of BEC attempts while maintaining low-latency processing and reducing administrative overhead.

DEPTH ■■■■■12 min read12 days ago

Sign in to save and rate

15Stack Overflow · architecture

Part 2: Evals as a deployment gate — and how to know when they drift

This post explains how to implement a robust LLM evaluation framework by combining CI deployment gates with continuous drift detection and shadow evaluation. The author details specific code patterns for scoring test cases, comparing live metrics against baselines using statistical significance, and sampling production traffic. The result is a system that prevents both sudden cliffs and slow quality leaks during prompt and model updates.

DEPTH ■■■■■8 min read2 weeks ago

Machine Learning

Sign in to save and rate

16Cloudflare · architecture

Under the hood, off the hot path: How D1 scales to millions of databases

Cloudflare scaled its serverless SQLite database, D1, to support millions of databases by decoupling the control plane from the data plane. The team used distributed metadata caching via Workers KV, optimized database connections through Hyperdrive, and implemented a self-correcting heartbeat mechanism. This architecture prevents centralized bottlenecks and allows D1 to handle rapid on-demand database creation efficiently.

DEPTH ■■■■■22 min read12 days ago

Databases

Sign in to save and rate

17Amazon Web Services · architecture

Bulk Executor for Amazon DynamoDB, Part 2: Lessons we learned the hard way

Engineering a bulk execution tool for Amazon DynamoDB using AWS Glue required addressing several architectural challenges, including cost estimation, graceful job termination, and error handling across hundreds of parallel workers. The team overcame limitations in CloudWatch logging and the Spark connector by combining serverless execution with direct API calls and distributed rate limiting. As a result, the tool provides a scalable, script-like interface for managing large-scale table operations safely.

DEPTH ■■■■■14 min read12 days ago

Sign in to save and rate

18Stack Overflow · architecture

Part 1: Make your AI agents boring: the determinism layer

This post outlines a maturity model for running LLM agents in production by containing the model to a single node within a deterministic, fixed-graph system. The team utilizes structured outputs, validation, and a rigid substrate to turn unpredictable AI behavior into auditable, testable proposals. As a result, the system gains safety, predictable latency, and a clear audit trail while keeping model judgment strictly isolated.

DEPTH ■■■■■13 min read2 weeks ago

Machine Learning

Sign in to save and rate

19Cloudflare · scaling

Rewinding the exponential: How we keep hardware usage down

Cloudflare tackled exponential traffic growth against linear hardware growth by continuously evolving their data pipeline architecture over five years. Key optimizations included breaking up monolithic schemas, switching to zstd compression, decoupling pipeline transit from database replication, decentralizing sampling, and moving aggregations earlier in the pipeline. These strategies significantly reduced backbone bandwidth usage, network congestion, and storage requirements while preserving data quality.

DEPTH ■■■■■12 min read12 days ago

Infrastructure

Sign in to save and rate

20Amazon Web Services · architecture

Agentic antimicrobial resistance surveillance with AWS HealthOmics and Amazon Bedrock AgentCore

Processing public bacterial sequencing data for antimicrobial resistance surveillance typically requires specialized bioinformatics and SQL skills. To streamline this process, the team built a reference architecture using AWS HealthOmics, Amazon S3 Tables, and an Amazon Bedrock AgentCore conversational agent that ingests raw reads, normalizes findings, and answers plain-language questions with cited evidence. The resulting system automates the path from public accession numbers to natural-language epidemiological answers.

DEPTH ■■■■■10 min read13 days ago

Sign in to save and rate