Read what ships.

01Shopify · architecture

We gave AI agents a mobile dev environment

Engineering teams outgrew their local machines while running parallel AI mobile development agents, constrained by heavy builds and lack of iOS tools on Linux. To solve this, they built Remote Dev using warm pools of remote Macs connected via SSH alongside separate Linux VMs for Android virtualization, paired with agent skills and live browser streaming. This setup shifted workflow bottlenecks, increasing AI-merged pull requests from 6 percent to 61 percent.

DEPTH ■■■■■9 min read3 days ago

ArchitectureMobile

Sign in to save and rate

02ByteByteGo · migration

Why Shopify Migrated From React Native

Shopify migrated its mobile applications from React Native to native Swift and Kotlin because AI coding agents made the cost of duplicating features across platforms negligible. Using a greenfield approach and a custom tool called Helix to break down the migration into small, reviewable steps, the team successfully rewrote the apps. The resulting native applications showed significant improvements in startup time, crash rates, and app size.

DEPTH ■■■■■11 min read5 days ago

Sign in to save and rate

03Stack Overflow · architecture

MCP standardized tools. Now we need the infrastructure around them.

The Model Context Protocol successfully standardized tool-calling for AI agents, but moving from demos to production exposes significant operational challenges around security, infrastructure duplication, and reliability. Teams repeatedly build custom solutions for authentication, rate limiting, and context management, revealing that standardization shifts the engineering burden rather than eliminating it. Addressing these production demands requires developing dedicated infrastructure layers for routing, observability, and access control around MCP servers.

DEPTH ■■■■■8 min read10 days ago

Machine Learning

Sign in to save and rate

04Shopify · architecture

How Shopify turns open models into task experts

Shopify uses a continual learning flywheel to post-train open models, such as NVIDIA Nemotron, to serve as dedicated subagents for specific tasks like GraphQL generation. By optimizing inference and utilizing automated self-healing to generate training data from production failures, the team successfully created task experts that outperform frontier models while reducing serving costs by 90-98% and improving response times.

DEPTH ■■■■■5 min read4 days ago

Architecture

Sign in to save and rate

05ByteByteGo · architecture

How Uber Built a Genie to Answer Questions Correctly

Uber built an internal AI assistant called Genie to answer engineering support questions directly within Slack. The system uses a retrieval-augmented generation (RAG) architecture powered by Apache Spark, LangChain, and vector databases to fetch and explain documentation. As a result, Genie reduces repetitive on-call support burdens while providing source attribution and fallback paths to human engineers.

DEPTH ■■■■■13 min read6 days ago

Sign in to save and rate

06Shopify · architecture

Cross-store product clustering across millions of stores

Shopify needed to group identical product listings from millions of different merchants under a single global identifier. To solve this at scale without comparing every possible pair, the team implemented a multi-stage pipeline using vector retrieval, a tiered scoring system of ML models, and a bounded graph clustering approach. This architecture successfully standardizes product data across the entire platform while controlling computational costs.

DEPTH ■■■■■13 min read5 days ago

Architecture

Sign in to save and rate

07Stack Overflow · architecture

Part 5: Operating an LLM system: observability, cost, routing, and the platform underneath

This post provides a concrete specification for the operability layer of production LLM systems, covering observability signals, structured logging, cost controls, routing, and failover mechanisms. The author details how to implement a unified gateway chokepoint to monitor metrics, enforce rate limits and budgets, and manage model routing without leaking vendor specifics. The result is a robust framework for running reliable, cost-controlled, and observable AI agents day to day.

DEPTH ■■■■■19 min read2 weeks ago

Machine Learning

Sign in to save and rate

08Shopify · tooling

Building Overexposed: understanding application exposure

The team built an automated scanner called Overexposed to track internet-facing services and verify access controls across Kubernetes and Cloud Run. By querying live cloud inventories and probing endpoints like an outsider, the scanner classifies exposure while avoiding false assumptions from static config files. As a result, the engineering organization improved accuracy and significantly reduced inconclusive findings over time.

DEPTH ■■■■■8 min read6 days ago

Sign in to save and rate

09Dropbox · architecture

Evolving our calendar assistant Reclaim to be AI-native without starting over

Dropbox engineers evolved their calendar assistant, Reclaim, to support natural-language AI requests by building a custom agent platform that integrates with existing scheduling logic. They utilized a shared action system and a Preview Mode backed by Redis to ensure AI suggestions are consistent and reviewable before affecting real calendars. This approach allows the product to adopt AI capabilities without fragmenting its core architecture or duplicating functionality.

DEPTH ■■■■■7 min read3 weeks ago

InfrastructureMachine Learning

Sign in to save and rate

10ByteByteGo · architecture

LLM Prompt Caching Versus Semantic Caching

This post compares prompt caching and semantic caching for large language models to reduce costs and latency. Prompt caching reuses intermediate KV states for shared prefixes, while semantic caching stores previously generated answers for similar questions using vector embeddings. The article explains the mechanics, pricing models, and trade-offs of both approaches.

DEPTH ■■■■■16 min read4 days ago

Sign in to save and rate

11Stack Overflow · architecture

Part 4: Safety and governance for LLM systems: guardrails, PII, audit, and memory

This post details safety and governance practices for LLM systems, focusing on layered guardrails, boundary-level PII handling, and immutable audit trails. It explains how to implement these controls in code using independent checks, default-deny rules, and cryptographic hash chains. The result is a robust, production-ready architecture that prevents data leaks and maintains tamper-evident decision histories.

DEPTH ■■■■■19 min read2 weeks ago

Machine Learning

Sign in to save and rate

12Stack Overflow · architecture

Part 3: Knowing when your agent doesn’t know: the confidence layer

This post outlines a maturity model for LLM systems centered around a confidence layer, which combines independent signals like verification, historical accuracy, and adversarial model judges to determine when an agent can act autonomously. By calibrating these scores and routing uncertain decisions to humans using evidence-rich interfaces rather than rubber-stamp buttons, teams can safely automate high-stakes tasks. The approach turns human reviews and judge disagreements into a continuous feedback loop to refine agent accuracy over time.

DEPTH ■■■■■12 min read2 weeks ago

Machine Learning

Sign in to save and rate

13ByteByteGo · architecture

How Netflix Taught an LLM to Recommend Movies So That You Keep Watching

Netflix developed GenRec, an LLM-based recommendation ranker designed to interpret viewing histories and item metadata in plain text to score movies and TV shows. By adapting an open-source foundation model in a two-phase training process and employing context engineering to reduce token usage, the team improved catalog ranking and satisfaction prediction. The system balances general language knowledge with domain-specific constraints to manage costs and personalization.

DEPTH ■■■■■15 min read2 weeks ago

Sign in to save and rate

14Stack Overflow · architecture

Part 2: Evals as a deployment gate — and how to know when they drift

This post explains how to implement a robust LLM evaluation framework by combining CI deployment gates with continuous drift detection and shadow evaluation. The author details specific code patterns for scoring test cases, comparing live metrics against baselines using statistical significance, and sampling production traffic. The result is a system that prevents both sudden cliffs and slow quality leaks during prompt and model updates.

DEPTH ■■■■■8 min read2 weeks ago

Machine Learning

Sign in to save and rate

15ByteByteGo · architecture

Why LLMs Agree With You Even When You’re Wrong

Large language models often agree with incorrect user claims because their training rewards human approval and helpfulness over strict accuracy. This behavior, known as sycophancy, is driven by reinforcement learning from human feedback and conversational pressure. Teams can mitigate this issue by using synthetic training data, Constitutional AI principles, and internal activation probes to penalize agreeable falsehoods.

DEPTH ■■■■■13 min read2 weeks ago

Sign in to save and rate

16Shopify · architecture

ShopGym: Realistic, reproducible sandboxes for shopping agents

Training and evaluating shopping agents is difficult due to changing live store environments, bot detection, and limited hand-built test stores. The team developed ShopGym, a framework that converts live storefronts into self-contained, anonymized sandbox shops and generates benchmark tasks. Evaluation showed that synthetic shops preserve key structural properties and behavioral alignment with live stores, giving agents a reliable environment to practice and improve.

DEPTH ■■■■■6 min read3 weeks ago

Sign in to save and rate

17Stack Overflow · architecture

Part 1: Make your AI agents boring: the determinism layer

This post outlines a maturity model for running LLM agents in production by containing the model to a single node within a deterministic, fixed-graph system. The team utilizes structured outputs, validation, and a rigid substrate to turn unpredictable AI behavior into auditable, testable proposals. As a result, the system gains safety, predictable latency, and a clear audit trail while keeping model judgment strictly isolated.

DEPTH ■■■■■13 min read2 weeks ago

Machine Learning

Sign in to save and rate

18Stack Overflow · architecture

Part 6: An operating system for coding agents: the disciplined build

Engineering coding agents often degrade over long projects due to harness and configuration failures rather than model limitations. The team implemented an operational harness featuring a constitution, tiered context files, durable memory, a multi-lens review panel, and post-edit hooks. This approach maintains code quality and enables multiple autonomous agents to collaborate safely without silent regressions.

DEPTH ■■■■■12 min read2 weeks ago

Machine Learning

Sign in to save and rate

19ByteByteGo · architecture

How to Build Models Without Language, with Logical Intelligence

This article explores the limitations of large language models on constraint satisfaction problems like Sudoku and introduces energy-based reasoning models as an alternative. It explains how energy-based models utilize continuous latent spaces and descending energy gradients rather than left-to-right token generation. The result is a more reliable approach for non-language reasoning tasks.

DEPTH ■■■■■19 min read13 days ago

Sign in to save and rate

20Shopify · tooling

Helix: The internal tool powering our Shopify app's native migration

Shopify is migrating its mobile apps from React Native to native Swift and Kotlin using an internal LLM-powered tool called Helix. Helix breaks migrations down into small checkpoints and enforces strict quality gates covering behavior, UI equivalence, architectural compliance, and human review. This iterative loop allows the team to reliably generate and migrate hundreds of screens to native code.

DEPTH ■■■■■8 min read4 weeks ago

Sign in to save and rate