Enjoy Newsletter #113.

Migrating from Go to Rust

Matthias Endler writes a comprehensive guide for backend teams considering migrating services from Go to Rust. The central message isn’t “Rust is always faster than Go,” but that Rust pushes many classes of risk into the type system so the compiler catches them before production: nil values, error handling, data races, resource lifetimes, and ownership. Go optimizes for team velocity—fast compile times, a strong standard library, and easy-to-use goroutines. Rust trades that for a stricter compiler, no GC, Result, Option, traits, and Send/Sync checks. Familiar Go habits have to be remapped: if err != nil becomes Result with ?, nil becomes Option, interfaces become traits, and goroutines become tasks when genuinely needed.

The most practical section covers migration strategy: don’t rewrite everything at once. Start with a hot path, worker, or clearly bounded service, keep the existing API contract, and shift traffic gradually endpoint by endpoint through a gateway using the strangler pattern. While cgo works, it is often more complex than splitting the Rust portion into a separate service. The author is also candid about the costs: the borrow checker makes the first month rough, compile times are slower, async function coloring is annoying, and parts of the ecosystem are smaller. There is no need to abandon Go entirely: Go remains a great fit for Kubernetes tooling, CLIs, and “boring” glue services, while Rust is worth the investment for foundational services where reliability, P99 latency, and bugs like nil dereferences or data races carry higher operational costs than the learning and build overhead.

AI Engineering for Developers

Luca Cavallin writes a deep-dive guide to AI engineering for developers already comfortable with backends, HTTP, queues, Kubernetes, and production operations. His main argument: when bringing LLMs into a product, the model is not the whole product; what actually creates value is the system around it—prompts, RAG, evals, agents, inference, observability, security, and cost controls. That makes the discipline much closer to backend engineering than model training. Because AI outputs are non-deterministic and can shift across models, prompts, or data, a working demo isn’t enough to ship: you need eval datasets, regression checks on every prompt or model change, canary rollouts, and monitoring from day one.

The article progresses from fundamentals to practical execution: selecting models by task, cost, and latency; exhausting prompt engineering before reaching for RAG; treating RAG primarily as a search problem where hybrid search, rerankers, chunking strategy, and source citations matter more than stuffing more context; fine-tuning only after prompting and RAG hit their ceiling; and optimizing inference through caching, batching, and model routing. Agents are framed as tool-calling loops that require strict schemas, narrow permissions, end-to-end tracing, and human approval gates for risky actions. The production section emphasizes guardrails, dedicated CI/CD pipelines for prompt and model changes, per-feature cost attribution, and treating every agent as a service with its own identity, least-privilege access, and audit logs.

The invisible engineering behind Lambda’s network

This post on All Things Distributed recounts nearly a decade of networking optimizations behind AWS Lambda—the kind of “invisible” engineering customers only notice because functions start faster and more reliably. The original bottleneck was cold starts when Lambda connected to a customer VPC: beyond spinning up a Firecracker microVM and initializing the runtime, the system had to establish a private network path into the customer’s VPC, with Geneve tunnel creation sitting right on the hot path. Rather than maintaining custom kernel patches, the Lambda team turned to eBPF: pre-creating tunnels with a dummy VNI and overwriting the packet header once the real VNI arrived, cutting tunnel setup latency from ~150ms to 200 microseconds.

Even more instructive is how they scaled to 4,000 networks per worker. Instead of creating tap devices, veth pairs, namespaces, and routing rules on every function invocation, they pre-allocated everything at worker startup, turning a load-variable cost into a one-time fixed cost. They replaced stateful NAT with stateless eBPF packet rewriting, cutting NAT setup latency 100x; collapsed over 125,000 iptables rules in the root namespace down to 144 static rules by moving per-slot rules into their respective namespaces; and reduced RTNL lock contention by reordering network creation and attaching eBPF programs in batches. At scale, iptables rules, conntrack, or a single kernel lock can all become bottlenecks. The result is a unified topology serving both traditional workloads and SnapStart, boosting snapshot network throughput 20x and getting repackaged for reuse in Aurora DSQL.

Selective Test Execution at Stripe: Fast CI for a 50M-line Ruby monorepo

Aditya Anchuri describes how Stripe keeps CI fast for a ~50-million-line Ruby monorepo containing nearly 100,000 test files and 1.2 million test targets. Running the entire suite sequentially would take roughly four months, while Stripe runs around 50,000 builds per week. Their Selective Test Execution (STE) system solves this by running an average of just ~5% of the test suite per build (with a median under 0.5%), consuming less than 10% of the compute required by a run-everything strategy.

What makes Stripe’s approach clever is that they don’t attempt static dependency analysis for Ruby, where metaprogramming, dynamic dispatch, configuration files, and code generation make static analysis either miss dependencies or become overly conservative. Instead, they observe runtime behavior: an internal C++ library injected via LD_PRELOAD intercepts every file open, associates read files with the currently executing test, and propagates tracking across child processes. The open path writes only minimal logs, keeping aggregation and indexing off the hot path. Those logs are compiled into a roaring bitmap index mapping changed files to required tests, storing roughly three billion data points while remaining fast to query. File changes are detected using hashdeep so generated files are covered alongside git-tracked files. STE also relies on reproducible baseline data and practical CI safety nets: always re-running currently failing tests, always running tests that glob the filesystem, handling linters via changed-file lists, and storing metadata in MongoDB with Monotonic Revision IDs to quickly select baselines in commit order.

Being oncall taught me everything

Yao Yue looks back on 7.5 years of being on call for Twitter’s distributed caching layer—the highest-throughput service and the one involved in the most severe site-wide outages. The post doesn’t romanticize on-call life, but argues that living alongside production is what shaped the author’s mindset as an infrastructure engineer. On-call teaches you that at scale, predictability matters more than being fast in the average case, tail latency matters more than the median, simple and clear architecture is a lifeline during a crisis, and operational excellence (deep observability, consistent configuration, automation readiness, sensible defaults) comes from up-front design rather than frantic patching two weeks before launch.

Even more memorable are the human lessons. Periodically restarting Memcached to prevent site-wide cascading failures sometimes triggered incidents of its own, teaching the author to pick low-risk windows, give advance notice, and—most importantly—own mistakes immediately, since that is the fastest way to earn back teammates’ trust. Many outages were resolved only through doggedly narrowing down possibilities and leaning on colleagues across every layer of the stack: site ops, kernel engineers, upstream service owners, managers, and fellow on-call engineers. The conclusion is blunt: engineers don’t truly understand software until they see it run and break in production, and people who write software shouldn’t consider themselves above deploying, monitoring, or debugging what they build.

Claude as your performance analysis partner

Archana Ravindar experiments with using Claude as a partner for Go performance analysis across CPU profiles and execution traces, focusing on the Green Tea garbage collector on POWER10 using the sweet benchmark suite. Claude’s value doesn’t lie in immediately generating a working optimization patch, but in rapidly digesting massive, dense artifacts like pprof outputs, assembly dumps, and trace timelines. For example, Claude pinpointed a hot path in runtime.tryDeferToSpanScan, spotted an atomic Load8 followed by Or8 pattern, and suggested fusing them into a single Or32 instruction. That experiment actually ran slower, and Claude then helped explain why: false sharing and contention across GC workers—a clear illustration that fewer atomic instructions doesn’t automatically mean faster code.

Claude also proved useful for low-level analysis: cross-referencing assembly to see that the compiler wasn’t caching q.class.sizeclass, ranking GC TODO comments by profile heat, and suggesting patterns to look for in traces such as idle processors during GC, mark assist gaps, overly frequent GC cycles, or prolonged gcMarkDone phases. The author draws a clear distinction: profiles tell you where cost lives, while traces explain why the system is stalling, especially around GC and scheduling. Still, every hypothesis must be benchmarked, because optimizations tied to hardware architecture, cache lines, instruction sets, or register pressure are easy to get wrong without full context; the best use of AI here is narrowing the search space and building an investigation checklist, not replacing benchmarks.

Things you didn’t know about indexes

Jon Charter explains Postgres indexes with an approachable analogy: an index is like the index at the back of a book, letting the database locate rows through a sorted structure instead of scanning the entire table. That speed comes with a fundamental trade-off: faster reads mean slower writes. Data modifications (INSERT, UPDATE, DELETE) usually have to update indexes as well and incur cleanup overhead; indexes also consume disk space, compete for cache memory, and give the query planner more paths to evaluate. Consequently, “index everything” is rarely a good strategy.

The most useful section covers common pitfalls that prevent an index from being used. Composite indexes depend on column order, so the order must match real query patterns: (type_1, type_2) helps queries filtering on type_1 or both columns, but is generally ineffective when filtering only on type_2 (though PostgreSQL 18’s skip scan helps in some cases). Wrapping a column in a function like lower(name) causes a plain index on name to be ignored because the database needs a structure sorted by the exact expression lower(name); implicit type casts cause the same issue. The right way to verify index usage is running EXPLAIN or EXPLAIN ANALYZE to confirm what the planner actually chooses rather than guessing. Finally, the post introduces functional indexes, partial indexes for small slices of data like soft-deleted rows or is_legendary = true, and covering indexes with INCLUDE to enable Index Only Scan.


This post was rewritten by Claude Code with Opus 5.5 on 27/09/2026; reviewed and edited by Codex with GPT 6.1 Sol on 09/11/2026, and re-verified by Claude Code with Sonnet 5.5 on 16/11/2026.