Newsletter #14
Enjoy Newsletter #14.
Bloom Filter: A Deep Dive
Kirupa Chinnathambi explains Bloom filters through a classic real-world problem: checking whether a username is already taken during sign-up on a social network with tens of millions of users. Scanning the database on every keystroke is too slow, while keeping every username string in an in-memory hash table eats up too much RAM. A Bloom filter sits in front of the database as a compact probabilistic gatekeeper: instead of storing the strings themselves, it stores bit “fingerprints” inside a fixed-size bit array. Each inserted item is run through $k$ independent hash functions, each mapping to an index in the array whose bit is flipped to 1. On lookup, if even a single one of those $k$ bits is 0, the item is guaranteed not to exist; if all $k$ bits are 1, the item might exist and you fall back to a database query, because other items may have happened to set those same bits (a false positive).
The second half walks through the formulas for picking the optimal bit-array size $m$ and number of hash functions $k$ given an expected item count $n$ and target false-positive rate $p$. For 10 million usernames at a 1% false-positive rate ($p = 0.01$), a Bloom filter needs only about 95.85 million bits (~12 MB) and 7 hash functions—whereas an equivalent in-memory hash table would consume 160 MB to 240 MB, or 13x to 20x more memory. Trading a tiny, tunable chance of an extra database lookup for massive memory savings makes the Bloom filter one of the most practical probabilistic data structures in systems engineering.
Rethinking LLM inference: Why developer AI needs a different approach
Augment Code explains why AI coding assistants demand a fundamentally different LLM inference architecture than consumer chatbots: repository-aware coding requires feeding tens of thousands of tokens of codebase context into the model, and suggestion quality keeps climbing well past 10,000 input tokens. Unlike chat workloads (roughly ~100 input tokens to generate hundreds of output tokens), developer AI workloads are heavily prefill-dominated—thousands of context tokens in, only a few dozen completion tokens out—whereas off-the-shelf engines like vLLM and TensorRT-LLM are tuned primarily for decode-heavy workloads. Because of this, Augment optimizes ruthlessly for prefill throughput and time-to-first-token (TTFT). After showing why single-token autoregressive decoding leaves GPUs below 1% compute utilization (memory-bandwidth bound), the post explains token-level batching (known in academia as chunked prefill), where lightweight decode steps “piggyback” onto another request’s compute-bound prefill batch, sized right at the ridge point between memory-bandwidth bound and FLOPS bound.
The article also details production systems requirements like reliable request cancellation as a developer types rapidly, alongside low-level GPU optimizations: CUDA Graphs to eliminate CPU launch overhead, FP8 quantization, FlashAttention-3, and custom fused CUDA kernels. Together, these optimizations deliver a time-to-first-token under 300ms for a 10,000-token prompt on Llama 3 70B—a 3x speedup—while sustaining nearly 10 requests per second across an 8x H100 node. It’s a great case study in why LLM inference tuning has to match the exact input/output shape of your application.
Floyd’s Cycle Algorithm: Fraud Detection in Java Systems
This DZone article illustrates how Floyd’s Cycle-Finding Algorithm (the “tortoise and the hare”) can be applied to spot circular money transfers in banking systems. Bank accounts are modeled as vertices and wire transfers as directed edges in a graph; a loop like A → B → C → D → B forms a cycle, a classic pattern in money laundering (“round-tripping”) used to obscure the origin of funds. Floyd’s algorithm advances two pointers through the sequence—a slow pointer moving one step at a time and a fast pointer moving two steps at a time; if a cycle exists, the two pointers are guaranteed to meet inside the loop. The Java walkthrough provides a TransactionGraph backed by a HashMap adjacency list, a FraudDetection cycle checker, and a test harness with sample transfer data.
The appeal of Floyd’s algorithm is its simplicity, $O(N)$ time complexity, and $O(1)$ auxiliary memory footprint, making it attractive for lightweight real-time checks. However, an important caveat applies: Floyd’s tortoise-and-hare algorithm assumes each node has at most one outgoing edge (a functional graph or linked list). In a general transaction graph where an account sends money to multiple recipients, following only a single outgoing edge per node can miss cycles on other branches—meaning full Depth-First Search (DFS) or Johnson’s/Tarjan’s strongly connected components algorithms are needed for general multi-branch graphs, and any detected cycle is merely a heuristic flag for deeper investigation rather than proof of fraud on its own. The post also notes applications in network routing loop detection, DNA motif analysis, ML pipelines, and blockchain transaction graphs.
Succinct data structures
Martijn Faassen introduces succinct data structures—a family of data structures he was delighted to discover. Like compression algorithms, they store data in space close to the information-theoretic minimum; unlike standard compression, however, you don’t need to decompress them before use—you can run fast queries directly on the compact representation. The foundational building block is a bitvector supporting two constant-time ($O(1)$) operations with tiny memory overhead: rank (count how many 1 bits appear up to index $i$) and select (find the index of the $n$-th 1 bit). From that primitive, a Wavelet Matrix generalizes rank and select to larger alphabets like DNA bases or 256-byte text; an FM-index enables compact full-text search that counts and locates substrings rapidly (widely used in bioinformatics); and balanced parentheses bitvectors encode arbitrary trees using just 2 bits per node instead of bulky 64-bit pointers while still supporting fast navigation to parent, child, and sibling nodes.
Faassen walks through an XML querying prototype he built by combining a balanced-parentheses tree with sparse bitvectors to jump straight to elements matching a specific tag name, and sketches how the same trick could shrink compiler ASTs in memory. He closes with pointers to Rust crates in this space, including vers (blazing fast with minimal overhead), sucds (offering sparse bitvectors like SArray), and fm-index. Though still relatively niche outside bioinformatics, succinct data structures hold huge promise anywhere fitting a working set into CPU cache or RAM is the key to speed.
The art of engineering team focus: less is more
This GitHub Resources piece makes a counterintuitive case: if you want your engineering team to ship more, they need to work on fewer things at the same time. Unshipped work delivers zero value to users, yet leaders often struggle to say “no” to stakeholders, slicing the team across half a dozen parallel initiatives under the illusion that everything is moving forward. The antidote starts with making all work visible so leadership confronts the team’s true capacity and actively kills or pauses lower-priority tracks; breaking projects into 1–2 week milestones to keep momentum high, shrink pull requests, de-risk deployments, and create natural pivot points; and strictly limiting Work in Progress (WIP), since every context switch can cost up to 23 minutes to regain deep focus while driving up bug rates and burnout.
The author goes further and recommends “stacking” engineers onto the single highest-priority initiative until adding another person would literally cause them to trip over one another, and only then staffing the #2 priority. Finally, never plan a team to 100% utilization—leave roughly a 20% capacity buffer to absorb unexpected bugs and production interrupts, because a highway at 100% capacity is a parking lot. Teams with slack in the system consistently ship faster, innovate more, and burn out less.
Performance optimization, and how to do it wrong
The author, a contributor to the Rust burn machine learning framework, shares a humbling war story about optimizing direct 2D CPU convolution using SIMD intrinsics. Even after implementing an academic paper’s optimal loop ordering and register blocking scheme, the new SIMD kernel ran more than twice as slow as the naive baseline. Standard profilers like cargo-flamegraph, samply, and AMD μProf couldn’t pinpoint why, forcing the author to strip the kernel down piece by piece to isolate the regression. The culprits turned out to be a combination of SIMD register spilling (x86 AVX2 provides only 16 YMM registers) and excessive branching inside the hot inner loop to handle boundary padding. Splitting the loop into a branchless fast path for interior pixels and a separate edge path—plus using compile-time monomorphization (const generics) to specialize common cases like zero padding or stride 1—brought massive speedups.
The biggest plot twist came from the Rust compiler itself: adding one more specialized branch pushed the outer function past LLVM’s inlining size threshold, causing the compiler to stop inlining the inner SIMD routines and tanking performance all over again. Adding #[inline(always)] to the top-level functions immediately restored the speedup. It’s a classic lesson in low-level optimization: measure everything, understand your hardware’s register file, and never assume the compiler is doing what you think it’s doing.
Supercharging Discord Mobile: Our Journey to a Faster App
Discord shares a sweeping set of mobile performance improvements for their React Native app, aimed especially at power users in hundreds of servers. First, they virtualized the server list so only on-screen server icons stay mounted in memory, cutting startup memory usage by 14% and cold-start time by 10% for users in 100+ servers. For the chat message list (already a native component), they overhauled view recycling, lazily initialized rarely used message attachments, and pre-populated the recycle pool at startup with common row types—reducing slow frames by up to 60% and trimming memory by ~12%. On Android, they rewrote the emoji picker completely in native Kotlin views to eliminate blank white flashes during fast fling scrolls on budget devices, and switched animated emojis from GIF to WebP to hit a smooth 60 FPS.
For the channel list, the team actually migrated back from Shopify’s FlashList to their own in-house FastList after finding that FlashList frequently rendered blank cells on low-end Android phones, and went on to build FastestList backed by Android’s native RecyclerView. On iOS, the media picker now displays a fast low-resolution thumbnail first while streaming in the high-res asset and requests 256×256 previews to hit Apple PhotoKit’s fast system cache, eliminating a 4.5-second freeze when opening large camera rolls. Looking ahead to 2025, Discord plans to enable React Native’s New Architecture, experiment with Static Hermes, and move core client state logic to Rust.
Git without a forge
Simon Tatham, creator of PuTTY, explains why he still hosts his open-source projects on plain bare Git repositories on his own server rather than moving to a web forge like GitHub or GitLab. He trusts machines administered by friends more than corporate platforms whose ownership and policies can shift overnight; he has no desire to sysadmin a heavy self-hosted forge instance himself; he dislikes how forges bake in rigid opinions about issue tracking and pull-request workflows; he doesn’t want to force drive-by contributors to register yet another web account just to send a patch; and he worries about platform lock-in trapping years of issue and PR discussions outside the Git repository itself. He also candidly admits that sheer inertia plays a part.
In the second half, Tatham ranks ways contributors can send him patches without a forge, from best to worst: sending a public Git repo URL and branch name he can git fetch directly (giving all the benefits of a pull request with zero forge lock-in), emailing a git bundle binary file containing the exact commits, attaching files generated by git format-patch, sending a raw git diff (which loses commit messages and base-commit metadata), and dead last, git send-email, which turns a patch series into a burst of separate emails he has to reassemble by hand. While he acknowledges that public forges help newcomers gauge how welcoming a project is from past PR threads, he remains holding out for a lightweight, decentralized code-review protocol that keeps discussion portable.
40 Thoughts On Turning 40
Paul Millerd, author of The Pathless Path, shares 40 reflections on turning 40, spanning reinvention, work, money, relationships, fatherhood, and happiness. Looking back, he considers walking away from a conventional high-status consulting career into uncertainty the defining decision of his 30s, noting that real personal transformation is slow, messy, and gradual rather than a single cinematic epiphany. On work, he argues that the FIRE movement’s dream of “escaping work” through early retirement misses the point: you don’t transcend work by avoiding it, you transcend it by finding good work—work that feels meaningful both in the moment and in hindsight. Everyone should try freelancing or self-employment for at least a year, he suggests, adding that taking a steep pay cut after quitting his corporate job was the fastest way to discover what he actually valued.
On life outside work, he reflects on how lucky he feels to have met his wife and encourages anyone still looking for a partner not to give up, while calling fatherhood the most humbling experience of his life. His closing thoughts are pithy reminders: following breaking news has never made anyone happier, deep rest is a skill that takes practice, “doing less” is the most underrated way to get unstuck, and writing in public remains one of the most powerful forces for personal clarity—even if it forces you to confront what you truly want.
Bonus: Interesting images from ByteByteGo

Bonus 2: Interesting videos from ByteByteGo
What Is the Most Popular Open-Source AI Stack?
This post was rewritten by Claude Code with Opus 5.5 on 27/09/2026; reviewed and edited by Codex with GPT 6.1 Sol on 09/11/2026, and re-verified by Claude Code with Sonnet 5.5 on 16/11/2026.
Comments