Newsletter #69
Enjoy Newsletter #69. This post was created using Claude Code, Claude Code Router, iFlow Open Platform & GLM-4.6
How and Why Netflix Built a Real-Time Distributed Graph: Part 1 — Ingesting and Processing Data Streams at Internet Scale
As Netflix expanded beyond on-demand video streaming into ad-supported tiers, live events, and mobile games, understanding a member’s journey across devices and business lines grew increasingly difficult. A microservices architecture with hundreds of services—each managing its own data store—fragmented the data and forced analytics teams to stitch records together manually. In response, Netflix’s data engineering team built the Real-Time Distributed Graph (RDG): modeling domain entities as a graph so relationships can be queried via multi-hop traversals across nodes and edges rather than expensive relational JOINs, making it easy to onboard new entity types and spot behavioral patterns or anomalies.
Part 1 focuses on the ingestion and stream-processing pipeline. User interactions flow through the API Gateway into Kafka topics (some handling up to ~1 million messages per second, encoded in Avro and simultaneously archived to Apache Iceberg tables for historical backfills). Apache Flink jobs filter out noise, enrich events with metadata, transform raw events into graph nodes and edges, and batch and deduplicate updates within a short time window before writing over 5 million records per second into Netflix’s Data Mesh. A key operational lesson: running a single monolithic Flink job across all topics proved nearly impossible to tune, so the team moved to one Flink job per Kafka topic—accepting higher operational overhead in exchange for isolation and independent tuning.
Start Small, Scale Smart: The Real Value of Incremental Architecture
Incremental architecture is the practice of designing systems so they can evolve gracefully, grounded in Gall’s Law: if you start with a complex system from scratch, you will end up with a complex system that doesn’t work. In the author’s view, team organization matters more than technology: cross-functional teams that own a business domain end-to-end outperform layer-siloed teams, and if you want microservices, you must first build independent teams. Architects should act as mentors who still write code and steward system coherence rather than handing down mandates from an ivory tower, spreading knowledge through pair and mob programming. Instead of asking “how long will this take?”, ask “how can we make this smaller?”
On the technical side, the post advises applying architectural patterns only when you actually feel the pain they solve, using the Strangler Fig pattern to incrementally extract cleaner modules out of legacy monoliths, and building small, single-responsibility, replaceable components with firm boundaries—which naturally become candidates for future microservices. Event-driven architecture reduces coupling by letting components emit events instead of relying on central orchestration, while Domain-Driven Design and a shared Ubiquitous Language clarify events, consequences, and actors before anyone writes a line of code.
Java Interview Question - Why Collection doesn’t extend Cloneable and Serializable interfaces?
This interview question explores a classic Java design decision: why the root Collection interface does not extend Cloneable or Serializable. The core reason is that not every collection can—or should—be cloned or serialized, so forcing both interfaces at the top of the hierarchy would impose unrealistic constraints on specialized implementations. For example, Collections.unmodifiableList() is just a read-only view where deep cloning would break its semantics, WeakHashMap relies on JVM-managed weak references that don’t make sense to serialize, and a TreeMap configured with a non-serializable Comparator cannot be serialized either. (Note that ConcurrentSkipListMap actually does implement both Cloneable and Serializable, so it isn’t an example of this restriction.)
Furthermore, both Cloneable and Serializable are flawed marker interfaces from Java 1.0: Cloneable declares no methods of its own and encourages error-prone shallow copies, while Serializable creates versioning headaches and security vulnerabilities. When the Collections Framework was introduced in Java 1.2, its designers left it up to concrete classes like ArrayList or HashMap to opt in where appropriate, keeping the core abstraction clean. In modern Java, it’s better to copy collections via copy constructors like new ArrayList<>(list) or List.copyOf(), and use explicit formats like JSON instead of default Java serialization.
We stopped roadmap work for a week and fixed 189 bugs
The author shares the story of a quarterly “Fixit Week,” where an organization of roughly 45 engineers paused all roadmap work for a full week. Across two time zones, 40 participating engineers squashed 189 bugs—averaging more than 4 bugs per person, with the top contributor fixing 12. The ground rules were strict: no single bug could take more than 2 days, work had to focus on small end-user papercuts or developer-productivity improvements, and all regular meetings, design reviews, and roadmap tasks were canceled. The event was gamified with point values scaled by task size, a live leaderboard, daily recaps, and custom t-shirts for awards like “first bug fixed” and “most annoying bug squashed.”
Standout wins included closing a four-year-old feature request in a single day and shipping a 25-line GitHub Action that saved frontend engineers repetitive daily toil. Key ingredients for success were prepping a triaged, t-shirt-sized backlog ahead of time, reaching critical mass so the whole team felt the energy, keeping Fixit points strictly separate from performance reviews, and leaning on AI coding tools to lower the context-switching cost of jumping into unfamiliar directories. As the author concludes, Fixit Weeks restore pride in craftsmanship, the joy of shipping immediate improvements, and team camaraderie.
The Math of Why You Can’t Focus at Work
This post models knowledge-worker productivity using three parameters: λ, the rate of interruptions per hour (modeled as a Poisson process); Δ, the number of minutes needed to regain focus after each interruption; and θ, the minimum contiguous block of focused time required to make meaningful progress (typically 30–60 minutes for complex engineering work). Effective work capacity is the number of θ-sized blocks that fit inside the uninterrupted gaps—meaning that the same total number of hours, when chopped into fragments, yields drastically less output than contiguous blocks. Cited workplace studies suggest knowledge workers are interrupted every 2–3 minutes on average and take 10–16 minutes to refocus.
Plug those typical workplace numbers into the math and deep work becomes nearly impossible—yet small shifts in any of the three parameters produce massive gains. The author points to three practical levers: lower λ by setting communication boundaries and defending calendar blocks, shrink θ to fit your environment by decomposing tasks into smaller self-contained steps, and reduce Δ through better context management, such as leaving yourself breadcrumb notes so you can resume work quickly after an interruption.
Tech predictions for 2026 and beyond
Werner Vogels outlines five technology predictions for 2026 and beyond. First, physical AI will redefine companionship for vulnerable populations: loneliness affects one in six people worldwide and raises mortality risk by 32%, whereas clinical studies with the Paro robotic seal showed positive engagement in 95% of dementia patients. Second is the rise of the “renaissance developer”: generative AI will elevate rather than replace developers—just as compilers and cloud computing did before—because humans still own business constraints, customer empathy, and systems thinking. Third, quantum-safe security becomes urgent as the estimated qubit count needed to break 2048-bit RSA has dropped below one million, and Vogels predicts quantum computers capable of breaking RSA and elliptic-curve cryptography could arrive within roughly five years.
Fourth, modern defense tech is being built as dual-use from day one, crossing over into civilian applications far faster than the 10–20-year lag of previous decades, with companies like Anduril reaching $1 billion in revenue in 2024. Finally, AI-powered personalized education will go mainstream: AI tutors cost around $4 a month, Khanmigo reached 1.4 million students in its first year, and teachers using AI tools save roughly 5.9 hours per week to reinvest in direct human teaching.
Why I (still) love Linux
Stefano Marinelli reflects on nearly 30 years with Linux, starting in 1996 after his DOS and Commodore 64 days, when Linux gave him his first real taste of Unix and a sense of freedom no prior OS had offered. By 1998 he was actively contributing to the community as a speaker and translator. Even though the fabled “Year of the Linux Desktop” never quite arrived, Linux conquered mobile via Android, servers, and embedded devices, while Ubuntu helped bring Linux to everyday PCs and drew major enterprises into open source.
Marinelli doesn’t hold back on criticisms either: many modern distributions have drifted away from the Unix philosophy of “do one thing and do it well,” most notably with systemd; novelty for its own sake often sacrifices stability; corporate interests sometimes steer development toward their own agendas; and software quality has suffered from “vibe coding” and fragile dependency chains. Although he prefers BSD and illumos for many server workloads today, he still appreciates distributions like Alpine and openSUSE, notes how much better Linux hardware support has become, remains grateful to Linux for launching his career and lifelong learning, and expects to keep using it for another 30 years.
Why (Senior) Engineers Struggle to Build AI Agents
Phil Schmid explains why senior engineers often struggle more than juniors when building AI agents: traditional software engineering is deterministic, whereas LLM-backed agents are inherently probabilistic. Junior engineers tend to trust the model and ship quickly, while seasoned engineers instinctively try to “code away” the non-determinism. Schmid highlights five mindset shifts. First, natural language is the new state: a nuance like “the plan looks good, but focus on the US market” should be kept as rich text rather than squeezed into a rigid boolean or enum schema. Second, hand over control flow: let the model navigate unpredictable multi-turn conversations—like a user asking to cancel a subscription and changing their mind after hearing a retention offer—instead of hardcoding every branch.
Third, errors are just another input: catch tool failures and feed the error message back into the agent’s context so it can self-correct, rather than throwing an unhandled exception. Fourth, move from unit tests to evals: alongside deterministic tests, measure end-to-end reliability, use LLM-as-a-judge for output quality, and inspect intermediate reasoning traces. Fifth, agents evolve while APIs stay fixed: because agents interpret tool schemas literally, APIs need clear docstrings and explicit parameter names like user_email_address instead of email to prevent hallucinations. Ultimately, engineers must trade a degree of rigid certainty for semantic flexibility.
How Java Achieves Zero-Copy File Transfer
Zero-copy is the technique of transferring file data from disk to a network socket without copying bytes through JVM user-space memory. Traditional file serving reads data from disk into the OS kernel page cache, copies it into a user-space JVM buffer, and copies it right back into a kernel socket buffer before sending it to the NIC; zero-copy lets the kernel move data directly between kernel buffers (from page cache to socket buffer and out via DMA to the network card), cutting CPU usage, context switches, and JVM garbage-collection pressure from temporary byte[] allocations.
Java exposes three main mechanisms here. FileChannel.transferTo() delegates to the OS’s sendfile syscall when the platform and target channel support it (transferFrom() works in the opposite direction without the same universal syscall guarantee). MappedByteBuffer maps file pages directly into the process’s virtual address space with read-only, read-write, or copy-on-write modes, avoiding user-space read copies though not automatically creating a zero-copy path to a network socket. For high-performance network servers, Netty’s DefaultFileRegion wraps file channels to exploit native transport zero-copy. The technique does have limits: user-space TLS encryption usually breaks the direct kernel path because bytes must be encrypted before transmission, tiny files rarely justify the syscall setup overhead, and any in-flight data transformation like compression requires touching the bytes in memory anyway.
Bonus
Images:
This post was rewritten by Claude Code with Opus 5.5 on 27/09/2026; reviewed and edited by Codex with GPT 6.1 Sol on 09/11/2026, and re-verified by Claude Code with Sonnet 5.5 on 16/11/2026.
Comments