Reading Log

Started Designing ML Systems: mapped the book’s structure (four recurring questions, five components, five scenarios) and drafted the first post of a reading series. Got stuck on Figure 1-4 until it clicked: with batching, throughput = batch size ÷ latency, so higher latency and higher throughput can coexist — latency becomes a tradeoff you tune, not the inverse of throughput.

Open question: in a trading system, p99 latency describes ~1% of messages — but those cluster at moments of market stress, so how should jitter be specified and tested?

The DiT big picture finally clicked: inside a DiT there is no “image” versus “condition” — there is only the token sequence. Every form of control decomposes into three actions: add tokens, assign them coordinates, and decide who is allowed to look at whom. Claude’s refinement, which I’m keeping: a fourth action — global conditions like the timestep never enter as tokens at all, they act through adaLN modulation — plus two axes hiding inside “add tokens”: which weights process the new tokens (shared vs. per-modality), and when they exist (condition tokens are constant across denoising steps, so they can be cached).

SIGMA-GEN reads differently through this frame: single-pass multi-subject identity preservation, with structure guidance anywhere from coarse boxes to pixel-level depth, in one model. Next week’s project: map it and its neighbors onto these axes as one table.

Open question: where does SIGMA-GEN place each subject’s identity tokens in coordinate space so that multiple identities don’t blend?

Expanded into a full skeleton post: DiT Control Is a Few Decisions About Tokens.

Launched this site. I started reading SIGMA-GEN but got stuck partway in because my DiT fundamentals were shakier than I thought, so I spent the evening working through DiT basics instead.

Open question: in a DiT block, why does adaLN conditioning work so much better than simply concatenating the conditioning tokens?