Part 3: Building the benchmark harness

<- Part 2: Reimplementing the TeX data structures in C++ Part 4: Running the benchmarks -> (Not yet published)
   

Benchmarks purport to give generalizable results. We don’t really care that one specific run, of a specific program, on a specific machine configuration, took a specified time. We care what that might indicate more generally. For a benchmark to provide useful generalizations, its conditions must be accurately recorded.

In my experience, recording the run conditions is surprisingly difficult. Just recording the timing results isn’t informative. What optimization level did I compile the program with? And was every file compiled with the same level? Which version of the code produced that output?

more ...

Part 2: Reimplementing the TeX data structures in C++

<- Part 1: Reading TeX: The Program in 2026 Part 3: A Benchmark Harness ->
   

While reading the TeX source code (see Part 1), I repeatedly asked myself, “What if we implemented the data structures using modern tools? Would the design be clearer? Would it perform better?”. Knuth took enormous pains to save memory. He also designed TeX’s dynamic memory allocator to be compatible with the early virtual memory systems of the 1980s. While he did his best, the resulting code is obscure compared to how we would implement this design using modern language facilities.

more ...

Reading "TeX: The Program" in 2026

[Start of series] Part 2: Reimplementing the TeX data structures in C++ ->
   

According to the order slip tucked into my copy, I ordered TeX: The Program on June 5, 1986. For four decades as I moved through two countries and nine mailing addresses, that book lived in boxes and bookshelves. I took it out several times, intending to read it, each time setting it aside because I wanted to understand TeX’s syntax first. So I took down its companion volume, The TeXbook, read a few chapters in, set that aside and never returned to either one.

more ...

How much information is retained in an average of percentiles?

Experienced performance analysts caution against taking averages of percentiles. Gil Tene’s famous 2015 Strange Loop talk, “How NOT to Measure Latency”, features the tip, “You can’t average percentiles. Period.” (at 9:15). Tene offers a brief example using the 100th percentile, the maximum, but does not provide many details. You might think this is enough—simply never average percentiles and you’re safe, right? But as Baron Schwartz points out, the metric pipeline probably averages metrics before you see them, and even displaying metrics as pixels on the screen is a form of averaging. Why is it so bad to average percentiles and how much information does the result retain, if any at all?

more ...

AWS Outpost vs. Oracle Region @ Customer: How do they differ?

Cloud vendors are rolling out products that place proprietary hardware in customer’s datacentres, delivering a selection of the vendor’s cloud services from the customer’s datacentre. These new products require a rethinking of current guidelines regarding tradeoffs between data residency, latency, and failure tolerance. As a start towards this, I compare two specific products, Amazon Outpost in 42U rack form factor and Oracle Dedicated Region Cloud @ Customer, according to these criteria. What do these comparisons tell us about the new possibilities such products create?

more ...