All my marbles in one place

A blog about course design, data display, C++, and Python. Yes, these are related, at least as I see them.

© 2026. All rights reserved.

Reading "TeX: The Program" in 2026

03 Sep 2026 Tags: Programming

According to the order slip tucked into my copy, I ordered TeX: The Program on June 5, 1986. For four decades as I moved through two countries and nine mailing addresses, that book lived in boxes and bookshelves. I took it out several times, intending to read it, each time setting it aside because I wanted to understand TeX’s syntax first. So I took down its companion volume, The TeXbook, read a few chapters in, set that aside and never returned to either one.

I made my last attempt sometime in the 1990s. Despite this lack of progress, the books survived multiple bookshelf culls, winding up in the much-reduced “technical” shelf of my latest home office. Never mind how few pages I’d ever read or how many years had passed without even opening them, these books were keepers.

I retired in December of 2025, ready to leave programming behind and explore other interests. Yet after six months I felt an absence, a lack. I kept looking over to my technical bookshelf, seeking the right balance of difficulty and readability.

And so, nearly forty years to the day after I made my initial order, I pulled out TeX: The Program and opened to the first page. I had some misgivings. Would it hold up? Could I learn anything from it? Or is it just an artifact of a long-past technical era, an application coded in a disused language, useful only for laying out an outdated media format, whose data structures address portability for obsolete machine architectures, developed in a team structure—one person working several years—totally unreplicable in modern, team-based, high-velocity environments?

This is the story of what I found.

Relating the TeX of 1986 to the TeX of 2026

The version of TeX describe in the book’s 1986 edition is TeX 2.0, while the current “Jubilee” version is 3.141592653. I have reviewed the code for the more recent version and the changes from 2.0 are modest. The data structures and program organization I describe here are the same across both versions.

In the final section of this article, I will review how modern extensions to TeX, such as adding Lua as a scripting language, are organized.

With the version issues out of the way, we can consider the book.

The book flows

My first impression was that the book reads as a book, quite unlike surveying a repo on GitHub or GitLab. It flows, it tells a story. I started on Page 1 and have read to Page 124 (so far), almost never feeling the need to flip ahead or behind.

Far from a limitation, the presentation of the code as a book is used to good effect. The basic unit of presentation is the two-page spread of the opened book. The bottom of the right-hand page is a cross-reference section, linking most identifiers not defined on these two pages with its defining section. If the identifier’s value is a WEB constant, its value is provided directly and you don’t need to leave the current page.

The cross-referencing falls a bit short in two ways. First, although most identifiers used on a page are cross-referenced, a few are not and have to be looked up in the full index. For example, Section 120 (p. 52) refers to mem_end, which is not listed in the cross-reference on the facing page (p. 53). I have never learned the criteria for whether an identifier will be listed or not.

Second, if the identifier is a Pascal constant, such as mem_max in the same section, the cross-reference only describes it as const, not its defined value. This likely results from the cross-reference being generated by WEB, which parses its own macros completely but not Pascal syntax.

The two-page spreads are grouped into sections, each beginning with a summary of this code’s role in the overall program and then laying out the relevant routines in a sequence of spreads. Each page repeats the section title in its header, connecting that page’s routines to its larger purpose.

The sections in turn proceed in the order most readers would expect: First the basic definitions, then memory management, then data structures building on that memory managment, and so forth.

Now this is just one way to read a computer program, where the reader wants an overview of the code, its consituent parts, and key algorithms. It’s far from a deep understanding, more of a warm feeling of having understood.

For other forms of reading, such as modifying the code, the book format is less suitable: The reader would need to see routines from widely-separated parts of the code. The format is terrible for that purpose, as the reader could only see one page at a time and would have to flip back and forth between pages, never able to see the routines at the same time. A multi-window editor with a backing index locating all the identifiers is a far better match for modifying the code. As I describe in Part 2, this is exactly what happened when I went on to re-implement some of these algorithms in C++.

My reaction to the code

The potential lessons from this code are going to come from its structure and the design choices that Knuth made. The detailed code has little to teach us because the Pascal language is dated, the code is written to maximize portability across a range of architectures that no longer exist, and the data structures were chosen with an eye to absolutely minimizing memory size.

Pascal is dated

For the past forty years I’ve been programming in C and languages derived from it, with particular emphasis on those featuring some form of objects, such as C++, Java, C#, and JavaScript. Although those languages have wide variations in memory management, they have a generally consistent syntax, including generalized for loops, break and continue statements for altering loop flow, and a return statement for immediately returning from a function. They all support breaking a program into multiple files.

Classic Pascal featured none of these. Although specific implementations provided some of these features, none was standard, and so Knuth opted not to use them. To gain equivalent functionality, Knuth defined macros in his WEB markup language.

For example, to provide the effect of a return statement, he had to build macros on top of Pascal’s numeric line labels and goto statement.

First, he globally defines a constant (indicated by an equals sign) and a macro (indicated by a defined-as sign ≡):

define exit = 10
define return ≡ goto exit

Then in any function or procedure (Pascal distinguishes these) in which he wishes to use a return, he defines exit as a label:

function init_terminal: boolean; { Section 37, p. 17 }
  label exit;
  ...
  loop begin { loop is another Knuth macro }
      if ....then begin
         ...
         return
         end
      end
  ...
exit: end

This annoyed me the first few times I read it but I quickly acclimated and interpreted return normally, ignoring the noise of the exit label. I’ve never gotten used to the way he indents his trailing ends, though.

The TeX code has reminded me of one feature of Pascal (as well as its successors Modula and Ada) that I do miss: explicit specification of fixed-range integer values:

ASCII_code = 0 .. 127; { Section 18 }

(Modern versions of TeX support full 8-bit ASCII, extending the range to 0 .. 255.)

The code maximizes portability to nonexistent architectures

At the time Knuth wrote TeX, there was a much wider range of extant computer architectures. For example, IBM mainfames used the EBCDIC character encoding rather than ASCII. Of more direct significance to Pascal’s design was the architecture of CDC’s scientific computers, which were amongst the primary machines used at the university where Wirth defined Pascal. These machines featured a 60-bit word length (double precision was 120 bits, by the way), with no way to directly address smaller units within those words. To fit characters into words of that length, CDC used a 6-bit character set.

At the same time, a growing number of machines featured the design that has became nearly universal: byte-addressable memory and the ASCII character set.

Knuth wanted to write genuinely universal code, so he strove to accept the widest range of input characters, translated to 7-bit ASCII for TeX’s internal representation. He also chose data types that compilers of the era could translate to the widest range of underlying word lengths extant in the early 1980s. The resulting data types feel a bit “off” to a modern reader, as they reflect machine architectures that have long since fallen out of use.

These anachronisms only have mild effects on readability. Far more difficulty arises from the design choices for TeX’s memory layout.

TeX’s memory management is complex and brittle

Much of the code in the book’s first 19 sections (120 pages) lays out the basic data structures in which TeX will represent its data. Several factors forced Knuth to craft intricate code to implement the structures. These factors prevailed when he wrote the program in the early 1980s but do not apply so strongly, if at all, today.

At the time Knuth wrote TeX, machines had less memory than they do now. Far, far less memory. A program’s memory footprint was also correlated with its speed: Smaller was typically faster. Large size could even prevent running TeX altogether, as many of the writers interested in using it might only have access to small machines and would not be able to run a large program at all. Squeezing out every extraneous byte of memory was essential and, as Knuth was the man who wrote the book on fundamental data structures, it was almost foreordained that TeX’s foundational data structures be intricate.

TeX uses custom dynamic memory

A key contributor to the code’s complexity was the lack of dynamic memory allocation, such as C’s malloc() function, in the standard Pascal of the time. To ensure portability, Knuth had to build his own dynamic memory from scratch by preallocating a large array of words and allocating TeX’s dynamic data structures within that.

Knuth defines the memory_word type (Section 113) as a variant record that is either an int, a real, two halfwords, or four quarter_words (bytes). The constraints on the value ranges of quarter_words and halfwords (Section 111) require a memory_word to be at least 32 bits wide, although it could be larger.

In TeX, all dynamic values are allocated from an array mem (Section 116) of memory_words. A pointer to a dynamically-allocated value is an index into mem and the pointer type is simply a macro defined (Section 115) as halfword. The equivalence between pointers and halfwords is a deep expectation of TeX’s data structures, which often tuck a pointer together with another halfword-sized value into a single machine_word.

For a machine with a 32-bit word, this definition limits the total dynamic memory (the largest possible size of mem) to 65,535 words (the value 0 is reserved as the null) of 32 bits, or 262,140 bytes. The only way to increase that range is to define memory_word to be larger, such as 64 bits, resulting in a much larger total footprint even if you only wanted a small increase in the mem array. For example, even just increasing the pointer range to 100,000 would require allocating that many 64 bit words, for a total of 800,000 bytes, more than tripling the footprint.

The version of TeX described in this book specifies the pool size to be 30,000 words (mem_max, Section 11), though production versions can specify larger pools.

Two distinct memory allocators

TeX allocates dynamic memory using two distinct allocators, one for the general case and one specifically for characters.

The general case is a standard allocator for variably-sized blocks, similar to C’s malloc(). He implements this using a sophisticated algorithm from Ex. 2.5–19 of The Art of Computer Programming. As the author of both the low-level memory management and its higher-level callers, Knuth could micro-optimize the memory API to require that the caller specify the size of a box at both allocation and deallocation time, eliminating the need for a field in the box itself to record its size. This saves memory at the expense of greater complexity in the caller.

Second, he wrote a special case for individual characters. These are by far the most common type of data in a text processor and also require only two bytes to represent, one for the font and one for the character. Note that this design will not suffice for CJK ideographs or other large character sets. In Section 134, Knuth suggests a way to handle what we would now call the Unicode Basic Multilingual Plane but that is insufficient for the full CJK set.

Character sequences are represented by singly-linked lists, with the links being the two-byte indices into the mem array. Combining the links with the two-byte character representation allows character sequences to be represented as a list of memory_words.

Note that unlike nodes allocated by the general allocator, character nodes do not include a type field. TeX distinguishes general nodes from character nodes by placing them at different ends of the mem array: The general allocator works from the bottom up, while the special character allocator works from the top down. A character node’s “type field” is simply its location at the high end of the array. This minimizes the size of character nodes, by far the most common type in TeX.

The custom dynamic memory cannot use regular Pascal types

This approach precludes using Pascal’s strict data types, which cannot be dynamically mapped on to array elements of a different underlying type. Instead of defining new Pascal data types for dynamically-allocated values, Knuth defines WEB macros that expand to an underlying machine memory unit. For example, the free pool of single-word values is represented as a singly-linked list, with each word comprising a pointer and an info field. The WEB macros define these as (Section 118)

define link(#) ≡ mem[#].hh.rh
define info(#) ≡ mem[#].hh.lh

where .hh.rh and .hh.lh identify the two halfwords of the underlying word.

Using WEB macros rather than actual Pascal types has serious consequences. First, it makes the definitions inaccessible to type-checking, bypassing an important feature of modern languages. Second, it ties the range of possible values of these structures to the range expressible by the underlying memory unit, as we saw above with the limited range of pointers. Third, it prevented Knuth from using even the basic data structuring provided by Pascal records.

Consider the most important data type in all of TeX, the box, defined in the nine pages of Part 10 (Sections 133–161). At its core, TeX is program assembling a page as a list of boxes, each of which might contain other boxes, in turn potentially containing more boxes, and so on. In Pascal or any other high-level language, an individual box would be represented by a type definition such as:

TYPE Box = RECORD
  width: real;
  depth: real;
  next: ^Box; { Pointer to next box in the list }
  ...
END;

Knuth cannot use this feature however, because his dynamic memory routines only allocate memory_words. So Knuth has to effectively hand-compile the above logical record structure into a collection of offsets into an allocated block of these words, then write WEB macros that look like accessing a record type.

The code in Section 135 implements this approach:

define box_node_size = 7 { Number of memory_words for a box}
define width_offset = 1 { The width field is in the second word }
define width(#) ≡ mem[# + width_offset].sc { the .sc field interprets a memory_word as real }

So where in Pascal you would access the width of a box referred to by the pointer next using the straightforward next^.width, in TeX next would be an integer index into the array mem and you would access the box width by the macro call width(next), expanding to mem[width + width_offset].sc.

This approach adds bookkeeping code and cannot be type-checked (what if the integer next is not the index of a box node or not an index into mem at all?). But it’s the only approach available using a custom-written memory allocator rather than the Pascal implementation’s built-in one.

Pascal did not support an object hiearchy

Dynamically-allocated values form a natural hierarchy in TeX. With the exception of glue specifications, which I’ll return to later, all dynamic values are a type of box. Boxes are arranged in singly-linked lists, with the link accessible via the link() macro. Each type of box, such as vertical or horizontal lists or rules, extend this basic type with fields specific to their type. For example, a vertical list contains a sublist of boxes, with the sublist accessed through the list_ptr() macro.

Modern readers, versed in object-oriented programming, would immediately design this as a class type hiearchy. Although object-oriented programming is facing increasing criticism for overuse in cases where it obscures more than clarifies, this case is a natural fit to that technique.

But the Pascal of the 1980s had no object features—indeed, at that time object-oriented programming was mostly an experimental feature rather than mainstream—and Knuth wanted to stay within the most portable features of the language, so he essentially implemented a basic object system in WEB macros. Every box has a two-byte type field (specified via the type() macro) and all its fields are defined using the technique described above of an offset and a macro.

The resulting code includes textbook examples of the awkward code that class hierarchies eliminate. For example, the show_box() procedure (Part 12) that recurses through a hierarchy of linked boxes includes a long case statement with a case for each box type. Defining a class hierarchy would define a show_box() procedure within each box class and the case would be eliminated.

Knuth made the best of the language facilities available to him at the time but a modern reader is likely to find themselves screaming, “Just define a class hiearchy!” when reading Section 10.

Other dynamic structures

In addition to lists of dynamically-allocated nodes, TeX has three other dynamic data structures.

First, glue specifications (Section 150) are allocated from the node array but have a different structure than a node. They do not link to any other nodes but can be linked from multiple nodes. For this reason, the field that would ordinarily be used for a link is instead a reference count.

Second, control sequences are implemented by a standard hash table (Part 18). This is a standard algorithm and I will not describe it further.

The third dynamic structure is the equivalence table (Part 17) and its save stack (Part 19). This table records the current values of variables local to the current group. These variables include available control sequences and parameters such as skip registers. These values are local to the current group; when a group ends, any value local to it is replaced with the value for the containing group.

The equivalence table has 5,076 values. A simple design would just implement a stack of these, pushing a new table upon group entry and popping it on exit. Instead, Knuth chose a more complicated design with a single 5,076-entry table (requiring 20K bytes) supplemented with a 600-entry stack (requiring 2K bytes). Whenever a value is defined local to the current group, the older value for the enclosing group is pushed on the stack. When the current group exits, any values it changed are restored from the stack.

I do not know the TeX code well enough to say whether this more complex design is simply to save a small bit of space or is required by subtle semantics of TeX groups. I note that the save stack alone consumes seven pages of code, a level of extra complexity that requires substantial justification. I suspect a modern implementation would just stack the full equivalence table and be done with it in a few lines.

WEB factors code into parts but not scopes

TeX makes extensive use of the WEB feature for breaking longer routines into smaller pieces. For example, the get_node() function, TeX’s counterpart to malloc(), includes the lines (Section 125):

if lo_mem_max + 1 ≤ mem_bot + max_halfword then
  ⟨ Grow more variable-size memory and goto restart 126 ⟩;

Then Section 126 expands the description between angle brackets into seven lines of Pascal. Logically, Section 126 functions as a subroutine called by Section 125—or nearly so. But this mechanism lacks the scoping provided by actual subroutines. Section 126 has complete access to every variable contained in Section 125 and can read or even update them at will. In fact, the lines in Section 126 can also access any variable contained in any enclosing scope of Section 125, all the way up to the global scope.

This unlimited scope is not simply inadvertent, it is essential to how TeX’s data structures operate. Recall from the discussion of dynamic memory that TeX simulates the fields of records via WEB macros that expand into direct references into the global array mem. For these references to succeed, every line of code has to be able to access that array.

TeX lacks clear separation into levels

The unrestricted scoping is a facet of a larger issue, the lack of separation of TeX into separate levels. TeX is clearly the product of one developer, working over a period of years, intimately familiar with every line of code and its contribution to the overall application.

The lack of separation has myriad effects throughout the code. I’ll highlight two here.

Reading the first commands from a file versus the console

Upon starting, TeX’s first console prompt is a pair of asterisks. Ignoring some special cases, the user will respond one of two ways (p. 23 of The TeXbook):

The decision between these two options is partly implemented deep in the code, in the scan_file_name routine (Section 526). If this routine encounters a command as the first token it reads, it sets the global name_in_progress to false and returns. For its part, the \relax command gets special treatment as the only primitive command in Section 265 that is assigned an operand, ensuring that the file name routine will recognize it as a command and abandon scanning for a name.

The description of this choice in The TeXbook implies that this design may have been chosen late in the implementation. Handling the relax command deep in the file name scanner may have been a last-minute solution.

The values chosen for enumerations are themselves significant

TeX makes extensive use of enumerated values: lists of integer constants representing commands, modes, and other internal states. Modern programming languages, including Pascal, support enumerated types (for example, enum in C/C++).

The subset of standard Pascal used by Knuth does not permit specifying the integer values associated with a given identifier, so he defines his enumerations instead via long sequences of WEB define macros. See, for example, Sections 207–210, which define the 119 codes identifying every TeX primitive command.

Knuth needs explicit control of the values assigned each constant because he wants to be able to make simple decisions between categories of constants:

These deliberate value choices allow the higher-level routines to distinguish categories using simple if statements rather than long switch sequences. The negative consequence of these choices is that the low-level decision of code assignment must be made in accord with the requirements of higher-level semantic choices; the levels blend together.

This blending is exacerbated by the choice to combine multiple codes into a single integer and then make semantic choices based on the range of the combined value. For example, Section 211 adds the current mode to the current command for later use in if comparisons. To make this work, the three mode values have to be assigned multiples of max_command:

define vmode = 1
define hmode = vmode + max_command + 1
define mmode = hmode + max_command + 1

See Section 211 for the details. Section 289 does a similar computation for token codes.

These combined values compound the level blending, in that low-level constants must be precisely chosen to ensure that higher-level decisions work correctly.

TeX in 2026

Reading TeX’s code, I was reminded again and again that this was the product of a single, master programmer, refining his code for years without deadlines, and with an extraordinary focus on saving every byte of memory. This approach is almost unrecognizable today, where programs are written by teams, updated incrementally, and feature velocity is more important than memory efficiency almost every time. The recent move to AI-aided programming takes us even further from TeX’s development model.

Language features such as nested scoping and design approaches such as strong separation of levels are essential to contemporary development. They permit a developer to work locally with the assurance that they don’t have to understand the full architecture, only the code within their interface, and that implementation decisions they make in the local code are hidden behind the interface’s abstraction. When team members are entering and leaving the project with some regularity, they have to be able to get up to speed quickly. There’s no opportunity for them to gain the sort of detailed, line-by-line understanding that Knuth had of TeX.

Of course, TeX is ultimately tiny compared to most modern programs, with a narrow scope. It has no networking component, it doesn’t serve online requests, and its interface is entirely console-based. It simply takes a few files, formats them—elegantly, to be sure—and writes out a binary file that drives the display. It doesn’t have a single external dependency, other than the most basic file and console operations afforded by the base language.

In all these regards, it is a relic of a long-gone era. And yet, astonishingly, forty years later it remains a widely-used, well-liked tool for writing technical documents. As recently as July of 2026, Dr. Angela Collier described how virtually all research papers in Physics use Tex/LaTeX (video on Nebula, on YouTube). The accomplishment is even more striking when we note that code for the current version has remained unchanged for forty years, with only modest extensions to handle more recent technologies such as UTF-8 encoding and TrueType fonts.

TeX may be the only consumer-facing program from the 1980s that remains in active use. I certainly can’t think of another. There may well be programs within specialized, vertical markets, or longstanding IT code running on the mainframes of large organizations, but to my knowledge no other programs from the 1980 with a wide customer base remain in use essentially unchanged.

TeX’s influence on contemporary technical writing extends beyond its direct use. The MathJax project allows authors to define formulas in the notation of TeX’s math mode for display in Web pages. The LATEX.js project goes even further, providing a complete LaTeX-to-HTML5 translator in JavaScript. Whatever the future of the original TeX program, the notation it established for writing mathematical formulas seems destined to persist long into the 21st century.

The Tao of TeX: The TeX you read is not the TeX you use

When we read Tex: The Program, we are seeing the reference version of the program. This specifies the exact algorithms to be used and sets the standard for any variant. But end users of the program have always used slight variants, even from the earliest days.

These variants do not change the algorithms or computations—they must produce identical outputs to the reference version, after all—but they will be modified to reflect local requirements:

Beyond local adaptations of basic TeX are more ambitious extensions, such as pdfTeX, XeTeX, and LuaTeX. I have skimmed the changes made by each of these and didn’t find any major changes to the architecture and data structures. LuaTeX seems to have made the biggest updates, using C’s realloc() to dynamically allocate the one-word-length and variable-length node pools. LuaTeX also manages the string pool differently from standard TeX but as I didn’t consider that data structure in my analysis above I won’t go into details.

What did I gain from reading this code?

So what have I gained from reading and reviewing this code? The most striking effect was the sense of opening a time capsule, peering into a programming past so different from our present that it seems quaint. At times, its use of the Pascal language and its definition of records via WEB macros gave an impression of an outdated dialect, much like reading Shakespearean English. Understandable but requiring a bit of effort.

At the same time, I was struck by the care evident in every line, the consciousness behind each choice. It simply makes everything I’ve written in recent years seem a bit slapdash. It got me thinking, if I had the chance to work on something until it was really right, how would the result look? And how would I feel while crafting it?

It also engendered a sense of loss. TeX seems unique, an artifact entirely specific to its moment of creation and its creator, conditions that will simply never recur in the contemporary environment. The consummation of a programming era that has long passed, it sits like a vase of cut crystal on a pedestal, an artifact more to be admired from a distance than handled.

That loss is compounded by the reminder of how fun programming used to be.

Coming in Part 2: Reimplementing the node data structures

I finished the data structure sections with a question in the back of my mind: Would reimplementing the base structures in a more modern language and paradigm improve the code? Would it be easier to read? Would it improve or worsen performance? The memory managers underlying modern languages are written with an awareness of the role of locality in the modern memory hierarchy, such as the importance of cache coherence. Would these routines perform better than Knuth’s code, written well before such considerations arose?

In Part 2, I reimplement TeX’s node data structures twice in C++ 23, first using the original approach, second using the full capacities of the language. Along the way, I just may rediscover the fun in this work.