Appendix B: Glossary

Axis. One entry of a tensor’s shape, such as n in [b n D]. An axis has a size and may have a name.

as. An assertion: x as T claims x already has type T, which the checker then believes and the program checks at run time. The only way to tell the checker something it can’t prove. See Chapter 7.

Batch. [b] Digit: b examples of a data record, whose fields each get the axis b first.

Broadcasting by name. How elementwise operations line up tensors: by axis name when one operand’s names are all in the other, otherwise from the right, where two named axes in a pair must have the same name. See Chapter 6.

Buffer. A model field that’s saved but not trained.

Controller. With several devices, the one process that runs your program; the workers run its tensor operations.

Data record. A data declaration: the fields of one example of a dataset.

Devices. The devices line: where a program’s tensors live, such as cpu, mps, cuda[0], or several, cuda[0 1 2].

Dim. A named constant size, dim D = 768. Writing D in a type both sets the axis’s size and names it.

Domain. What values a tensor may hold, such as Unit, Prob over C or Index[V]. The checker infers most domains from the code. See Chapter 8.

Dtype. How values are stored: f32, bf16, u8, bool and so on.

Elementwise function. A function whose parameters have no shape, such as fn swish(x: Real) -> Real, which applies to tensors of any shape.

Export. Turning a function into a torch.export program that runs without ShapeScript or Python code. See Chapter 15.

extern fn. A typed declaration of a Python function, whose signature the checker trusts.

Generic axis. A name in a parameter’s type that isn’t declared anywhere, such as b in x: [b D]. Each call chooses its size.

Generic function. A function with a type variable, such as M in m: M, checked again for the types each call gives it.

Grad. Grad[M], a gradient with respect to a model M’s params, with a field for each.

Group. (gh 16): one axis whose size is the product of its parts.

Grouped size. 16gh: shorthand for the group (gh 16), an axis whose size is a multiple of 16.

Host scalar. An int, float or bool: a plain Python value, kept on the host, for counters, rates and flags.

Logs record. A logs declaration: named metrics and how each combines.

Mask. A boolean tensor, from comparisons or causal[n m].

Mode. Train(seed=s) or Eval, passed to everything random, such as dropout.

Model. A model declaration: a record of params and buffers, the state of a network.

mut. Marks a parameter a function may change, and the argument at every call.

Param. A model field that’s trained.

Pattern. b n (h d) -> b h n d: the einops-style description an operation such as rearrange or einsum takes, written as syntax.

Pipe. a |> f(args), which means f(a, args).

Prelude. The extern fn declarations every file sees, such as exp, gelu and layer_norm.

Python value. A value from import python, which the checker knows nothing about until an as gives it a type.

Schedule. A learning rate that changes with the step, such as warmup(100, cosine(3e-4, 10000)).

Shape alias. dim Images = [b 3 16gh 16gw]: a name for a list of axes.

Sink. A Python object that logs records are written to, such as shapescript.sinks.csv("log.csv").

to. A conversion, such as u8 to Unit or a mask to Additive. Unlike as, it changes the values.

Type variable. A single capital letter, optionally followed by digits, in a parameter’s type that nothing declares: M, E.

Unknown size. ?k: a definite size the checker can’t see, such as the number of rows a filter keeps.