Foreword

Every PyTorch programmer knows this bug. A tensor that should be (batch, heads, seq, seq) is (batch, seq, heads, seq), a transpose went the wrong way, or a softmax(dim=-1) normalizes the wrong axis. Nothing complains. The model trains, a little worse than it should, and you find out days later, if at all.

The shapes were in your head the whole time. They were in the comments too: # x: (B, T, C). But nothing read the comments.

ShapeScript is a small language that reads them. Axis names and sizes are part of its types, every operation takes axes by name, and a checker verifies the whole program before any of it runs, in your editor, as you type. When it’s happy, it compiles to plain Python that calls PyTorch, so everything you know and every library you use still works.

This book teaches the language from the beginning. It assumes you’ve written some PyTorch: you know what a tensor is, what x @ w does and roughly what a training loop looks like. It doesn’t assume you’ve used einops, a typed language or a compiler.

Like The Rust Programming Language, which it borrows its shape from, the book is meant to be read in order, with your editor open. Every ShapeScript example in it is checked by the project’s test suite, so if the book says a program is fine, the checker agrees, and if the book shows an error, that is the error you’ll see.

ShapeScript is in early development. The book only teaches what works today. Where something is planned but not built yet, it says so, and calls it coming.