Runtime checks

shapescript build compiles a file to a Python module. Every function checks its arguments when it is called, from Python or from ShapeScript: each parameter with a shape gets a rank check and a check of its sizes, and the function’s generic sizes are read from the arguments’ shapes (the body may use them, as in causal[n m] or sqrt(Dh)). One read from an axis that isn’t just it, as h is read from (h Dh), 16h or h+3, is checked first: some whole h of at least 0 must give the axis’s size, as for a call the checker sees, and the error says which sizes fit, as in whose axis 3 is a multiple of Dh = 16, got shape (2, 5, 72): 72 isn't divisible by 16. Every as with a shape compiles to a shape check, and so does as int, as float or as bool on a Python value. A parameter whose type is a model checks that its argument is a model of that name whose fields have the declared shapes and dtypes, and one whose type is Grad[M] that its argument is a gradient of M whose fields have the params’ shapes and dtypes. A parameter whose type is a batch of a data record checks each field as a parameter of the field’s batched type would be checked, and a dataset parameter wraps its argument so each item is checked as it’s loaded (Loading). A parameter whose type is optimizer state, such as AdamW[Mlp], checks that its argument is state of that optimizer for a model of that name with the same params, and a Schedule parameter that its argument is a schedule or a number (TypeError if it isn’t). A Mode parameter checks that its argument is a mode, raising TypeError if it isn’t. A parameter whose type is a logs record checks that its argument is a record of that name, and a LogSink parameter that its argument has a write method. Each argument that is a tensor is also checked to be on the declared device. Parameters without a shape, and host scalar parameters, aren’t checked.

Each as with a domain, and each parameter with a shape and a domain that has a test, also gets a value check. A value check tests what the domain promises: that values are within a range’s finite ends; that Prob over C sums to 1 over C, as does the exp of LogProb over C (to within the square root of the dtype’s epsilon); that Binary holds only 0 and 1, Additive only 0 and $-\infty$, and Index[V] integers in $[0, V)$; and that a Mask is stored as bool. Real and Logit have no check, and other dtypes aren’t checked.

debug skips the checks when Python runs with -O. A failing shape check, host check or model check raises shapescript.ShapeError, a failing device check shapescript.DeviceError, and a failing value check shapescript.DomainError. They name the .ss location; ShapeError and DomainError are ValueErrors, and DeviceError is a RuntimeError. shapescript run (python -m shapescript) shows these errors, and a .ss file’s compile errors, as shapescript check shows its diagnostics, without a traceback: the line of the .ss file and the call that ran it, and for an argument, the argument in the call with the parameter it was given for. Every other exception keeps its traceback, and so do these with SHAPESCRIPT_TRACEBACK=1 and under a debugger, which stops where the error is raised. On a CUDA tensor, value checks use torch._assert_async, so a failure shows up later as a device-side assert.