Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Modules and generic functions

Importing other files

Each .ss file is a module with its own namespace. import attention makes the declarations of attention.ss available as attention.name: its dims, domains, models, data and logs records, functions and extern fns, all with their full types. A qualified name can be used wherever the plain name could: attention.causal_attention(q, k, v), m: attention.Mlp, attention.Mlp.init(seed=0), [b attention.D], Dataset[attention.Digit], AdamW[attention.Mlp], attention.StepLogs(loss=l). A name the file doesn’t declare is E2304. Messages show another file’s types with its name, as attention.Mlp, and its shape aliases are written the same way, x: attention.Images.

Another file’s dims are named as this file names them wherever their axes go: the result of attention.f(x), or a field of an attention.Mlp, whose types say [b D] in attention.ss, is [b attention.D] here, so softmax(y, over=attention.D) finds the axis. Another file’s D is a different dim from this file’s D, even when both have the same size, just as two dims of one file are: [n C] + [n attention.C] is E0404 until an as renames one. If a file is imported under two names, its dims take the name of the import without as.

  • import models.mlp imports models/mlp.ss and binds the last part, mlp, as import models.mlp as mlp would. Unlike import python, which binds the first part as Python does, an import always binds one name, so every qualified name has exactly one dot.
  • import shapescript.std.fit as std binds std. The name an import binds is a module-scope name like any other: a fn fit and an import shapescript.std.fit in one file are E0302.
  • There is no from attention import f: names of another file are always written with its name, so where each comes from is visible where it’s used.

The file is found next to the importing file, then in each folder of the Python path, as the import hook finds modules: sys.path when compiling from Python, $PYTHONPATH for shapescript check and build, and the workspace’s folders then $PYTHONPATH in the language server; then, from the command line and the language server, among installed packages. The standard library’s files, shapescript.std.*, are also built into the compiler, so they’re found without Python. No file is E2301. Files that import each other, directly or through others, are E2302.

An imported file is checked with the file that imports it, as one program; an error in it is reported once, as E2303 at the import that leads to it, with the errors as notes. Its warnings are left to checking that file itself.

Generated code. The generated module imports the other file through the import hook, _ss_mod_attention = _ss_rt.import_ss(__name__, __file__, "attention"), which looks next to the module’s own file, then on sys.path, and installs the hook if it isn’t; the name attention is bound to the module too, so Python sees main.attention. Each file’s Python is its own module, compiled once, so models and records of one file are the same classes everywhere. The hook’s cache entry for a file records the text of every file it imports, and which file each import found, so the importing files are compiled again when an imported file is edited, and when an import would now find another file: in a copy of the folder, with another sys.path, or where a new file hides the one it found. A --standalone build can’t import other files (E2305), and neither can a notebook cell (which can’t call generic functions either) or a program on several devices, whose workers run only the file’s own code, nor an exported function that calls another file’s (E2306). A batch loop doesn’t pad a batch it passes to another file’s function, which is compiled without knowing.

Overloaded functions

Several functions of a file may share a name when each takes a different model as its first parameter, so each model can have its own forward and loss. A call picks the one whose first parameter takes its first argument’s model, at compile time. It looks among the functions of that name the calling file sees and those of the file that declares the model, so forward(m, x) finds attention.forward for an attention.Mlp without the qualified name. Two with the same model are E2307, and so is sharing a name with a function whose first parameter isn’t a model, or with a generic function. Files are namespaces, though: when a call’s name means a generic function of the calling file, that’s the function, and the model’s file isn’t looked in. So the standard library’s fit calls its own run and train_step even if the model’s file has functions of those names, and finds only names it doesn’t declare, such as loss, next to the model. A library can add helpers without changing what its users’ code means; a call whose first argument is a model none of them takes, or not a model, is E2308.

Each one is its own Python function, _ss_forward__Mlp, named after its first parameter’s type as written, and the name forward is a function for calls from Python that calls the one for its first argument’s model class.

In the editor, typing m. after a value whose type is a model offers the model’s fields and every function whose first parameter takes that model: in the file, in the files it imports, and in the other .ss files of the workspace’s folders, even those it doesn’t import yet (but not those that import it, which would be a cycle), with generic functions whose first parameter is a type variable. A field completes as m.w1. A function rewrites m.forward as the call forward(m, …), with a tab stop for each parameter after the first that has no default, mut where the parameter is (train_step(mut m, mut opt, batch, mode)), qualified as attention.forward(m, …) when it’s declared in another file, and adds import attention after the file’s imports (or before its first declaration) when the file doesn’t import it yet. A generic function is marked when a function it calls on its first parameter, directly or through the file’s other generic functions, has no version for the model in the generic function’s file or the model’s, the functions a call would look in: calling it would be E2311.

Generic functions

A parameter type that is a single capital letter, optionally followed by digits, that nothing in scope declares (no dim, domain, model, record, logs record or built-in type) is a type variable: M in m: M, Grad[M], AdamW[M] and SGD[M] stands for a model, and E in [b] E, E and Dataset[E] for a data record. A function with a type variable is generic:

dim D = 8
dim C = 3

model A:
    param w: [D C] = normal(std=0.1)

data Ex:
    x: [D]
    y: [] Index[C]

fn loss(m: A, batch: [b] Ex) -> []:
    return cross_entropy(batch.x @ m.w, batch.y, over=C)

## One SGD step on any model that has a `loss`.
fn step(mut m: M, mut opt: SGD[M], batch: [b] E) -> []:
    l, g = value_and_grad(loss, m, batch)
    update(mut m, mut opt, g)
    return l

fn train(mut m: A, batch: [b] Ex) -> []:
    opt = sgd(m, lr=0.1)
    return step(mut m, mut opt, batch)

This is compile-time duck typing, as C++ templates are: there are no traits or interfaces to declare. A generic function isn’t checked on its own. Each call works out its type variables from the arguments (a parameter’s model, record, dataset or batch; two different types for one variable, or a variable no argument gives, is E2310) and checks an instance, a copy of the function with the types put in, so the function may do anything the types it’s called with allow. Its calls with a value of a type variable as the first argument, such as loss(m, …) and value_and_grad(loss, m, …), find functions as overloaded calls do, in the generic function’s file and the model’s, so a model brings its own forward and loss. A function it needs that the type doesn’t have, wherever the call is in its body, including in an if never taken, is E2311 at the call to the generic function, naming the function and where the generic function uses it; any other error in an instance is E2312 there, with the error as a note. A type variable that also names an axis of the signature, or one only the return type uses, is E2309. Instances that make other instances can go 16 levels deep (E2313).

The language server’s hover on a generic function lists its type variables and the functions it calls on them.

Generated code. Each instance is a Python function of its own in the module whose code calls it, as _ss_fit__Mlp__Digit, so mut, gradients, padding and placement are worked out for it as for any other function. The generic function’s name is a function for calls from Python that calls the instance for its first model argument’s class; one that no ShapeScript code calls has no instances, and calling it from Python raises TypeError saying so.

Packages

A ShapeScript package is an ordinary Python package whose folders have .ss files, and pip installs it like any other: pip install shapes_extra, from a wheel file, a local folder or a git+https:// URL. Its modules are imported with their dotted names, import shapes_extra.layers, which finds site-packages/shapes_extra/layers.ss and binds layers, with every declaration’s full types, as for a file of one’s own; an error in it is E2303 like any other imported file’s. docs/packages.md is the guide to making, building and publishing one; shapescript new NAME makes a package to start from.

Finding installed packages. An import looks next to the importing file, then on the search path, then in the standard library built into the compiler, then in the folders of a Python interpreter’s sys.path, where pip installs packages. From Python, the search path is already sys.path. The command line asks an interpreter: --python PYTHON, else the active virtual environment’s ($VIRTUAL_ENV), else $SHAPESCRIPT_PYTHON (which the shapescript command installed by pip sets to its own Python), else python3 (python on Windows). It asks only when an import isn’t found otherwise, and keeps the answer in the user’s cache folder (below) until a folder of the path changes, as installing a package with a .pth file does. The language server asks the interpreter it knows from its python option or shapescript/setPython (the VS Code extension sends the Python extension’s selected interpreter; see Python interop). An installed package’s files are read-only to the language server: opening one publishes no diagnostics, nothing is written beside it, and the VS Code extension opens .ss files under site-packages read-only (files.readonlyInclude).

Building. A package’s pyproject.toml names the build backend shapescript.build, which the shapescript package provides:

[build-system]
requires = ["shapescript>=0.4"]
build-backend = "shapescript.build"

It builds with hatchling (its [tool.hatch.build] options apply, and every file of the package’s folders goes in the wheel, .ss files included), checks every .ss file of the wheel with the others, failing the build with their errors, and adds to the wheel the Python module of each .ss file, a py.typed in each top-level package with .ss files, and a shapescript-build.toml in each folder with .ss files that names the compiler that built them. So Pylance, pyright and mypy know the installed package’s types, and from shapes_extra.layers import forward works without shapescript.install(). A hand-written .py file of a module’s name stays in the wheel instead, with W2501 as a warning. The sdist and editable installs (pip install -e .) are hatchling’s own: an editable install imports the .ss files from the source tree, beside which the editor writes the modules.

Running from site-packages. The import hook and the generated modules compile an installed .ss file like any other. Where __pycache__ beside it can’t be written, as in a read-only site-packages, the compiled Python goes to the user’s cache folder instead: $SHAPESCRIPT_CACHE_DIR, else $XDG_CACHE_HOME/shapescript, else ~/Library/Caches/shapescript on macOS, %LOCALAPPDATA%\shapescript\Cache on Windows and ~/.cache/shapescript elsewhere, under modules/, in a folder named by a hash of the file’s path. Generated modules of installed packages are never rewritten, since pip doesn’t keep the files’ times.

Versions. A package’s .ss files are written in the language of the compiler that built it, which shapescript-build.toml records (compiler = "0.4.1"). Patch releases don’t change the language, so a package needs at least the major and minor version that built it: an older compiler reports E2901 at the import instead of failing on what it doesn’t know, and so does the import hook when Python imports the module. A package’s own requirement on shapescript (dependencies = ["shapescript>=0.4"], as shapescript new writes) makes pip install a compiler new enough. A pre-release of the compiler writes itself instead, as in shapescript>=0.1.2b1, since pip takes pre-releases only for a requirement that names one.