16: Checkpoints

E1601

Value that save can’t write to a checkpoint. A checkpoint holds models, optimizer state and int, float and bool values. A tensor, a gradient, a Mode, a Schedule, a logs record, a dataset or several results can’t be saved. A tensor that training needs belongs in a model, as a buffer if it isn’t trained; a schedule is saved with the optimizer state that uses it.

dim D = 64

model Mlp:
    param w: [D D] = zeros

fn f(m: Mlp, x: [b D]):
    save("ckpt.pt", m, x)

See save.

E1602

load given something that isn’t a type it can read. After the path, load takes the type of each value in the file: a model’s name, optimizer state such as AdamW[Mlp], or int, float or bool. A value, as in load("ckpt.pt", m), another kind of type, such as a tensor type, Grad[Mlp] or Schedule, or an expression that isn’t a type is this error.

dim D = 64

model Mlp:
    param w: [D D] = zeros

fn f(m: Mlp) -> Mlp:
    return load("ckpt.pt", m)

Write the type: load("ckpt.pt", Mlp). See load.

E1603

Checkpoint path that isn’t a string. save and load take the file’s path first, by position: a string, or a Python value, which is checked when the call runs. Anything else there, such as a model or a tensor, or a path given by name, is this error.

dim D = 64

model Mlp:
    param w: [D D] = zeros

fn f(m: Mlp):
    save(m)

Write save("ckpt.pt", m). See save.