Logs and sinks

Logs declarations

logs Name: followed by an indented block of metrics declares a logs record, the numbers a program reports:

## How each value is combined when several are reported together, such as over an epoch.
logs StepLogs:
    loss:      [] mean
    grad_norm: [] max

Each metric is a name, a type, and how its values combine: mean (the arithmetic mean), sum, max, min or last (the latest value). A metric is a single number: its type is [], with any domain and dtype, as in [] NonNeg or [] i64, or a host scalar int, float or bool. Anything else, a missing or unknown combining word, and a name that is a Python keyword or starts with _, are E1402. A record’s metrics have different names (E0302). A logs record and each metric can have a doc comment.

Records

A logs record’s name is a type, as in -> StepLogs or r: StepLogs; another record or a tensor there is E1403. StepLogs(loss=l, grad_norm=n) makes a record. Every metric is given, by name: a metric the record doesn’t declare is E1401, and one given by position, twice or not at all is E0406. Each value must fit its metric’s type as an argument fits a parameter’s: a [] tensor or a number for a [] metric (E0403 for a tensor with axes, E0501 and E0502 for its domain and dtype), and a host scalar for a host one (E0805). r.loss reads a metric, with the metric’s type; a misspelled one is E1401.

A record holds its values as they are, so a tensor stays on the device until the record is written. It compiles to a subclass of shapescript.rt.Logs, and from Python its metrics are attributes.

Sinks

LogSink is the type of a sink, a Python object with a write(dict) method that records are written to. Sinks come from Python, as arguments, and ShapeScript code can only pass them on and call two methods on them:

• sink.write(record) copies the record’s values to the CPU, turns each into a Python number (float, int or bool, after the metric’s dtype), and calls the sink’s write with a shapescript.rt.LogRecord: a dict from each metric’s name to its number, in declaration order, whose name attribute is the record’s name and whose combine attribute maps each metric to how it combines. Writing anything but a logs record is E1403.
• sink.flush() calls the sink’s flush(), if it has one, and does nothing otherwise.

Any other method is E0301, and a wrong number of arguments is E0406. Something other than a sink where a LogSink is expected is E1404. A LogSink parameter checks at run time that its argument has a write method (Runtime checks).

Combining. The program writes a record whenever it likes, such as after every step, and a sink decides how often to report: it combines the records it has been given, metric by metric, by each metric’s rule. The sinks in shapescript.sinks take every=: with every=1, the default, each record is a row; with every=n, each row combines n records; and with every=None, a row is written only when the program calls sink.flush(), so flushing at the end of each epoch gives one row per epoch, with the epoch’s mean loss and largest gradient norm. A flush or close() writes a row for the records since the last row, if there are any. shapescript.sinks.combine(records) combines a list of records the same way, for sinks of one’s own.

• shapescript.sinks.csv(path, every=1) writes a CSV file: a header of the metrics’ names, then a row per combined group. The file is flushed after every row.
• shapescript.sinks.tensorboard(log_dir, every=1) writes each metric as a TensorBoard scalar tagged with its name, at the step of the last record in the row, counting records from 0. It needs the tensorboard package, and raises ImportError saying so without it.

Both are context managers, and close() flushes them.

Printing

print(a, b, ...) prints any values, separated by spaces, as Python’s print does: tensors on any device, host scalars, strings, models and logs records. It copies tensors to the CPU itself. A [] tensor is printed as a number, and a logs record as a dict of numbers. Its arguments are positional; a keyword is E0406. It has no result.