Evaluating Beyond Forecast Accuracy
We explore how to evaluate temporal models beyond a single forecast score, including their behavior under uncertainty and changing conditions.
Accuracy is a starting point
A forecast score captures one part of model behavior. It may leave open questions about uncertainty, robustness to change and whether a representation supports useful reasoning.
We are exploring evaluation that examines these capabilities together, while keeping the conditions of each test visible.
Testing a changing world
Evaluation can include unfamiliar regimes, missing data and shifts in the relationships between variables. These settings help reveal where a model depends on patterns that do not persist.
Making evaluation reproducible
A useful evaluation records the dataset, split, baseline, measurement conditions and assumptions behind a result. We would compare capabilities separately so a strong average score does not conceal a failure under a specific condition.