When forecast models are trained, they withhold part of the historical data (actuals) from the training process. The amount of data withheld is equal to the length of the forecast horizon.
For example:
You have 12 months worth of historical data and a 3 month forecast horizon.
- The model training process uses the first 9 months worth of the historical data and then predicts the last 3 months.
The remaining 3 months of historical data are withheld. - It then compares the predicted last 3 months with the actuals. This is to assess the model performance, and calculate the forecast model quality metrics (advanced metrics).
The closer the forecasted values are to the actuals, the more accurate the model's predictions were, and the better the performance.