> ## Documentation Index
> Fetch the complete documentation index at: https://nixtlaverse.nixtla.io/llms.txt
> Use this file to discover all available pages before exploring further.

# Evaluation

> Model performance evaluation

### `evaluate`

```python theme={null}
evaluate(df, metrics, models=None, train_df=None, level=None, id_col='unique_id', time_col='ds', target_col='y', cutoff_col='cutoff', agg_fn=None, weights=None)
```

Evaluate forecast using different metrics.

**Parameters:**

| Name | Type | Description | Default |
| - | - | - | - |
| `df` | <code>pandas, polars, dask or spark DataFrame</code> | Forecasts to evaluate. Must have `id_col`, `time_col`, `target_col` and models' predictions. | *required* |
| `metrics` | <code>list of callable</code> | Functions with arguments `df`, `models`, `id_col`, `target_col` and optionally `train_df`. | *required* |
| `models` | <code>list of str</code> | Names of the models to evaluate. If `None` will use every column in the dataframe after removing id, time and target. Defaults to None. | <code>None</code> |
| `train_df` | <code>pandas, polars, dask or spark DataFrame</code> | Training set. Used to evaluate metrics such as `mase`. Defaults to None. | <code>None</code> |
| `level` | <code>list of int</code> | Prediction interval levels. Used to compute losses that rely on quantiles. Defaults to None. | <code>None</code> |
| `id_col` | <code>[str](#str)</code> | Column that identifies each serie. Defaults to 'unique\_id'. | <code>'unique\_id'</code> |
| `time_col` | <code>[str](#str)</code> | Column that identifies each timestep, its values can be timestamps or integers. Defaults to 'ds'. | <code>'ds'</code> |
| `target_col` | <code>[str](#str)</code> | Column that contains the target. Defaults to 'y'. | <code>'y'</code> |
| `cutoff_col` | <code>[str](#str)</code> | Column that identifies the cutoff point for each forecast cross-validation fold. Defaults to 'cutoff'. | <code>'cutoff'</code> |
| `agg_fn` | <code>[str](#str)</code> | Statistic to compute on the scores by id to reduce them to a single number. Defaults to None. | <code>None</code> |
| `weights` | <code>str, pandas or polars DataFrame</code> | Weights to use when `agg_fn='weighted_mean'`. If 'auto', weights are computed as the sum of the target in the forecast/evaluation window, not from `train_df`. If a dataframe, it must contain `id_col` and `weight`; if `cutoff_col` is present in `df`, it can also contain `cutoff_col` for cutoff-level weights. If `df` has `cutoff_col` but the weights dataframe does not, the same per-series weight is used for every cutoff. | <code>None</code> |

**Returns:**

| Type | Description |
| - | - |
| <code>[AnyDFType](#utilsforecast.compat.AnyDFType)</code> | pandas, polars, dask or spark DataFrame: Metrics with one row per (id, metric) combination and one column per model. If `agg_fn` is not `None`, there is only one row per metric. |


## Related topics

- [Hierarchical Evaluation](/hierarchicalforecast/evaluation.html.md)
- [Probabilistic Forecast Evaluation](/hierarchicalforecast/examples/tourismlarge-evaluation.html.md)
- [NumPy Evaluation](/neuralforecast/losses.numpy.html.md)
- [Amazon Forecast vs StatsForecast](/statsforecast/docs/experiments/amazonstatsforecast.html.md)
- [Geographical Aggregation (Prison Population)](/hierarchicalforecast/examples/australianprisonpopulation.html.md)