TimesFM 2.5: Time Series Forecasting with Transformers

TimesFM 2.5 is a pretrained time-series foundation model from Google Research. It provides accurate zero-shot forecasts across different domains, forecasting horizons, and temporal granularities.

This notebook demonstrates:

1. Zero-shot forecasting on synthetic and real-world time series
2. Quantile (probabilistic) predictions with uncertainty estimation
3. Attention backend comparison (eager, SDPA, Flash Attention 2/3, Flex Attention)
4. Fine-tuning with PEFT (LoRA) on a custom dataset
5. Forecasting with exogenous covariates using in-context linear regression (XReg)

Key advantages of using TimesFM 2.5 via Transformers:

• Native support for all attention backends (SDPA, Flash Attention 2, Flex Attention) through the standard attn_implementation API
• Standard from_pretrained / save_pretrained workflow
• Easy fine-tuning with PEFT adapters (LoRA, QLoRA, etc.)
• Compatible with the broader Transformers ecosystem

Setup

Install the required packages. We need the latest transformers from main for TimesFM 2.5 support.

1. Zero-Shot Forecasting

TimesFM 2.5 can forecast time series out-of-the-box without any training. Let's load the pretrained model and run inference on some synthetic signals.

Forecasting synthetic signals

The model accepts a list of 1D tensors as input (variable-length time series) and returns:

• mean_predictions: point forecasts of shape (batch_size, horizon_length)
• full_predictions: quantile forecasts of shape (batch_size, horizon_length, num_quantiles)

Forecasting real-world data

Let's try the model on the Electricity Transformer Temperature (ETTh1) dataset, a commonly used benchmark for time series forecasting.

Batch forecasting multiple columns

TimesFM 2.5 accepts a list of variable-length time series, making it easy to forecast multiple series at once.

2. Attention Backend Comparison

One of the key benefits of using TimesFM 2.5 via Transformers is native support for different attention implementations:

• Eager: Standard PyTorch attention (baseline)
• SDPA: PyTorch's scaled_dot_product_attention (fused kernels)
• Flash Attention 2: Memory-efficient attention via kernels-community/flash-attn2
• Flash Attention 3: Latest Flash Attention via kernels-community/flash-attn3
• Flex Attention: PyTorch's flex_attention with torch.compile (first call incurs a one-time compilation cost)

All backends produce numerically equivalent outputs but differ in speed and memory usage. We benchmark each on a batch of 8 series of length 2048.

3. Fine-Tuning with PEFT (LoRA)

TimesFM 2.5 can be fine-tuned on custom datasets using parameter-efficient methods like LoRA. This is especially useful when you have domain-specific data and want to adapt the model without updating all 200M parameters.

We'll fine-tune on a retail demand forecasting dataset (weekly store sales), the same dataset used in the Chronos-2 quickstart notebook. The goal is to forecast the next 13 weeks of sales for each store.

Compare zero-shot vs fine-tuned

Let's compare the zero-shot model with the LoRA fine-tuned model on the held-out test set for a few stores.

Loading and reusing a saved LoRA adapter

The LoRA adapter weights can be saved and loaded separately from the base model, making them easy to share and version.

4. Forecasting with Exogenous Covariates (XReg)

TimesFM 2.5 can incorporate exogenous covariates (known future variables) to improve forecasts using a post-hoc in-context linear regression approach (xreg_lib):

1. Fit a ridge regression on the context: target ~ covariates
2. Subtract the regression fit from the context to get residuals
3. Feed the residuals to TimesFM for forecasting
4. Combine: final_forecast = regression_horizon_prediction + TimesFM_residual_forecast

This works best when covariates have an approximately linear relationship with the target (e.g., price → sales, marketing spend → demand). For complex nonlinear relationships (e.g., temperature → electricity load), neural approaches like Toto that pass covariates through the network are more appropriate.

We demonstrate this using a synthetic retail demand dataset with a known price effect, adapted from the TimesFM covariates PR.

Summary

In this notebook, we demonstrated the key capabilities of TimesFM 2.5 in Transformers:

• Zero-shot forecasting: The model provides accurate forecasts on unseen data without any training, including quantile uncertainty estimates.
• Batch forecasting: Multiple time series of different lengths can be forecast in a single call.
• Attention backends: Native support for eager, SDPA, Flash Attention 2/3, and Flex Attention, with numerically equivalent outputs and different performance characteristics.
• PEFT fine-tuning: LoRA adapters can be applied to adapt the model to domain-specific data (retail sales) with minimal trainable parameters.
• Exogenous covariates (XReg): Known future variables (e.g., price) can be incorporated via in-context ridge regression to improve forecast accuracy without retraining. The linear regression can be extended with polynomial features (e.g., price², price³) or interaction terms to capture non-linear covariate effects while keeping the closed-form solve. For highly non-linear relationships (e.g., temperature → electricity load), neural approaches like Toto that pass covariates through the network are more appropriate.

For more information, see:

• Model card
• Transformers documentation
• Original paper