Learnable Orthogonal Decomposition for Non-Regressive Prediction for PDE

What is it about?

Modeling the spatio-temporal evolution of complex physical systems remains a fundamental challenge in both deep learning and scientific computing. Although recent approaches—such as Transformers and Neural Operators—have demonstrated strong performance in learning PDE solutions, their reliance on auto-regressive forecasting introduces substantial computational overhead and leads to error accumulation over time. In this paper, we introduce Learnable Orthogonal Decomposition (LOD), a non-regressive framework that combines the principles of classical Proper Orthogonal Decomposition (POD) with modern deep learning techniques.

Featured Image

Photo by Pawel Czerwinski on Unsplash

Photo by Pawel Czerwinski on Unsplash

Why is it important?

Unlike the aforementioned methods that rely on auto-regressive predictions, we aim to address these limitations by developing a non-regressive framework. Inspired by POD, we introduce Learnable Orthogonal Decomposition (LOD) as a novel approach to tackle these challenges. LOD effectively decomposes and processes the temporal and spatial structures inherent in PDE data. Instead of constructing a single set of fixed spatial bases, LOD adopts a parameter-wise POD strategy: at each time step, POD is applied to an ensemble of PDE solutions across varying parameters, yielding a time-indexed set of spatial bases. These time-varying bases then initialize a learnable dictionary within the model. The deep learning model then focuses exclusively on the system’s temporal dynamics, predicting the coefficients associated with these bases and thereby enabling non-regressive, full-sequence prediction of the entire time series in a single pass.

Perspectives