EddyFormer: Accelerated Neural Simulations of Three-Dimensional Turbulence at Scale
Simulating turbulence in fluid dynamics is a massive computational challenge. Traditional methods like Direct Numerical Simulation (DNS) are incredibly accurate but also prohibitively expensive, limiting their use for large-scale or real-time applications.
We develop EddyFormer, a neural network architecture that accelerates turbulence simulation by up to 30x while maintaining the accuracy of high-resolution DNS. EddyFormer combines the strengths of spectral methods for high accuracy and the Transformer architecture for scalability.
Our model learns to reproduce complex turbulent dynamics across a wide range of physical conditions, even generalizing to domains four times larger than what it was trained on. This work opens up new possibilities for using machine learning in critical scientific and engineering applications, from weather forecasting to aerospace design.
Architecture
EddyFormer's architecture is designed to efficiently capture the complex, multi-scale nature of turbulence. It integrates the high accuracy of spectral methods with the scalability and long-range dependency modeling of the Transformer architecture.
A core innovation is our Spectral Element Method (SEM) tokenization. Instead of treating the simulation domain as a uniform grid of pixels, we partition it into a smaller number of coarse elements. Each element, or 'token', contains a rich, high-order polynomial representation of the flow within it. This compact representation allows the attention mechanism to operate efficiently, capturing long-range interactions between distant parts of the flow without the quadratic complexity of standard Vision Transformers.
The architecture further employs a separation of scales, inspired by classical Large Eddy Simulation methods. The model processes the flow through two specialized streams: a Large Eddy Scale (LES) stream that uses attention on the SEM tokens to model global, coherent structures, and a Sub-Grid Scale (SGS) stream that uses localized spectral convolutions to capture the fine-scale, local eddy dynamics. This two-stream design allows each component to specialize, leading to a more accurate and efficient simulation.
The EddyFormer architecture, showing the SEM tokenization and the two-stream (LES and SGS) design.
Results on Isotropic Turbulence
We validate EddyFormer on three-dimensional homogeneous isotropic turbulence (Re ≈ 94), a key benchmark for capturing the core physics of the turbulent energy cascade. The Q-criterion visualization below, which highlights vortex structures, shows that EddyFormer accurately captures the vortical features of the flow, while the baseline F-FNO model fails to resolve these fine structures.
The model also preserves key physical invariants, such as the energy spectrum and structure functions, over long simulation rollouts. These results indicate that EddyFormer can achieve a level of accuracy comparable to a 256³ DNS. For a detailed analysis of all error metrics and physical statistics, please refer to the full paper.
Q criterion of reference solution.
Q criterion of our EddyFormer.
Q criterion of baseline F-FNO.
Domain Generalization
A key feature of EddyFormer is its ability to generalize to larger domains than it was trained on. We trained the model on a 2D Kolmogorov flow and tested it on domains up to 4x larger. The results show that EddyFormer maintains its accuracy and correctly captures the energy spectrum, unlike baseline models which fail to generalize.
Illustration of the attention mask used by domain generalization.
Energy spectra of model predictions on 2x and 4x larger domains of KF4.
Results on The Well Dataset
To test EddyFormer's robustness on a wider range of challenging scenarios, we benchmark it on several turbulence cases from The Well dataset. This dataset includes complex, physically-driven flows such as buoyancy-driven instabilities and shear flows. Across all tested cases, EddyFormer consistently outperforms baseline models, successfully capturing complex phenomena where other models fail to converge to a physical solution.
Density of a Rayleigh-Taylor flow.
EddyFormer's prediction.
Baseline FNO's prediction.
Density field of a Rayleigh-Benard instability test case.
Visualization of the tracer field for a shear flow test case.