Pretraining and Transformers for Accelerating Solutions to Partial Differential Equations

<p dir="ltr">Solving Partial Differential Equations (PDEs) is the cornerstone of many fields of science, engineering, and mathematics. There are many challenges associated with developing solutions to complex sets of equations. Namely, no universal mathematical theory of PDEs exists, so equations are often difficult or impossible to solve analytically. Computational techniques have been developed based on analytical theory to compute solutions, overcoming analytical limitations. These computational techniques, however, often require significant computational resources, human input, or both. Specifically, constructing the spatial mesh over which equations are solved normally requires human input and intuition. Additionally, solving the equations given a mesh often requires tailor-made numerical solvers, and can be prohibitively slow. This thesis presents four works to address challenges in computational science. First, Mesh Deep Q Network uses Deep Reinforcement Learning to remove vertices in a mesh while maintaining accuracy in coarse property calculation. Physics Informed Contrastive learning utilizes pretraining to improve model performance across multiple systems simultaneously. Lastly, Physics Informed Token transformer and Explain Like I’m Five explore text-based multimodality in PDEs through end-to-end training and using pretrained Large Language models, respectively. Multimodality proves to be a valuable approach to incorporate system information in a flexible way. These works provide an important step towards developing large scale, general purpose PDE solvers.</p>