Implementing Advanced Differential Equation Solvers and Neural ODEs with Diffrax and JAX

A Coding Guide to Implement Advanced Differential Equation Solvers, Stochastic Simulations, and Neural Ordinary Differential Equations Using Diffrax and JAX

This guide details the integration of Diffrax and JAX for solving complex dynamical systems using adaptive solvers like Tsit5. It demonstrates how to utilize JAX’s XLA backend to perform batched simulations and train Neural Ordinary Differential Equations from noisy synthetic data.

Why This Matters

Numerical solvers for differential equations are traditionally decoupled from machine learning frameworks, creating bottlenecks in gradient-based optimization of physical systems. Diffrax addresses this by providing a JAX-native library that supports automatic differentiation through solvers, allowing engineers to integrate physical constraints directly into neural network architectures. This approach is critical for modeling systems where underlying laws are partially known, enabling the transition from purely black-box models to physics-informed machine learning.

Key Insights

• Adaptive step-size solvers like Tsit5 and Dopri5 utilize PID controllers to maintain precision while optimizing computational steps (Diffrax, 2026).
• Dense interpolation via sol.evaluate allows querying solution states at arbitrary time points without the overhead of additional solver steps.
• PyTree-based states enable the modeling of complex mechanical systems, such as spring-mass-dampers, using structured data instead of flat arrays.
• Stochastic Differential Equations (SDEs) are implemented using VirtualBrownianTree for consistent Brownian motion paths across different time intervals.
• Neural ODEs utilize Equinox-based MLPs and Optax optimizers to learn system dynamics from data, achieving convergence on MSE loss through JIT-compiled training loops.

Working Examples

Implementation of a logistic growth ODE using an adaptive Tsit5 solver with dense interpolation enabled.

def logistic(t, y, args):
    r, k = args
    return r * y * (1 - y / k)

sol_logistic = diffrax.diffeqsolve(
    diffrax.ODETerm(logistic),
    diffrax.Tsit5(),
    t0=0.0, t1=10.0, dt0=0.05,
    y0=jnp.array(0.4),
    args=(2.0, 5.0),
    saveat=diffrax.SaveAt(ts=jnp.linspace(0.0, 10.0, 300), dense=True),
    stepsize_controller=diffrax.PIDController(rtol=1e-6, atol=1e-8)
)

Stochastic Differential Equation simulation for an Ornstein-Uhlenbeck process using VirtualBrownianTree.

def solve_ou(key):
    bm = diffrax.VirtualBrownianTree(t0=0.0, t1=6.0, tol=1e-3, shape=(1,), key=key)
    terms = diffrax.MultiTerm(diffrax.ODETerm(ou_drift), diffrax.ControlTerm(ou_diffusion, bm))
    sol = diffrax.diffeqsolve(terms, diffrax.EulerHeun(), t0=0.0, t1=6.0, dt0=0.01, y0=jnp.array([0.0]), args=(1.2, 1.5))
    return sol.ys

Practical Applications

• High-throughput dynamical system analysis: Using jax.vmap to run thousands of parallel simulations for sensitivity analysis; avoids the performance penalty of Python-level loops.
• Physics-Informed Neural Networks: Training Neural ODEs to recover latent dynamics from noisy sensor data; pitfall includes using non-differentiable solvers which prevents backpropagation.

References:

• https://www.marktechpost.com/2026/03/19/a-coding-guide-to-implement-advanced-differential-equation-solvers-stochastic-simulations-and-neural-ordinary-differential-equations-using-diffrax-and-jax/