1 Normalizing Flows

2 Flow Matching

As we saw, learning forward and reverse SDE using score function changed the dynamics under which diffusion-based models operate. Differential equations turned out to be the underlying language. Because diffusion is just transporting particles from one state to another where start and end states follows certain distribution. The question arose, “whether we can use ODEs to follow the exact same trajectory as the learned SDE in score-based models?”. Flow matching based generative models constructed a suitable ODE,
solving which gets the flow of the equation. Solving the ODE implies learning the associated vector field.

So, the goal of flow matching based generative model is to design a vector field that transforms a sample-able distribution (say Gaussian) to target distribution .

Vector field and ODE

Vector field assigns a vector to each position in the space, and represents instantaneous movement at all locations, and we want to create an ODE, solution of which gives a trajectory that follows the vector field. Formally, with the initial condition, .

Before moving forward, We should also be confident about the existence of a solution to such an ODE. Fortunately, it’s already proven that when the vector field is in with a bounded derivative then the ODE has a unique solution given by flow.

Flow map, denotes the location of a particle in the space at time t. Solution to the ODE, , with initial condition . Trajectory of the ODE is recovered via . Thus, solving an ODE, or finding the vector field, or flow map is equivalent. Intuitively, Flow map is the solution to the ODE defined by vector fields.

We don’t know the final distribution, so we need a way to estimate the flow map using only samples from . An intermediate solution is to pick a conditional sample and let all trajectories end at . We define a conditional interpolating probability path as the set of distribution such that . Think of probability path as a straight line from noise distribution and data distribution . Taking the example of Gaussian probability path, trajectory is defined a simple straight line trajectory and are the noise schedulers such that . So eventually, , and . For intermediate time, is shrinking Gaussian.

The probability path can define intermediate distributions along the trajectory, but we don’t know the marginal vector field corresponding to all points . So, we can define the conditional vector field for every data point such that solving the ODE gives the conditional probability path .

Taking an example of conditional Gaussian probability path .

Todo

replace it with something I make

So, we have the conditional vector field, but as explained before, what we need is marginal vector field. Define marginal vector field as the average of all conditional fields for all data points weighed by how likely it is to get from . We use continuity equation to get marginal from conditional vector fields.

Informally, it states that change of probability mass at each location over time equals the net inflow (negative divergence) of mass change according to the vector field at x (Each particle follows the field scaled by total mass currently residing at ). Interpret as as vector-valued function, as scalar valued function, then , and

We know that conditional vector field correctly transports conditional probability path . But we need to know whether marginal vector field will also correctly solve the marginal probability path.

Define marginal vector field as the average over all data points that takes velocity weighted by the probability of generating from , and we have to prove that it follows the marginal probability path, i.e . We can prove this using continuity equation:

Continuity equation is satisfied when marginal vector field equals the field defined above, and transports to such that the endpoint of the flow . Now, we see how do we learn the marginal vector field .

We parametrize the vector field as neural network approximation and use MSE loss as the flow matching loss:

But we can’t directly sample from , so we use ancestral sampling .

Informally, the loss says to draw a random time , then draw a random point from the data set, and sample , and compute , and then finally compute the mean squared error between the output from the neural network and true marginal vector field . But computing the marginal vector field requires that the integral is tractable, which is often untrue. Solution to this is to use a known term and replace with unknown term in the loss formula, and then use the surrogate loss. We will, of course, have to prove that minimizing the surrogate loss is equivalent to minimizing the original one. We use the conditional velocity field , and define the new loss as:

In the next result, we show that marginal flow matching loss equals the conditional flow matching loss up to a constant, . Therefore, . Proof works by expanding the mean squared error into components and removing constants:

Now, let’s take a look at the second summand:

We plug the result into the equation for ,

Therefore, flow matching training consists of minimizing the conditional flow matching loss. We can see that the loss equation turns out to be simple regression, equivalent to supervised learning. We also thus, don’t need to simulate any ODE during training, and can directly train the neural network on the loss.

For the Gaussian conditional probability paths, the loss function simplifies to

3 Flow and Score Matching

We will aim to prove the equivalence between learning score and the velocity field . Similar to conditional and marginal velocity field, we can define conditional score and marginal score .

Prove marginal score formulation .

For gaussian probability paths we know that . We can write marginal velocity vector field with respect to score as by using the conditional velocity field . This establishes the fact that learning vector field is equivalent to learning score .
Further note that, both conditional vector field and conditional score used to derive marginals are linear functions of z and x. Thus, marginalizing term fall naturally as an average (weighted) of conditional term multiplied by how likely it is to arrive at noisy term given clean data () given all data points . Formally, plugging the conditional term back into marginalizing, we get a linear reparameterization of the posterior mean .

add a diagram for conditional and marginal score field

To rephrase, any quantity that can recover can be used to learn vector field or score field. We interpret the term that gives estimate of clean data z given noisy data as Denoiser. For conditional scenario, is equal to the clean data , .

Therefore, denoising diffusion models, flow matching models and score models, all are equivalent and learning one allows to recover the other.

3.1 Training Score Models

We proved using continuity equation that trajectory follows the desired probability path if it follows the ODE . Learning the vector field allows us to follow the trajectory that leads to . For diffusion models, we use SDEs to learn the score distribution.

Proposition

We will similarly prove that we can take flow ODE and extend it using stochastic dynamics with diffusion coefficient such that

Setting for the above SDE implies that marginal distribution is preserved across the whole process.

We use Fokker-Planck equation to prove the above proposition.

where is Laplacian of distribution defined as .

Proof

Explain Fokker-planck. Attach a proof in appendix.

Langevin dynamics is a special case of above SDE with (constant) distribution, which implies . In conclusion, for langevin dynamics, . In practice, for any Markov Process, the dynamics converge to stationary distribution even if we start and , then .

Training score matching: We use a neural network to learn the score. We define the marginal score matching loss and conditional loss as

We use similar training dynamics as flow matching models, i.e. train a conditional score model and prove that which implies . So, training the network to predict conditional score is equivalent to predicting the score and for the minimizer .

For Gaussian probability path,

Summary

  1. We defined the marginal and conditional score.
  2. Proved relationship to velocity vector field.
  3. To get desired probability path, extended the ODE to SDE for any diffusion coefficient .
  4. Using Fokker-Planck equation, proved that marginal distribution is preserved for .
  5. Defined and proved that learning a conditional score network finds the optimal minimizer.
  6. After training, sample from the SDE to obtain approximate sample given .

4 References

  1. Helbling, Alec. “A Visual Introduction to Rectified Flows.” (2025) [Link]
  2. Gao, Ruiqi and Hoogeboom, Emiel and Heek, Jonathan and Bortoli, Valentin De and Murphy, Kevin P. and Salimans, Tim. “Diffusion Meets Flow Matching: Two Sides of the Same Coin.” (2025) [Link]
  3. Gagneux, Anne and Martin, Ségolène and Emonet, Rémi and Bertrand, Quentin and Massias, Mathurin. “A Visual Dive into Conditional Flow Matching”. (2025) [Link]
  4. Fjelde, Tor and Mathieu, Emile and Dutordoir, Vincent. “An Introduction to Flow Matching”. (2024) [Link]
  5. Scott H. Hawley. “Flow With What You Know”. (2024) [Link]

  1. Lipman, Yaron, et al. “Flow matching for generative modeling.” arXiv preprint arXiv:2210.02747 (2022).
  2. Liu, Xingchao, Chengyue Gong, and Qiang Liu. “Flow straight and fast: Learning to generate and transfer data with rectified flow.” arXiv preprint arXiv:2209.03003 (2022).
  3. Albergo, Michael S., and Eric Vanden-Eijnden. “Building normalizing flows with stochastic interpolants.” arXiv preprint arXiv:2209.15571 (2022).
  4. Albergo, Michael, Nicholas M. Boffi, and Eric Vanden-Eijnden. “Stochastic interpolants: A unifying framework for flows and diffusions.” Journal of Machine Learning Research 26.209 (2025): 1-80.
  5. Lipman, Yaron, et al. “Flow matching guide and code.” arXiv preprint arXiv:2412.06264 (2024).
  6. Li, Tianhong, and Kaiming He. “Back to basics: Let denoising generative models denoise.” Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
  7. Esser, Patrick, et al. “Scaling rectified flow transformers for high-resolution image synthesis.” Forty-first international conference on machine learning. 2024.
  8. Kornilov, Nikita, et al. “Optimal flow matching: Learning straight trajectories in just one step.” Advances in Neural Information Processing Systems 37 (2024): 104180-104204.
  9. Chen, Ricky TQ, and Yaron Lipman. “Flow matching on general geometries.” International Conference on Learning Representations. Vol. 2024. 2024.