Stochastic Differential Equations

• ODEs are equations defined using derivatives of variables. In the simplest form, how a physical object position as a function of time $x(t)$ is updated can be defined using some function $f(t,x(t))$.
• What does this equation mean?
• Write the differential equation in a shorter way, and explain it.
  • $dx(t)=f(t,x(t))dt$
  • This states that infinitesimal change in position $x(t)$ is equal to the function $f(t,x(t))$ scaled by infinitesimal difference $dt$
• An equation of the form $y=f(x)$ gives the value of y according to function f at any point inside the support of the function. While a differential equation is a vector field that gives the change of position at any point and at any time t.
• The way to solve a differential equation is to take an initial point, and sum the infinitesimal changes over time. $x(T)=x(0)+{\int }_0^{T}dx(t)=x(0)+{\int }_0^{T}f(t,x(t))dt$
• To make the integral tractable, above formulation can be discretized to $\Delta x(t)$ and $\Delta t$, and turned into a summation.

Before talking about SDEs, let’s talk about Stochasticity over time: Weiner Process.

• Weiner Process models position of a particle with a purely random trajectory in a Euclidean space. Mathematically, we can represent the position as: $dx(t)=ϵdt$, where $ϵ$ is a probability distribution.
• The above equation is a differential equation with a non-deterministic function that follows some distribution, and we equate the infinitesimal change in position to random move in space scaled times the infinitesimal time that move is followed.
• Since $ϵ$ is a random variable, each position also follows the same distribution and, we treat each position in time itself as a random variable, governed by equation above.
• We can choose any distribution for $ϵ$, but Good ol’ friend Gaussian doesn’t like that, and is staring right at us through the Central limit hole! We model the probability of random movement with a standard Gaussian, $ϵ\sim \mathcal{N}(0,I)$.
• We choose amount of infinitesimal time to be $\sqrt{dt}$. We’ll understand the reason why shortly.
• Thirdly, we want the starting position of the differential equation to be $x_0=0$.

$$\begin{array}{c}{x}_T=x_0+{\int }_{t=0}^{T}d{x}_t={\int }_0^Tϵ\sqrt{dt}\end{array}$$

• Since $ϵ$ is a random variable, we don’t know its integral. Rather we look at the first two moments:
  • $\mathbb{E}[x_T]=\mathbb{E}\left[{\int }_{t=0}^{T}d{x}_t\right]={\int }_{t=0}^T\mathbb{E}[ϵ]\sqrt{dt}=0$
  • $\mathbb{V}[x_T]=\mathbb{V}\left[\int x_T\right]={\int }_0^{T}dt\mathbb{V}[ϵ]=T$.
• So, all paths stay around initial position 0, and start to spread further proportional to the length of the path. Also, the derivation turned out to be straightforward due to our choice of infinitesimal time $\sqrt{dt}$.
• The process defined so far is an elementary stochastic process called Wiener process $W_t$ that defines the path of infinitesimal movement of particle.

add a brownian motion picture

Let’s define the properties of the Wiener process:

• Initial position: $W_0=0$
• Independent increments: $\text{Cov}[W_{t+u}-W_s,W_s]=0,u\ge 0,s\le t$. Any future motion is independent of past motion. This can be seen as continuous time analog of iid steps in random walks.
• Stationary Gaussian increments: $W_{t+u}-W_t\sim \mathcal{N}(0,u)⟹d{W}_t\sim \mathcal{N}(0,\mathbf{dt})$. The displacement of the particle over an interval is Gaussian distributed with mean 0 and variance proportional to the interval. We just proved this above.
• Continuous time paths but nowhere differentiable. Paths are continuous, but this being a stochastic process, Wiener process is not differentiable because $W_{t+u}-W_t\sim \mathcal{N}(0,u)$, so ${\lim }_{u\to 0}\frac{W_{t+u}-W_t}{u}=\frac{\sqrt{u}}{u}\to \infty$.
• Other properties include the covariance structure: $\mathbb{E}[W_s{W}_t]=\min (s,t)$, Markov property: $p(W_t|\text{past})=p(W_t|W_s)$ for $s<t$, Martingale property: $\mathbb{E}[W_t|{\mathcal{F}}_s]=W_s$.

We write the Brownian motion SDE by equating the infinitesimal change in position with Brownian motion which is defined by a normally distributed random variable scaled by $\sqrt{dt}$.

$$d{x}_t=d{W}_t=ϵ\sqrt{dt},ϵ\sim \mathcal{N}(0,\mathbf{I})$$

Interestingly, Wiener process is continuous but non-differentiable. Intuitively, we know that differential of a continuous function in classical calculus measures the rate of change of the function on an infinitesimal change in domain. But for Wiener process, $d{W}_t=ϵ\sqrt{s}$, where $ϵ$ is a random variable (usually Gaussian) which does not depend on time. So, no matter how much we zoom into the domain to get the infinitesimal value, we’ll always have a random brownian motion on the finer axis, which explains the fractal property of Brownian motion.

Enter fractal picture from wikipedia

SDE

An SDE is just combination of ODE and stochastic process. We defined a stochastic differential equation which is the infinitesimal change in random variable $X_t$ as sum of deterministic change ${\mu }_{t}dt$ and stochastic process (infinitesimal Wiener process) ${\sigma }_{t}d{W}_t\sim \mathcal{N}(0,{\sigma }_t^{2}dt)$

$$d{X}_t=\underset{\text{deterministic}}{\underbrace{{\mu }_{t}dt}}+\underset{\text{stochastic}}{\underbrace{{\sigma }_{t}d{W}_t}}$$

This is also known as Ito drift-diffusion process. Solving the SDE, we’ll assume constant ${\mu }_t=\mu$ and ${\sigma }_t=\sigma$.

$$\begin{array}{rl}{X}_T& ={\int }_{t=0}^{T}d{X}_t\\ & ={\int }_{t=0}^T\mu dt+\sigma d{W}_t\\ & =\mu T+\sigma {\int }_0^{T}d{W}_t\\ & =\mu T+\sigma W_T\end{array}$$

$W_T\sim \mathcal{N}(0,T)$ is a random variable. Let’s calculate analytical mean and variance of the process.

$$\begin{array}{r}\mathbb{E}[X_T]=\mathbb{E}[\mu T+\sigma W_T]=\mu T+\sigma \mathbb{E}[W_T]=\mu T\\ \mathbb{V}[X_T]=\mathbb{V}[\mu T+\sigma W_T]={\sigma }^2\mathbb{V}[W_T]={\sigma }^{2}T\end{array}$$

• Ito’s Process: Ito’s lemma that tells how to differentiate an SDE: Ito’s (Di)Lemma
• Ornstein-Uhlenbeck process: Solving (Some) SDEs
• Fokker-Planck Equation
  • FokkerPlanck_Equation

Questions that we're going to answer in this document:

1. Write Fokker-Planck equation. Understand the physical intuition behind it.
2. Derive FP equation, and reverse SDE using FP equation.
3. Why FP equation is necessary to show that stochastic equation used in score-based models preserve the intermediate distribution?
4. Derive PF-ODE from the SDE and FP equation.

For the drift function $f:{\mathbb{R}}^d\times \mathbb{R}\to {\mathbb{R}}^d$ and diffusion function $g:{\mathbb{R}}^d\times \mathbb{R}\to {\mathbb{R}}^{d\times d}$, we let $\{X_t{\}}_{t\in [0,T]}$ evolve with the forward SDE,

$\boxed{d{X}_t=f(X_t,t)dt+g(X_t,t)d{W}_t}$

Why is $g\in {\mathbb{R}}^{d\times d}$?

Notice that $W_t$ is a d-dimensional Wiener process. So, there exists d-independent noise sources of $\{d{W}_t^{(j)}{\}}_{j\in [0,d)}\in {\mathbb{R}}^d$. To find out the effect of these noise channels on coordinate $i$ of $X_t$, $[gd{W}_t{]}_i={\sum }_j{g}_{ij}d{W}_t^{(j)}$. Thus, $g$ is the map that takes $d-$dim noise vector and produces $d-$dim perturbation, where each row $i$ of g calculates effect of the noise sources on coordinate $i$.

Time reversal of diffusion/reverse SDE

• Why do we want to reverse an SDE?

Score estimation using SDE

Stochastic Calculus

Introduction to Stochastic Calculus | Ji-Ha’s Blog
Itô Processes and The Fundamental Theorem of Stochastic Calculus
ItoSDE_Tutorial
LW - Blog