Interests

Flow-based generative models (flow matching, diffusion/score models, and related formulations)
Sampling and learning algorithms
One-step and few-step models; distillation and consistency-style training
Optimal transport perspectives
Connections with other generative paradigms
Discrete flow and diffusion models, and applications to language modeling, protein modeling
Applications including vision, text, multimodal modeling, and scientific settings
what’s after diffusion (markov-process based iterative noising and denoising), score (navigating the data manifold through score approximation), and flow matching (approximating the optimal transport map)? Energy based matching?

Questions

Why high-dimensional gaussians’ density is concentrated on a sphere?
What’s fisher divergence, and what’s the geometry like? Compare it to more general bregman divergence?
What’s tweedie’s identity? How to derive it? Explain it in plain words.
Write the continuous time version of langevin dynamics? Why is there a square-root 2 in the diffusion term? Why is it useful for diffusion models?
Score Matching
- What’s the difference between score matching (Hyvärinen and Dayan, 2005) and denoising score matching (Vincent, 2011)?
- What’s the issue with Score matching?
- How does sliced score matching sidestep the problems in score matching? And what’s the problem with sliced score matching?
- Write the loss function for denoising score matching (DSM). Express denoiser. How does it connect with tweedie’s identity?
- How does NCSN improves upon previous iteration of score matching? Write training objective of NCSN.
- can i explain the difference between VE-SDE, and VP-SDE?
- Why is NCSN VE-SDE, and DDPM VP-SDE?
- How are NCSN and DDPM losses connected?
- What’s the difference between denoising score matching and NCSN?
- Illustrate DDPM, NCSN.
- Proof of affine-drift conditional forward kernel closed-form analytical formulation as gaussian.
- Why does forward marginal density converge to prior distribution?
- Write reverse SDE dynamics equation. Explain the reason for diffusion coefficient in the drift coefficient term.
- How does f,g in forward and reverse SDE vary with time?
- What’s PF-ODE? How to convert between other representations of the same thing? i.e. going from SDE to ODE to discretization.
- What’s the algorithm/pseudocode for annealed langevin dynamics? How is it different from Unadjusted langevin algorithm?
- Proof of fokker-planck.
- How does $σ$ vary from $t = 0 \to t = T$ ? Is it more at the start or the end?
Flow matching
- NF, continuous NF, NODE
- Illustrate flow matching models.
Efficient solvers and samplers
- What are the different samplers for ODEs and SDEs that are used?
- What are the different ODE solvers used for sampling the diffusion models?
- Write the equation for euler-maruyama?
- What’s the sde solver beside euler-maruyama?
- What are the main takeaways from EDM paper?
Guided diffusion
- How will you explain DPS really quickly?
- guidance: classifier-based, classifier free
Multimodal diffusion
- What’s the problem with CLIP? What are other better multimodal encoders?
Architecture
- Lay out the architecture for U-Net and DiTs. How are they different? Which to prefer? What’s the pitfalls?
- Write the architecture for Image generators: SD2, SD3, Flux 1-2, Nano Banana
- Illustrate the architecture for multimodal DiT
- Write the architecture for Video Gen models. Meta Movie gen, google omni
- What’s the current SOTA architecture for Any-to-Any generative model?
Discrete diffusion or flow matching
- Illustrate discrete flow matching
- What’s the difference between MDLM
- Block diffusion modeling
- CTMC theory. Why is it useful?
What is the design space over which diffusion models can be categorized?