Architecture
MoE
- [2401.04088] Mixtral of Experts: Introduces Mixtra-8B MoE models that outperformed GPT3.5 using 2-of-8 expert strategy.
Reasoning
- “Stealing Reasoning Traces from Proprietary LLM APIs”, arxiv 2608
- Fine-tuning LFM2.5-1.2B-Instruct with GRPO – Leonie Monigatti
Inference
- Inside vLLM: Anatomy of a High-Throughput LLM Inference System - Aleksa Gordić
- Why your local LLM feels dumber than it is - Machine Learning, LLMs, & AI - Level1Techs Forums
- LLM Inference Calculator
Speculative Decoding
- How speculative decoding makes LLMs go brrr – Leonie Monigatti
- Accelerating Generative AI with PyTorch II: GPT, Fast – PyTorch
- Speculative Decoding From Scratch | SR Cookbooks
- Speculative Decoding Tutorial | Pramodith Dissects
- An Introduction to Speculative Decoding for Reducing Latency in AI Inference | NVIDIA Technical Blog
- Exploring Speculative Decoding in vLLM on AMD GPUs | vLLM Blog
- [2211.17192] Fast Inference from Transformers via Speculative Decoding
- [2302.01318] Accelerating Large Language Model Decoding with Speculative Sampling
- [2305.09781] SpecInfer: Accelerating Generative Large Language Model Serving with Tree-based Speculative Inference and Verification
- [2401.10774] Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads
- [2402.02057] Break the Sequential Dependency of LLM Inference Using Lookahead Decoding
- [2404.19124] Accelerating Production LLMs with Combined Token/Embedding Speculators
- [2404.19737] Better & Faster Large Language Models via Multi-token Prediction
- [2401.15077] EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty
- [2406.16858] EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees
- [2503.01840] EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test
- [2503.09573v3] Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models
- [2602.06036] DFlash: Block Diffusion for Flash Speculative Decoding
- [2607.05147] DSpark: Confidence-Scheduled Speculative Decoding with Semi-Autoregressive Generation
To-Read
- Jane Street Blog - Using group theory to explore the space of positional encodings for attention
- [2511.05963] Next-Latent Prediction Transformers Learn Compact World Models
- [2605.27734] Learn from your own latents and not from tokens: A sample-complexity theory,
- [2606.06479] Pretraining Recurrent Networks without Recurrence
- [2604.12946] Parcae: Scaling Laws For Stable Looped Language Models
- [2606.18206] Fixed-Point Reasoners: Stable and Adaptive Deep Looped Transformers
- The Annotated Kolmogorov-Arnold Network (KAN) | Alex L. Zhang
- [2511.05963] Next-Latent Prediction Transformers Learn Compact World Models
- State of RL for reasoning LLMs | A. Weers
- RLHF Book: Reinforcement Learning from Human Feedback and LLM Post-Training
- NanoGPT Pro — Multi-Architecture NanoGPT Training & Evaluation Suite
- [2512.07805] Group Representational Position Encoding
- [The Smol Training Playbook - a Hugging Face Space by HuggingFaceTB](
- https://huggingface.co/spaces/HuggingFaceTB/smol-training-playbook)
- [2505.24832] How much do language models memorize?: Got ICML honorable mention.