Xiaotian Han — Posts https://ahxt.github.io/ 2025-06-17T00:00:00Z Xiaotian Han Group Relative Policy Optimization (GRPO) https://ahxt.github.io/post/2025-06-17-grpo/ 2025-06-17T00:00:00Z 2025-06-17T00:00:00Z A practical derivation of Group Relative Policy Optimization (GRPO), including its objective, implementation details, and relationship to PPO. Rotary Position Embeddings (RoPE) https://ahxt.github.io/post/2025-03-18-rope/ 2025-03-18T00:00:00Z 2025-03-18T00:00:00Z A visual and mathematical guide to rotary position embeddings (RoPE), from the rotation formulation to efficient implementation. Optimizers: SGD, Momentum, and Adam https://ahxt.github.io/post/2025-01-22-optimizer/ 2025-01-22T00:00:00Z 2025-01-22T00:00:00Z A from-scratch guide to SGD, momentum, and Adam, with PyTorch implementations and performance comparisons. LLM Technical Reports https://ahxt.github.io/post/2025-01-21-llm-tech-reoport/ 2025-01-21T00:00:00Z 2025-01-21T00:00:00Z A continuously updated collection of technical reports and notes on large language models. Scaling Test-Time Compute https://ahxt.github.io/post/2024-12-30-inference-time-scaling-exp/ 2024-12-30T00:00:00Z 2024-12-30T00:00:00Z Experiments on scaling test-time compute for language models, comparing majority voting, best-of-N, and FLOPs trade-offs. Softmax: Math, Implementation, and GPU Optimization https://ahxt.github.io/post/2024-12-19-softmax/ 2024-12-19T00:00:00Z 2024-12-19T00:00:00Z A mathematical and practical guide to softmax, including gradients, PyTorch implementations, and Triton GPU optimization. Optimizing Cross-Entropy Loss for LLMs https://ahxt.github.io/post/2024-12-12-celoss/ 2024-12-12T00:00:00Z 2024-12-12T00:00:00Z A guide to implementing and optimizing cross-entropy loss for large language models with PyTorch and Triton. Graph Convolution ≈ Mixup https://ahxt.github.io/post/2024-11-20-graph-convolution-mixup/ 2024-11-20T00:00:00Z 2024-11-20T00:00:00Z An intuitive explanation of graph convolution as a specialized form of Mixup, with experiments and visualizations. FlashAttention with Trainable Bias Terms https://ahxt.github.io/post/2024-10-20-attention/ 2024-10-20T00:00:00Z 2024-10-20T00:00:00Z A derivation and implementation of FlashAttention with trainable bias terms, including forward and backward passes.