Blog

An in-depth look at how knowledge distillation transfers capabilities from large teacher models to smaller, efficient student models covering response-based, feature-based, and attention-based techniques.

Explores how reinforcement learning addresses the core limitations of standard LLM training for long-horizon tasks covering MDPs, policy gradients, RLHF, DPO, and hierarchical planning for multi-step decision-making.

Inter-annotator agreement is a core measure of data quality in machine learning. When multiple annotators label the same data, agreement levels reveal how consistently a task can be interpreted and how reliable the resulting labels are. Low agreement often indicates unclear guidelines, task ambiguity, or expertise mismatches rather than annotator error. Measuring and monitoring inter-annotator agreement helps teams detect label noise early, improve annotation design, and produce datasets that lead to more stable and generalizable models.

A recent paper from MIT CSAIL researchers Alex L. Zhang, Tim Kraska, and Omar Khattab introduces Recursive Language Models (RLMs) as a solution. Published as a preprint in late 2025 and updated in January 2026, RLMs enable models to handle inputs far beyond standard context limits, often with better accuracy and comparable or lower cost.