Engineering Notes

Engineering Notes

Thoughts and Ideas on AI by Muthukrishnan
27 Feb 2026

Grounded Language Agents Connecting Words to Actions in the Physical World

How AI agents learn to connect language to perception and physical action, from the symbol grounding problem to modern vision-language-action models
27 Feb 2026

Offline Reinforcement Learning Training Agents from Fixed Datasets

Learn how agents can master complex tasks from pre-collected experience logs without ever touching a live environment, using conservative Q-learning, implicit Q-learning, and the Decision Transformer.
26 Feb 2026

Active Inference and the Free Energy Principle How Agents Minimize Surprise Instead of Maximizing Reward

Explore Karl Friston's Free Energy Principle: a unified theory where agents minimize surprise through belief updating and action, offering an alternative foundation to reward-based reinforcement learning
25 Feb 2026

Code Writing Agents and Program Synthesis Teaching AI to Build Its Own Tools

How AI agents generate, execute, and refine code as a reasoning medium, from classical program synthesis to modern REPL-based agent loops and SWE-bench architectures
24 Feb 2026

Continual Learning and Catastrophic Forgetting How Agents Remember Without Forgetting

How AI agents can learn continuously across tasks and environments without overwriting what they already know — the science and practice of lifelong machine learning
23 Feb 2026

Multi-Agent Reinforcement Learning Teaching Agents to Cooperate Compete and Coexist

Explore multi-agent reinforcement learning: how multiple RL agents learn simultaneously, coordinate under uncertainty, and produce emergent strategies in cooperative, competitive, and mixed-motive settings
22 Feb 2026

Temporal Abstraction and the Options Framework How Agents Learn to Think in Subgoals

Understand how AI agents escape the curse of shortsightedness by learning reusable subgoals and temporally extended actions through the Options Framework
21 Feb 2026

Automatic Prompt Optimization with DSPy Building Self-Tuning Agent Pipelines

Learn how DSPy reframes prompt engineering as a compilation problem, letting agents automatically discover better instructions, few-shot examples, and reasoning strategies through optimization
20 Feb 2026

Causal Reasoning and Intervention Planning in AI Agents

How AI agents can move beyond correlation to understand cause and effect, enabling more robust planning, better tool use, and reliable interventions in the real world
19 Feb 2026

Agent Evaluation and Benchmarking for Measuring What Matters

Learn how to systematically evaluate AI agent performance using benchmarks, metrics, and evaluation frameworks that go beyond simple accuracy