Engineering Notes
Thoughts and Ideas on AI by Muthukrishnan
Tag: AI Agents
Least Privilege and Capability Containment Designing Agents That Cannot Exceed Their Mandate
How to apply the principle of least privilege to AI agents through tool allowlisting, permission tiers, and sandboxed execution environments that enforce safety at the architecture level.Mechanism Design Teaching Agents to Cooperate Through Incentives
How to design rules that make self-interested agents collectively produce desirable outcomes, with applications to LLM-based multi-agent systemsIndirect Prompt Injection How Untrusted Content Hijacks AI Agents
How indirect prompt injection attacks compromise AI agents, why agentic systems are uniquely vulnerable, and the defense patterns that actually work.Landmark Based Planning Using Obligatory Subgoals to Guide Search
How landmark-based heuristics guide classical and LLM planners by identifying facts that must be true in every valid planMixture of Agents Building Collaborative LLM Pipelines That Outperform Any Individual Model
How layered LLM collaboration in the Mixture-of-Agents architecture produces outputs that consistently outperform any single model, and how to build it.Corrigibility and Interruptibility Building Agents That Accept Human Override
How to design AI agents that accept correction, allow safe interruption, and remain under human control even as their capabilities growReceding Horizon Planning and Plan Commitment in Agent Reasoning Loops
How agents manage the tradeoff between committing to a plan and staying responsive to new information using receding horizon planningDependency Aware Task Scheduling How Agents Execute Plans as Parallel DAGs
How agents represent plans as dependency graphs and execute independent tasks in parallel for better performance and fault toleranceAnytime Heuristic Search How Agents Find Good Plans Fast and Improve Them Over Time
Anytime heuristic search, including Weighted A* and ARA*, lets agents commit to a suboptimal plan immediately and refine it as time allows, with provable bounds on solution quality at every step.Classical Planning with STRIPS and the Delete Relaxation Heuristic
How STRIPS formalizes agent planning problems and how the delete relaxation trick produces powerful heuristics that guide search toward goals efficientlyWorking Memory Compression and Context Distillation in Long Horizon Agents
How long-running agents compress, distill, and selectively retain working memory to operate effectively within finite context windowsOpponent Modeling and Theory of Mind in Multi-Agent Systems
How agents reason about other agents' beliefs, goals, and strategies, from k-level thinking to neural Theory of Mind and LLM-based recursive reasoningCurriculum Learning and Automatic Curriculum Generation in Reinforcement Learning
How agents learn faster and more robustly by training on the right task at the right time — from hand-crafted curricula to adversarial environment generationMean Field Games and Mean Field Reinforcement Learning for Massive Agent Populations
How mean field theory lets you solve game-theoretic problems with millions of agents by replacing individual interactions with a statistical summary of the crowdCounterfactual Regret Minimization for Solving Imperfect Information Games
How CFR and its variants taught AI to master poker and navigate the fog of war in multi-agent decision makingMaximum Entropy Reinforcement Learning and the Soft Actor-Critic Algorithm
Understand how entropy maximization leads to more robust, exploration-efficient agents and underpins the industry-standard Soft Actor-Critic algorithmCooperative Game Theory and Shapley Values for Fair Credit Assignment in Multi-Agent Systems
Learn how cooperative game theory and Shapley values provide a mathematically principled way to assign credit among collaborating agents, with practical Python implementations and connections to modern LLM agent teams.Reward Machines and Automata-Based Task Specification for AI Agents
How to specify complex, multi-step tasks for AI agents using finite-state automata called reward machines, enabling non-Markovian rewards and compositional task structureSafe Reinforcement Learning Teaches Agents to Optimize Without Violating Constraints
How constrained MDPs, Lagrangian methods, and safety critics enable agents to maximize reward while staying within hard operational boundariesModel-Based Reinforcement Learning How Agents Simulate Experience to Learn Faster
Explore how agents that build internal environment models can plan, simulate, and learn orders of magnitude faster than model-free approachesSuccessor Representations a Map of Where You Are Going
Understand successor representations — the elegant middle ground between model-free and model-based RL that enables fast adaptation and transfer across tasks.Goal-Conditioned Reinforcement Learning and Hindsight Experience Replay Turn Failures Into Training Opportunities
Learn how goal-conditioned RL and Hindsight Experience Replay allow agents to master hard tasks with sparse rewards by treating every failure as a lesson toward a different goal.Spec-Driven Development with spec-kit: Stop Vibe Coding, Start Specifying
A complete tutorial on using GitHub's spec-kit to bring structure to AI-assisted development — from install to your first specification.RLHF and Preference Learning Teaching Agents What Humans Actually Want
Master reinforcement learning from human feedback — the algorithm behind ChatGPT and modern aligned agents — from reward modeling and PPO to Direct Preference Optimization.Distributional Reinforcement Learning and Learning the Full Return Distribution
Move beyond expected returns — learn why modeling the full distribution of rewards unlocks risk-aware agents, better exploration, and state-of-the-art performanceGrounded Language Agents Connecting Words to Actions in the Physical World
How AI agents learn to connect language to perception and physical action, from the symbol grounding problem to modern vision-language-action modelsOffline Reinforcement Learning Training Agents from Fixed Datasets
Learn how agents can master complex tasks from pre-collected experience logs without ever touching a live environment, using conservative Q-learning, implicit Q-learning, and the Decision Transformer.Active Inference and the Free Energy Principle How Agents Minimize Surprise Instead of Maximizing Reward
Explore Karl Friston's Free Energy Principle: a unified theory where agents minimize surprise through belief updating and action, offering an alternative foundation to reward-based reinforcement learningCode Writing Agents and Program Synthesis Teaching AI to Build Its Own Tools
How AI agents generate, execute, and refine code as a reasoning medium, from classical program synthesis to modern REPL-based agent loops and SWE-bench architecturesContinual Learning and Catastrophic Forgetting How Agents Remember Without Forgetting
How AI agents can learn continuously across tasks and environments without overwriting what they already know — the science and practice of lifelong machine learningMulti-Agent Reinforcement Learning Teaching Agents to Cooperate Compete and Coexist
Explore multi-agent reinforcement learning: how multiple RL agents learn simultaneously, coordinate under uncertainty, and produce emergent strategies in cooperative, competitive, and mixed-motive settingsTemporal Abstraction and the Options Framework How Agents Learn to Think in Subgoals
Understand how AI agents escape the curse of shortsightedness by learning reusable subgoals and temporally extended actions through the Options FrameworkAutomatic Prompt Optimization with DSPy Building Self-Tuning Agent Pipelines
Learn how DSPy reframes prompt engineering as a compilation problem, letting agents automatically discover better instructions, few-shot examples, and reasoning strategies through optimizationCausal Reasoning and Intervention Planning in AI Agents
How AI agents can move beyond correlation to understand cause and effect, enabling more robust planning, better tool use, and reliable interventions in the real worldAgent Evaluation and Benchmarking for Measuring What Matters
Learn how to systematically evaluate AI agent performance using benchmarks, metrics, and evaluation frameworks that go beyond simple accuracyRole Specialization and Crew Based Multi Agent Architectures
How assigning specialized roles to AI agents and orchestrating them as a crew produces better outcomes than monolithic single-agent systemsInverse Reinforcement Learning Inferring Goals From Behavior
Learn how inverse reinforcement learning lets AI agents discover hidden reward functions by observing expert behavior, and why it matters for agent alignment and autonomous systemsWhat If GitHub Was Built for AI Agents
Reimagining source code management from the ground up for AI agents, with intent based commits, simulation before merge, agent reputation, and automatic rollback contractsError Recovery and Graceful Degradation in AI Agents
Learn how to build resilient AI agents that recover from failures, degrade gracefully, and maintain reliability in production environmentsImitation Learning Teaching Agents by Watching Experts
Learn how AI agents can acquire complex behaviors by observing and mimicking expert demonstrations, from classical behavioral cloning to modern LLM agent distillationKnowledge Distillation Teaching Smaller Agents From Larger Ones
Learn how knowledge distillation enables large, expensive AI agents to teach smaller, faster ones — reducing cost and latency while preserving capabilityBuilding Reusable Agent Capabilities with Skill Libraries
How AI agents acquire, store, and compose reusable skills—from hierarchical reinforcement learning to LLM-based skill synthesis and the emerging paradigm of lifelong learning agents.Improving Decisions Through Multi-Agent Debate and Deliberation
How AI agents can reach better decisions by arguing with each other—exploring debate protocols, deliberation architectures, and the surprising power of constructive disagreement.Diffusion Models and Iterative Refinement in AI Agent Planning
Explore how diffusion models enable AI agents to generate and refine complex action plans through iterative denoising, revolutionizing long-horizon planning and decision-making.Beam Search and Parallel Decoding for Multiple Solution Paths in AI Agents
Master beam search—a powerful technique for exploring multiple solution paths simultaneously in AI agents, from classical NLP to modern LLM reasoning systems.Reward Shaping and Credit Assignment for Guiding Agent Learning
Master the art and science of designing reward functions and solving the credit assignment problem—the key to training agents that learn efficiently and align with human intentions.Tool Composition and Chaining for Complex Agent Capabilities
Master the art of combining simple tools into sophisticated agent capabilities through composition patterns, chaining strategies, and intelligent orchestration.Curiosity-Driven Learning and Intrinsic Motivation in AI Agents
Discover how curiosity-driven learning enables AI agents to explore, learn, and adapt in sparse-reward environments through intrinsic motivation mechanisms.Constraint Satisfaction Problems for Valid Solutions in Complex Agent Planning
Master constraint satisfaction problems (CSP) - a fundamental technique for agent planning, scheduling, and configuration tasks where finding any valid solution is the goal.Consensus Algorithms for Coordinating Agreement in Distributed Agent Systems
When multiple AI agents need to make a collective decision (agreeing on a shared belief, coordinating a distributed tra...Markov Decision Processes as the Foundation of Sequential Decision-Making
**Markov Decision Processes (MDPs)** provide the mathematical framework for sequential decision-making under uncertaint...How AI Agents Learn from Themselves Through Self-Play and Iterative Refinement
Explore how AI agents improve through self-play and iterative refinement—from AlphaGo's game mastery to modern LLM self-correction techniques.Test-Time Compute and o1-Style Reasoning for Scaling Intelligence Through Inference
**Test-time compute scaling** is the principle that allowing a model to reason longer during inference, rather than jus...Multi-Armed Bandits and the Exploration-Exploitation Dilemma in Agent Learning
Master the fundamental problem of sequential decision-making under uncertainty and learn how AI agents balance trying new actions versus exploiting known rewardsConstrained Decoding and Structured Output for Agent Reliability
Master how modern AI agents ensure reliability through controlled generation—from JSON schemas to grammar constraints and finite-state machines.Verification and Validation Loops for Agent Reliability Through Runtime Checks
Learn how modern AI agents use verification and validation loops to ensure output quality, catch errors at runtime, and build reliable production systems.Symbolic Reasoning and Neural-Symbolic Integration in AI Agents
Modern AI agents face a fundamental challenge: how to combine the pattern-matching prowess of neural networks with the ...Contextual Bandits for Balancing Exploration and Exploitation in Real-Time
Master contextual bandits—the algorithm behind personalized recommendations, A/B testing, and adaptive agents. Learn how to balance exploration and exploitation in real-time decision-making.Human-in-the-Loop Agents for Bridging Autonomy and Oversight
Master the patterns and practices of human-in-the-loop agent systems that balance automation with human judgmentPlanning with Backtracking and Replanning for Resilient Adaptive Agents
Learn how backtracking and replanning enable AI agents to recover from failures and adapt to dynamic environmentsSemantic Routing and Intent Classification in AI Agent Systems
## Concept Introduction **Semantic routing** determines which tool, model, or workflow should handle a user request. I...Meta-Learning and Few-Shot Adaptation in AI Agents
How agents learn to learn, adapting rapidly to new tasks with minimal examples through meta-learning and few-shot learning techniques.Agent Communication Protocols and Message-Passing Patterns for Coordination
Master the protocols and patterns that enable AI agents to communicate, coordinate, and collaborate effectively in multi-agent systemsHow Transformers Power Modern AI Agents with Attention and Reasoning
Understand how attention mechanisms enable AI agents to reason, maintain context, and make intelligent decisions in modern LLM-based systems.Agent Debugging and Observability for Seeing Inside the Black Box
Master the art of understanding, debugging, and monitoring AI agents through tracing, logging, and observability patternsTemporal Difference Learning and Q-Learning as the Engine of Agent Intelligence
**Temporal Difference (TD) Learning** and its most prominent variant, **Q-Learning**, are among the most influential al...Scaling Agent Intelligence Through Specialization with Mixture of Experts
**Mixture of Experts (MoE)** is an architectural pattern built around dynamic routing: instead of one generalist model ...Policy Gradient Methods and Actor-Critic Architectures
Master the mathematics and implementation of policy gradient methods and actor-critic architectures—the foundation of modern agent learning systems.Decision-Making Under Partial Observability with POMDPs
**Partially Observable Markov Decision Processes (POMDPs)** are the mathematical framework for making optimal decisions...Emergence and Self-Organization When Simple Rules Create Complex Intelligence
Discover how simple local interactions between agents can spontaneously produce sophisticated global behaviors—from ant colonies to distributed AI systems.Building Rational Agents with Human-Like Reasoning Using BDI Architecture
Master the BDI architecture pattern that models rational agent behavior through beliefs, desires, and intentions—a bridge between philosophy and practical AI systems.Bellman Equations and Value Functions for Optimal Decision-Making
## Concept Introduction A **value function** V(s) assigns a numerical score to each state s, representing the expected...Vector Embeddings and Semantic Search as the Foundation of Agent Memory
Master the mathematical and practical foundations of vector embeddings—the technology that enables AI agents to remember, search, and reason over vast knowledge bases.Collaborative Problem-Solving in Multi-Agent Systems with the Blackboard Architecture
Explore how the blackboard pattern enables multiple specialized agents to work together on complex problems through shared knowledge spaces and opportunistic reasoning.Goal-Oriented Action Planning for Dynamic Problem Solving in Adaptive Agents
Discover how Goal-Oriented Action Planning (GOAP) enables AI agents to dynamically create flexible plans that adapt to changing conditions, from game NPCs to modern autonomous systems.Auction-Based Task Allocation in Multi-Agent Systems
Learn how auction mechanisms from economics power efficient task distribution in multi-agent systems, from robot swarms to cloud computing.Minimax and Adversarial Search for Decision-Making in Competitive Environments
Master the minimax algorithm and adversarial search - the foundation of game-playing AI agents and competitive decision-making systems.Building Stateful AI Systems with LangGraph and Agentic Workflow Graphs
Learn how to build complex, stateful AI agent systems using graph-based architectures with LangGraph—a paradigm shift from linear chains to cyclic, controllable workflows.Teaching Agents to Learn from Mistakes Through Reflection and Self-Critique
## Concept Introduction **Reflection** in AI agents is a meta-cognitive process where the agent: 1. Executes an actio...Prompt Chaining and Workflow Orchestration for Reliable Multi-Step Agents
When you ask an AI to perform a complex task like "analyze this dataset and create a comprehensive report with visualiz...Behavior Trees for Modular Decision-Making in AI Agents
## Concept Introduction A **Behavior Tree** (BT) is a mathematical model of plan execution represented as a directed a...Hierarchical Task Networks for Planning with Decomposition
## Concept Introduction **Hierarchical Task Networks (HTN)** planning works by decomposing abstract goals into progres...Intelligent Pathfinding for AI Agents with A-Star Search
## Concept Introduction **A\*** is a **best-first search algorithm** that finds the shortest path from a start point t...Monte Carlo Tree Search from Games to LLMs
## Concept Introduction **Monte Carlo Tree Search (MCTS)** is a heuristic search algorithm for finding optimal decisio...Programming Multi-Agent Conversations with AutoGen
## Concept Introduction **AutoGen** is a framework for simplifying the orchestration, optimization, and automation of ...Advanced Problem-Solving with Tree of Thoughts
## Concept Introduction **Tree of Thoughts (ToT)** allows an agent to explore a tree of possibilities rather than foll...An Introduction to Reinforcement Learning for AI Agents
## Concept Introduction Reinforcement Learning (RL) is a paradigm of machine learning where an **agent** learns to mak...Architecting Short-Term and Long-Term Memory in AI Agents
## Concept Introduction AI agents require structured memory to operate effectively across turns and sessions. Without ...Beyond the Training Data with Retrieval-Augmented Generation for AI Agents
## Concept Introduction **Retrieval-Augmented Generation (RAG)** gives an AI agent an external knowledge base that it ...Ensuring Agent Safety with Constitutional AI Guardrails
## Concept Introduction **Constitutional AI (CAI)** is a method for training an AI to supervise itself. Rather than re...How AI Agents Use Tools with Function Calling
## Concept Introduction **Tool Use** is the mechanism that allows a language model to interact with and affect the out...Improving Agent Reliability with Plan-and-Solve Prompting
## Concept Introduction **Plan-and-Solve (PS) Prompting** is a structured approach to AI agent reliability. Instead of...Self-Improvement and Autoformalization in AI Agents That Build Themselves
## Concept Introduction We have explored how agents can reason, plan, remember, act, and collaborate. The final, ultim...Solving Complex Problems with AI Agent Swarms
## Concept Introduction An **AI Agent Swarm** is a system composed of a large number of relatively simple, often ident...Task Allocation for Machine Teamwork with the Contract Net Protocol
## Concept Introduction The **Contract Net Protocol** is a decentralized method for assigning tasks in a multi-agent s...Understanding Multi-Agent Behavior with Game Theory and Nash Equilibrium
## Concept Introduction **Game Theory** is the mathematical study of strategic decision-making among rational, self-in...Fusing Reasoning and Action in LLM Agents with ReAct
### Concept Introduction **ReAct** (Reason + Act) is a prompting framework where the LLM generates output in a specifi...Chain-of-Thought Reasoning and Giving LLMs a Thinking Process
### Concept Introduction **Chain-of-Thought (CoT) Reasoning** is a prompting strategy that encourages a model to decom...The Cognitive Loop of AI Agents with Planner Executor and Memory
## Concept Introduction The **Planner-Executor-Memory (PEM)** pattern is an architectural pattern for autonomous agent...