Engineering Notes
Thoughts and Ideas on AI by Muthukrishnan
Tag: Llm
Mixture of Agents Building Collaborative LLM Pipelines That Outperform Any Individual Model
How layered LLM collaboration in the Mixture-of-Agents architecture produces outputs that consistently outperform any single model, and how to build it.Automatic Prompt Optimization with DSPy Building Self-Tuning Agent Pipelines
Learn how DSPy reframes prompt engineering as a compilation problem, letting agents automatically discover better instructions, few-shot examples, and reasoning strategies through optimizationImproving Decisions Through Multi-Agent Debate and Deliberation
How AI agents can reach better decisions by arguing with each other—exploring debate protocols, deliberation architectures, and the surprising power of constructive disagreement.Beam Search and Parallel Decoding for Multiple Solution Paths in AI Agents
Master beam search—a powerful technique for exploring multiple solution paths simultaneously in AI agents, from classical NLP to modern LLM reasoning systems.How AI Agents Learn from Themselves Through Self-Play and Iterative Refinement
Explore how AI agents improve through self-play and iterative refinement—from AlphaGo's game mastery to modern LLM self-correction techniques.How Transformers Power Modern AI Agents with Attention and Reasoning
Understand how attention mechanisms enable AI agents to reason, maintain context, and make intelligent decisions in modern LLM-based systems.Advanced Problem-Solving with Tree of Thoughts
## Concept Introduction **Tree of Thoughts (ToT)** allows an agent to explore a tree of possibilities rather than foll...Beyond the Training Data with Retrieval-Augmented Generation for AI Agents
## Concept Introduction **Retrieval-Augmented Generation (RAG)** gives an AI agent an external knowledge base that it ...How AI Agents Use Tools with Function Calling
## Concept Introduction **Tool Use** is the mechanism that allows a language model to interact with and affect the out...Fusing Reasoning and Action in LLM Agents with ReAct
### Concept Introduction **ReAct** (Reason + Act) is a prompting framework where the LLM generates output in a specifi...