Can AI build better AI ? // AI agents can design better agents
This twitter thread describes a new research area called “Automated Design of Agentic Systems” (ADAS) and introduces a method called “Meta Agent Search.” Here’s a summary of the key points:
1. ADAS aims to automatically design powerful AI agents, potentially replacing manually designed solutions.
2. Meta Agent Search is introduced as the first ADAS method that enables automated agent design in code, allowing for the creation and use of any conceivable component.
3. The method involves a meta agent that iteratively programs new agents based on an archive of previous designs.
4. The algorithm was evaluated on the ARC logic puzzle task, where it discovered novel agents that outperformed state-of-the-art hand-designed agents.
5. Testing across various reasoning and problem-solving domains (math, science) showed that the algorithm consistently discovered agents outperforming baselines.
6. Surprisingly, the invented agents maintained superior performance when transferred across domains and models, demonstrating robustness and generality.
7. The researchers suggest that since the meta agent used in ADAS is itself an agent, it could potentially be improved through ADAS, leading to a recursive improvement process (meta-meta agents and beyond).
In essence, this research explores the possibility of AI systems designing better AI systems, potentially leading to more efficient and powerful agent designs across various domains.
Researchers are investing substantial effort in developing powerful general-purpose agents, wherein Foundation Models are used as modules within agentic systems (e.g. Chain-of-Thought, Self-Reflection, Toolformer). However, the history of machine learning teaches us that hand-designed solutions are eventually replaced by learned solutions. We formulate a new research area, Automated Design of Agentic Systems (ADAS), which aims to automatically create powerful agentic system designs, including inventing novel building blocks and/or combining them in new ways. We further demonstrate that there is an unexplored yet promising approach within ADAS where agents can be defined in code and new agents can be automatically discovered by a meta agent programming ever better ones in code. Given that programming languages are Turing Complete, this approach theoretically enables the learning of any possible agentic system: including novel prompts, tool use, control flows, and combinations thereof. We present a simple yet effective algorithm named Meta Agent Search to demonstrate this idea, where a meta agent iteratively programs interesting new agents based on an ever-growing archive of previous discoveries. Through extensive experiments across multiple domains including coding, science, and math, we show that our algorithm can progressively invent agents with novel designs that greatly outperform state-of-the-art hand-designed agents. Importantly, we consistently observe the surprising result that agents invented by Meta Agent Search maintain superior performance even when transferred across domains and models, demonstrating their robustness and generality. Provided we develop it safely, our work illustrates the potential of an exciting new research direction toward automatically designing ever-more powerful agentic systems to benefit humanity [paper]
The design of new alloys is a multi-scale problem that requires a holistic approach that involves retrieving relevant knowledge, applying advanced computational methods, conducting experimental validations, and analyzing the results, a process that is typically slow and reserved for human experts. Machine learning (ML) can help accelerate this process, for instance, through the use of deep surrogate models that connect structural and chemical features to material properties, or \textit{vice versa}. However, existing data-driven models often target specific material objectives, offering limited flexibility to integrate out-of-domain knowledge and cannot adapt to new, unforeseen challenges.
Here, we overcome these limitations by leveraging the distinct capabilities of multiple AI agents that collaborate autonomously within a dynamic environment to solve complex materials design tasks. The proposed physics-aware generative AI platform, AtomAgents, synergizes the intelligence of large language models (LLM) the dynamic collaboration among AI agents with expertise in various domains, including knowledge retrieval, multi-modal data integration, physics-based simulations, and comprehensive results analysis across modalities that includes numerical data and images of physical simulation results.
The concerted effort of the multi-agent system allows for addressing complex materials design problems, as demonstrated by examples that include autonomously designing metallic alloys with enhanced properties compared to their pure counterparts. Our results enable accurate prediction of key characteristics across alloys and highlight the crucial role of solid solution alloying to steer the development of advanced metallic alloys. Our framework enhances the efficiency of complex multi-objective design tasks and opens new avenues in fields such as biomedical materials engineering, renewable energy, and environmental sustainability.
