A study conducted by Shanghai Jiao Tong University and the SII Generative AI Research Lab highlights that training large language models (LLMs) for complex tasks may not necessitate extensive datasets. The research introduces a framework named LIMI (Less Is More for Intelligent Agency), asserting that effective machine autonomy stems from the strategic curation of high-quality demonstrations rather than sheer data quantity.
In their experiments, the researchers discovered that a curated dataset of only 78 examples enabled LLMs to achieve superior performance compared to models trained on thousands of examples when evaluated against key industry benchmarks. This finding could be significant for enterprises facing challenges in data collection due to scarcity or cost.
The concept of “agency” is defined by the researchers as the capability of AI systems to act autonomously by identifying issues, proposing solutions, and engaging with various tools and environments. Current methodologies often link higher intelligence in AI to larger datasets, which can complicate training processes and demand considerable resources. However, prior studies suggest that achieving training goals in LLMs can be possible with limited data, highlighting the idea that quality outperforms quantity.
The LIMI framework emphasizes the importance of collecting high-quality demonstrations of autonomous behavior. Each demonstration includes a user query and a detailed trajectory illustrating the AI’s thought processes, interactions, and actions taken to fulfill the query. To develop their dataset, the researchers utilized queries based on real-world scenarios and expanded these using GPT-5 to generate further instances.
Testing using the AgencyBench benchmark and other established metrics demonstrated that the LIMI-trained model outperformed several baseline models, scoring an average of 73.5% compared to 45.1% for the highest-performing conventional model. The study showed that the LIMI’s approach to training, with significantly less data, marks a shift in developing autonomous AI systems.
The researchers shared their findings and released the associated code and model weights, offering a new strategy for organizations to create tailored AI agents using smaller, high-quality datasets that align with specific needs.
Source: https://venturebeat.com/ai/new-ai-training-method-creates-powerful-software-agents-with-just-78