Memento-Skills: A new framework that helps AI write skills automatically without retraining the model.

One of the major challenges when deploying automated AI agents is adaptability to new environments. Often, when the system changes, engineering teams have to retrain large language models (LLMs) or rebuild manual skills, a process that is time-consuming and costly.

 

Memento-Skills – a new framework developed by researchers from various universities – is seeking to address this problem. What makes Memento-Skills unique is that it allows AI agents to develop and update their skills independently without needing to retrain the underlying model.

According to Jun Wang, co-author of the study, Memento-Skills adds continuous learning capabilities to existing agent systems like OpenClaw or Claude Code. Instead of directly changing the language model, the framework acts as an external memory that can evolve over time, helping agents improve their capabilities based on real-world feedback.

This is especially important for businesses deploying AI agents in production environments. If fine-tuning models or building skills manually were required, operating costs would increase significantly. Memento-Skills avoids both of these problems by allowing the system to improve itself over time.

Why is self-evolution of AI agents important?

Large language models often become 'frozen' after deployment—that is, their parameters don't change. This limits them to the initial training data and the amount of information within the context window.

The addition of external memory allows agents to learn continuously without needing to retrain the entire model. However, current methods still have many limitations. Much of the system relies on manually designed skills, or simply generates optimal, prompt-based text instructions that don't truly improve performance.

 

Furthermore, many systems struggle to retrieve relevant skills. Most use semantic similarity-based search methods. However, semantic similarity doesn't necessarily mean the skill is behaviorally useful. For example, a system might mistakenly retrieve a 'reset password' script when processing a 'refund' request simply because the two documents share similar business terminology.

How do Memento-Skills work?

To overcome these limitations, researchers developed Memento-Skills as an agent system capable of designing skills independently.

Instead of simply saving conversation history, Memento-Skills creates skills as structured markdown files. Each skill includes three main components: a description of its function and usage, instructions to help the model reason, and executable code or supporting scripts to solve the task.

The system learns continuously through a "Read-Write Reflective Learning" mechanism. When faced with a new task, the agent retrieves the most appropriate behavioral skills, then executes the task and receives feedback. Based on this result, the system will automatically adjust existing skills or create new skills if needed.

If a task fails, the system will analyze the cause and directly fix the relevant code or prompt. Additionally, the skill router is updated using reinforcement learning, enabling more effective skill selection in the future.

To ensure safety in a production environment, Memento-Skills also uses an automated testing system. Before saving changes, the system creates test cases and checks the results to avoid errors.

 

Practical results: AI learns significantly faster.

The research team tested Memento-Skills on two complex benchmarks: GAIA and Humanity's Last Exam (HLE). The entire system used Gemini-3.1-Flash as the underlying model.

The results show that the self-evolving system outperforms the static skill library. On the GAIA benchmark, accuracy increased from 52.3% to 66%. On HLE, performance even doubled from 17.9% to 38.7%.

Specifically, the new skill adapter increases the task completion rate to 80%, compared to 50% when using the traditional retrieval method.

Remarkably, the system started with just 5 basic skills such as web searching and terminal operations. After the learning process, the agent automatically expanded to 41 skills in GAIA and 235 skills in the HLE benchmark.

When should businesses implement this?

Researchers suggest that Memento-Skills is best suited to structured workflow environments where tasks are repetitive and skills are reusable.

If tasks are disjointed and have little connection to each other, the potential for skill transfer will be limited. Conversely, when tasks have a similar structure, the agent can learn faster and significantly improve performance.

However, the research team also cautioned that complex physical agents or long-term tasks still require further research. In these cases, multi-agent systems may be a more suitable solution.

The future of self-improving AI agents

As AI agents begin to rewrite their own skills and production code, governance and security issues become more critical than ever. While Memento-Skills already has automated testing mechanisms, businesses still need a comprehensive evaluation and monitoring system.

According to the research team, the self-improvement process needs guidance rather than allowing AI to change uncontrollably. Feedback and evaluation are crucial to ensuring the system evolves in a safe and efficient manner.

Memento-Skills represents a new leap forward in the AI ​​agent era: instead of simply executing tasks, AI can now learn, correct errors, and expand its capabilities over time. This could help businesses build more flexible and sustainable AI systems in the future.