MIT Unveils SEAL: The AI Framework That Learns and Updates Itself Instantly

MIT laboratories have unveiled a framework called **Self-Adapting Language Models (SEAL)**. This isn’t merely another tool for training models; SEAL enables AI to learn from itself. A language model equipped with SEAL receives new information and, instead of merely absorbing it, initiates a process of self-editing.

The model autonomously:

— Reconstructs the acquired information in multiple ways.
— Generates its own data based on this information for fine-tuning.
— Writes optimization instructions for itself.
— Applies data augmentation and updates through gradient-based approaches.

Subsequently, using these self-generated educational resources, the model engages in reinforcement learning with the help of the **ReST^{EM}** algorithm developed by the team at Google DeepMind. The core idea of ReST^{EM} revolves around running numerous learning cycles and selecting the most effective outcomes. This allows the model to learn **efficiently** by choosing the best updates for itself.

The MIT team tested SEAL on Alibaba’s Qwen2.5-7B model. The results were striking. Initially, the «barebones» **Qwen2.5-7B** (represented by the red line on the graph) lagged behind the robust **GPT-4.1** (represented by the green line). However, after applying SEAL and allowing the model to undergo just two cycles of self-learning, it not only caught up but surpassed GPT-4.1 in its ability to assimilate new knowledge.

Although it seems feasible to endlessly pursue self-learning, the phenomenon known as **catastrophic forgetting** began to emerge. In simple terms, after numerous update cycles, the model started to lose previously acquired knowledge. Researchers acknowledge the necessity for a mechanism that allows the model to retain its old skills while acquiring new ones.

*References:*

Official post and details: [Self-Adapting Language Models (SEAL)](https://jyopari.github.io/posts/seal);

Article on arXiv: [2506.10943 Self-Adapting Language Models](https://arxiv.org/abs/2506.10943);

Source code on GitHub: [Continual-Intelligence/SEAL](https://github.com/Continual-Intelligence/SEAL).

Subscribe to our Telegram channel [BotHub AI News](https://t.me/bothub) to stay updated on IT news.