Alibaba Group has introduced a breakthrough architecture of large language models-QwenLong-L1, capable of efficiently processing texts of almost unlimited length. This development opens up broad prospects for the application of artificial intelligence in such industries as finance, law, research and corporate analytics.
A new standard for LLM in processing long documents
The key feature of QwenLong-L1 is the ability to quickly analyze multi-page documents: from financial reports and legal contracts to research publications. Traditionally, large language models have experienced difficulties when working with texts exceeding 4000 tokens. The new architecture eliminates this limitation with multi-step training and a hybrid reward system.
Innovative learning process
QwenLong-L1 goes through three stages:
- Supervised fine tuning — SFT) - training on samples with long contexts.
- Reinforcement learning (RL) - incrementally increases the length of input data to adapt to increasingly complex tasks.
- Selecting complex examples-encourages the model to learn advanced reasoning strategies and self-test.
Unique rating system
The model uses a hybrid reward system that combines strict formal rules for checking accuracy, and semantic evaluation conducted by another LLM, which allows you to take into account acceptable variations in responses in complex contexts.
During testing on seven DocQA sets, QwenLong-L1 showed outstanding results:
The QwenLong-L1-32B demonstrated performance comparable to Claude-3.7 Sonnet Thinking from Anthropic, outperforming the OpenAI o3-mini and Qwen3-235B-A22B.
The lighter version of QwenLong-L1-14B outperformed Google's Gemini 2.0 Flash Thinking and even the larger Qwen3-32B model.
Experts pay special attention to the fact that the model has acquired the skills of deep analysis of long texts:
accurate correlation of responses with parts of the document;
- formulation of interim conclusions;
- independent error detection and correction;
- verification and refinement of responses.
Alibaba has already published the source code and weights for QwenLong-L1, giving developers, researchers, and enterprise users access to cutting-edge technology. This solution can radically transform areas such as audit, legal support, banking analytics and intelligent service assistants.
