Listen to this Post
Introduction
For years, the world of large language models (LLMs) has been dominated by English. Despite the global spread of AI technology, other languages — even widely spoken ones like French — have been left behind, with fewer datasets, weaker benchmarks, and less specialized research. This imbalance has limited the potential of multilingual AI and created a performance gap that affects global accessibility to high-quality AI tools.
The Luth project takes aim at this problem head-on, delivering specialized small-scale French language models that not only close this gap but also maintain — and in some cases improve — English performance through clever training and model merging. By focusing entirely on French data while leveraging smart engineering techniques, Luth proves that linguistic diversity and high performance can go hand in hand.
the Original
The Luth project introduces two compact, non-reasoning French-specialized causal LLMs, both instruction-tuned exclusively on French datasets. Recognizing the scarcity of French training data, the team built Scholar, a high-quality dataset drawn from Baccalauréat and Classes Préparatoires (CPGE) exams, covering mathematics, physics, computer science, and general scientific knowledge. This dataset was combined with other curated French corpora to create Luth-SFT, a 338-million-token resource designed for post-training in French instruction following.
To ensure French specialization without losing cross-lingual skills, the models were trained using the Axolotl framework with DeepSpeed over three epochs, employing full fine-tuning rather than LoRA for better results. Learning rates were carefully tuned, and training focused only on assistant outputs to optimize efficiency.
The team then applied model merging with MergeKit, using SLERP and linear merging methods to blend the specialized model with its base version. This allowed the retention — and in some benchmarks, improvement — of English performance alongside strong French results. For example, Luth-0.6B-Instruct and Luth-1.7B-Instruct not only achieved state-of-the-art performance in French for their size but also outperformed their base models in several English benchmarks.
The evaluation process used LightEval, with added custom French benchmark tasks adapted from popular English sets like Math-500, MMLU, IFEval, and GPQA-Diamond. The results showed remarkable gains:
In French, Luth-1.7B scored 64.00 on math-500-fr, far surpassing its base Qwen3-1.7B score of 60.80.
In English, it reached 70.00 on math-500-en, again beating the base score.
This cross-lingual improvement suggests that focusing on a low-resource language can have positive spillover effects on English capabilities — a finding with big implications for multilingual AI.
The authors conclude that their approach can be replicated for other underrepresented languages worldwide by combining targeted datasets, domain-specific resources, and model merging techniques. All code, models, and datasets are released openly to encourage further research and development.
What Undercode Say:
From a technical and industry perspective, the Luth project is more than a French-language success story — it’s a proof-of-concept for solving one of AI’s most persistent problems: the English monopoly on training data.
Why It Matters
In today’s AI ecosystem, performance gaps between English and non-English languages are not just a technical inconvenience; they represent a cultural and accessibility barrier. Millions of people face subpar AI results simply because their native language is underrepresented in training corpora. Luth’s results show that deliberate linguistic focus can break this cycle without sacrificing global applicability.
The Data Strategy
The cornerstone of Luth’s success is dataset craftsmanship. Instead of relying solely on translations of English benchmarks, the team invested in building authentically French datasets, rich in subject-specific depth. This not only increased model accuracy but also created resources that reflect French cultural and academic contexts — something simple translation could never achieve.
By incorporating multilingual sources like smoltalk2 and CroissantLLM alongside their Scholar dataset, they struck a balance between diversity and domain expertise. This ensures the model can handle general instruction following while excelling in academic problem-solving.
Model Merging as a Game-Changer
One of the standout innovations is strategic model merging. Using SLERP merging allowed the team to blend fine-tuned French models with their base Qwen models, producing cross-lingual gains. In fact, some merged models outperformed both parent versions in English and French benchmarks, defying the common belief that specialization comes at the cost of generalization.
This technique is potentially transformative for other languages: a Japanese, Arabic, or Swahili-focused project could follow the same approach and expect similar performance boosts.
Implications for Small Models
Luth is also a statement about small model efficiency. At just 0.6B and 1.7B parameters, these models compete with — and sometimes beat — much larger counterparts. This makes them cheaper to train, easier to deploy, and more sustainable, while still delivering state-of-the-art results in their target domain.
The Future Outlook
If other language communities adopt similar strategies, we could see the multilingual AI gap narrow significantly in the next few years. This would mean not only more equitable AI access but also improved global AI robustness, as models trained on linguistically diverse data tend to develop richer semantic understanding overall.
In short, Luth demonstrates that with precision engineering, careful dataset creation, and smart merging techniques, AI can become truly multilingual without compromising performance.
✅ Fact Checker Results
Claim: Luth models outperform base models in French benchmarks. True — Benchmark tables confirm significant gains.
Claim: Specializing in French harmed English performance. False — Some English scores improved after specialization.
Claim: All resources are open-source. True — Code and datasets are publicly available on GitHub.
🔮 Prediction
Within the next two to three years, more AI research teams will adopt Luth’s specialized-plus-merging approach, leading to a wave of high-performing small models for underrepresented languages. This could result in a global multilingual AI renaissance, where smaller but more diverse models outperform generic giants in real-world tasks.
🕵️📝✔️Let’s dive deep and fact‑check.
References:
Reported By: huggingface.co
Extra Source Hub:
https://www.linkedin.com
Wikipedia
OpenAi & Undercode AI
Image Source:
Unsplash
Undercode AI DI v2
🔐JOIN OUR CYBER WORLD [ CVE News • HackMonitor • UndercodeNews ]
📢 Follow UndercodeNews & Stay Tuned:
𝕏 formerly Twitter 🐦 | @ Threads | 🔗 Linkedin | 🦋BlueSky | 🐘Mastodon
