unsloth/LFM2-2.6B-Exp-GGUF
Text Generation
•
3B
•
Updated
•
1.36k
•
8
Best Codes
Best-codes
AI & ML interests
Text Generation.
Recent Activity
liked
a model
3 days ago
unsloth/LFM2-2.6B-Exp-GGUF
liked
a model
3 days ago
LiquidAI/LFM2-2.6B-Exp-GGUF
liked
a model
3 days ago
LiquidAI/LFM2-350M-GGUF
Organizations
None yet
liked
5
models
3 days ago
liked
a
Space
3 days ago
liked
3
models
3 days ago
liked
2
Spaces
3 days ago
liked
a
model
3 days ago
reacted to
codelion's
post with 🔥
3 days ago
Post
5624
Introducing Dhara-70M: A diffusion language model that achieves 3.8x higher throughput than autoregressive models!
Key findings from our research on optimal architectures for small language models:
→ Depth beats width: 32 layers outperforms 12 layers at the same parameter count
→ Best-in-class factuality: 47.5% on TruthfulQA
→ 10x training efficiency using WSD (Warmup-Stable-Decay) conversion
→ Canon layers add only 0.13% parameters but improve reasoning
We trained on 1B tokens using the optimal 50-30-20 dataset mix (PDFs + filtered web + educational content), then converted to diffusion with just 100M additional tokens.
Blog: https://huggingface.co/blog/codelion/optimal-model-architecture
Model: codelion/dhara-70m
Key findings from our research on optimal architectures for small language models:
→ Depth beats width: 32 layers outperforms 12 layers at the same parameter count
→ Best-in-class factuality: 47.5% on TruthfulQA
→ 10x training efficiency using WSD (Warmup-Stable-Decay) conversion
→ Canon layers add only 0.13% parameters but improve reasoning
We trained on 1B tokens using the optimal 50-30-20 dataset mix (PDFs + filtered web + educational content), then converted to diffusion with just 100M additional tokens.
Blog: https://huggingface.co/blog/codelion/optimal-model-architecture
Model: codelion/dhara-70m
- 1 reply
liked
a
model
3 days ago
upvoted
a
collection
3 days ago
liked
a
model
3 days ago
liked
a
model
7 days ago
liked
3
models
12 days ago
