Hugging Face H4

Team

company

https://github.com/huggingface/alignment-handbook

Activity Feed

AI & ML interests

Aligning LLMs to be helpful, honest, harmless, and huggy (H4)

Recent Activity

alozowski authored a paper 1 day ago

YourBench: Easy Custom Evaluation Sets for Everyone

SaylorTwift new activity 9 days ago

HuggingFaceH4/MATH-500:adds_eval_yaml

SaylorTwift new activity 19 days ago

HuggingFaceH4/aime_2024:Create eval.yaml

View all activity

sergiopaniego

posted an update about 10 hours ago

Post

386

ICYMI, you can fine-tune open LLMs using Claude Code

just tell it:
“Fine-tune Qwen3-0.6B on open-r1/codeforces-cots”

and Claude submits a real training job on HF GPUs using TRL.

it handles everything:
> dataset validation
> GPU selection
> training + Trackio monitoring
> job submission + cost estimation
when it’s done, your model is on the Hub, ready to use

read more about the process: https://huggingface.co/blog/hf-skills-training

sergiopaniego

posted an update about 17 hours ago

Post

425

We just released TRL v0.26.0!

It comes packed with updates:
> Agent training with tools in GRPO
> New CISPO & SAPO losses + reasoning rewards
> vLLM quantization in colocate mode
> Dataset shuffling in SFT
> Lots of NEW examples
> Tons of fixes and documentation improvements

3 replies

sergiopaniego

posted an update 1 day ago

Post

2302

NEW: @EssentialAI just released Rnj-1, their first 8B model.

You can easily fine-tune it with GRPO using TRL to add reasoning capabilities to a compact mode

Free Colab link: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_rnj_1_instruct.ipynb

More free TRL notebooks: https://huggingface.co/docs/trl/main/en/example_overview#notebooks

alozowski

authored a paper 1 day ago

YourBench: Easy Custom Evaluation Sets for Everyone

Paper • 2504.01833 • Published Apr 2 • 22

sergiopaniego

posted an update 6 days ago

Post

2765

Want to get started with fine-tuning but don’t know where to begin? 🤓☝️

We’re expanding our collection of beginner-friendly free Colab notebooks so you can learn and fine-tune models using TRL at no cost

🔬 Check out the full list of free notebooks: https://huggingface.co/docs/trl/main/en/example_overview#notebooks

🔬 If you want more advanced content, we also have a lot to cover in the community tutorials: https://huggingface.co/docs/trl/community_tutorials

And now the obvious question: what would you like us to add next?

sergiopaniego

posted an update 7 days ago

Post

2309

NEW: @mistralai released a fantastic family of multimodal models, Ministral 3.

You can fine-tune them for free on Colab using TRL ⚡️, supporting both SFT and GRPO

Link to the notebooks:
- SFT: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/sft_ministral3_vl.ipynb
- GRPO: https://colab.research.google.com/github/huggingface/trl/blob/main/examples/notebooks/grpo_ministral3_vl.ipynb
- TRL and more examples: https://huggingface.co/docs/trl/index

2 replies

sergiopaniego

posted an update 9 days ago

Post

2139

ICYMI, transformers v5 is out!

Grab a coffee ☕ and go read the announcement blog https://huggingface.co/blog/transformers-v5

SaylorTwift

in HuggingFaceH4/MATH-500 9 days ago

adds_eval_yaml

#5 opened 9 days ago by

SaylorTwift

sergiopaniego

posted an update 10 days ago

Post

3075

want to use open models easily through an API?

Inference Providers might be exactly what you’re looking for sooo here’s a complete beginner-friendly walkthrough 🧐

https://www.youtube.com/watch?v=oxwsizy1Spw

2 replies

sergiopaniego

posted an update 13 days ago

Post

1722

nanochat is now in transformers!

The LLM by @karpathy is officially in the library, and we wrote a blog covering: how did we port the model, differences from the original, and how to run or train it.

go read it 🤓

nanochat-students/transformers

sergiopaniego

posted an update 15 days ago

Post

3943

you gotta go fast and go read the latest blog by @ror et al. explaining Continuous Batching in depth

https://huggingface.co/blog/continuous_batching

sergiopaniego

posted an update 17 days ago

Post

1709

Interested in RL training environments?

We just released a beginner-friendly walkthrough notebook!

Train a model to play Wordle using TRL + OpenEnv (TextArena) + GRPO + vLLM.

happy learning! 🌱

Notebook: https://github.com/huggingface/trl/blob/main/examples/notebooks/openenv_wordle_grpo.ipynb

OpenEnv guide in TRL: https://huggingface.co/docs/trl/main/en/openenv

SaylorTwift

in HuggingFaceH4/aime_2024 19 days ago

Create eval.yaml

#3 opened 19 days ago by

SaylorTwift

cmpatino

in HuggingFaceH4/on-policy-distillation 20 days ago

Hyperparameters for replication

#1 opened 24 days ago by

thomasip

sergiopaniego

posted an update 21 days ago

Post

327

Ya está disponible el vídeo de la charla del otro día en @nerdearla sobre IA abierta, por si queréis verla! 🤠

https://www.youtube.com/watch?v=p-JLn4xAkMw

1 reply

sergiopaniego

posted an update 21 days ago

Post

2580

we've just added several example scripts to TRL showing how to train models with GRPO using some of the new OpenEnv environments

train a model to interact with a browser (🎮 BrowserGym Env), play Wordle (🎮 Wordle Env) and moooore!

TRL (GRPO + vLLM) + OpenEnv! ⚡️

📝 go play with them: https://github.com/huggingface/trl/tree/main/examples/scripts/openenv

📝 examples list: https://huggingface.co/docs/trl/main/en/example_overview#scripts