Fewer parameters.Finer behaviour.Better measured.
We are a research lab working on the stage after pre-training: reinforcement learning, fine-tuning, distillation, alignment and evaluation, all aimed at small, efficient models.
Thesis
Pre-training sets what a model could know. Post-training settles what it reliably does.
The steps after pre-training (rewards, demonstrations, preferences and evaluations) turn broad capability into dependable behaviour: following intent, reasoning carefully, using tools, and declining when it should.
We study those steps on small models. They are cheap to run, quick to iterate on and practical to deploy near the people who use them. We want to learn which methods truly hold at that scale and report the results plainly.
Research areas
Reinforcement learning
Learning from human and AI feedback and from verifiable rewards. Reward models and policy optimisation suited to compact models.
RLHF · RLAIFVerifiable rewards
Fine-tuning & distillation
Data-efficient supervised tuning, preference optimisation, and transferring skill from larger teachers to smaller students.
SFT · DPODistillation
Alignment
Instruction following, honesty, calibrated refusals and behavioural specifications that survive at small scale.
Specs · HonestyRefusals
Evaluation
Task-grounded, contamination-aware measurement that tells us whether an improvement is real.
BenchmarksHeld-out tasks
Writing
Reward signals for small policies
What makes a reward model useful when the policy it trains is small.
Distilling behaviour, not just answers
How much of a larger model's aligned behaviour carries over through distillation.
A compact evaluation suite
Held-out, contamination-aware tasks for measuring small models.
Upcoming topics. Nothing has been published yet; notes, papers and releases will be listed here as they appear.
Principles
iMeasure first
A result is only as strong as the evaluation behind it. We publish both, limitations included.
iiDesign for the budget
Compute and memory are constraints we design around. Efficient models reach more people.
iiiShare the method
We intend to release methods, code and negative results so others can check and extend them.
ivStay narrow
One stage of the pipeline, studied in depth. Post-training is enough to keep us busy.
Interested in small models done properly?
We'd like to hear from researchers and engineers who care about RL, evaluation and the craft of training models well, and from potential collaborators.