RL post-trainingfor small data
Sample efficient. Model agnostic.
> smalldataregime train --budget 200 --reward prefs.jsonlHeld-out win rate · small-data regime
71.4%
+9.2 vs base model · KL 0.04 · reward spec v3
qwen-32b → sdr-07
The foundation for small-data post-training
Shape a reward, train on a small set, and ship only the policies that pass your evals — on a cluster, a lab machine, or an air-gapped host.
01Signal
Deliberately small, high-leverage data
Choose the traces, rankings, and corrections that enter the loop. The set stays small because each example has to earn its place.
02Governance
A reward spec with a release gate
Every update is checked against held-out evals, a written reward, and a human review before it can leave the lab.
03Value
Lift per labeled example
Tie win rate, sample use, and task quality to the cost of the next preference — not to the size of the corpus.
Foundational research
Built on decades of our own research applied at scale.
2006 – 2011
Our chief scientist identifies a problem and provides a groundbreaking solution.
“Improved Algorithms for Linear Stochastic Bandits”
2,900 citations
OFUL introduces foundational online learning research.
OFUL becomes a basis for, and helps shape, billion-user-scale recommendation systems and methods now widely used across tech, science, pharmaceutical, manufacturing, and other industries.
Implemented for recommendation, personalization, and digital marketing by
- Netflix
- Microsoft
- AstraZeneca
- Instacart
- Adobe
- Wayfair
OFUL / LinUCB citations
Google Scholar
2025
Hierarchical Reasoning Model
Breakthrough AI architecture: a major milestone in recursive and hierarchical models.
2026
OpenAI Astra is using recurrent reasoning.
Resolves a decades-old bottleneck in training recursive models. Impressive performance on hard reasoning tasks.
Proprietory methods
Our recursive self-improving factory.
Optimizer
Input
- Instructions + documents
- Success criteria
Reasoning system
Output
Our planner vs typical agentic loops
Uncertainty
Goal
Sample evaluation results
Our systems created by our factory outperform today's leading AI models on expert reasoning tasks.
Corporate M&A Agent against Claude and ChatGPT in the Harvey Labs Benchmark
Performance score
92.43
93.8
97.9
Cost, USD
$6.67
$6.62
$1.93
Agent cost vs performance by task
- AIR Agent97.9% · $1.93
- Codex + GPT 6 Astra92.43% · $6.67
- Claude + Fable 5.193.8% · $6.62
- DeepSeek88.2% · $0.35
Task key
Build a post-training regime
Turn a short stack of demonstrations and preferences into a policy that holds up outside the notebook. SmallDataRegime.ai plans the run, applies the update, and validates it against the evals you wrote.
Lift per preference
- Task writer86%
- Support judge64%
- Tool use41%
- Tone pair18%
12 of 200 pairs carry most of the gain.
Yield
See which preferences actually move the policy, by task, labeler, and team.
Lab machine · on prem
Weights stay localLab machines
Dedicated machines that keep weights, traces, and reward models on hardware you control.
Readiness · level 3
72
- Reward specWritten
- Held-out set40 traces
- Base gapCloseable
Readiness
A score for whether a task can be post-trained from a small set, tracked as the spec changes.
Run r-1842
- 00:12 collect 14 new rankings
- 00:18 fit reward on 186 pairs
- 00:41 policy step · KL 0.04
- 00:44 waiting on safety eval
Run control
Live view of active runs, worker progress, and the eval gate sitting in front of release.