SmallDataRegime.ai

RL post-trainingfor small data

Sample efficient. Model agnostic.

> smalldataregime train --budget 200 --reward prefs.jsonl

Held-out win rate · small-data regime

71.4%

+9.2 vs base model · KL 0.04 · reward spec v3

qwen-32b → sdr-07

200 prefs
Reward spec passedStyle eval passedSafety eval waiting

The foundation for small-data post-training

Shape a reward, train on a small set, and ship only the policies that pass your evals — on a cluster, a lab machine, or an air-gapped host.

  1. 01Signal

    Deliberately small, high-leverage data

    Choose the traces, rankings, and corrections that enter the loop. The set stays small because each example has to earn its place.

  2. 02Governance

    A reward spec with a release gate

    Every update is checked against held-out evals, a written reward, and a human review before it can leave the lab.

  3. 03Value

    Lift per labeled example

    Tie win rate, sample use, and task quality to the cost of the next preference — not to the size of the corpus.

Foundational research

Built on decades of our own research applied at scale.

2006201120252026

2006 – 2011

Our chief scientist identifies a problem and provides a groundbreaking solution.

“Improved Algorithms for Linear Stochastic Bandits”

2,900 citations

OFUL introduces foundational online learning research.

OFUL becomes a basis for, and helps shape, billion-user-scale recommendation systems and methods now widely used across tech, science, pharmaceutical, manufacturing, and other industries.

Implemented for recommendation, personalization, and digital marketing by

  • Google
  • Netflix
  • Microsoft
  • AstraZeneca
  • Instacart
  • Adobe
  • Wayfair

OFUL / LinUCB citations

Google Scholar

20112026

2025

Hierarchical Reasoning Model

Breakthrough AI architecture: a major milestone in recursive and hierarchical models.

2026

OpenAI Astra is using recurrent reasoning.

Resolves a decades-old bottleneck in training recursive models. Impressive performance on hard reasoning tasks.

Proprietory methods

Our recursive self-improving factory.

Optimizer

Optimizes the components of the system to improve success criteria over training tasks. Based on Optimism in the Face of Uncertainty (OFU).

Input

  • Instructions + documents
  • Success criteria

Reasoning system

Infers goal and obligations, uncertainties, and memory, then solves a reasoning problem by exploring the solution space.

Output

The output is shaped according to the criteria. It could be a simple answer, a complex report, or a data room.

Our planner vs typical agentic loops

ChatGPT
Tools + skills
State
Uncertainty
Goal
Policy
Action

Sample evaluation results

Our systems created by our factory outperform today's leading AI models on expert reasoning tasks.

Corporate M&A Agent against Claude and ChatGPT in the Harvey Labs Benchmark

Performance score

100500

92.43

93.8

97.9

ChatGPTClaudeAIR

Cost, USD

73.50

$6.67

$6.62

$1.93

ChatGPTClaudeAIR

Agent cost vs performance by task

  • AIR Agent97.9% · $1.93
  • Codex + GPT 6 Astra92.43% · $6.67
  • Claude + Fable 5.193.8% · $6.62
  • DeepSeek88.2% · $0.35

Task key

Build a post-training regime

Turn a short stack of demonstrations and preferences into a policy that holds up outside the notebook. SmallDataRegime.ai plans the run, applies the update, and validates it against the evals you wrote.