PostTrainBench: Can LLM Agents Automate LLM Post-Training?

We introduce PostTrainBench, a benchmark that measures the ability of CLI agents to post-train pre-trained large language models (LLMs). In PostTrainBench, the agent's task is to improve the performance of a base LLM on a given benchmark. The agent is given access to an evaluation script and 10 hours on an H100 GPU. Performance is measured by the benchmark score of the post-trained LLM. This setup naturally evaluates an agent's ability to conduct AI R&D.

[!IMPORTANT] Run it on cloud GPUs via Harbor. This repository's reference pipeline targets our HPC cluster (HTCondor), but src/harbor_adapter/ runs the full benchmark (same prompt, judges and evaluation) on Modal, with no cluster needed. See its README.

Scaffolds

Agents are run through one of 4 CLI scaffolds: Claude Code, Codex CLI, Gemini CLI, and OpenCode.

Evaluation Tasks

PostTrainBench includes 6 benchmarks spanning reasoning, knowledge, math, health, and code:

1. AIME 2025 - Math competition problems
2. Arena Hard Writing - Creative writing benchmark adapted from ArenaHard v2
3. GPQA - Graduate-level science questions
4. GSM8K - Grade school math
5. HealthBench Easy - Medical knowledge and reasoning
6. HumanEval - Code generation

Quick Start

On an HTCondor cluster (the reference setup); to run on cloud GPUs instead, see src/harbor_adapter/.

# 1. Install requirements (apptainer, fuse-overlayfs)

# 2. Build the container
bash containers/build_container.sh standard

# 3. Download HuggingFace cache
bash containers/download_hf_cache/download_hf_cache.sh

# 4. Set up environment variables
cp example.env .env
# Edit .env and fill in your API keys and paths

# 5. Run jobs
bash src/commit_utils/commit.sh

Environment Setup

Copy example.env to .env and fill in your values:

cp example.env .env

The .env file contains API keys and configuration. See example.env for all available variables:

Variable	Description	Default
OPENAI_API_KEY	OpenAI API key	-
ANTHROPIC_API_KEY	Anthropic API key	-
GEMINI_API_KEY	Google Gemini API key	-
OPENCODE_API_KEY	OpenCode API key (used by the opencode agent)	-
ZAI_API_KEY	Z.AI API key (used by the opencode and glm5 agents)	-
HF_HOME	HuggingFace cache directory	$HOME/.cache/huggingface
POST_TRAIN_BENCH_RESULTS_DIR	Directory for results	results
POST_TRAIN_BENCH_CONTAINERS_DIR	Directory for containers	containers
POST_TRAIN_BENCH_CONTAINER_NAME	Container name	standard
POST_TRAIN_BENCH_PROMPT	Prompt variant	prompt
POST_TRAIN_BENCH_JOB_SCHEDULER	Job scheduler (htcondor or htcondor_mpi-is)	htcondor

Environment variables already set in your shell take precedence over .env values.

Currently, we only support the HTCondor job scheduler. Harbor support is planned.

API-based agents

Most agents authenticate via API keys set as environment variables (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, GEMINI_API_KEY). These are passed into the container automatically by run_task.sh. Set them in your environment before running commit.sh.

Subscription-based agents (non-API)