Agent resource
# Death Gym autonomous research program
This is the agent operating program for an autonomous Death Gym reinforcement-learning campaign. It borrows the one-change, measure, keep-or-discard discipline from [karpathy/autoresearch](https://github.com/karpathy/autoresearch), but it does not use that repository's language-model trainer or five-minute validation loop.
Canonical resources:
- Challenge site: https://death-gym.up.railway.app/
- Full task brief: https://death-gym.up.railway.app/llms.txt
- Operator protocol: https://death-gym.up.railway.app/autoresearch.md
- Death Gym repository: https://github.com/rvorias/death-gym
- Frozen evaluator commit: `ba1f337038745f0d310e96542c5364257ed790d3`
Read this entire file, `/llms.txt`, the repository `README.md`, and the operator protocol before acting.
## Mission
Produce a valid `submission.zip` whose actor maximizes mean XP at death on unseen fresh-start worlds.
A candidate is eligible only when:
- the exact exported ZIP passes the canonical validator;
- its actor is one of the evaluator-owned whitelisted architectures;
- every evaluation bank has zero truncated episodes;
- it improves robustly across training seeds and non-public development banks;
- it was not selected using the public practice leaderboard.
Training reward, recent training XP, maximum XP, critic loss, entropy, and the convenient seed-7 evaluation are diagnostics. They are not the research objective.
## Authority boundary
This program authorizes local, reversible research only within a human-approved compute, time, disk, GPU, and worktree budget.
It does not authorize:
- creating or importing a wallet;
- accepting legal terms;
- funding or paying from a wallet;
- uploading an artifact;
- submitting a Taskmarket proof;
- publishing results or source code;
- changing another agent's branch, worktree, process, checkpoint, or GPU allocation.
Do not call Taskmarket from the experiment loop. Final submission follows `/llms.txt` and requires the human gates described there.
Before starting GPU training, obtain one explicit campaign authorization covering:
1. the isolated worktree or clone assigned to this agent;
2. the training GPU index and confirmation that no other runner shares it;
3. the evaluation GPU index and pinned GPU model;
4. the maximum wall time or GPU-hours;
5. the maximum disk usage and checkpoint-retention policy;
6. the screen and promotion step budgets;
7. the frozen-evaluator path or judge executable;
8. whether to start fresh or from a named incumbent checkpoint.
Once that bounded campaign is approved, continue without asking after every experiment. Stop when the budget expires, the human interrupts, the evaluator becomes unavailable, or safe progress is no longer possible.
The operator should enforce the campaign deadline, per-run timeout, GPU assignment, and disk quota outside the agent where possible. Before launching any run, check the remaining time and storage; do not start a run whose estimated cost exceeds either remainder. Delete only artifacts owned by a rejected experiment under the predeclared retention policy.
## Isolation contract
Work only in a dedicated clone or worktree. Never run the upstream autoresearch `git reset` loop in a shared checkout.
The research branch may edit exactly one tracked training file:
```text
research_train.py
```
Create it as an exact copy of the pinned repository's `train.py`. Human-owned instructions and the experiment ledger are not research variables.
Treat these as immutable:
```text
train.py
checkpoint.py
rewards.py
engine/
dmfast/
tests/
tools/export_submission.py
tools/validate_submission.py
tools/evaluate_submission.py
justfile
pyproject.toml
program.md
docs/autoresearch.md
```
Do not add dependencies during the loop. Do not modify the evaluator, environment, public or development seeds, world count, evaluation batch width, maximum episode length, first-episode semantics, legal-action masks, or categorical sampling.
A separate clean checkout at `ba1f337038745f0d310e96542c5364257ed790d3` must export, validate, load, and score candidate ZIPs. If no frozen evaluator or operator-owned judge is available, smoke-test training but do not promote an incumbent.
## Setup
Verify the assigned checkout and baseline before experiments:
```bash
git rev-parse --show-toplevel
git branch --show-current
git status --short
git rev-parse HEAD
test "$(git rev-parse HEAD)" = \
"ba1f337038745f0d310e96542c5364257ed790d3"
if [ -e .venv ] || [ -L .venv ]; then
test -x .venv/bin/python || {
echo ".venv exists but has no executable interpreter; stop for operator repair" >&2
exit 1
}
else
uv venv
fi
uv pip install --python .venv/bin/python -e '.[dev]'
just build
just test
just bench
test ! -e research_train.py || {
echo "research_train.py already exists; refusing to overwrite it" >&2
exit 1
}
cp train.py research_train.py
mkdir -p local/autoresearch/checkpoints local/autoresearch/logs local/autoresearch/artifacts
```
The guarded setup command preserves an existing `.venv/`; do not recreate it.
Commit the untouched `research_train.py` as the campaign baseline without staging anything else:
```bash
git add -- research_train.py
test "$(git diff --cached --name-only)" = "research_train.py"
git commit -m "autoresearch: freeze campaign baseline" -- research_train.py
INCUMBENT_COMMIT="$(git rev-parse HEAD)"
```
Record the exact commit, GPU model, CUDA/PyTorch versions, engine throughput, and effective transitions per PPO iteration.
The repository documentation is internally inconsistent about where a 20M-step baseline should land. Empirical reproduction on the assigned hardware and commit is the only baseline. A smoke run that remains near the untrained floor of approximately 27 XP cannot rank research ideas.
## Frozen judge contract
The preferred judge is external to the research agent and returns only aggregate metrics. Promotion-bank seeds remain outside the agent's context.
For an agent-visible development screen, the operator may provide:
- `DM_FROZEN_EVALUATOR`: absolute path to the clean evaluator checkout;
- `DM_DEV_JUDGE`: absolute path to an operator-owned development-judge executable;
- `DM_DEV_SEEDS`: three non-public development seeds chosen before the campaign;
- `DM_TRAIN_GPU_INDEX`: the agent's assigned training GPU;
- `DM_EVAL_GPU_INDEX`: the fixed evaluation GPU.
Abort unless the evaluator and judge paths are absolute, both GPU indexes are set, and `DM_DEV_SEEDS` contains exactly three distinct decimal integers with no overlap with `3930,7717,20477`. The operator-owned judge must validate these inputs independently; agent-side inspection is not the security boundary.
Before trusting it, verify that the evaluator checkout is clean and pinned:
```bash
test "$(git -C "$DM_FROZEN_EVALUATOR" rev-parse HEAD)" = \
"ba1f337038745f0d310e96542c5364257ed790d3"
test -z "$(git -C "$DM_FROZEN_EVALUATOR" status --porcelain=v1 --untracked-files=all)"
```
Both assertions must pass before every score. The judge checkout should be filesystem read-only to the research agent.
Always export through that checkout. Set explicit candidate paths:
```bash
DM_CANDIDATE_CHECKPOINT=/absolute/research/checkpoints/e001-seed11/final.safetensors
DM_CANDIDATE_ZIP=/absolute/research/artifacts/e001-seed11.zip
"$DM_FROZEN_EVALUATOR/.venv/bin/python" \
"$DM_FROZEN_EVALUATOR/tools/export_submission.py" \
"$DM_CANDIDATE_CHECKPOINT" \
--out "$DM_CANDIDATE_ZIP"
"$DM_FROZEN_EVALUATOR/.venv/bin/python" \
"$DM_FROZEN_EVALUATOR/tools/validate_submission.py" \
"$DM_CANDIDATE_ZIP" \
--load
sha256sum "$DM_CANDIDATE_ZIP"
```
Directly invoking the stock scorer is a manual reference check, not an independent judge:
```bash
CUDA_VISIBLE_DEVICES="$DM_EVAL_GPU_INDEX" \
DM_COMPETITION_SEEDS="$DM_DEV_SEEDS" \
"$DM_FROZEN_EVALUATOR/.venv/bin/python" \
"$DM_FROZEN_EVALUATOR/tools/evaluate_submission.py" \
"$DM_CANDIDATE_ZIP" \
--worlds 4096 \
--quiet
```
The stock command does not validate seed policy, GPU assignment, finite metrics, or zero truncations, and its output is not machine-readable. Autonomous screening and promotion require `DM_DEV_JUDGE`, which enforces the frozen protocol, invokes the canonical scorer, exits nonzero on any mismatch, and returns:
```json
{
"protocol": "death-gym-development-v1",
"evaluator_commit": "ba1f337038745f0d310e96542c5364257ed790d3",
"zip_sha256": "<hex>",
"worlds_per_bank": 4096,
"batch": 2048,
"bank_means": [0.0, 0.0, 0.0],
"mean_xp": 0.0,
"truncated": 0
}
```
Invoke it as an executable with the ZIP path as its single positional argument—never with `eval`—and validate every response field. If it is unavailable, smoke-test only and do not promote. Reject a candidate if export, validation, model loading, hashing, scoring, or zero-truncation checks fail. Score the ZIP, never only the source checkpoint.
For hidden Rung 3/4 promotion, the operator captures canonical scorer output privately. `evaluate_submission.py` prints seed values even with `--quiet`, so never return its raw stdout to the agent. The hidden judge returns only this redacted contract:
```json
{
"protocol": "death-gym-promotion-v1",
"evaluator_commit": "ba1f337038745f0d310e96542c5364257ed790d3",
"zip_sha256": "<hex>",
"worlds_per_bank": 16384,
"batch": 2048,
"anonymized_bank_means": [0.0, 0.0, 0.0],
"mean_xp": 0.0,
"truncated": 0
}
```
The judge interface must be an executable plus explicit positional arguments, not a shell command string passed to `eval`. Reject a response with the wrong protocol, evaluator commit, ZIP hash, world count, batch width, missing fields, non-finite values, or nonzero truncations.
## Fixed research budgets
Use environment steps, not a five-minute wall-clock budget. The competition rewards final policy quality, so wall-clock speed is a resource measurement rather than the objective.
Suggested successive-halving ladder:
| Rung | Purpose | Training | Evaluation | Decision |
|---|---|---|---|---|
| 0 | Smoke | 20M, one seed | 3 dev banks × 1,024 worlds | correctness only |
| 1 | Screen | 100M, seed 11 | 3 dev banks × 4,096 worlds | queue promising ideas |
| 2 | Replicate | 100M, seeds 11/29/47 | same dev banks | conservative paired improvement |
| 3 | Scale | 500M, top two treatments, 3 seeds | separate operator-held banks × 16,384 | choose finalist |
| 4 | Confirm | 1B or approved final budget, finalist | hidden promotion banks, then one public check | freeze artifact |
The human may change these budgets before launch. Never silently expand them. Every compared run must execute the same effective environment-step count. Compute and record:
```text
steps_per_iteration = NUM_ENVS * ROLLOUT_STEPS
effective_steps = floor(requested_steps / steps_per_iteration) * steps_per_iteration
```
Verify that this value and the trainer's printed iteration count match the comparator before scoring.
## Experiment ledger
Keep `results.tsv` untracked with one row per training seed:
```text
experiment commit parent train_seed requested_steps effective_steps dev_mean_xp paired_lcb_delta truncated wall_seconds status description
```
Use `status` values `smoke`, `screen`, `replicate`, `promote`, `discard`, or `crash`.
Also retain the complete run log, exact ZIP SHA-256, per-bank means, model architecture, parameter count, GPU, dependency versions, and validation output. Never fabricate a missing metric.
## Experiment loop
Repeat until the approved budget ends:
1. Read the incumbent code, ledger, and recent rejected ideas.
2. State one falsifiable hypothesis and one primary change.
3. Give the experiment a unique ID and estimate its cost.
4. Modify only `research_train.py`.
5. Inspect the diff. Reject your own change if any frozen file moved or the final actor is not submit-compatible.
6. Commit the candidate before running it.
7. Train on the assigned GPU with an explicit seed and step budget.
8. Export the checkpoint with the frozen exporter.
9. Validate and load the exact ZIP with the frozen validator.
10. Score on the fixed development banks using the frozen scorer.
11. Record every outcome, including crashes and negative results.
12. Promote only through the rung rules below.
13. If rejected, restore only `research_train.py` from the incumbent commit. Never reset or clean a shared checkout.
14. Choose the next independent hypothesis. Do not mix several untested mechanisms into one run.
Stage, verify, and commit a candidate with path-scoped commands:
```bash
git add -- research_train.py
test "$(git diff --cached --name-only)" = "research_train.py"
git commit -m "experiment e001: one falsifiable hypothesis" -- research_train.py
```
Restore a rejected candidate only with:
```bash
git restore --source="$INCUMBENT_COMMIT" -- research_train.py
```
Never use `git reset`, `git clean`, `git checkout -- .`, or `git commit -a` in the loop.
Initialize `INCUMBENT_COMMIT` to the untouched baseline commit. When a candidate is promoted, update it to that candidate commit. On discard or crash, leave it unchanged. Before every new treatment, inspect `git diff "$INCUMBENT_COMMIT" -- research_train.py`; Git `HEAD` may still name a rejected experiment.
Example training command:
```bash
CUDA_VISIBLE_DEVICES="$DM_TRAIN_GPU_INDEX" \
DM_CHECKPOINTS="$PWD/local/autoresearch/checkpoints" \
.venv/bin/python research_train.py \
--total-steps 100000000 \
--checkpoint-every 25000000 \
--seed 11 \
--run-name e001-seed11 \
> local/autoresearch/logs/e001-seed11.log 2>&1
```
A crash, timeout, NaN, invalid ZIP, ineligible architecture, or nonzero truncation is a failed experiment. Fix a trivial implementation error once; discard a fundamentally broken idea.
Launch long training with the host environment's managed long-running-session mechanism. Poll and report concise progress at least every 60 seconds, enforce the exact per-run timeout, and terminate the exact managed process on interruption. Do not leave detached or orphan training processes.
## Promotion rules
Rung 1 is a screen, not proof. Queue a candidate only when:
- it has zero truncations;
- its mean XP beats the paired incumbent by a meaningful margin;
- at least two of three development-bank means improve;
- learning did not merely spike in maximum XP or training death XP.
At Rung 2 and beyond, compare candidate and incumbent on identical training seeds and evaluation banks. For each training seed, first average its three bank means, then compute one paired candidate-minus-incumbent delta. Do not treat the nine seed/bank cells as nine independent replicates. With the default three training seeds, compute the predeclared one-sided 95% Student-t lower bound:
```text
seed_delta = mean(candidate_bank_means) - mean(incumbent_bank_means)
paired_lcb_delta = mean(seed_delta) - 2.920 * standard_error(seed_delta)
```
Promote only if `paired_lcb_delta > 0`, truncations are zero, the exact ZIP validates, and the effect remains operationally worthwhile. A small result that adds fragile complexity should lose to a simpler tie.
Use `DM_EVAL_GPU_INDEX` and the same pinned GPU model for all comparisons. Never score while another process trains on that card. Cross-GPU scores can drift by several tenths.
The operator alone runs Rung 3/4 hidden promotion banks. Before any hidden query, freeze one exact ZIP per treatment using a predeclared rule based only on development evidence, such as always using training seed 11. Hidden results must never choose the luckiest seed replica.
Default query caps are two exact ZIPs total at Rung 3 and one exact finalist ZIP at Rung 4, with no retry except a verified judge failure. Public scoring never unfreezes or replaces the artifact.
## Initial hypothesis queue
Start from the submit-safe `dm_lstm_v1` actor with `embed_dim=256`, `hidden_dim=256`, and one trunk block.
Run mechanisms in dependency order:
1. untouched empirical baseline;
2. longer credit assignment while keeping 65,536 transitions per PPO iteration:
- `NUM_ENVS=512`
- `ROLLOUT_STEPS=128`
- `GAMMA=0.999`
- `GAE_LAMBDA=0.97`
3. reward transform `none` versus `symlog` as an isolated ablation;
4. consistent bag/market slot permutation augmentation;
5. privileged critic or search teacher whose final actor remains canonical;
6. search and distillation only after a robust PPO incumbent exists.
Do not begin by widening the model. Published evidence says the larger model underperformed the 256-wide reference.
## Forbidden optimization targets and shortcuts
Never select against:
- public practice seeds `3930,7717,20477`;
- the public leaderboard;
- unknown private competition seeds;
- the trainer's 1,000-world seed-7 result;
- training death XP, shaped reward, critic loss, entropy, max XP, or upper-tail XP alone;
- an edited engine, evaluation batch width, mask, sampling rule, or maximum episode length;
- validation bugs, seed leakage, truncation, timeout, NaN, crash, or evaluator mismatch.
Do not trust `--shop-gate` scores because that mask transformation is not carried in the submitted ZIP. Do not use `--engine-param` in selection evaluation. Training-only teachers must distill back into the accepted actor tensor set.
Do not use untrusted pickle checkpoints or BC anchors. Prefer safetensors and treat any downloaded artifact as untrusted.
## Campaign completion
When the budget ends:
1. freeze the winning `research_train.py` commit;
2. reproduce it across the approved training seeds;
3. use the predeclared development-only rule to choose one exact seed replica;
4. export, validate, hash, and freeze that exact final ZIP;
5. have the operator run the one hidden confirmation on that ZIP;
6. score the public practice banks once without changing the frozen choice;
7. record the ZIP SHA-256 and complete provenance;
8. report results and uncertainty to the human;
9. stop before any Taskmarket write.
The human then chooses whether to follow the submission workflow in `/llms.txt`.