CLAUDE.md

Agent quick start

This repo is a C→Rust translation benchmark: 200 C programs, scored by pass@1 (compile the Rust translation, run it against the reference C program's outputs over argv/stdin, pass if they match).

1. Read README.md for the run flow and the environment variables (SACTOR_HOME, SERVE_VENV, CFG, TAG, PORT).
2. Read SETUP.md before touching the environment — it lists what must be built and on PATH, and the failure modes that look like model quality but are not.

Rules that matter when running it:

• Keep total eval workers ≤ ~15 across all concurrent jobs; watch /proc/loadavg. Overload causes false timeouts that silently corrupt results.
• Always average 5 seeds. A single run at temperature 0.6 is worth ±2–3 points.
• Never compare numbers across different test_pass_threshold / CoT / thinking / max_tokens settings — those move the result by several points on their own.
• bash fix_paths.sh once per checkout, before the first run.