capstone-mem-artifacts

Data and analysis artifacts for the SMU MITB capstone study "Does Cross-Instance Memory Help an LLM Software-Engineering Agent?" (2026). The study runs matched-pair experiments (memory-ON vs memory-OFF) with a Kimi K2 agent on SWE-bench Pro instances across six repositories.

Layout

• ARCHIVES.md - registry mapping every experiment run to its GitHub Release, SHA-256, and restore command.
• Releases - ten checksummed full-minus-pcap zstd bundles of raw run results (per-instance trajectories and logs; packet captures dropped), one release per batch.
• manifests/ - per-release provenance manifests (per-instance instance_id, base_commit_date, resolved verdict, LLM cost).
• checksums/ - the .sha256 files for all release assets.
• analysis/ - manifest generation and paired pass-count scripts.
• vmine_ew125.py, vmine_qb23.py, vmine_ew23.py, vmine_wc23.py, vmine_ol23.py, wc_fidelity.py, ew23_matrix.py, ew23_classify.py - mining and classification scripts behind the behavioural results (report Section 5 and Appendix D).

Related repositories: harness fork simkimsia/mem-comp-26 (upstream mem-comp/mem-comp-26); agent simkimsia/kimsia-mem-agent.

Restore and reproduce

gh release download <tag> -R simkimsia/capstone-mem-artifacts
shasum -a 256 -c *.sha256
zstd -dc *.tar.zst | tar x

vmine_ew125.py and vmine_qb23.py take the directory holding the extracted run dirs as their only argument. vmine_ew23.py walks run directories at the path set at the top of the script. vmine_wc23.py and vmine_ol23.py parse the harness master log (run_*.log) named at the top of each script. wc_fidelity.py, ew23_matrix.py, and ew23_classify.py document their inputs in their docstrings.