Skip to content

Constrain PHPUnit discovery to selected suites - #2153

Merged
chubes4 merged 1 commit into
mainfrom
fix/phpunit-testsuite-discovery
Jul 29, 2026
Merged

Constrain PHPUnit discovery to selected suites#2153
chubes4 merged 1 commit into
mainfrom
fix/phpunit-testsuite-discovery

Conversation

@chubes4

@chubes4 chubes4 commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • parse --testsuite name and --testsuite=name before XML-backed file discovery
  • support comma-separated suite names with stable deduplication
  • constrain suite directories, files, excludes, and matchers before loading tests
  • fail deterministically when a requested suite is absent

Fixes #2150.

Verification

  • npm run build
  • npm exec -- tsx tests/phpunit-project-autoload.test.ts
  • npm run test:phpunit-structured-evidence
  • npm run test:phpunit-runtime-rejection
  • npm run test:playground-worker-runtime-rejection
  • WPCOM comparator SHA 7ae9ae21ca30f7fc9724aecfb54a16f674f82a23: api-v2-agentic-commerce discovery changed from 7,018 API-v2 tests to exactly 6 files under bin/tests/api-v2/suites/AgenticCommerce
  • Combined with Reuse Playground direct input mounts #2154, the full parity execution passed 677/677 suites: 674 WP Codebox and 3 native, with zero failures and zero timeouts at 4 shards x 6 workers
  • Reproduction: run node scripts/run-wpcom-teamcity-parity.mjs --mode execute --allow-execute true --manifest artifacts/phpunit-profiles/teamcity-parity-manifest-7ae9ae21-corrected.json --concurrency 6 --shards 4 --timeout-multiplier 3 from the WPCOM Codebox parity workspace with the certified runtime inputs

Two broader diagnostics commands remained red on unchanged origin/main behavior: the exact generated-wrapper shape assertion and the local Playground fixture mount visibility integration. Raw run artifacts remain in private Lab storage and are intentionally not linked as public evidence.

AI Assistance

OpenAI gpt-5.6-sol via OpenCode implemented the change, added executable generated-PHP regression coverage, ran local verification, and executed the Lab WPCOM proof. Chris Huber remains responsible for the change.

@chubes4
chubes4 merged commit 832f53f into main Jul 29, 2026
4 checks passed
@chubes4

chubes4 commented Jul 29, 2026

Copy link
Copy Markdown
Collaborator Author

Full compatibility proof completed after merge, combined with #2154:

  • 677/677 TeamCity suites passed
  • 674/674 WP Codebox suites passed
  • 3/3 native suites passed
  • zero failures and zero timeouts
  • topology: 4 shards x 6 workers, timeout multiplier 3
  • comparator SHA: 7ae9ae21ca30f7fc9724aecfb54a16f674f82a23

Reproduction: run node scripts/run-wpcom-teamcity-parity.mjs --mode execute --allow-execute true --manifest artifacts/phpunit-profiles/teamcity-parity-manifest-7ae9ae21-corrected.json --concurrency 6 --shards 4 --timeout-multiplier 3 from the WPCOM Codebox parity workspace with the certified runtime inputs. Raw artifacts remain in private Lab storage.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

PHPUnit --testsuite does not constrain XML discovery

1 participant