Summary
currently at the moment some stuff is extremly hard with the M.T.P or even before. and that is running things before all tests projects did run and also after all test projects did run.
the problem is even amplified because its now possible to run tests in multiple ways, there is:
- dotnet test
- dotnet run --project TestProj.csproj
- server mode
probably even more cases i do not now about.
now if you do not have a single csproj that will run as a cordinater you are basically in a game over statement if developers use either one and you want to create a test solution that basically warms stuff up or builds grouped reports at the end.
its also impossible to correctly build solutions to limit parallel execution of tests since, test projects run in parallel and tests inside the projects can also run in parallel. in a big monorepo that gets ugly insanly fast.
things that are also not possible because that feature is missing is test sharding, if a global hook would exists, it might even be possible to provide a way to introduce new args that would allow these features.
also running a single testcontainer, for example a database is basically impossible, running all migrations once before all tests, basically impossible, you basically have no concurrency control at all.
there can be a "hacky" solution for the "start", but not for the finish. the start can be done by creating a file level lock (urgh) the end can be done with a file level lock, if you only need to merge files, but that breaks for stopping containers.
Background and Motivation
we basically want more concurrency control and starting containers and migrations once for the whole test session no matter how many test projects we have.
Proposed Feature
add a way of providing a before all/after all hook, that does run exactly once, but not twice, if you have multiple projects that contain tests. maybe even provide a way to announce possible new arguments, however that would be hard if there are multiple extensions that provide the same arg, so that would need further design work.
the hook should run before anything, maybe for the server mode it can be done when the server mode is launched, its not like test containers would need to be reloaded all the time.
Alternative Designs
provide a solution to stop running test projects in parallel, however that would fix some issues it would mean that we need to pay the bootup multiple times.
Summary
currently at the moment some stuff is extremly hard with the M.T.P or even before. and that is running things before all tests projects did run and also after all test projects did run.
the problem is even amplified because its now possible to run tests in multiple ways, there is:
probably even more cases i do not now about.
now if you do not have a single csproj that will run as a cordinater you are basically in a game over statement if developers use either one and you want to create a test solution that basically warms stuff up or builds grouped reports at the end.
its also impossible to correctly build solutions to limit parallel execution of tests since, test projects run in parallel and tests inside the projects can also run in parallel. in a big monorepo that gets ugly insanly fast.
things that are also not possible because that feature is missing is test sharding, if a global hook would exists, it might even be possible to provide a way to introduce new args that would allow these features.
also running a single testcontainer, for example a database is basically impossible, running all migrations once before all tests, basically impossible, you basically have no concurrency control at all.
there can be a "hacky" solution for the "start", but not for the finish. the start can be done by creating a file level lock (urgh) the end can be done with a file level lock, if you only need to merge files, but that breaks for stopping containers.
Background and Motivation
we basically want more concurrency control and starting containers and migrations once for the whole test session no matter how many test projects we have.
Proposed Feature
add a way of providing a before all/after all hook, that does run exactly once, but not twice, if you have multiple projects that contain tests. maybe even provide a way to announce possible new arguments, however that would be hard if there are multiple extensions that provide the same arg, so that would need further design work.
the hook should run before anything, maybe for the server mode it can be done when the server mode is launched, its not like test containers would need to be reloaded all the time.
Alternative Designs
provide a solution to stop running test projects in parallel, however that would fix some issues it would mean that we need to pay the bootup multiple times.