Repository navigation
more fine grained categorisation for tests #4370
Description
Activity
@DaanHoogland is this still on hold?
yes, becoming more actual, but might not be ready for 4.16
Moving it to unplanned based on @DaanHoogland 's comment.
Closing this - this has been open for 2+yrs; any categorisation effort won't speed up the CI/CD, will create new PRs/issues to track work towards improving CI/CD timings; one idea I want to explore is if it's possible to even run tests in parallel (or certain groups that can run in parallel).
this is valid as long as we use integration tests and really not yet implemented to satisfaction
1 remaining item
This issue is stale because it has been open for 120 days with no activity. It may be removed by administrators of this project at any time. Remove the stale label or comment to request for removal of it to prevent this.
as a tester, one would want a more fine grained categorisation than "smoke", "component", "integration" to be able to quickly test a PR against features it touches on.
ISSUE TYPE
COMPONENT NAME
CLOUDSTACK VERSION
CONFIGURATION
OS / ENVIRONMENT
SUMMARY
when a PR is tested it is usually tested against a set of smoke tests in travis or a private cloud with s.moke tests and than maybe avery feature specific (often manual) test. A better categorisation would help in testing more related code and would allow for bypassing any tests that have no baring on the subject of the change.
A blueorangutan smoke test run takes anywhere between 8 and 15 hours and fails only once after the run, albeit with multiple reports. It would be helpful to run a smaller set of basic tests. if this set passes than a more high level batch could be choosen for a next step.
STEPS TO REPRODUCE
go-testcould be any test infra structure like blueorangutan. It is used to illustrate the categorisation, not the test-infrastructure.EXPECTED RESULTS
ACTUAL RESULTS