[ARCHIVE] Test Suite Versioning

Owned by Damian Borowiecki
Last updated: Oct 03, 2023 by Aneta Pałczyńska (Deactivated)
1 min read

A new version of Test Suite Versioning can be found in the same folder.

Different test suite versions will be determined through GitHub releases.

Structure of repository remains the same

/examples     /mock     /registrationApp     /… /test     /openAPI         …    /nextTestSuite         …

Test suite versions are determined using release tags, which helps maintain a clean repository structure.

In CircleCI, we employ a dynamic matrix to identify the candidate application and the test suites provided in a given BB repository. This matrix can be expanded by using release git tags that match specific regex patterns.

For example, let's assume BB has three test suite releases:

  • V1.0.0-test-suite

  • V2.3.3

  • V3.0.0

The dynamic matrix will execute tests for each suite and the latest version of the repository (main branch), using a candidate app, test suite and version:

suite × candidate × version

With multiple releases, the number of executed test harnesses may increase significantly. To address this, we could consider using dedicated tags for releases that correspond to changes in test suites.

Additionally, the test suite version could be passed as an input value to the candidate application's ./test_entrypoint.sh --bb-version latest, allowing the setup to be tailored for a specific release.