January 19, 2022
Date
Jan 19, 2023
Participants
@Dominika Bieńkowska (Deactivated)
@Damian Borowiecki (Deactivated)
@Taylor Downs
@Paweł Gesek
Goals
Discuss our approach to data used in the gherkin tests
Discuss CircleCi configuration
Discussion topics
Agenda | Discussion |
---|---|
Data used in tests | If there is particular data that we need to pass the test then apps and candidate app authors should use the json blob data during test_entrypoint.sh execution. Data models can vary between candidates. App author is responsible for configuring the app in the way to pass the tests and check compliance. We can encourage them to use our json blob and make functionality creating schema based on our spec - therefore, during any data changes test result shouldn’t change. However, it’s not a mandatory approach. |
Use cases - how do we want to approach tests to make sure that use cases can be passed. | At the moment we have following structure: bb-folder/ /test/<gherkin app> /examples/ candidate-a/ test_entrypoint.sh … When we’ll start to execute tests against multiple test suites, we’ll need to change structure:
/test/ test_suite_A/<gherkin app> test_suite_B/<gherkin app> … With separate definition for all of the test suites (Post Partum, OpenAPI spec, ect.)
/examples/ /candidate-a/test_entrypoint.sh –config test_suite_A /candidate-b/test_entrypoint.sh –config test_suite_B Type of test suite would be determined by the execution attribute. Note: This has to be discussed with the committee as we come up with a solution during the meeting.
|
Integration with Sandbox team work | The Sandbox team will be creating adaptors and creating setup and deployment scripts for particular use cases. We should have a conversation with the Sandbox team to decide when and how we can align our work to make sure we’re going in the right direction. |
CircleCi configuration - test entry points | Test entry points should not run in the background! Taylor did the change on his PR. The test entry point has to be finished first and so does the whole setup before running the tests. Taylor removed the background process and it seems to be working well so we should align our approach with it. |
CircleCi configuration - Security checks | How can we avoid contributors making changes? We need to think about it in the future. We need to be careful when we do PRs reviews. |
CircleCi configuration - Backend | We need to slightly improve the way test execution results are passed to the database. In the current solution we’re executing the tests and then aggregate results, which are sent directly to the mongodb. This has several drawbacks, among others:
We decided to change the solution and make aggregations and insertions of test results on express.js backend site. This will allow us to keep better control over the structure and make it easier to test results. |
Action items
Action Items | Responsible party | Date |
---|---|---|
|
|