GovStack Gherkin scenarios - user guide + template

This document provides an explanation of the Gherkin scenarios used in the project. The scenarios are written in the Given-When-Then format and are aimed at testing the functionality related to retrieving the openAPI description of a specified REST service.

Scenario Structure

The scenarios follow the Given-When-Then structure:

Given: The initial state of the scenario before an action is taken.
When: A specific action that the user takes.
Then: A testable outcome caused by the action in the When step.
And: This keyword adds more steps to an operation (Given, When, Then). Note that it’s discouraged to use And in When operations, you probably want to split the Scenario into several Scenarios.

Scenario vs. Scenario Outline

Scenario: Represents a specific behavior and is executed once.
Scenario Outline: Executed for each row of test data from the Examples section. It requires test data to be provided in the Examples section.

Examples

The Examples section contains the test data matrix used by the scenarios. The first row serves as a header describing the parameter names, while subsequent rows provide values assigned to those parameters.

Types of Tests

The feature files should contain three types of tests: Smoke tests, Positive tests, and Negative tests.

Smoke Tests

Smoke tests are basic checks to verify if the API is responding. They serve as preliminary checks to determine if further testing is feasible. Smoke tests should be quick and focus on the following aspects:

Does the API respond within an acceptable time frame?
Does the API return a proper status code (e.g., 200 OK)?
Does the API response body adhere to the expected schema?

Positive Tests (Happy Path Testing)

Positive tests validate the application's behavior with correct and valid data. They ensure that the application meets specifications. Positive tests should cover:

Proper status code in the API response.
Adherence of the API response body to the expected schema.
Timely response from the API.
Presence of appropriate headers in the API response.
Optional parameters specified according to requirements (e.g., filters, sorting).

Negative Tests

Negative tests focus on invalid inputs to ensure that the system handles them appropriately. They verify that the software does not perform undesired actions. Negative tests should cover:

All aspects covered in positive tests.
Presence of clear, descriptive error messages in the API response when invalid inputs are provided.

Required Tags in Feature Files

Tags are used for grouping scenarios, running subsets of scenarios, and passing data to the web application that displays the results.

All feature files must declare the API endpoint and HTTP method they are testing using the following tags:

@method=: Indicates the endpoint's method (e.g., GET, PUT, POST) for reporting test results.
@endpoint=: Describes the endpoint name being tested and will be displayed in the web application.

By including these tags, the necessary information is provided to collect test data and report the results accurately.

Feature file example:

@method=GET @endpoint=/{GovStackInstance}/{memberClass}/{memberCode}/{applicationCode}/getOpenAPI
Feature: Retrieve openAPI description of the specified REST service

  Retrieve openAPI description of the specified REST service

  @smoke
  Scenario: Retrieve the openAPI description of the specified REST service smoke type test

    Given User wants to retrieve the openAPI description of the specified REST service
    When User sends GET request with given "identifier" as GovStackInstance "identifier" as memberClass "alp4aNum3r1c" as memberCode "alp4aNum3r1c" as applicationCode
    Then User receives a response
    And The response should be returned in a timely manner
    And The response should match json schema
    And The response should have status 200
    

  @positive
  Scenario Outline: Retrieve the openAPI description of the specified REST service

    Given User wants to retrieve the openAPI description of the specified REST service
    When User sends GET request with given "<GovStackInstance>" as GovStackInstance "<memberClass>" as memberClass "<memberCode>" as memberCode "<applicationCode>" as applicationCode
    Then User receives a response
    And The response should match json schema
    And The response should be returned in a timely manner
    And The response should have status 200
    And The response header content-type should be "application/json; charset=utf-8"
    And The response should return memberCode as in the request

    Examples: Valid data
    | GovStackInstance | memberClass | memberCode   | applicationCode |
    | identifier       | identifier  | alp4aNum3r1c | alp4aNum3r1c    |
    | a11a11a11a       | asd2211a    | 12as12       | m3mb3rc0d3      |
    | a552example      | zzz321a     | m3mb3rc0d3   | aa15            |
    | valid1dentifi3r  | rand0mid3nt | str1ng       | azxcv55         |


  @negative
  Scenario Outline: Unable to retrieve the openAPI description of the specified REST service

    Given User wants to retrieve the openAPI description of the specified REST service
    When User sends GET request with given "<GovStackInstance>" as GovStackInstance "<memberClass>" as memberClass "<memberCode>" as memberCode "<applicationCode>" as applicationCode
    Then User receives a response
    And The response should be returned in a timely manner
    And The response should have status 400
    And The response should match json schema
    
    Examples: Invalid data
    | GovStackInstance | memberClass | memberCode   | applicationCode |
    | 1nval1d          | identifier  | alp4aNum3r1c | alp4aNum3r1c    |
    | identifier       | 1nval1d     | alp4aNum3r1c | alp4aNum3r1c    |
    | identifier       | identifier  | _invalid     | alp4aNum3r1c    |
    | identifier       | identifier  | str1ng       | _invalid        |
    | 1nval1d          | 1nval1d     | _invalid     | _invalid        |