bdd helper scripts

BDD utility scripts

In order to make the BDD process as friction-less as possible the following scripts have been added:

xtp-pull: Command line script that can retrieve/create feature files for specified ticket.
xtp-push: Command line script that can upload feature files for specified ticket.
xtp-compare: Command line script that can compare feature files locally with the corresponding one in Jira.
xtp-xray-upload: Command line script that posts the results of the cucumber json file to Jira.

These scripts will ensure tests, test sets and test results in Jira are often updated and kept in sync with what is stored in git repos.

Jira Test Set vs Test

A Jira Test ticket consists of one test scenario. Multiple Test tickets can be associated with a Test Set ticket.

If the cucumber formatted file is exported from a Test Set then it will include all the test scenarios of the Tests associated to it. If the cucumber formatted file is exported from a Test ticket then it will only include the scenario from that Test ticket.

Install

Install the ska-ser-skallop Python package.

pip3 install -U ska-ser-skallop --extra-index-url https://artefact.skao.int/repository/pypi-internal/simple

Once installed the xtp scripts will be available in your Python bin directory.

Scripts

xtp-pull - Retrieve feature file

Used to export a cucumber formatted test file from either a Jira Test or Test Set ticket.

Usage

usage: xtp-pull [-h] [-u USERNAME] [-p PASSWORD] [-t XTP_TICKET] [-o OUTPUT_PATH] [-v]

Retrieve a feature file from an XTP ticket. Your JIRA username and password will be required to perform this operation

optional arguments:
  -h, --help            show this help message and exit
  -u USERNAME, --username USERNAME
                        JIRA account username
  -p PASSWORD, --password PASSWORD
                        password for specified user
  -t XTP_TICKET, --xtp-ticket XTP_TICKET
                        JIRA XTP ticket number
  -o OUTPUT_PATH, --output-path OUTPUT_PATH
                        preferred path to store feature file. Default is current path
  -v, --verbose         Verbose output

Example

xtp-pull -u "<USER>" -t XTP-1834
Password:

head ./XTP-1834.feature
Feature: Default


@XTP-1839 @XTP-1834 @XTP-1834 @XTP-1834.feature @XTP-1839 @another_label @XTP-<ID>.feature
Scenario Outline: Upload .feature file test ticket
  Given I have entered <input_1> into the calculator
  And I have entered <input_2> into the calculator
  When I press <button>
  Then the result should be <output> on the screen

output-path is optional and would default to your current path if not specified.

~/.local/bin/xtp-pull --help
usage: xtp-pull [-h] [-u USERNAME] [-p PASSWORD] [-t XTP_TICKET]
                [-o OUTPUT_PATH] [-v] [--dry-run]

Retrieve a feature file from an XTP ticket. Your JIRA username and password
will be required to perform this operation

optional arguments:
  -h, --help            show this help message and exit
  -u USERNAME, --username USERNAME
                        JIRA account username
  -p PASSWORD, --password PASSWORD
                        password for specified user
  -t XTP_TICKET, --xtp-ticket XTP_TICKET
                        JIRA XTP ticket number
  -o OUTPUT_PATH, --output-path OUTPUT_PATH
                        preferred path to store feature file. Default is
                        current path
  -v, --verbose         Verbose output
  --dry-run             Run the script without downloading the XTP-*.feature file.

xtp-push - Push feature file to Jira

This utility uploads a feature file (cucumber format) to Jira. It will create/update Test tickets based on the scenarios in the file and associate them to the Test Set.

To use this you have to

Ensure that a test set exists in Jira that you want to upload the feature file to.
Ensure that a feature file is in the format below. Make sure the first line is the tag of the test set.

@XTP-1834
Feature: Test Upload feature file 3. As a user, I can calculate the sum of 2 numbers

    Scenario Outline: Upload .feature file test ticket
        Given I have entered <input_1> into the calculator
        And I have entered <input_2> into the calculator
...

If the Scenario Outline in the feature file matches the summary attribute on a Jira ticket that ticket will be updated. Otherwise a new Test ticket will be created.

The Test ticket will be associated with the Test Set.

Usage

xtp-push --help
usage: xtp-push [-h] -f FEATURE_FILE [-u USERNAME] [-p PASSWORD] [-v]
                [--dry-run]

Uploads a cucumber test file to JIRA by way of the XRAY extension. Either use username/password or set the environment variable JIRA_AUTH for authentication.

optional arguments:
  -h, --help            show this help message and exit
  -f FEATURE_FILE, --feature-file FEATURE_FILE
                        Path to the feature file.
  -u USERNAME, --username USERNAME
                        JIRA account username
  -p PASSWORD, --password PASSWORD
                        Password for specified user. If not specified you will be prompted for one
  -v, --verbose         Verbose output
  --dry-run             Run the script without uploading the XTP-*.feature file.

Example

xtp-push -u <USER> -f ./XTP-<ID>.feature  -v
Password:

Checking feature file ./XTP-<ID>.feature
OK

Getting the test set XTP ticket from XTP-<ID>.feature
OK. Test set XTP ticket is XTP-2925

Checking that test set XTP-2925 exists
OK. Result: [{'key': 'XTP-2926', 'id': 63270, 'rank': 1, 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63270', 'reporter': 'V.Naram', 'precondition': [], 'type': 'Cucumber', 'status': 'TODO', 'definition': 'Given the Telescope is in OFF state\nWhen the user sends the ON command\nThen the Telescope System is tuned ON'}, {'key': 'XTP-2932', 'id': 63276, 'rank': 2, 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63276', 'reporter': 'V.Naram', 'precondition': [], 'type': 'Cucumber', 'status': 'TODO', 'definition': 'Given the Telescope is in ON state\r\nWhen the user sends the OFF command\r\nThen the Telescope System is tuned OFF'}, {'key': 'XTP-2933', 'id': 63277, 'rank': 3, 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63277', 'reporter': 'V.Naram', 'precondition': [], 'type': 'Cucumber', 'status': 'TODO', 'definition': 'Given the Telescope is in <start_state> state\nWhen the user sends the <command> command\nThen the Telescope System goes to <end_state>\n\n\tExamples: \n\t\t| start_state | command | end_state | \n\t\t| OFF         | ON      | ON        | \n\t\t| ON          | OFF     | OFF       | '}, {'key': 'XTP-3080', 'id': 63871, 'rank': 4, 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63871', 'reporter': 'V.Naram', 'assignee': 'V.Naram', 'precondition': [], 'type': 'Cucumber', 'status': 'TODO', 'definition': 'Given the Telescope is in <start_state>\nWhen the user sends the <command> command\nThen the Telescope goes to <end_state>\n\n\tExamples: \n\t\t| start_state | command | end_state | \n\t\t| OFF         | ON      | ON        | '}, {'key': 'XTP-3081', 'id': 63872, 'rank': 5, 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63872', 'reporter': 'V.Naram', 'assignee': 'V.Naram', 'precondition': [], 'type': 'Cucumber', 'status': 'TODO', 'definition': 'Given the Telescope is in <start_state>\nWhen the user sends the <command> command\nThen the Telescope goes to <end_state>\n\n\tExamples: \n\t\t| start_state | command | end_state | \n\t\t| ON          | OFF     | OFF       | '}, {'key': 'XTP-3082', 'id': 63873, 'rank': 6, 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63873', 'reporter': 'V.Naram', 'assignee': 'V.Naram', 'precondition': [], 'type': 'Cucumber', 'status': 'TODO', 'definition': 'Given the Telescope is in <start_state> state\nWhen the user sends the <command> command\nThen the Telescope System goes to <end_state>\n\n\tExamples: \n\t\t| start_state | command | end_state | \n\t\t| OFF         | ON      | ON        | \n\t\t| ON          | OFF     | OFF       | '}]

Uploading feature file ./XTP-<ID>.feature
Result: [{'id': '63278', 'key': 'XTP-2934', 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63278', 'issueType': {'id': '11600', 'name': 'Test'}}, {'id': '63279', 'key': 'XTP-2935', 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63279', 'issueType': {'id': '11600', 'name': 'Test'}}]
Associating test ticket XTP-2934 to test set XTP-2925
Result: b'[]'
Test XTP-2934 associated with XTP-2925
Associating test ticket XTP-2935 to test set XTP-2925
Result: b'[]'
Test XTP-2935 associated with XTP-2925

xtp-compare - Compare a local feature file with one on Jira

This utility ensures that the feature file locally is in line with what is in Jira. The feature file should have a tag with a XTP ticket number at the top of the file. A feature file that has been exported from Jira would have a number of scenarios that are tagged with XTP-tickets. E.g

@XTP-1834
Feature: Default

    @XTP-1834
    Scenario Outline: Upload .feature file test ticket
        Given I have entered <input_1> into the calculator

These tags are used to pull the Jira ticket from Jira and this is then compared to the local feature file.

Usage

xtp-compare --help
usage: xtp-compare [-h] (-d DIRECTORY | -f FEATURE_FILE) [-u USERNAME] [-p PASSWORD] [-v]

optional arguments:
  -h, --help            show this help message and exit
  -d DIRECTORY, --directory DIRECTORY
                        Path to the location of a directory of '*.feature' files
  -f FEATURE_FILE, --feature-file FEATURE_FILE
                        Path to the location of a 'XTP-*.feature' file
  -u USERNAME, --username USERNAME
                        JIRA account username
  -p PASSWORD, --password PASSWORD
                        When authenticating with a username, you will be prompted for a password if you don't use this option
  -v, --verbose         Verbose output

Example

xtp-compare -f ./XTP-1834.feature -u <USER>
Password:
--- XTP-1834-local.feature
+++ XTP-1834-remote.feature
@@ -1,136 +1,136 @@
 Feature: Default


    @XTP-1839 @XTP-1834 @XTP-1834 @XTP-1834.feature @XTP-1839 @another_label @XTP-<ID>.feature
    Scenario Outline: Upload .feature file test ticket
        Given I have entered <input_1> into the calculator
        And I have entered <input_2> into the calculator
        When I press <button>
        Then the result should be <output> on the screen
+
+           Examples:
+               | input_1 | input_2 | button | output |
+               | 20      | 30      | add    | 50     |
+               | 2       | 5       | add    | 7      |
+               | 0       | 40      | add    | 40     |
+               | 4       | 50      | add    | 54     |

xtp-xray-upload - Upload xray results to JIRA

The xtp-xray-upload script posts the results of the cucumber json file to JIRA. It is setup to run as a job in the skampi pipeline but can be installed and executed from the terminal.

Usage

xtp-xray-upload --help
usage: xtp-xray-upload [-h] -f CUCUMBER_FILE [-u USERNAME] [-p PASSWORD] [-v]

Uploads a cucumber results file to JIRA by way of the XRAY extension. Either use username/password or set the environment variable JIRA_AUTH for authentication.

optional arguments:
  -h, --help            show this help message and exit
  -f CUCUMBER_FILE, --cucumber-file CUCUMBER_FILE
                        Path to the cucumber JSON results file.
  -u USERNAME, --username USERNAME
                        JIRA account username
  -p PASSWORD, --password PASSWORD
                        Password for the JIRA user. If not specified and the environment variable `JIRA_AUTH` is not set then you will be prompted for one.
  -v, --verbose         Verbose output

Example

xtp-xray-upload -f build/cucumber.json -v

Uploading cucumber file build/cucumber.json to https://jira.skatelescope.org/rest/raven/1.0/import/execution/cucumber
Using JIRA_AUTH environment variable as credentials
Result: {'testExecIssue': {'id': '63850', 'key': 'XTP-3072', 'self': 'https://jira.skatelescope.org/rest/api/2/issue/63850'}, 'testIssues': {'success': [{'id': '37799', 'key': 'XTP-1081', 'self': 'https://jira.skatelescope.org/rest/api/2/issue/37799'}, {'id': '50123', 'key': 'XTP-1156', 'self': 'https://jira.skatelescope.org/rest/api/2/issue/50123'}]}, 'infoMessages': []}