Jenkins

Jenkins & CloudBees

If you use Jenkins continuous integration system for your builds. There is a TestQuality plugin available on Jenkins that makes it easy to automate your test upload.

To get plugin, within Jenkins go to Manage Jenkins > Plugin Manager, select the Available tab and search for Test Quality. Select “TestQuality Updater” and install. If you prefer to use the CLI method for uploading test results to TestQuality you can skip the Jenkins plugin and use the CLI method described later in this article.

Once you have the plugin, simply configure by going to Manage Jenkins > Configure System. You will find a section for TestQuality.

Fill in your user name and password. Then for any job, enter Configure. Select “Add post-build action”, and select TestQuality Updater. Select the Project, Test Plan and location of test results file. On your next build check TestQuality to set see Test Plan changes and Test Plan Run added.

REST

Test results can be uploaded via TestQuality REST interface. The following example shows how to do this via curl a popular command line tool.

Step 1, Authenticating

TestQuality API uses OAuth2 for authentication. Please contact Support to obtain a client key for using the API.

curl -X POST \

-H 'Content-Type: application/json' \

-H 'Accept: application/vnd.testquality.v1+json' \

-d '{"client_id": "2","client_secret":"23MBS86X7JrK4Mrr1mk4PKfo6r4zRVx9MrmxXfdX", "grant_type":"password", "username":"your@email.com", "password":"your_password"}' \

https://api.testquality.com/api/oauth/access_token

This command will return your token.

{

  "user_id":7,

  "beta_mode": true,

  "verification_ends_at": "null",

  "trial_ends_at": "2017-11-08T21:42:26+00:00",

  "subscription_ends_at": null,

  "client_id": 1,

  "client_name": "tq",

  "token_type":"Bearer",

  "expires_in":86400,

  "access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImp0aSI6IjE2NDU4MmE5YWQwZjIwOTBjNDI2OTMwMWQ5MmUyYzQ4NDQwNTkxZTNhNmE5ZmNhMzVmYjg0ZmY1NGQ4YzAwOTI4OTdmZDYxZTk4NTlhNTExIn0.eyJhdWQiOiIyIiwianRpIjoiMTY0NTgyYTlhZDBmMjA5MGM0MjY5MzAxZDkyZTJjNDg0NDA1OTFlM2E2YTlmY2EzNWZiODRmZjU0ZDhjMDA5Mjg5N2ZkNjFlOTg1OWE1MTEiLCJpYXQiOjE1MDg4ODEzNDUsIm5iZiI6MTUwODg4MTM0NSwiZXhwIjoxNTA4OTY3NzQ1LCJzdWIiOiI3Iiwic2NvcGVzIjpbXX0.n9UGtd4UpN2hfFi4pnAcossv_24J0IJjcFHoPNETbIXuJcZUziLa1S1iJjvqklq3J4GX2GzIG0l0Mwh6Ek_lrXej0aBxB-j1eYz2_VtTvr92IyaMcJVQ6iVb_y5WEIjM8K_xxseFFUQB2cBUzTgBudi1FjhSVby7M7DoEdzsRkiT5RlbszwjpWhrcvs9YD08u-m286AjmWeOdD1vPA837wh0eU1Ay5r_sWMdlmcmTdWHcCkqz50Z12DH3cIwtxByi8nYdYCwnW0ubcNoKRaY8FDjWUQBCfVezYjHjIzQy82FbvZxwRWaJ2KKeo7f6HZT3Wt-xUis-ci0Ac2z4D8TkLgifBPeloCNUbAl2mNDp3RlOCGwWcHPnZUS7282V887mF6OHWSzGJ2uzKqC_EgJPGmz2-CeILpm13wO-cy_0DIiTtjCZCezpLJt95El5NRVOouymSJHCgRrCZ0y33-iy9EJqYiTb1m6QSyyXd3aH90rPJZ_VukKyrPLHHMrMaggp_mEv8Dx49w_6My297J8ew9bqO9y_-IO2IU5O_fmtJFWz6LPcoiw4a4954G6O_XBqjD9BzuYej34jjwmmPLkCOGlo-U4NW70QdTqQ-v61Jij7L9KV69nlTFoYriAB9RJC9ncxXyw1Oi_I4OeYYXIQYe1WXyUSu3z2rRvYy678Xg",

  "refresh_token":"def5020009f4bd152abf47c3f61c8a5dd1153c570bebf374d8d2a012afa0496c13b86d83344f873bc41759f9681d9529472ea30a19dd5f5a995863b1a9d9bcc0272720be94474bc4812fb46a6df8c8dda9343812d089c8945a693fa93a313b9aff7c0dd9cc1e49fde6e117db77e9ae614bdf8d49872c728c966577996ae23bbd552870ac0c4183946e65e027aaa12fb119742275359360ccb01a2ca970582c1e7f5db11a6d0921a7e31d33461acffb60db4f56276b7b9197c925e20e5f4a976fa20319788014fddc08122f84aa1a394e56a2182a4a0fabe25ab1a1e2b5a72fd9d2aafc2af22bfb117e68bc6efec8b004bab5c285af90a84e6ef6550fc16ada5c441218af9f3114a9297605b5d0f7b7540c1f45b940c50d5d202c66933a33eda3ff5534df14af09060b4442a65c8135cc0bd2f2cebe963d18868788555f7d215111bcda8cd303b60b563a6a27bcd3e25aa0f259bfeda5c26b8c057547bea7c9014c"

}

The access token is in the “access_token” property.

 

Step 2, Obtain Test Plan ID

To obtain the Test Plan identification number, we will query using the plan name (could use key if you prefer).

curl -X GET \

-H 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImp0aSI6IjE2NDU4MmE5YWQwZjIwOTBjNDI2OTMwMWQ5MmUyYzQ4NDQwNTkxZTNhNmE5ZmNhMzVmYjg0ZmY1NGQ4YzAwOTI4OTdmZDYxZTk4NTlhNTExIn0.eyJhdWQiOiIyIiwianRpIjoiMTY0NTgyYTlhZDBmMjA5MGM0MjY5MzAxZDkyZTJjNDg0NDA1OTFlM2E2YTlmY2EzNWZiODRmZjU0ZDhjMDA5Mjg5N2ZkNjFlOTg1OWE1MTEiLCJpYXQiOjE1MDg4ODEzNDUsIm5iZiI6MTUwODg4MTM0NSwiZXhwIjoxNTA4OTY3NzQ1LCJzdWIiOiI3Iiwic2NvcGVzIjpbXX0.n9UGtd4UpN2hfFi4pnAcossv_24J0IJjcFHoPNETbIXuJcZUziLa1S1iJjvqklq3J4GX2GzIG0l0Mwh6Ek_lrXej0aBxB-j1eYz2_VtTvr92IyaMcJVQ6iVb_y5WEIjM8K_xxseFFUQB2cBUzTgBudi1FjhSVby7M7DoEdzsRkiT5RlbszwjpWhrcvs9YD08u-m286AjmWeOdD1vPA837wh0eU1Ay5r_sWMdlmcmTdWHcCkqz50Z12DH3cIwtxByi8nYdYCwnW0ubcNoKRaY8FDjWUQBCfVezYjHjIzQy82FbvZxwRWaJ2KKeo7f6HZT3Wt-xUis-ci0Ac2z4D8TkLgifBPeloCNUbAl2mNDp3RlOCGwWcHPnZUS7282V887mF6OHWSzGJ2uzKqC_EgJPGmz2-CeILpm13wO-cy_0DIiTtjCZCezpLJt95El5NRVOouymSJHCgRrCZ0y33-iy9EJqYiTb1m6QSyyXd3aH90rPJZ_VukKyrPLHHMrMaggp_mEv8Dx49w_6My297J8ew9bqO9y_-IO2IU5O_fmtJFWz6LPcoiw4a4954G6O_XBqjD9BzuYej34jjwmmPLkCOGlo-U4NW70QdTqQ-v61Jij7L9KV69nlTFoYriAB9RJC9ncxXyw1Oi_I4OeYYXIQYe1WXyUSu3z2rRvYy678Xg' \

-H 'Content-Type: application/json' \

-H 'Accept: application/vnd.testquality.v1+json' \

https://api.testquality.com/api/plan?name=Smoke

This will return a result of:

{

  "current_page":1,

  "data":[

    { 

      "id":53,

      "created_by":7,

      "updated_by":7,

      "epoch":1,

      "project_id":22,

      "assigned_to_tester":7,

      "virtual":null,

      "name":"Smoke",

      "description":"Basic Smoke Tests",

      "client_id":1,

      "key":18,

      "retain_runs":0,

      "created_at":"2017-09-28T23:15:32.533000+00:00",

      "updated_at":"2017-10-24T14:52:40.938000+00:00",

      "metadata_model":"Plan"

    }

  ],

  "first_page_url":"http:\/\/testquality:8001\/api\/plan?name=Smoke&page=1",

  "from":1,

  "last_page":1,

  "last_page_url":"http:\/\/testquality:8001\/api\/plan?name=Smoke&page=1",

  "next_page_url":null,"path":"http:\/\/testquality:8001\/api\/plan",

  "per_page":15,

  "prev_page_url":null,

  "to":1,

  "total":1

}

We just need the id, in this case 53.

 

Step 3, Upload XML File

The curl form function that is used to upload file.

curl -X POST \

-F 'file=@/Users/jamespitts/Dev/testQualityApp/test-results/karma/Chrome_61.0.3163_(Mac_OS_X_10.12.6)/test-results.xml' \

-H 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImp0aSI6ImQ1OTdhMjNiNjY2NDMwYWMyYjM5MWRhMDRkYzNjYjhiMzZlNmZmNzRmMTYxOGNiYTM3MjQ3NjNkMjIxMDMyZDk2ZDIyM2NkM2Q5MWRiNjc0In0.eyJhdWQiOiIyIiwianRpIjoiZDU5N2EyM2I2NjY0MzBhYzJiMzkxZGEwNGRjM2NiOGIzNmU2ZmY3NGYxNjE4Y2JhMzcyNDc2M2QyMjEwMzJkOTZkMjIzY2QzZDkxZGI2NzQiLCJpYXQiOjE1MDg4ODIwNTIsIm5iZiI6MTUwODg4MjA1MiwiZXhwIjoxNTA4OTY4NDUyLCJzdWIiOiI3Iiwic2NvcGVzIjpbXX0.RQl8acC5HtfI7LGCYdrEPRkgeJ52-hRU9-sN7o1Rc-ITrnynlMdd60J8oDqRGAfQrppKx7MEM4S9ygznpzsNrHcqbKPnTnu6tfqws6k1kJcglGlgTXH_YRIbR8V7kao_pcUouj5tUgNLnKWVbHRBpoKTy1xO9nQZEjtWkWCkZCDg0xd6MRtyyYDINLf-K9QQZtpbqKLBPWQVYKk74YoDA75S19HzbcQQhI1J7XqqizWdaUmfzsbK74GcS5ve3v_VYeXafOnuC8dSmytsOoFohODJz9urzKtIKVKiD-tXXGf7opLKrW44XJVFrZ9kImYujC6bo4Bberg-1UGvScgd6AJ2p2NueMUiUGbs7CYEW-DP_12viVnxGgfiMhmwBmjjwgHlsLGZG8JzUIL_gHRzpyGTTQY56RAXuAslMGDct5WmEz2ErQxWucLf1q294t6PjCXPICIUL41IqKQMe_X0koPyA1RzcJEom9Sk5T3KyMg0bJnYICX0lseVT7ul2tMjP8CWOCvVVpMI81OcCoRuP9Oxw-xYZsDT7hY1RM1phrHk0gR5tPZyLzd_LZP7DmlENbjA0yioqBlZTBSGL-Rvbn0GZPREc87Fr0K5YB8NsAKgcdG-i6U7Y08Bw2kvTrCz2ly08J6qM7FJADP0_4Budh6CsuzaNxj-iWqB_lr301U' \

-H 'Accept: application/vnd.testquality.v1+json' \

https://api.testquality.com/api/plan/53/junit_xml

Note: 53 in /plan/53/junit_xml

Success!

 

{

  "run_url":"http:\/\/localhost:3000\/project\/36\/run\/414",

  "total":368,

  "passed":366,

  "failed":0,

  "skipped":2,

  "time":"9.452",

  "start_time":"2017-10-25T01:22:12.817583+00:00",

  "end_time":"2017-10-25T01:22:12.452"

}

 

Other Scripts

This flow can easily be used in gulp/grunt using “request” framework. Or PowerShell, basically anything that supports REST API.

Integrating Jenkins or Cloudbees via the TestQuality Command Line Interface (CLI)

The TestQuality Command Line Interface allows you to upload your automated test results from Jenkins or Cloudbees to TestQuality. Automated test results must be output in JUnit XML format, which most test automation tools will provide. Test result attachments and related defects are also supported through test name tags or console outputs.

For DEFECTS we expect the following tag format for both test name tags and console ouptuts.

• GitHub Defects [[DEFECT|22]]
• Jira Defects [[DEFECT|TQ-123]]

For ATTACHMENTS the following format is expected:

• Attachments from test name tag [[ATTACHMENT|ScreenshotFileName.png]]
• Attachments from console output [[ATTACHMENT|path/to/file]]

Note that for attachment test name tag usage you will neet to use ‘run_result_output_dir’ option to specify the test result output directory.

Download TestQuality Command Line Interface

Download CLI tools from http://cli.testquality.com/

CLI tools are available for Windows, MacOS, Linux, and Alpine.

Requirements

In order to upload xml you will need to:

1. Create a target Test Plan
2. Authenticate with TestQuality
3. Run command to upload files

Compiled Commands

There are compiled commands for

• Windows
• MacOS
• Linux
• Alpine

Commands can be downloaded from cli.testquality.com

Note: For alpine you must install libstdc++

apk add –no-cache libstdc++

Usage

For list of commands

yarn start –help

or

testquality-macos –help

For command help

yarn start login –help

or

testquality-macos login –help

Save

Include –save to save tokens to use with other commands.

Example

Example workflow.

testquality-macos login larry@bitmodern.com *password* –save

testquality-macos upload_test_run ‘sampleXml/*.xml’ –project_name=Test –plan_name=Test

You can also create a manual test plan run.

testquality-macos create_manual_run –project_name=My_Project –plan_name=My_Test_Plan –run_name=Test_Run_Name

CSV Files

testquality-macos upload_csv ./test_run_results.csv –cf ./test_run_results.config