Jenkins & CloudBees
If you use Jenkins continuous integration system for your builds. There is a TestQuality plugin available on Jenkins that makes it easy to automate your test upload.
To get plugin, within Jenkins go to Manage Jenkins > Plugin Manager, select the Available tab and search for Test Quality. Select “TestQuality Updater” and install. If you prefer to use the CLI method for uploading test results to TestQuality you can skip the Jenkins plugin and use the CLI method described later in this article.
Once you have the plugin, simply configure by going to Manage Jenkins > Configure System. You will find a section for TestQuality.

Fill in your user name and password. Then for any job, enter Configure. Select “Add post-build action”, and select TestQuality Updater. Select the Project, Test Plan and location of test results file. On your next build check TestQuality to set see Test Plan changes and Test Plan Run added.
REST
Test results can be uploaded via TestQuality REST interface. The following example shows how to do this via curl a popular command line tool.
Step 1, Authenticating
TestQuality API uses OAuth2 for authentication. Please contact Support to obtain a client key for using the API.
curl -X POST \
-H 'Content-Type: application/json' \
-H 'Accept: application/vnd.testquality.v1+json' \
-d '{"client_id": "2","client_secret":"23MBS86X7JrK4Mrr1mk4PKfo6r4zRVx9MrmxXfdX", "grant_type":"password", "username":"your@email.com", "password":"your_password"}' \
https://api.testquality.com/api/oauth/access_token
This command will return your token.
{
"user_id":7,
"beta_mode": true,
"verification_ends_at": "null",
"trial_ends_at": "2017-11-08T21:42:26+00:00",
"subscription_ends_at": null,
"client_id": 1,
"client_name": "tq",
"token_type":"Bearer",
"expires_in":86400,
"access_token":"eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImp0aSI6IjE2NDU4MmE5YWQwZjIwOTBjNDI2OTMwMWQ5MmUyYzQ4NDQwNTkxZTNhNmE5ZmNhMzVmYjg0ZmY1NGQ4YzAwOTI4OTdmZDYxZTk4NTlhNTExIn0.eyJhdWQiOiIyIiwianRpIjoiMTY0NTgyYTlhZDBmMjA5MGM0MjY5MzAxZDkyZTJjNDg0NDA1OTFlM2E2YTlmY2EzNWZiODRmZjU0ZDhjMDA5Mjg5N2ZkNjFlOTg1OWE1MTEiLCJpYXQiOjE1MDg4ODEzNDUsIm5iZiI6MTUwODg4MTM0NSwiZXhwIjoxNTA4OTY3NzQ1LCJzdWIiOiI3Iiwic2NvcGVzIjpbXX0.n9UGtd4UpN2hfFi4pnAcossv_24J0IJjcFHoPNETbIXuJcZUziLa1S1iJjvqklq3J4GX2GzIG0l0Mwh6Ek_lrXej0aBxB-j1eYz2_VtTvr92IyaMcJVQ6iVb_y5WEIjM8K_xxseFFUQB2cBUzTgBudi1FjhSVby7M7DoEdzsRkiT5RlbszwjpWhrcvs9YD08u-m286AjmWeOdD1vPA837wh0eU1Ay5r_sWMdlmcmTdWHcCkqz50Z12DH3cIwtxByi8nYdYCwnW0ubcNoKRaY8FDjWUQBCfVezYjHjIzQy82FbvZxwRWaJ2KKeo7f6HZT3Wt-xUis-ci0Ac2z4D8TkLgifBPeloCNUbAl2mNDp3RlOCGwWcHPnZUS7282V887mF6OHWSzGJ2uzKqC_EgJPGmz2-CeILpm13wO-cy_0DIiTtjCZCezpLJt95El5NRVOouymSJHCgRrCZ0y33-iy9EJqYiTb1m6QSyyXd3aH90rPJZ_VukKyrPLHHMrMaggp_mEv8Dx49w_6My297J8ew9bqO9y_-IO2IU5O_fmtJFWz6LPcoiw4a4954G6O_XBqjD9BzuYej34jjwmmPLkCOGlo-U4NW70QdTqQ-v61Jij7L9KV69nlTFoYriAB9RJC9ncxXyw1Oi_I4OeYYXIQYe1WXyUSu3z2rRvYy678Xg",
"refresh_token":"def5020009f4bd152abf47c3f61c8a5dd1153c570bebf374d8d2a012afa0496c13b86d83344f873bc41759f9681d9529472ea30a19dd5f5a995863b1a9d9bcc0272720be94474bc4812fb46a6df8c8dda9343812d089c8945a693fa93a313b9aff7c0dd9cc1e49fde6e117db77e9ae614bdf8d49872c728c966577996ae23bbd552870ac0c4183946e65e027aaa12fb119742275359360ccb01a2ca970582c1e7f5db11a6d0921a7e31d33461acffb60db4f56276b7b9197c925e20e5f4a976fa20319788014fddc08122f84aa1a394e56a2182a4a0fabe25ab1a1e2b5a72fd9d2aafc2af22bfb117e68bc6efec8b004bab5c285af90a84e6ef6550fc16ada5c441218af9f3114a9297605b5d0f7b7540c1f45b940c50d5d202c66933a33eda3ff5534df14af09060b4442a65c8135cc0bd2f2cebe963d18868788555f7d215111bcda8cd303b60b563a6a27bcd3e25aa0f259bfeda5c26b8c057547bea7c9014c"
}
The access token is in the “access_token” property.
Step 2, Obtain Test Plan ID
To obtain the Test Plan identification number, we will query using the plan name (could use key if you prefer).
curl -X GET \
-H 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImp0aSI6IjE2NDU4MmE5YWQwZjIwOTBjNDI2OTMwMWQ5MmUyYzQ4NDQwNTkxZTNhNmE5ZmNhMzVmYjg0ZmY1NGQ4YzAwOTI4OTdmZDYxZTk4NTlhNTExIn0.eyJhdWQiOiIyIiwianRpIjoiMTY0NTgyYTlhZDBmMjA5MGM0MjY5MzAxZDkyZTJjNDg0NDA1OTFlM2E2YTlmY2EzNWZiODRmZjU0ZDhjMDA5Mjg5N2ZkNjFlOTg1OWE1MTEiLCJpYXQiOjE1MDg4ODEzNDUsIm5iZiI6MTUwODg4MTM0NSwiZXhwIjoxNTA4OTY3NzQ1LCJzdWIiOiI3Iiwic2NvcGVzIjpbXX0.n9UGtd4UpN2hfFi4pnAcossv_24J0IJjcFHoPNETbIXuJcZUziLa1S1iJjvqklq3J4GX2GzIG0l0Mwh6Ek_lrXej0aBxB-j1eYz2_VtTvr92IyaMcJVQ6iVb_y5WEIjM8K_xxseFFUQB2cBUzTgBudi1FjhSVby7M7DoEdzsRkiT5RlbszwjpWhrcvs9YD08u-m286AjmWeOdD1vPA837wh0eU1Ay5r_sWMdlmcmTdWHcCkqz50Z12DH3cIwtxByi8nYdYCwnW0ubcNoKRaY8FDjWUQBCfVezYjHjIzQy82FbvZxwRWaJ2KKeo7f6HZT3Wt-xUis-ci0Ac2z4D8TkLgifBPeloCNUbAl2mNDp3RlOCGwWcHPnZUS7282V887mF6OHWSzGJ2uzKqC_EgJPGmz2-CeILpm13wO-cy_0DIiTtjCZCezpLJt95El5NRVOouymSJHCgRrCZ0y33-iy9EJqYiTb1m6QSyyXd3aH90rPJZ_VukKyrPLHHMrMaggp_mEv8Dx49w_6My297J8ew9bqO9y_-IO2IU5O_fmtJFWz6LPcoiw4a4954G6O_XBqjD9BzuYej34jjwmmPLkCOGlo-U4NW70QdTqQ-v61Jij7L9KV69nlTFoYriAB9RJC9ncxXyw1Oi_I4OeYYXIQYe1WXyUSu3z2rRvYy678Xg' \
-H 'Content-Type: application/json' \
-H 'Accept: application/vnd.testquality.v1+json' \
https://api.testquality.com/api/plan?name=Smoke
This will return a result of:
{
"current_page":1,
"data":[
{
"id":53,
"created_by":7,
"updated_by":7,
"epoch":1,
"project_id":22,
"assigned_to_tester":7,
"virtual":null,
"name":"Smoke",
"description":"Basic Smoke Tests",
"client_id":1,
"key":18,
"retain_runs":0,
"created_at":"2017-09-28T23:15:32.533000+00:00",
"updated_at":"2017-10-24T14:52:40.938000+00:00",
"metadata_model":"Plan"
}
],
"first_page_url":"http:\/\/testquality:8001\/api\/plan?name=Smoke&page=1",
"from":1,
"last_page":1,
"last_page_url":"http:\/\/testquality:8001\/api\/plan?name=Smoke&page=1",
"next_page_url":null,"path":"http:\/\/testquality:8001\/api\/plan",
"per_page":15,
"prev_page_url":null,
"to":1,
"total":1
}
We just need the id, in this case 53.
Step 3, Upload XML File
The curl form function that is used to upload file.
curl -X POST \
-F 'file=@/Users/jamespitts/Dev/testQualityApp/test-results/karma/Chrome_61.0.3163_(Mac_OS_X_10.12.6)/test-results.xml' \
-H 'Authorization: Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiIsImp0aSI6ImQ1OTdhMjNiNjY2NDMwYWMyYjM5MWRhMDRkYzNjYjhiMzZlNmZmNzRmMTYxOGNiYTM3MjQ3NjNkMjIxMDMyZDk2ZDIyM2NkM2Q5MWRiNjc0In0.eyJhdWQiOiIyIiwianRpIjoiZDU5N2EyM2I2NjY0MzBhYzJiMzkxZGEwNGRjM2NiOGIzNmU2ZmY3NGYxNjE4Y2JhMzcyNDc2M2QyMjEwMzJkOTZkMjIzY2QzZDkxZGI2NzQiLCJpYXQiOjE1MDg4ODIwNTIsIm5iZiI6MTUwODg4MjA1MiwiZXhwIjoxNTA4OTY4NDUyLCJzdWIiOiI3Iiwic2NvcGVzIjpbXX0.RQl8acC5HtfI7LGCYdrEPRkgeJ52-hRU9-sN7o1Rc-ITrnynlMdd60J8oDqRGAfQrppKx7MEM4S9ygznpzsNrHcqbKPnTnu6tfqws6k1kJcglGlgTXH_YRIbR8V7kao_pcUouj5tUgNLnKWVbHRBpoKTy1xO9nQZEjtWkWCkZCDg0xd6MRtyyYDINLf-K9QQZtpbqKLBPWQVYKk74YoDA75S19HzbcQQhI1J7XqqizWdaUmfzsbK74GcS5ve3v_VYeXafOnuC8dSmytsOoFohODJz9urzKtIKVKiD-tXXGf7opLKrW44XJVFrZ9kImYujC6bo4Bberg-1UGvScgd6AJ2p2NueMUiUGbs7CYEW-DP_12viVnxGgfiMhmwBmjjwgHlsLGZG8JzUIL_gHRzpyGTTQY56RAXuAslMGDct5WmEz2ErQxWucLf1q294t6PjCXPICIUL41IqKQMe_X0koPyA1RzcJEom9Sk5T3KyMg0bJnYICX0lseVT7ul2tMjP8CWOCvVVpMI81OcCoRuP9Oxw-xYZsDT7hY1RM1phrHk0gR5tPZyLzd_LZP7DmlENbjA0yioqBlZTBSGL-Rvbn0GZPREc87Fr0K5YB8NsAKgcdG-i6U7Y08Bw2kvTrCz2ly08J6qM7FJADP0_4Budh6CsuzaNxj-iWqB_lr301U' \
-H 'Accept: application/vnd.testquality.v1+json' \
https://api.testquality.com/api/plan/53/junit_xml
Note: 53 in /plan/53/junit_xml
Success!
{
"run_url":"http:\/\/localhost:3000\/project\/36\/run\/414",
"total":368,
"passed":366,
"failed":0,
"skipped":2,
"time":"9.452",
"start_time":"2017-10-25T01:22:12.817583+00:00",
"end_time":"2017-10-25T01:22:12.452"
}
Other Scripts
This flow can easily be used in gulp/grunt using “request” framework. Or PowerShell, basically anything that supports REST API.
Integrating Jenkins or Cloudbees via the TestQuality Command Line Interface (CLI)
The TestQuality Command Line Interface allows you to upload your automated test results from Jenkins or Cloudbees to TestQuality. Automated test results must be output in JUnit XML format, which most test automation tools will provide. Test result attachments and related defects are also supported through test name tags or console outputs.
For DEFECTS we expect the following tag format for both test name tags and console ouptuts.
- GitHub Defects [[DEFECT|22]]
- Jira Defects [[DEFECT|TQ-123]]
For ATTACHMENTS the following format is expected:
- Attachments from test name tag [[ATTACHMENT|ScreenshotFileName.png]]
- Attachments from console output [[ATTACHMENT|path/to/file]]
Note that for attachment test name tag usage you will neet to use ‘run_result_output_dir’ option to specify the test result output directory.
Download TestQuality Command Line Interface
Download CLI tools from http://cli.testquality.com/
CLI tools are available for Windows, MacOS, Linux, and Alpine.
Requirements
In order to upload xml you will need to:
- Create a target Test Plan
- Authenticate with TestQuality
- Run command to upload files
Compiled Commands
There are compiled commands for
- Windows
- MacOS
- Linux
- Alpine
Commands can be downloaded from cli.testquality.com
Note: For alpine you must install libstdc++
apk add –no-cache libstdc++
Usage
For list of commands
yarn start –help
or
testquality-macos –help
For command help
yarn start login –help
or
testquality-macos login –help
Save
Include –save to save tokens to use with other commands.
Example
Example workflow.
testquality-macos login larry@bitmodern.com *password* –save
testquality-macos upload_test_run ‘sampleXml/*.xml’ –project_name=Test –plan_name=Test
You can also create a manual test plan run.
testquality-macos create_manual_run –project_name=My_Project –plan_name=My_Test_Plan –run_name=Test_Run_Name
CSV Files
testquality-macos upload_csv ./test_run_results.csv –cf ./test_run_results.config