How to Run Continuous API Testing

Software testing is an integral part of the software development life cycle. It is a process by which a software’s utmost quality is both insured and assured simultaneously. There are mainly 2 ways of testing a software; one is by conducting the test manually and the other by automation.

The designation of the person who is responsible for performing the tests is generally called a Software Quality Assurance Engineer (a.k.a SQA) in most companies around the world. SQAs usually runs the manual tests by eyeballing the software’s particular feature and verifies that the behaviour is as written in the requirements specifications. Software test automation does exactly the same thing, except the feature behaviours are verified programmatically. With the initiation of agile methodology, the need for software test automations have significantly increased over time. This is because in the era of agile methodology, testing is often required compared to that of the old waterfall methodology, and due to resource and time limitations not all tests can be covered manually.

The earlier the tests are conducted the less the software would be prone to any bugs. Nowadays, one of the earliest phases of feature development is API development. If the functionalities of the APIs are thoroughly tested then there would be fewer probability of discovering bugs during the end to end integration testing and acceptance testing. This article will elaborate on how certain tools can be leveraged to achieve the goal of continuous API testing.

Postman

This is a tool that an SQA can use to create API tests using javascript. At first, the SQA has to create a suite of APIs to test their functionality and it’s known as a Collection in Postman. Once the test scripts are created, the tests can be conducted in Command Line Interface (CLI) using Newman, which is the non-gui mode of Postman. Once the test results have been verified in Newman, the SQA can then push the test collection files, which consists of the APIs and their test cases written in javascript, in a github repository. The github project will then need to be configured in a way that with every merged commit in the repository it will then trigger CircleCI, which is a continuous integration-development pipeline tool, to run the tests.

By following the steps below, one can actualize the above statement. To run postman tests in CircleCI leveraging AWS S3 instance:

Step 1) Create a postman test and validate it manually in your local machine using postman runner.

Step 2) Export your collection to your local machine. This article from Postman about importing and exporting can help you to learn how to export collection in your local machine

N.B. Please make sure that the collection is in .json format.

Also, don't forget to download the environment file.

Step 3) Link up your local machine to our AWS S3 instance. View the quick reference Connecting your local machine to an S3 bucket on GitHub on how to do so.

‍Step 4) Copy your postman collection & environment file from your local machine to AWS S3 Instance.
AWS CLI Command to copy a file from local machine to AWS S3:
aws s3 cp <path to="" file_here.extension=""></path> s3://[Our-S3-bucket-name-here/]

Step 5) Sign up for CircleCI trial version from circleci.com. Use your GitHub account to sign up in Circle CI.

Step 6) Create a circle.yml file in your local machine. The following is a sample command to insert in the circle.yml file:

{% c-block language="JavaScript" %}
machine:
node:
version: 6.11.1
dependencies:
post:
- npm install -g newman
- aws s3 cp s3://required-installation-packages/CI_Test_postman_collection.json ~
- newman run ~/CI_Test_postman_collection.json
{% c-block-end %}

Explanation of the above circle.yml file:

{% c-block language="JavaScript" %}
machine:
node:
version: 6.11.1
I'm stating CircleCI to download node version 6.11.1. NPM gets installed by default with node.js installation. NPM required for installing Newman. In order to run the Postman test from CLI, Newman is required.
dependencies:
post:
- npm install -g newman
- aws s3 cp s3://required-installation-packages/CI_Test_postman_collection.json ~
- newman run ~/CI_Test_postman_collection.json
{% c-block-end %}

npm install -g newman : Installs Newman in CircleCI using NPM.

aws s3 cp s3://required-installation-packages/CI_Test_postman_collection.json ~ : Copies postman collection from AWS S3 to CircleCI's home directory (~).

newman run ~/CI_Test_postman_collection.json : Runs postman test

Step 7) Login to your Github account. Create a project. Make your first commit. The instruction to commit for the first time will be clearly provided by GitHub as soon as you create a project inside it. Once the first commit takes place, then you will have the option to upload files in your project. Upload the circle.yml file in your project.

Step 8) Login to your CircleCI account and go to the app. Then Go To App link which should be located at the top right corner of CircleCI once you log in. Click on the Builds tab located at the left top sidebar of CircleCI. You should be able to see your GitHub project name in this page and a small settings icon beside it. Click on that settings icon. Then click on Follow Project. Then click back on the Builds tab on the left sidebar of CircleCI. Also, insert the AWS credentials (Access Key ID and Secret Access Key) in CircleCI; click on Builds tab, then click on the project settings, click on AWS Permission and you can insert the AWS credentials in this page. The builds will start automatically as soon as it detects a change in your GitHub project.

You’re all done!