Usage of REST APIs has been gaining popularity in the Networking industry as more vendors are realizing its ease of use and the standard interface it provides. With HTTP based REST API, a vendor can provide a standard interface to the hardware’s features that helps in management, verification and automation.
The primary interface to OpenDaylight controllers features is the YANG model based REST API, backed by the standard IETF RESTCONF specification.
As a Senior QA Engineer working on QA aspects of OpenDaylight controller, I will describe various techniques of automating OpenDaylight REST API testing using Postman tools.
A typical automation testing workflow of OpenDaylight can be classified as:
- Install a feature
- Verify that the feature is installed
- Execute a a northbound REST API call against an underlying fabric or application
- Assert the impact of the call
- Run a test (positive, negative, performance, scale etc)
- Goto 3
OpenDaylight primarily uses Robot Framework for automation testing. Although Robot Framework is very easy to learn as it uses plain English as keywords for test cases, it gets complicated for parsing data structures or executing loops.
I will share my experience on using Postman and its features, where test cases are vastly simplified and as a side effect, its much faster than executing same tests in Robot Framework.
Most of the SDN community is familiar with Postman, the UI tool for executing REST APIs. But not many are familiar with its companion tool newman, which is available as an NPM module (Node Package Manager). Using the newman utility, all Postman collections can be run on CLI as a bulk action. Some of the features of Postman are:
- grouping a set of similar logical APIs through collections
- write tests on API to parse correct return codes and values
- ability to create environments for a call and switch environments
- ability to generalize constants through environment
- running a set of collections workflows using Runner feature
- generate reports and documents
- collaborate with various teams by sharing collections and environments. Sharing can be done through a shareable link or json files
Postman can be either installed through application or as a Google Chrome browser extension. The Google Chrome extension will be deprecated some time soon. So it is recommended to install the native package, which is available for most platforms. If you sign up an account with Postman, it allows you to share collections between multiple systems, which can be a great benefit to test from various platforms.
A typical request in Postman has many options like selecting type of request, authorization, headers, body (request payload), and a very easy way to store the response of the request the user sends. This can be quite handy while writing tests. More about tests would be covered in the subsequent sections.
Collection is a powerful feature of Postman. We can group a set of calls for a particular workflow and make it a Postman collection. Each collection can result in a set of APIs that verify a feature or functionality or a sequence of steps. For example following screenshot shows a collection for verifying data migration feature in OpenDaylight.
The environment feature in Postman can be used to define variables and change them for different scenarios. These variables values would be substituted at runtime from the environment. A typical call on ODL inventory would be like this.
Testing the Response
Let’s say you have defined a collection with set of REST APIs for an environment. The response would then have to be inspected for its values. This can be tedious especially if the response runs into hundreds of lines. Even though Postman can prettify json response, it still takes time to ensure a particular key-value is what is expected.
The Collection Runner feature is very useful to verify a particular postman collection and a specific environment with various control options. Using this feature, a user can select a collection and an environment with options like, iterations, delay between each request and logging. When the runner is invoked on a collection with all these parameters set, it would generate a report which can be shared across. A typical collection runner looks like the following:
While the Postman testing is packed with features, it is still a chore to execute tests via UI tools. For effective automation, testing should be performed via cli. This is exactly what Newman helps with. You have to install NodeJS first to use newman. Goto NodeJS and install the latest LTS version of NodeJS for your platform.
From the terminal, execute
npm install newman -g
to install newman globally. The -g parameter will ensure newman is installed in ~/.npm and a binary link is created to execute newman from any directory.
The newman tool can run any collection with a given environment in cli. We can source the collection and environment in the form of an URL or a json files to the newman client. There are many options provided by newman to control your output. See
newman for a complete list of options. The newman utility can also generate a report in html, xml, plain text format. A typical newman run looks like this:
newman run intf_snake_test_collection -e specifci_env -r cli --reporter-cli-export clirun --disable-unicode
Integrating Newman with Robot Framework
At LuminaNetworks, we use Jenkins as our CI tool and Robot Framework as our primary automation testing tool. Robot Framework is a keyword based framework built on top of Python. From Robot Framework you can call nemwan tool, execute the collections, grab the results and inspect for success / failure and post the status to Jenkins.
Well, lets say you have run all your tests successfully. How do you let the world know about the APIs used? Documentation can be a tedious job, but Postman provides a very nice feature for automating documentation and assisting in automating REST API calls. For every collection in postman, there is an option to generate documentation. Once we generate the documentation, a public url for that particular documentation is generated that is shareable. Using this link, we can generate snippets of code to automate the APIs in that particular collection. A good use of this is to share the urls to customers. Here is a screenshot that shows how to generate and publish documentation:
In summary, Postman (and newman) is a very fast and efficient tool to verify, automate, document and publish the REST APIs. These Postman features help to collaborate and build on existing work very easily and without any hassle. Since it tool can also be integrated into CI easily, I consider it a one stop shop for the API development and automation testing.