Automated Blackbox and Whitebox Testing of RESTful APIs with EvoMaster

Abstract

RESTful APIs are very popular in industry, especially when developing enterprise systems using a microservice architecture. Testing such APIs is challenging, as tests will be composed of not only HTTP calls, but also settings of the environment, like databases. Different blackbox testing techniques have been shown to easily find real faults in many RESTful APIs, with very little human effort from software engineers. However, whitebox techniques could lead to much better results, although having an up-front cost for the engineers. In this paper, we report on the use of the open-source tool EvoMaster, on eight RESTful APIs. We show how EvoMaster can be used to automatically generate test cases that can find several bugs, even when using a naive blackbox approach. When enhancing the search with whitebox information, significantly better results are achieved. However, there are several challenges that need to be taken into account when an engineer wants to use a tool such as EvoMaster to test their projects.