Tosca meets Studiosus Group

Creating sustainable test automation
in an agile world.

Studiosus Group is a Munich based tour operator with focus on cultural tours. Founded in 1954 it flourished to produce a €276 million turnover in 2019, serving 103,430 customers. As any established tour operator, Studiosus Group faced the challenge of developing an IT infrastructure and web presence for its subsidiary company Marco Polo Reisen to fulfill modern customer expectations and to differentiate themselves from competitors.

The agency’s goal was to rebuild the online travel market for one of its major product lines Marco Polo Young Line Travel. Implementation of the front and back end software components was assigned to experienced PRODYNA developers whilst Qnit was trusted with establishing the testing and test automation processes due to their expertise in the field of quality assurance.

The overall perspective:

In general, the project was a huge leap in terms of technological transition. One core task was to completely rework the front end design and technologies aiming to achieve an attractive modern look and improved usability.

Full device responsiveness had to be guaranteed and was achieved using the React framework. With a vast amount of new features that needed to be implemented, the overall challenge was further increased by the fact that the underlying content management system was simultaneously shifted to a new technology.

Incorporating the Magnolia CMS enabled current and future content editors to manage content in an easy and flexible way.

The testing perspective:

Any test execution, especially when automated, depends on predictable test results. Handling a dynamic test object, in this case a production-synched travel market with constantly changing offerings, is challenging.

Furthermore, the test had to detect unique errors in distinct travel combinations, or in specific search results. Not only the functionality but also the content, assembled from the new CMS, had to be verified.
Since exhaustive testing is not possible due to limited test resources, a balance had to be found between testing broad and testing deep in order to cover a maximum of the overall risk.

Working in an agile project with highly dynamic test objects required the ability to automate tests for new features at high speed while being able to quickly amend the existing automation in case the last build broke it.

Maintainability, as well as stability of the automation, was not only of importance for the development cycles. Since the customer intended to reuse test cases for regression tests in further developmental efforts, they needed to be clean and understandable for other people that might be working with the artefacts in the future.

Test approach:

Implementing the test automation required, first of all, a clear analysis of the requirements in order to identify where automation can add value and where it can’t. The process of verifying the URLs of several thousand images is more predestined for automation than the correct positioning of a design element, for example.

In some cases, the dynamics within the expected results made it necessary to use the productive environment as a test oracle and apply production results for further verification against the system under test. Also, API cases were set up and integrated to facilitate the execution of the high number of required test cases.

Technologies

After setting the scope of the approach, the right tooling had to be defined. For this specific project, a clear decision was made for Tricentis Tosca, because it:

…supports a risk-based approach: Tosca provided full traceability between risk weighted requirements (in our case user stories) and execution results of test cases that covered them.

…combines UI and non-UI tests: Testing a huge number of individual journeys including different variants would have been slow and unstable in UI execution. Tosca’s integrated API Scan enabled a fast creation of non-UI test cases which execute extremely quickly.

…ensures maintainability and reusability: Developing in an agile environment, the tested system was under constant change. Not only new features had to be tested, but also existing tests had to be maintained. The model-based approach of Tosca reduced efforts for adjusting tests after sprints and before hand-over for regression testing.

…creates business-readable test cases: All test activities should focus on facilitating a proper evaluation of the product’s quality. Being scriptless, Tosca enabled to present the automation to non-Tosca developers and business owners.

Keeping up with the rapid yet very clean implementation was intense but feasible. The test team was able to support the development team with constant feedback regarding issues and the current quality of their work. Decision makers could be provided with high transparency in terms of their product’s quality. Supported by the proper tooling the test achievements where accomplished roughly at a one to five cost ratio between test and development, being far below the average proportion.