Optimization model for an automatic system testing framework
|has title::Optimization model for an automatic system testing framework|
|Master:||project within::Software Engineering|
|Student name:||student name::Anca Gentiana Coman|
|Second supervisor:||Eric Raijmakers|
|Second reader:||has second reader::Hans van Vliet|
Océ is one of the leading providers of document management and printing for professionals. The Océ offering includes office printing and copying systems, high speed digital production printers and wide format printing systems for both technical documentation and color display graphics. Part of the Canon group, the company was founded in 1877, is active in over 100 countries and employs more than 20,000 people worldwide. At the company headquarters, in Venlo, The Netherlands, the Research & Development department is responsible with developing Océ own basic technologies and the majority of its product concepts.
Their line of digital document systems comprises several models, out of which a select few use the PRISMAsync controller, an Océ product which enables a single point of control, efficient task splitting, media synchronization, intelligent color management and advanced editing capabilities. The controller is a software component which runs on dedicated hardware and is the heart of the printer.
The PRISMAsync controller is developed and tested within the Océ R&D facility in Venlo. The testing of the product involves both an automated framework and manual tests. The FAT (framework for automated testing) is responsible for testing controller capabilities such as printing, connectivity, copying, device management, error handling and workflow management. Individual test scripts are run by the framework every night with test run results being presented the following day.
The testing framework has been under development for several years and it is still undergoing improvement. In its current state it provides the ability to add test cases without disrupting the framework, to run any test case linked in the database, to run tests by levels, to obtain results after level completion and to see detailed description of failures.
The problems the testing framework is facing are related to the lack of knowledge about what is being tested, how much is tested, which modules are executed during a test run, what capabilities are not being tested enough. Because the framework is developed by a separate team, there is no knowledge on which modules have been modified since the last test run in order to select the appropriate tests to be executed. Therefore, the framework lacks the ability to provide regression test selection. Without a clear connection between what is tested and how much, there is also the possibility of having duplicate tests and no manner of obtaining coverage measures.
Taking into consideration the capabilities and problems of the automated testing framework, the following main research question can be posed:
What optimization model can be created and applied to the automated testing framework ?
The main research question can also be accompanied by a series of sub-questions in relation to the testing framework:
•What type of coverage measure could be obtained from test runs?
•What techniques could be employed for generating relevant test cases?
•How can regression test selection capability be added to the framework?
•How can the optimization model be automated?