Automatic Test Case Generation

• Evaluation of existing test input generators for sequential software
  In recent years significant progress was achieved in automatic test generation which is reflected in an increased number of available tools. This bachelor’s thesis rigorously evaluates existing tools on benchmark programs to assess the state-of-the-art in automatic test generation. The tasks are the collection of existing test generators and verification tools (which might be used for test generation as well), selecting a representative set of programs as benchmarks, as well as running the tools on these benchmark programs. An evaluation of the test generators includes statistics on resource consumption (time, memory), supported languages/language features, found bugs, and achieved code coverage.
  [Contact: Andreas Holzer]
• Evaluation of existing approaches for testing concurrent software
  In recent years significant progress was achieved in automatic testing of concurrent software which is reflected in an increased number of available tools. This bachelor’s thesis rigorously evaluates existing tools on benchmark programs to assess the state-of-the-art in testing concurrent software. The tasks are the collection of existing testing and verification tools for concurrent software, selection of a representative set of programs as benchmarks, as well as running the tools on these benchmark programs. An evaluation of the test generators includes statistics on resource consumption (time, memory), supported languages/language features, found bugs, and achieved coverage of thread/process interleavings.
  [Contact: Andreas Holzer]
• Evaluation of existing approaches for Model-Based Testing
  In recent years significant progress was achieved in automatic model-based test generation which is reflected in an increased number of available tools. This bachelor’s thesis rigorously evaluates existing tools on benchmark models and programs to assess the state-of-the-art in automatic model-based test generation. The tasks are the collection of existing test generators, selecting a representative set of models and programs as benchmarks, as well as running the tools on these benchmarks. An evaluation of the test generators includes statistics on resource consumption (time, memory), supported languages/language features, found bugs, achieved model/code coverage, and the quality of test oracle generation.
  [Contact: Andreas Holzer]
• Implementation of an exact coverage measurement tool
  The task is the integration of a program analysis which can track single program executions into the test generator CPA/Tiger to enable a coverage analysis for C programs which contain bit-operations, pointers, and heap manipulation. A performance evaluation of the coverage analysis on a set of benchmark programs as well as performance improvements of the underlying program analysis are part of this thesis, too.
  [Contact: Andreas Holzer]
• Implementation of a logging facility for concurrent C software
  The topic of this thesis is the implementation of a tool that logs program executions of concurrent C software and generates C programs which represent the logged program traces. In this thesis different existing frameworks for logging of program executions shall be examined and an experimental evaluation of the introduced execution overhead has to be done.
  [Contact: Andreas Holzer]

Master’s Thesis Topics

• Adaption and evaluation of existing program analyses for test generation
  In static test generation classic program analyses for program verification are used to generate inputs for the program under investigation. One important difference of test input generation to program analysis is that usually a huge number of different reachability analyses have to take place whereas static program analysis normally involves only a single reachability analyses. This huge amount of reachability analyses requires the adaption of existing program analyses with respect to reuse of information gathered during previous analysis runs as well as different strategies for the refinement of the precision of a used program analysis (in static analysis usually an analysis only gets more precise whereas test generation requires an adjustment towards precision as well as imprecision).
  [Contact: Andreas Holzer]
• Systematic combination of different test generation approaches by program rewriting
  In this thesis, different approaches to test generation should be combined by rewriting C programs into a sequence of simplified C programs. This simplification reflects the abilities of the used test generators, i.e., we generate simplified programs which can be handled easily by the used test generator. Then, generated inputs are fed into the original C program and a new simplified C program is generated for different test generation approaches which are capable of generating inputs for the remaining input variables. This approach should investigate the combination of static and dynamic test generation approaches.
  [Contact: Andreas Holzer]
• Model generation for automatic test generation
  In this thesis C programs involving heap manipulations shall be translated into integer programs (C programs involving only operations on integer variables). After the translation, a test suite for the resulting integer programs shall be generated using the test generator CPA/Tiger. This test suite is generated based on a coverage specification given in the coverage specification language FQL. The translation of the original C program into the integer program has to track all information necessary to translate an FQL specification on the original program into an FQL query on the integer program as well as enabling a back-translation of the generated test inputs into inputs on the original program.
  [Contact: Andreas Holzer]
• Combination of Random and Systematic Test Generation
  Test input generation for programs involving complex heap manipulations remains a challenging problem. The goal of this thesis is to combine random test generation (a fast and scalable test generation approach) with systematic test generation (a precise but much less scalable test geneeration approach) using machine learning techniques. This thesis involves the implementation of a prototype tool as well as an experimental evaluation on benchmark programs.
  [Contact: Andreas Holzer]
• Reasoning on Coverage Specifications
  The coverage specification language FQL enables programmers to succinctly and precisely specify code and state coverage criteria (a very simple example would be basic block coverage or condition coverage) which has to be achieved by a test suite. The test generator CPA/Tiger exploits the structure and meaning of FQL queries to reason on the feasibility and infeasibility of single test goals (an FQL query specifies a set of test goals). The goal of this thesis is to extend the reasoning capabilities of CPA/Tiger. This involves a systematic development of a calculus on test goals, which originate from FQL specifications, as well as the integration of this reasoning framework into CPA/Tiger and an experimental evaluation.
  [Contact: Andreas Holzer]
• Testing of Web-based Applications
  The coverage specification language FQL and its backends CPA/Tiger and FShell2 were designed and developed for imperative programs. Especially, the test generators support only the programming language C. In this thesis, testing of web-based applications using the concepts of FQL (i.e., query-driven program testing) shall be investigated. By extracting models for web-based applications which can be translated into C programs, our existing test generators shall be used for test generation. An experimental evaluation compares existing tools with the newly developed approach with respect to resource consumption (time, memory), bugs found, and achieved code coverage.
  [Contact: Andreas Holzer]