Monthly Archives: February 2017

Publication at A-MOST 2017

Another publication has been accepted at A-MOST 2017.

Bernhard K. Aichernig, Silvio Marcovic and Richard Schumi: “Property-Based Testing with External Test-Case Generators“, In IEEE 10th International Conference on Software Testing, Verification, and Validation Workshops (ICSTW), 13th Workshop on Advances in Model Based Testing (A-MOST 2017), Tokyo, Japan, 13-17 March, 2017, pages 337–346. IEEE Computer Society, 2017. (PDF)(doi:10.1109/ICSTW.2017.62)

The paper will be presented on the 13th Workshop on Advances in Model Based Testing A-MOST 2017 in Tokyo, Japan on 17 March 2016.  The workshop is part of the 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017).


Previous work has demonstrated that property-based testing (PBT) is a flexible random testing technique that facilitates the generation of complex form data. For example, it has been shown that PBT can be applied to web-service applications that require various inputs for web-forms. We want to exploit this data generation feature of PBT and combine it with an external test-case generator that can generate test cases via model-based mutation testing. PBT already supports the generation of test cases from stateful models, but it is limited, because it normally only considers the current state during exploration of the model. We want to give the tester more control on how to produce meaningful operation sequences for test cases. By integrating an external test-case generator into a PBT tool, we can create test cases that follow certain coverage criteria. This allows us to reduce the test execution time, because we do not need a large number of random tests to cover certain model aspects. We demonstrate our approach with a simple example of an external generator for regular expressions and perform an industrial case study, where we integrate an existing model-based mutation testing generator.

Publication at ICST 2017

A new publication has been accepted at the ICST 2017.

Bernhard K. Aichernig and Richard Schumi: “Statistical Model Checking Meets Property-Based Testing“, In 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017), Tokyo, Japan, 13-17 March, 2017, pages 390-400. IEEE Computer Society, 2017. (PDF)(doi:10.1109/ICST.2017.42)

The paper will be presented at the 10th IEEE International Conference on Software Testing, Verification and Validation (ICST 2017) in Tokyo, Japan (13-17 March 2017).


In recent years, statistical model checking (SMC) has become increasingly popular, because it scales well to larger stochastic models and is relatively simple to implement. SMC solves the model checking problem by simulating the model for finitely many executions and uses hypothesis testing to infer if the samples provide statistical evidence for or against a property. Being based on simulation and statistics, SMC avoids the state-space explosion problem well-known from other model checking algorithms. In this paper we show how SMC can be easily integrated into a property-based testing framework, like FsCheck for C#. As a result we obtain a very flexible testing and simulation environment, where a programmer can define models and properties in a familiar programming language. The advantages: no external modelling language is needed and both stochastic models and implementations can be checked. In addition, we have access to the powerful test-data generators of a property-based testing tool. We demonstrate the feasibility of our approach by repeating three experiments from the SMC literature.