About

TRUCONF is a national Austrian research project funded by FFG (Österreichische Forschungsförderungsgesellschaft).

The project has a duration of three years: from November 2014 to October, 2017. Three partners contribute to the project: two research partners and one industrial partners.

Project Summary

When integrating ICT systems into systems of systems, system providers are facing the problem of emerging effects – the overall system does not behave as expected from the behaviour of the system parts. In many cases, especially when the system parts are loosely coupled, these effects stem from the shared use of limited resources (usually, but not necessarily, computing and communication related resources like bandwidth, memory, handles, power dissipation). Due to the emergent nature of the effects and the overall complexity of the systems, they are hard to predict systematically. This leaves testing as an approach to gain trust in the assembled system. The project is motivated by the use case provided by AVL, where up to twenty out of over hundred different test devices are combined into engine and power train test beds defined by customer needs. The software, which controls the testbed (called automation system) and all the devices “plugged in”, needs to work flawlessly with any combination of devices. Currently, testing the automation system is a manual, time consuming process that cannot guarantee to cover a worst-case. Hence, a lot of testing experience and significant manual efforts of explorative testing are required prior releasing a set of configurations to the market. This project aims to speed up this process by a methodical shift from manual to automated testing with even higher accuracy by applying and extending techniques successfully used in model-based test-case generation. Since sharing of limited resources is one of the main causes of emerging unwanted behaviour, high overall consumption of a resource is considered to be a good indicator for reaching possibly problematic scenarios. Therefore, test cases shall be generated that drive both the single systems and the combined system of systems to high resource consumption levels within a short time frame. The project will develop and demonstrate an approach to extend an existing model based test case generator with support for resource consumption testing – thereby moving from testing of functional properties to testing non-functional properties. To do so, a modelling approach for expressing how resource consumption (costs) is related to the behaviour model will be elaborated, together with the test case generator itself. Said costs are usually not obvious and tedious to derive manually. Therefore, an accompanying approach and tool support is provided that automatically derives resource consumption data from the system under test and connects it to a given behaviour model (cost model learning). As outlined above, the outcome of the project will thus substantially improve the situation of validating non-functional properties of complex systems of systems. It is motivated by the needs of the industrial partner and will increase the competitiveness of the Austrian industry in a global market once successfully finished. Knowledge transfer will be promoted systematically in both directions, from research to industry and from industry to research. The scientific results will further establish the role of the research partners within the European model based testing community and strengthen Austria’s cluster of scientists working in software verification, validation and formal methods.

Project goals and results

The three main goals of TRUCONF are as follows.

1. Test-case Generation For Non-Functional Properties Of Systems Of Systems
TRUCONF will create a technique accompanied with tools for testing non-functional properties in a system of systems. The project will conduct research to address the stated challenges of Modelling Language, Coverage and test purposes, and Test-case generation and deliver necessary theory, models, and a prototype-version of a test-case generator as results of the first goal.

2. Automated Cost-Function Refinement Via Learning
TRUCONF will take advantage of the data available from test-case execution and tune the model so there is a close relationship between the observed, real behaviour and the modelled one. In order to deliver to this promise, the project will conduct research to address the challenges stated under Refinement of Costs. The project will deliver theory and tools for learning and cost-function refinement as results of the second goal.

3. Practicability of the Method & Know-How Transfer
TRUCONF will try hard and make sure the adopted formalism can be presented in a way non-experts can use it, cf. the challenges stated under Acceptance of Model-based approaches. This is demonstrated best by the fact that the models, although initially developed by the research partners, will be maintained by the industry partner in the end. Hence as a result of TRUCONF, the academic partners get a better feeling on what holds back rigorous system engineering approaches based on formal methods in industry, while the industry partner will acquire knowledge and practice in formal modelling.