Category DASIA Presentations
Automation of Requirements-based Testing
Date | 4 Jun 2019 - 6 Jun 2019 |
Event | DASIA 2019 |
Location | Torremolinos, Spain |
Manual requirements-based testing is time-
consuming: Input data must cover the requirements and
observed output data must be checked for their compatibility
with the requirements. Testcases can also be automatically
generated from test models. However, these models first have to
be established manually. In contrast, the approach to be
presented here uses simpler ways of formalizing requirements
to automatically map test data generated for automatic
robustness testing using massive stimulation to requirements and
to check the results for correctness.
Permalink
Generating Random Telecomand Test Data Using Genetic Algorithms
Date | 29 May 2018 - 31 May 2018 |
Event | DASIA 2018 |
Location | Oxford, United Kingdom |
Generating useful test data is one of the big
challenges in automatic software testing. While random test
data generation is the easiest method, the test inputs generated
by it may fail to exercise the software under test properly if the
internal structure of the data is unknown to the generator and
at the same time relevant for the decisions taken in the code.
Handling of telecommands in space onboard software is one
example where this is the case. We investigate a method of
generating test data for these cases using genetic algorithms.
Permalink
Evaluation of Verification Tools Continued: More Tools, More Software, More Aspects
Date | 30 May 2017 - 1 Jun 2017 |
Event | DASIA 2017 |
Location | Gothenburg, Sweden |
In a previous study six software verification tools
have been applied to a representative space software package.
The findings reported by each tool have been compared in
order to derive footprints regarding fault identification. In a
continuation three more tools were applied to the previously
selected application software and to another application
together with two tools previously used in order to broaden the
base of evaluation. More aspects were considered regarding
the evaluation of results: an additional evaluation criterion was
added and a comparison of reported defects with the outcome
of unit tests was performed. Due to a higher degree of
formalization and automation the manual evaluation effort
could be decreased while extending the number of considered
reports and the number of tools. The encountered evaluation
and verification issues are discussed in detail. All results
together shall provide a detailed view on the defect
identification capabilities of the considered tools w.r.t. current
software base. Altogether, the high quality of reports as
obtained in the previous study was not obtained again: in
context of a different set of tools and another (object-oriented)
language a lot of trivial reports were observed.
Permalink
Challenges Regarding Automation of Requirements-based Testing
Date | 30 May 2017 - 1 Jun 2017 |
Event | DASIA 2017 |
Location | Gothenburg, Sweden |
Testing as a method of software verification is
limited in that it can only prove the presence of defects, not
their absence. To be useful, a large number of test cases may
be needed, a strategy that is often in conflict with project
constraints such as available time and funds. Test automation
may be considered as an interesting approach to alleviating
this conflict. However, test automation requires accurate and
computer-accessible information about the system to be tested,
both in terms of the interfaces by which the system is to be
stimulated as well as the desired properties of these interfaces.
Within the FASTII activity (FAST=Flow-optimised Automated
Source-code based Testing) the possibility of deriving this
information from available requirements and design
documents is being investigated. Preliminary results of this
investigation as well as suggestions for future changes in the
process are presented in this paper.
Permalink
Early Results from Characterizing Verification Tools through Coding Error Candidates Reported in Space Flight Software
Date | 10 May 2016 - 12 May 2016 |
Event | DASIA 2016 |
Location | Tallinn, Estonia |
Six software verification tools have been applied to
space flight software and the findings reported by each tool
have been compared in order to derive footprints of the tools
regarding capabilities of fault identification. Currently
available results are provided in this paper: sensitivity and
precision of individual tools and combinations of pairs of tools
out of the set. A reader should bear in mind that the results as
presented here depend on the spectrum of fault types as
present in the reference software and on the configuration of
tools towards real defects and fault types which are of interest
for embedded systems and space flight software.
Permalink
Characterizing Verification Tools through Coding Error Candidates Reported in Space Flight Software
Date | 19 May 2015 - 21 May 2015 |
Event | DASIA 2015 |
Location | Barcelona, Spain |
Mastering the continuously increasing amount of
software requires identification of more efficient strategies for
software verification. Currently, fault coverage is only
indirectly addressed, e.g. by code coverage. The idea as
presented in this paper is to get a better understanding of fault
coverage by a systematic classification of software fault types,
derivation of footprints of verification tools regarding coverage
of such fault types, and recording of required effort. A number
of issues regarding fault identification and classification are
discussed in this context.
Permalink
Automated Source-code-based Testing of Object-Oriented Software
Date | 3 June 2014 - 5 June 2014 |
Event | DASIA 2014 |
Location | Warsaw, Poland |
With the advent of languages such as C++ and Java
in mission- and safety-critical space on-board software, new
challenges for testing and specifically automated testing arise.
In this paper we discuss some of these challenges, consequences
and solutions based on an experiment in automated source-
code-based testing for C++.
Permalink
A Case Study on Automated Source-Code-Based Testing Methods
Date | 14 May 2013 - 17 May 2013 |
Event | DASIA 2013 |
Location | Porto, Portugal |
We present results of a case study on a test
generation approach called Flow-optimized Automated
Source-code-based unit Testing (FAST) which generates test
stimuli from information available in the source code, in
particular taken from the detailed software interfaces. This
allows automation of a significant part of testing, ranging from
the test stimuli generation to the generation of the test report.
A huge number of stimuli can be generated exploring the
behaviour of the software under test under nominal and non-
nominal conditions. Symptoms like timeouts, unexpected
termination, run-time exceptions, out-of-range conditions and
missing coverage are applied for defect detection. The goal of
this study was to evaluate the FAST process in context of a real
spacecraft flight software application and to get a feedback on
its scalability regarding larger applications, its sensitivity on
detecting defects in the code, the achievable test coverage, its
compliance with software standards and potential limitations.
We also consider the impact of coding style on suitability for
automated testing. The results confirm that the approach (1)
provides acceptable code coverage results without requiring
manual intervention for test preparation and execution, (2)
raises the probability of activation of exotic fault conditions,
(3) may provide hints on locations in the code where
robustness needs to be verified, and (4) identifies defects not
found before by static analysis and intensive testing
Permalink
Potentials of Constraint-Based Methods in Software Verification and Validation
Date | 14 May 2012 - 16 May 2012 |
Event | DASIA 2012 |
Location | Dubrovnik, Croatia |
We give an overview over the principles of constraint-
based test data generation, discuss its limitations and
potentials and touch some of the domains which may be
interesting to combine with constraint-based testing
techniques. Automated generation of test data is an
example where this technique can be applied and
significantly increase the degree of automation, but it is
not limited to. This paper is intended to give interested
readers a quick entry into the methods and applications
to allow a deeper understanding and an informed verdict
about the actual capabilities and potential future
directions.
Permalink
Lessens Learned on Quality (of) Standards
Date | 17 May 2011 - 20 May 2011 |
Event | DASIA 2011 |
Location | Malta |
Standards are used to describe and ensure the quality of
products, services and processes throughout almost all
branches of industry, including the field of software
engineering. Contractors and suppliers are obligated by
their customers and certification authorities to follow a
certain set of standards during development. For
example, a customer can easier actively participate in
and control the contractor’s process when enforcing a
standard process.
However, as with any requirement, a standard may also
impede the contractor or supplier in assuring actual
quality of the product in the sense of fitness for the
purpose intended by the customer.
This is the case when a standard defines specific quality
assurance activities requiring a considerable amount of
effort while other more efficient but equivalent or even
superior approaches are blocked. Then improvement of
the ratio between cost and quality exceeding miniscule
advances is heavily impeded.
While in some parts being too specific in defining the
mechanisms of the enforced process, standards are
sometimes too weak in defining the principles or goals
on control of product quality.
Therefore this paper addresses the following issues: (1)
Which conclusions can be drawn on the quality and
efficiency of a standard? (2) If and how is it possible to
improve or evolve a standard? (3) How well does a
standard guide a user towards high quality of the end
product?
One conclusion is that the analyzed standards do
interfere with technological innovation, though the
standards leave a lot of freedom for concretization and
are understood as technology-independent.
Another conclusion is that standards are not only a
matter of quality but also a matter of competitiveness of
the industry depending on resulting costs and time-to-
market. When the costs induced by a standard are not
adequate to the achievable quality, industry encounters a
significant disadvantage.
Permalink
Tool-Guided, Domain-Specific, Systematic Requirements Management
Date | 1 Jun 2010 - 4 Jun 2010 |
Event | DASIA 2010 |
Location | Budapest, Hungary |
The importance of the quality of requirements for
successful execution and completion of a project from a
technical and contractual point of view is being
recognized more and more. Many methods are targeted
to improve the support for collecting requirements
while still focusing on natural language. However, the
ambiguities in the semantics of natural language are the
biggest obstacles towards success. The approach
presented in this paper focuses on the elements of a
domain while keeping the expressiveness of natural
names and terms and introducing clear semantics. This
brings the advantage that immediate verification of the
human-provided inputs is possible, immediate
contributions to validation are available and
inconsistencies can be detected by a tool immediately.
This leads to guidance of an engineer by a tool towards
consistent, complete and correct requirements -
requirements of high quality - and eases maintenance
for the same reasons. As most of the complexity is
handled by the tool due to its good knowledge on the
domain, the approach is scalable towards large
specifications. Several examples of application domains
are described which illustrate the universality and
feasibility of the approach across domain boundaries.
Permalink
Fault Identification Strategies
Date | 26 May 2009 - 29 May 2009 |
Event | DASIA 2009 |
Location | Istanbul, Turkey |
Various strategies for fault identification exist - e.g.
based on formal analysis of code or on testing - of
which each focuses on certain identification aspects and
fault types. This paper characterises the strengths and
weaknesses of methods in theory and practice -
focusing on application-independent identification
strategies, and it suggests strategies to maximise the
number of detected faults while minimising the related
effort. Fault activation conditions are discussed in
detail, resulting in an extended scope on stimulation
needs. In particular, the contribution of automation in
raising the activation probabilities is investigated.
Various examples of fault activation mechanisms and
statistics on fault types vs. identification methods are
provided as observed in practice. An interesting result is
the identification of application-dependent test cases by
application-independent test strategies.
Permalink
Automated Verification of Code Generated from Models: Comparing Specifications with Observations
Date | 27 May 2008 - 30 May 2008 |
Event | DASIA 2008 |
Location | Palma de Mallorca, Spain |
Certification is based on compliance of the code of the
code generator with given standards. Such compliance
never can guarantee correctness of the whole chain
through transformation down to the environment for
execution, though the belief is that certification implies
well-formed code at a reduced fault rate.
The approach presented here goes a direction different
from manual certification.. It is guided by the idea of
automated proof: each time code is generated from a
model the properties of the code when being executed in
its environment are compared with the properties
specified in the model. This allows to conclude on the
correctness of the whole chain for every application and
related generated code.
Permalink
Model Transformation in Practice
Date | 29 May 2007 - 1 Jun 2007 |
Event | DASIA 2007 |
Location | Naples, Italy |
The intention of this paper is to highlight the benefits of
model exchange between different tools, methods and
notations on one side, and to identify issues of proper
modelling on the other side which have been detected
during model transformation and code generation from
models.
Permalink
Evaluation of Auto-Test Generation Strategies and Platforms
Date | 29 May 2007 - 1 Jun 2007 |
Event | DASIA 2007 |
Location | Naples, Italy |
As the test effort takes a significant part of the software
development lifecycle, efficient test strategies are a precondition
for reduction of development costs and time.
In this respect two main issues exist: firstly, the tuning
of the test track from test case identification to
evaluation, secondly, the reduction of number of test
cases to be processed and evaluated. Both aspects were
considered in the work presented in this paper.
Permalink
Improving Test Automation by Deterministic Methods in Statistical Testing
Date | 22 May 2006 - 25 May 2006 |
Event | DASIA 2006 |
Location | Berlin, Germany |
This paper was presented at the DASIA 2006 conference in Berlin, Germany.
It lays out principal approaches for tackling the "small target" and "oracle" problems in statistical automatic testing by applying deterministic methods for assistance.
Permalink
Software Diversity by Automation
Date | 30 May 2005 - 2 Jun 2005 |
Event | DASIA 2005 |
Location | Edinburgh, Scotland |
This paper was presented at the DASIA 2005 conference in Edinburgh, Scotland.
It shows the result of an Automated ISVV (AISVV) activity executed on the
Flight Application Software (FAS) of the Autonomous Transfer Vehicle (ATV).
Permalink
Automated Software Production - Quo Vadis
Date | 30 May 2005 - 2 Jun 2005 |
Event | DASIA 2005 |
Location | Edinburgh, Scotland |
Permalink
Can We Provide Better Protection against Budget Overruns of Software Projects?
Date | 28 Jun 2004 - 1 Jul 2004 |
Event | DASIA 2004 |
Location | Nice, France |
This paper was presented at the DASIA 2004 conference in Nice, France.
The answer to this hypothetic question is "yes", of course. The paper will
approach the problem in two steps: firstly, we will discuss if and which
measures exist to identify an overrun early enough, secondly, we will analyse
the sources of overruns and which means may be applied not to exceed the
planned budget.
Permalink
Benchmarks on Automated System and Software Generation
Date | 13 May 2002 - 16 May 2002 |
Event | DASIA 2002 |
Location | Dublin, Ireland |
This paper was presented at the DASIA 2002 conference in Dublin, Ireland. It
shows results of benchmarks on projects carried out using ISG/ASaP.
Permalink
Platform-Dependent (Cost) Impacts on Portability, Software Reuse and Maintenance
Date | 28 May 2001 - 1 June 2001 |
Event | DASIA 2001 |
Location | Nice, France |
This paper describes the challenges of maintaining software on different platforms.
Permalink
Automated generation of real-time software from datasheet-based inputs: The process model, the platform and the feedback from the MSL project activities
Date | 22 May 2000 - 26 May 2000 |
Event | DASIA 2000 |
Location | Montreal, Canada |
This paper describes how ISG was used to built MSL software.
Permalink
Software Development for the Material Science Laboratory on ISS by Automated Generation of Real-time Software from Datasheet-based Inputs
Date | 22 May 2000 - 26 May 2000 |
Event | DASIA 2000 |
Location | Montreal, Canada |
This paper was presented during the DASIA 2000 conference in Montreal, Canada by
Kayser-Threde.
It describes the organisation of the database from which MSL software was
automatically generated by ISG/ASaP.
Permalink
Organizing Incremental, Reusable and Automated Software Development
Date | 17 May 1999 - 21 May 1999 |
Event | DASIA'99 |
Location | Lisbon, Portugal |
Description of an approach for incremental software development and
validation applying automated generation of an executable system
from a minimum of system engineering information.
Permalink
Lessons learned by the use of (C)OTS
Date | 25 May 1998 - 28 May 1998 |
Event | DASIA'98 conference |
Location | Athens, Greece |
This paper was presented during the DASIA'98 conference, May 25th-28th 1998 in
Athens. It shows BSSE's experience with the use of
(Commercial-)Off-The-Shelf-Software based on a project dealing with the
integration of a number of (C)OTS packages and software developed from scratch.
Permalink
Experience with Validation by Simulation, Automated Code Generation and Integration
Date | 26 May 1997 - 29 May 1997 |
Event | DASIA'97 |
Location | Sevilla, Spain |
The presentation included an online demonstration on a Sparc Workstation and a PC laptop.
Permalink
From CASE to CIVE: A Future Challenge!
Date | 20 May 1996 - 23 May 1996 |
Event | DASIA'96 |
Location | Rome, Italy |
This paper was presented during the DASIA'96 conference, May 20th - 23th 1996
in Rome.
Permalink