Empirical Software Engineering

An overview of the research of the Empirical Software Engineering Group on software quality, human aspects, empirical studies, automotive software and AI-based software.

Software is the material for the 21st century. All innovations are somehow driven or influenced by software. Therefore, building high-quality software continues to be a central societal and economic challenge. As Empirical Software Engineering group, we see our responsibility in finding and evaluating techniques, methods and tools to better engineer software. To achieve this, we focus on three perspectives of software engineering methods:

  1. Software quality: Software is only useful if it is of high quality. Quality can mean many things, and our core competency is that we can bring different views and quality attributes together in comprehensive quality assurance. This includes definitions, measurements of quality as well as quality assurance techniques such as testing and static analysis.
  2. Human aspects: Yet, achieving high-quality software is not only a technical problem. We work from the perspective that humans need to be the centre of software engineering processes – be it the engineers themselves or end-users or other stakeholders. We need to understand them and explicitly consider them in everything we do.
  3. Empirical studies: All techniques, methods and tools need to reliably show their positive impact on the goal of high-quality software. The only way to do this is to run empirical studies, such as case studies, experiments or surveys, to objectively evaluate these impacts. Therefore, most of our contributions are empirical or contain empirical validations.

We apply these three perspectives to any kind of software in basic and applied research projects. We like to collaborate with industry on these topics, as empirical evaluations can be most interesting in a practical context. Yet, there are two focus domains in which we have special expertise:

  1. Automotive software: The automotive industry has been in an impressive transformation for the last two decades and is still evolving. With current trends, such as autonomous driving, connected cars and software-defined cars, software for vehicles has become a complex but vital challenge for the automotive industry. We have been working with automotive companies for two decades on various aspects including software architectures and quality assurance.
  2. AI-based software: Our group is also part of Cyber Valley, Europe’s largest AI research consortium. In this, we use AI techniques for software engineering problems. Yet, more importantly, we focus on how to build complete and operational software systems that contain strong AI components. These aspects are essential for the success of AI in practice, but are rarely considered by core AI researchers. Therefore, we bring our expertise to AI systems and, thereby, enable its application in new areas.

Large Research Collaborations

  • Artificial Intelligence Software Academy (AISA) is funded by the Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg to push research and teaching at the intersection of artificial intelligence, software engineering and applications. It is part of the Cyber Valley ecosystem. Prof. Dr. Stefan Wagner is speaker of the board of directors of AISA and is PI in three doctoral projects.
  • Cyber Valley is Europe's largest AI research consortium, which includes besides the University of Stuttgart, the University of Tübingen and the Max Planck Institute for Intelligent Systems partners such as Fraunhofer Gesellschaft, Robert Bosch GmbH, Amazon, BMW AG, Mercedes-Benz Group AG or Dr. Ing. h.c. F. Porsche AG. Prof. Dr. Stefan Wagner is a Cyber Valley Faculty.
  • Graduate School Intelligent Methods for Test and Reliability (GS-IMTR) is funded by Advantest Corporation to push research on applications of AI methods for future test and reliability methods in chip testing. Prof. Dr. Stefan Wagner is member of the board of directors and PI in two projects.
  • Graduate School of Excellence advanced Manufacturing Engineering (GSaME) is funded by the Ministerium für Wissenschaft, Forschung und Kunst Baden-Württemberg as continuation from the excellence initiative. It promotes research on advanced manufacturing engineering, bringing together disciplines such as mechanical engineering, computer science and economics. Prof. Dr. Stefan Wagner is a scientific member and PI in three doctoral projects.

Ongoing Projects

  • AutoDevSafeOPS, 2022–2025
    Software Campus, German Federal Ministry for Education and Research (BMBF)
  • Centralisation and Service Orientation in Automotive software architectures, 2022–2025
    External PhD student at Daimler Truck AG
  • iDevOps – Microservices, 2021–2022
    Siemens AG
  • Integrated approach for software Development for upgradable vehicles, 2022–2023
    Innovation Campus Future Mobility, State of Baden-Württemberg
  • Integrated AI in Teaching at the University of Stuttgart, 2021–2024
    German Federal Ministry for Education and Research (BMBF)
  • Microservices: Rejuvenation by Refactoring, 2022–2023
    Software Campus, German Ministry for Education and Research (BMBF)
  • Framework for the Design & Integration of AI Components, 2021–2023
    Project in the AI Software Academy (AISA), State of Baden-Württemberg
  • Efficient Testing of AI-Based Control Engineering, 2021–2023
    Project in the AI Software Academy (AISA), State of Baden-Württemberg
  • Visual Analytics for MLOPS, 2021–2023
    Project in the AI Software Academy (AISA), State of Baden-Württemberg
  • Artificial Intelligence in Vocational Training (KI B3), 2020–2024
    German Federal Ministry for Education and Research (BMBF)
  • Software Test Optimisation for Complex High-Volume Software, 2020–2023
    Project in the graduate school „Intelligent Methods for Test and Reliability“ funded by Advantest
  • Automated Generation of System-Level Test Programs for Characterization of Parametric Device Properties, 2021–2023
    Project in the graduate school „Intelligent Methods for Test and Reliability“ funded by Advantest
  • Requirements Engineering for AI Systems, 2021–2023
    Stipend by University of Kotli Azad Jammu and Kashmir
  • Domain-Specific Conversational Companion for Software Developers, 2021–2023
    External PhD project
  • Agile Release Planning, 2018–2021
    External PhD project in collaboration with Dr. Ing. h.c. F. Porsche AG
  • Communication in GitHub projects (Doctoral project by Verena Ebert M.Sc)
    Communication is essential when working together as a team. Especially in large projects with contributors from all over the world, like on GitHub. Therefore, I want to see how and which communication channels are offered in large GitHub projects and how maintainers and developers think about the given settings.
  • Improvement of static code analysis (Doctoral project by Dipl. Ing. Jan-Peter Ostberg)
    Static code analysis is an inexpensive way to increase software quality. Todays tools can provide valid and helpful warnings to the developers. Still these tools are underused. One problem leading to this is the lack of usability. In my research I am focusing on the helping of an individual developer gaining the best experience from the tool. The underlaying concept is based on psychological models. Also I want to explorer the connections between removal of issues of static analysis and their impact on software metrics.
  • Improvement of the usability of crypto in software engineering (Doctoral project by Kai Mindermann M.Sc)
    The realization of profound mathematical techniques through the modern cryptography is able to greatly improve information security. Many implementations lack easy usage; they are complicated to operate in general, their implementations are flawed and it is often hard to keep them up to date. In my research I want to reduce these problems within software technology by using software technology approaches themselves.
  • Application Modernization: Refactoring to Microservice-based Systems (Doctoral project by Jonas Fritzsch, M.Sc.)
    Modernizing applications is a widely discussed topic in the context of making existing applications cloud-ready. Cloud vendors offer a variety of innovative computing services and deployment models from hosting virtual machines to containers to serverless environments. For legacy applications that have evolved over time it can be difficult to find a clear guidance on the transformation process. This applies in particular to refactoring monolithic architectures into the recently emerged microservices pattern, which is widely seen as an optimal architecture for  cloud environments. My research project addresses this topic by investigating the transformation process from an architectural, as well as organizational perspective.

Completed Projects

  • Cybersecurity Static Analysis with Immediate Feedback (CRITICALMATE), 2019–2022
    German Federal Ministry for Education and Research (BMBF)
  • Integration Testing of Black-Box Systems, 2016–2021 (Doctoral project by Dominik Hellhake)
    In collaboration with BMW AG
  • The Scrum Master Role, 2018–2021
    External doctoral project in collaboration with Robert Bosch GmbH
  • Safety Analysis of Fail-Operational Systems, 2017–2021
    External doctoral project in collaboration with BMW AG
  • Evaluation and Control of the Value Provision of Complex IoT Systems, 2018–2021
    External doctoral project in collaboration with Robert Bosch GmbH
  • Inspection and assessment of spreadsheet programs (Doctoral project by Daniel Kulesz, M.Sc.)
    Today, a world without Spreadsheet programs is hard to imagine. Compared to traditional programs, spreadsheet programs provide a higher flexibility and allow end-users to develop the programs themselves. Unfortunately, anomalies in spreadsheet programs are very commong and can lead to severe damage. My research focuses on the inspection and assessment of spreadsheet programs.
  • Safety and security analysis in agile development processes (Doctoral project by Yang Wang, funded with a stipend of the state of Baden-Württemberg)
    Agile development methodologies are becoming a tendency in today’s changing software     development. However, due to a lack of quality assurance activities, especially safety and    security analysis, agile methods are criticized for being inadequate for the development of safe and security critical software. To apply agile methodologies into safety and security critical   systems, most research prefers combining agile methods with traditional development processes relying on standards. However, little emphysis is put on the nature of agile techniques. On the other side, safety and security are two interrelated quality aspects. Due to the growing trend of Internet of Things, more and more security vulnerabilities will also cause safety hazards. Thus, in my research, these two quality aspects would be taken into consideration in agile development processes.
  • In which way is software quality influenced by the personality of a software engineer? (Doctoral project by Dipl. math. oec. Erica Weilemann) 
    During the whole software engineering process, human beings are involved and thus influence the software product, also concerning different aspects of software quality. In my research I investigate, in which way the personality of every single participant in a software engineering process has an influence on the quality of the product of this process – the software. In a first step, I focus on the influence of the personality of a programmer on the maintainability of the software.
  • Evaluation and Improvement of Software Tests (Doctoral project by Rainer Niedermayr)
    • Automated and manual software tests are used in the quality assurance to discover faults in a software product at an early stage. In practice, code-coverage criteria are used to determine the effectiveness of the tests. However, code coverage only expresses which code chunks were executed by tests, but not, how effective the tests are in detecting faults. This work investigates, how the effectiveness of tests can be determined in a more meaningful way, and how ineffective tested areas can be prioritized.
    • In collaboration with CQSE GmbH
  • Doctoral Programme Services Computing
    • Services Computing is a Cooperative Doctoral Program at the Herman Hollerith Center (HHC) in Böblingen, Germany. It is a joint program of the University of Stuttgart and Reutlingen University, and supported by an initiative of the Ministry of Science and Art in the federal state of Baden-Württemberg, Germany.
    • Services Computing is an interdisciplinary approach to providing IT solutions that substantially support business-related strategies and increase entrepreneurial value. The essential goal of Services Computing is to holistically and consistently model complex information systems through a combination of computer science methods and information and communications technology to support design, development, delivery, operation, and monitoring of distributed services. Services Computing enables us to prevent and eliminate particular misalignments between business processes and information technologies, and to identify and develop improvements for digitized products and services. Fundamental concepts of Services Computing are, for example, Web and REST services, service-oriented architectures (SOA), cloud computing, or modeling and management of business processes. Services Computing covers the entire lifecycle of IT-based business-related services and refers to the development, implementation, and distribution of IT services, the associated service management, the alignment of IT and business, data management, data processing and analysis, and the design of innovative IT architectures.
  • System-Theoretic Analysis of Dependable Systems in the Automotive Domain (MISTI Global Seed Fund)
    Responsible researcher: Dr. Asim Abdulkhaleq
    In collaboration with: Prof. Dr. Nancy Leveson, Dr. John Thomas (MIT)
    The software-based functions in cars are increasing and becoming more complex at a breathtaking speed. We already have electronic/software components controlling safety-critical functions such as the anti-lock braking system. Yet, so far, these components individually were comparably small and classical embedded systems. With the two major new developments in the automotive domain to let cars communicate via wireless networks („connected car“) and to let them drive fully automatically („autonomous driving“), the software will reach a new dimension of complexity. Traditional methods to analyse safety in automobiles assume electromechanical systems and do not apply to software nor do they scale up to the complexity starting to emerge in these systems. Furthermore, new degrees of requirements on security and privacy will be needed for those future cars. The recent Tesla accidents involving what they call their “autopilot” is just the beginning.
    This project aims at building on the promising results with a system-theoretic approach to safety analysis and developing and evaluating an integrated analysis approach to safety, security and privacy for this new generation of automotive systems. System theory is especially suitable for identifying problems in the interaction of different components instead of individual component faults. Furthermore, system theory allows us to take the whole socio-technical system into account including humans. Similarly, security and privacy relies on a suitable interaction of system components and humans.
    The further potential application areas are far beyond automotive systems, but we want to be focused in this collaboration.
  • Concealing Patterns - Privacy in Stream Processing (PATRON, funded by the Baden-Württemberg Stiftung). 
    • Responsible research assistant: Kai Mindermann M.Sc.
    • This research project focuses on the development of approaches for the concealing of information from datastreams of sensors of various devices from the internet of things (IoT). The goal is to find a balance between the protection of data and the quality of service (QoS). It is relevant as can be seen by a simple example. Activity trackers sense for example your heart rate which, if tracked over time and analyzed, can reveal very sensitive information about you.
  • Hazard analysis in highly automated driving (funded by Continental Teves AG & Co. oHG)
  • Application of STPA to the Daimler Environment (funded by Daimler AG)
  • Specification of Software Modules (Doctoral project by Ivan Bogicevic, Dipl.-Inf.)
    If a software is not subdivided into precisely delimited parts, it is hard to maintain. Therefore, it is both essential and common practice to break the system down into modules. A well-documented software design contains precise module specifications that are up-to-date. But in practice, the quality of documents on this level is insufficient. The documents are outdated, or their state is unknown. This work investigates how developers can specify and document modules effectively. It presents a new process model that helps to describe modules under real-life conditions. A catalogue of module specifications that describe the modules of a software is a key element of the technique.
  • Experimentelle Verbesserung des Lernens von Software Engineering (EVELIN, subcontractor of HS Neu-ULM, founded by the BMBF)
  • Mining Software Repositories for Coupled Changes (Doctoral project of Jasmin Ramadani)
    Software repositories contain information about the history of a software system which can be used by develoopers during maintenance tasks. This includes the data in the versioning systgem, the issue tracking system and the documentation archives. One of the most used techniques to analyze software repositories is data mining whereby frequent itemset analysis has often been used to define sets of files which changed frequently together. The overall goal of this research is to help developers in their maintenance tasks by suggesting potential file changes based on previous modifications in the Git version history of a software product.
  • STPA Swiss: Software Safety Engineering Approach based on STPA for Software-Intensive Systems (Doctoral project by Asim Abdulkhaleq, M.Sc.)
    Software’s safety becomes a critical aspect in the development process of modern systems. However, safety is a system property and, hence, needs to be analysed in a system context to identify all potential hazardous software behaviours. The complexity of software makes defining appropriate software safety requirements with traditional safety analysis techniques difficult. STPA (Systems-Theoretic Processes Analysis) is a unique safety analysis approach which has been developed to identify system hazards, including the software-related hazards as well. Formal verification and testing are complementary approaches which are used in the development process to verify the functional correctness of software. However, the correctness of software cannot ensure the safe operation of safety-critical software systems. The software must be verified against its safety requirements which are identified by safety analysis to ensure potential hazardous causes cannot occur. My research focuses on developing a comprehensive safety engineering approach based on STPA including software testing and model checking approaches for the purpose of developing a safe software. The proposed approach can be embedded within a defined software engineering process or applied on existing software systems to help software and safety engineers to recognize the software risks.


For creating surveys we often use Rogator.

To the top of the page