Software Quality Good Practices in Open Source ‐ Proposal

State of the art

The established practice for the creation of electronic software-based control systems in regulated industries is based on methodological approaches that were developed for proprietary software development. The success of open source software in critical business and societal applications demonstrates its ability to deliver results comparable to proprietary development companies. Open source communities have proven their effectiveness and methodology, leading to widespread acceptance in areas with strict security regulations and certifications. This level of quality makes open source software a viable option for other regulated domains, including functional safety.

Challenge statement

However, to enable the adoption of open source software in other industries, it is crucial to ensure proper documentation of the processes executed by open source communities. This documentation will provide transparency and facilitate understanding of the methodologies employed, to be referred to as a quality management system for code driven development.

Industry standards for software quality do not prescribe any individual technique or method in detail. However, a lifecycle process is described that essentially corresponds to the V-model together with techniques that are rarely found in open source projects.

Intended work items

This document describes and justifies a research project to demonstrate equivalence of established open source development practices for the purpose of quality assurance (QA) with, for example, the V-model. The suitability of open source practices to deliver necessary work products equivalent to those from traditional design, implementation, integration and maintenance requires further analysis. This research will identify the capabilities and possible gaps to establish equivalence between open source practices and established industry software lifecycle standards.

Quality, Cyber Security and Functional Safety (FuSa) integrity standards aim for a certain level of reliability for the development of electronic devices with a protective function for hazardous machines and systems that reduces the risk to people and society when operating these machines and systems to an acceptable level. All these standards, although aiming for different verticals as well as different risk scenarios, address the risk reduction by raising the resilience and stability of a software project.

Electronic devices with protective or vulnerable functions are often controlled by software, which is prone to errors, whether intentional or unintentional. The size and complexity of the software directly impact the number of these errors, regardless of whether it is open source or proprietary. FuSa integrity standards like IEC 61508 or ISO 26262 provide methods and tools for developing electronic systems with safety-relevant functions, with which the unavoidable risk of errors is reduced to an acceptable level. These standards emphasize that carefully developed software, meeting specifications, is suitable for use in safety applications and cannot be significantly improved with reasonable effort.

Execution

In the execution of the research project, well established open source projects are examined as examples to methodically describe how the projects’ development processes are executed. This research aims to define quality criteria that align with a specific software quality goal through an in-depth analysis. The findings of this study contribute to the identification of key performance indicators (KPIs) that can effectively measure the maturity of quality in software development.

As research work, existing proposals and drafts of standards, which primarily focus on code-driven and/or continuous X code development, are examined for the adaptability and application to open source development. Furthermore, we will consider academic research on open-source development as a foundation for our proposal (starting from [1]). After analyzing the findings, we will determine whether to create our own process description or participate in an existing initiative. In any case, we will reference existing work whenever applicable.

Wider ecosystem relevancy

Additionally, this work will provide the authors of future versions of FuSa (integrity) standards with concrete access to such methods and metrics as a reference. This is a promising way that such methods will then also be accepted in practice in the approval procedures during functional safety assessments.

---

[1] Napoleão, B. M., Petrillo, F., & Hallé, S. (2020). Open Source Software Development Process: A Systematic Review. https://arxiv.org/pdf/2008.05015