Skip to main content

Challenges of Using Artificial Intelligence in Safety-Critical Systems

Artificial Intelligence (AI) has transformed the world of technology, enabling systems to learn, adapt, and make decisions without explicit programming. From autonomous vehicles to medical diagnostics and flight control systems, AI promises unprecedented efficiency and capability. However, when it comes to safety-critical systems—where failure could result in injury, loss of life, or significant damage—the use of AI introduces profound challenges that go far beyond traditional software engineering. Unlike conventional software, which behaves predictably according to its programmed logic, AI is built on learning and training. Its decisions and outputs depend heavily on the data it has been trained on and the patterns it recognizes during runtime. This adaptive, data-driven behavior means that an AI system’s responses may vary with changing inputs or environments, often in ways that are not explicitly defined or foreseen by developers. While this flexibility is a strength in many applica...

The Critical Importance of Software Version Control and Configuration Management in Safety-Critical Software Development

The Critical Importance of Software Version Control and Configuration Management in Safety-Critical Software Development

In the development of safety-critical software—whether for avionics, medical devices, rail signaling, nuclear systems, or industrial automation—the integrity and correctness of every software artifact is of paramount importance. Unlike general-purpose software, where defects may cause inconvenience or financial loss, failures in safety-critical domains can result in severe hazards, mission loss, or even loss of life. For this reason, robust software version control and configuration management (CM) are not optional tools—they are foundational pillars of system safety, mandated by standards such as DO-178C, IEC 61508, ISO 26262, and EN 50128.

These disciplines ensure that every change is traceable, every modification is intentional, and every release is precisely understood. Without them, even the most rigorously designed software can accumulate hidden risks that manifest during integration, deployment, or maintenance. This blog post discusses why version control and configuration management matter so deeply in safety-critical engineering and how they prevent systemic failures across design teams, vendors, and lifecycle stages.

Software Development at Scale: The Complexity of Multi-Vendor Safety-Critical Systems

Large-scale safety-critical software systems are rarely built by a single integrated team. Instead, they typically involve multiple vendors, each responsible for specific components:

  • Firmware for sensors, actuators, and processing modules

  • Application software for independent subsystems

  • Middleware or communication stacks

  • Diagnostic and maintenance utilities

  • Separate functional modules within the same avionics or embedded application

These independently developed components eventually converge during system integration. In many programs—particularly in avionics—vendors provide object files, libraries, or precompiled binaries instead of source code. Integration engineers are responsible for assembling these parts into the final application image.

This distributed development structure introduces significant configuration risks. During maintenance or incremental development, it becomes entirely possible for the integration team to inadvertently package the application with an outdated module or a previous revision of an object file—especially if proper configuration management processes are lacking or inconsistently followed. Such mismatches can lead to subtle functional errors, regression failures, or interface incompatibilities that may only surface in later testing phases or, worse, in the operational environment.

This risk is not theoretical; it is well documented in post-incident analyses across aerospace and defense programs. Without strict version control and CM, the system becomes vulnerable to integration inconsistencies that undermine safety and reliability.

Why Version Control Is Not Enough: The Broader Need for Configuration Management

While software version control (e.g., Git, SVN, ClearCase) is central to tracking source code changes, it is only one piece of the discipline. In safety-critical development, configuration management encompasses the entire ecosystem of artifacts, including:

  • High-level and low-level requirements

  • System and software architecture documents

  • Design descriptions and models

  • Source code, object code, libraries, and built images

  • Verification plans, test cases, procedures, and results

  • Review records and static analysis reports

  • Tool qualification data

  • Problem reports and change requests

  • Build environments and toolchains

This is why DO-178C refers not just to version control but to a full-lifecycle configuration management system. Its goal is to ensure:

1. Integrity and Authenticity of Artifacts

Every artifact must be the correct, approved version, free from unauthorized or accidental modification.

2. Repeatability of Builds

The exact binary must be reproducible at any point in the lifecycle, even years later.

3. Traceability of Changes

Every change must be justified, authorized, verified, and connected to a requirement or problem report.

4. Control of Baselines

Each release must correspond to a correctly documented baseline of components, documentation, and tests.

5. Prevention of Regression

Changes must undergo impact analysis to ensure they do not adversely affect previously verified behavior.

Version control on its own does not provide these guarantees. Only a comprehensive configuration management framework can.

How Poor Configuration Management Causes Safety and Integration Failures

The absence of robust CM becomes especially dangerous in distributed, multi-vendor environments. The example described earlier is an excellent illustration:

During software maintenance, an application may be packaged with an old file or outdated component if proper configuration is not rigorously maintained. This can introduce functional mismatches, regressions, and subtle bugs that manifest only during late-stage testing or system operation.

In a safety-critical context, the consequences of such errors can be severe. Some common failures caused by poor CM include:

  • Inconsistent interface behaviors when an old object file does not implement the latest protocol update

  • Failure of safety monitors if underlying assumptions change between versions

  • Mismatch between requirement versions and code versions

  • Loss of traceability, making certification audits difficult or impossible

  • Silent regression errors arising from outdated modules not covered by recent test updates

  • Integration crashes due to incompatible binary interfaces

DO-178C is explicit: configuration consistency is a required element of safety assurance. If even one component is outdated, unverifiable, or inconsistently traced, the integrity of the entire airborne system is considered compromised.

The Central Role of Configuration Management in DO-178C

DO-178C dedicates two configuration control categories and associated objectives to configuration management depending upon the Design Assurance Level (DAL). The intent is to guarantee that every aspect of the software lifecycle contributes to system safety and certification credibility. Key expectations include:

Identification of All Configuration Items

Every artifact that influences software behavior must be uniquely identified and versioned.

Baseline Establishment and Control

Formal baselines must be defined for development, integration, testing, and release phases.

Change Control and Authorization

No change can be made unless it is reviewed, approved, impact-analyzed, and traced.

Archival and Reproducibility

Developers must be able to regenerate a delivered binary using the exact versions of all contributing artifacts.

Configuration Audits

Independent audits must verify completeness, consistency, and correctness of the CM system.

Software Configuration Management Activities Under DO-178C

Figure 1: Software Configuration Management Activities Under DO-178C

These requirements exist not to satisfy auditors but to preserve the integrity of the safety case for the entire lifecycle of the system.

Version Control and CM as Enablers of Safe, Predictable, and Maintainable Engineering

When implemented properly, version control and CM provide numerous long-term advantages:

  • New engineers can quickly understand system history and rationale.

  • Integration becomes deterministic rather than error-prone.

  • Maintenance cycles become safer because changes are traceable and controlled.

  • Certification becomes smoother due to consistency of artifacts.

  • Accidental regressions are minimized, caught early, or prevented entirely.

  • Vendors and internal teams can collaborate on a shared baseline without conflict.

Most importantly, strong configuration management protects against the accidental introduction of unsafe behavior—a critical need when human lives depend on system reliability.

Figure 2: Git’s branching model is one of its most powerful features. It allows multiple developers to work on different parts of a project simultaneously without interfering with each other’s changes.

Conclusion

In safety-critical software development, version control and configuration management are not simply administrative tools—they are core safety mechanisms. They enforce discipline, preserve knowledge, maintain traceability, and ensure that every software decision is visible, intentional, and documented. In multi-vendor and large-scale engineering environments, these practices become even more vital, preventing outdated components or uncontrolled changes from undermining system integrity.

As safety-critical systems continue to grow in complexity and interdependency, the rigor of configuration management remains one of the most powerful safeguards against failure. By elevating CM and version control to first-class engineering priorities, organizations not only meet certification requirements—they build safer systems and protect lives. 

Comments