Incidents with rolling release distributions

List of incidents that lead to issues for users of rolling release distributions

This list may be incomplete. Please let us know if you are aware of further incidents.

2011–2012: Early Adoption of GNOME 3 and Incompatibilities

Start of Disruption: April 6, 2011

Duration of Disruption: Approximately 6 months

Early Adoption of GNOME 3 and Incompatibilities

Description:

Arch Linux and other rolling release distributions adopted GNOME 3 immediately upon its release. This led to significant compatibility issues and incomplete extensions, as many applications and GNOME extensions were not yet prepared for GNOME 3. Users experienced instability and a lack of functionality, while Ubuntu LTS continued to provide a stable desktop experience with GNOME 2.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: GNOME developers released updates to improve compatibility and stability, accelerating the development of GNOME Shell extensions and application support.

  • Arch Community: Increased communication about potential issues with major desktop environment upgrades, advising users to review release notes and consider the implications before updating.


2012: Kernel 3.5 and Network Driver Issues

Start of Disruption: July 21, 2012

Duration of Disruption: Approximately 4–5 weeks

Linux Kernel 3.5 Network Driver Issues

Description:

The release of Linux Kernel 3.5 introduced changes to network drivers, causing network interruptions and compatibility problems with certain older WLAN cards in rolling release distributions like Arch Linux. Users experienced dropped connections and unreliable network performance until patches were released.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Released patches to address driver compatibility issues in subsequent kernel updates.

  • Arch Community: Advised users on how to downgrade to previous kernel versions if necessary and increased testing of kernel updates with a variety of hardware configurations.


2012–2013: Systemd Adoption and Transition Issues

Start of Disruption: Late 2012

Duration of Disruption: Several months

Systemd Adoption and Transition Issues

Description:

Arch Linux transitioned early to systemd as its init system, leading to significant problems with init services and network tools that were not fully compatible. Users faced inconsistencies and errors over several months until systemd became more stable and compatible with existing services. Ubuntu LTS continued using the Upstart init system, avoiding these issues.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Continued to develop systemd for better compatibility and stability.

  • Arch Community: Provided detailed migration guides and support to help users transition to systemd, and improved documentation on handling init service issues.


2013: Early Adoption of MySQL 5.6 Leading to Incompatibilities

Start of Disruption: February 2013

Duration of Disruption: Approximately 2 months

MySQL 5.6 Incompatibilities

Description:

Rolling release distributions like Arch Linux adopted MySQL 5.6 soon after its release, which led to incompatibilities and performance issues in web applications expecting MySQL 5.5. This resulted in stability problems and crashes, particularly with older plugins or CMS systems. Ubuntu LTS remained on the stable 5.5 version, and users did not encounter these issues.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Released updates to MySQL 5.6 to improve compatibility and performance.

  • Arch Community: Offered guidance on downgrading to MySQL 5.5 or applying patches, and increased testing of major database updates for compatibility with popular web applications.


2013: PulseAudio 4.0 Causing Audio Issues

Start of Disruption: May 6, 2013

Duration of Disruption: Approximately 3–4 weeks

PulseAudio 4.0 Audio Problems

Description:

The release of PulseAudio 4.0 introduced changes that caused audio problems on Arch Linux and other rolling release distributions, especially with certain applications and Bluetooth devices. Users experienced audio dropouts and device connectivity issues until a stabilizing update was released. Ubuntu LTS users, on older PulseAudio versions, did not face these problems.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Issued updates to address the compatibility and stability issues in PulseAudio.

  • Arch Community: Provided temporary workarounds and increased scrutiny of audio subsystem updates, encouraging users to report issues promptly.


2014: Early Adoption of OpenSSL 1.0.1 and Heartbleed Vulnerability

Start of Disruption: April 7, 2014

Duration of Disruption: Approximately 1 week

Heartbleed Vulnerability in OpenSSL

Description:

Rolling release distributions implemented OpenSSL 1.0.1 quickly, making them more susceptible to the Heartbleed vulnerability when it was discovered. Users had to apply workarounds until a patch was released. Ubuntu LTS was using an earlier, unaffected version of OpenSSL and was less impacted.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Released OpenSSL 1.0.1g to fix the vulnerability.

  • Arch Community: Promptly updated the OpenSSL package and communicated the importance of immediate patching, emphasizing security update practices.


2014: OpenSSH 6.7 Causing SFTP Issues

Start of Disruption: October 6, 2014

Duration of Disruption: Approximately 2 weeks

OpenSSH 6.7 SFTP Problems

Description:

The release of OpenSSH 6.7 led to problems with SFTP on rolling release distributions like Arch Linux, causing decreased transfer stability and speed. Ubuntu LTS users remained on older, stable versions without these issues.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Addressed the issues in subsequent OpenSSH releases.

  • Arch Community: Advised users on downgrading or applying patches, and increased testing of critical network tools before rolling them out.


2014–2015: Early Adoption of KDE Plasma 5 Leading to Stability Issues

Start of Disruption: July 15, 2014

Duration of Disruption: Approximately 4–5 months

KDE Plasma 5 Stability Issues

Description:

Rolling release distributions like Arch Linux adopted KDE Plasma 5 early, resulting in instability and incompatibility with many KDE applications still developed for KDE 4. Users faced crashes and inconsistent behavior until applications were updated and Plasma became more stable. Ubuntu LTS stayed with KDE 4, providing a consistent user experience.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Worked on porting applications to KDE Frameworks 5 and improving Plasma stability.

  • Arch Community: Offered both KDE 4 and Plasma 5 packages to allow users to choose, and provided guidance on potential issues with early adoption.


2015: XFCE 4.12 Update Causing Panel Crashes

Start of Disruption: February 28, 2015

Duration of Disruption: Approximately 3 weeks

XFCE 4.12 Panel Crash Issues

Description:

The update to XFCE 4.12 introduced new features but caused panel crashes on rolling release distributions like Arch Linux, especially when using specific plugins. Ubuntu LTS continued to use XFCE 4.10, ensuring a stable desktop experience.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Released patches to fix the panel crashes in subsequent updates.

  • Arch Community: Advised users on troubleshooting and temporarily disabling problematic plugins, and emphasized the importance of community feedback for quick resolution.


2015: GRUB 2.02 Causing Boot Problems

Start of Disruption: December 25, 2015

Duration of Disruption: Approximately 1–2 weeks

GRUB 2.02 Boot Issues

Description:

An early version of GRUB 2.02 caused boot problems on Arch Linux and other rolling release distributions, particularly on dual-boot systems. Users often had to manually intervene to keep their systems bootable. Ubuntu LTS used a stable GRUB version and was unaffected.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Addressed the boot issues in subsequent GRUB releases.

  • Arch Community: Provided instructions for manual fixes and increased testing of bootloader updates, recognizing the critical nature of boot components.


2016: Early Adoption of Wayland in Fedora 25 (Not Ubuntu LTS)

Start of Disruption: November 22, 2016

Duration of Disruption: Several months

Wayland Adoption Issues

Description:

Distributions like Fedora adopted Wayland as the default display server early on, leading to compatibility issues with certain applications that did not yet support Wayland. Problems persisted for several months until software compatibility improved. Applications like GIMP only officially stabilized Wayland support around 2020. Ubuntu LTS continued using the X.Org Server, avoiding these issues.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by other distributions.

Measures Taken:

  • Upstream Developers: Worked on adding Wayland support to applications and toolkits.

  • Fedora and Arch Communities: Provided options to use X.Org Server as a fallback and assisted users in troubleshooting Wayland-related issues.


2016: LibreOffice 5.1 Bugs on Rolling Releases

Start of Disruption: February 10, 2016

Duration of Disruption: Approximately 3–4 weeks

LibreOffice 5.1 Bugs

Description:

LibreOffice 5.1 introduced bugs affecting stability and functionality on rolling release distributions. Users experienced crashes and data loss until bug fixes were implemented in later versions. Ubuntu LTS, using an earlier LibreOffice version, did not encounter these problems.

Categorization: (1) The incident did not affect Ubuntu because Ubuntu conducted integration tests that prevented the update to the new version.

Measures Taken:

  • Upstream Developers: Released updates to fix critical bugs.

  • Arch Community: Encouraged users to report issues and consider using the previous stable version until fixes were available.


2018: systemd 239 Bug in Arch Linux

Start of Disruption: July 3, 2018

Duration of Disruption: Approximately 2 weeks

systemd 239 Bug

Description:

A bug in systemd version 239 caused boot and service management issues on Arch Linux. Users encountered failed boots and service crashes until an update was released. Ubuntu LTS was on an earlier systemd version and did not experience these problems.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Fixed the bug in a subsequent systemd release.

  • Arch Community: Provided instructions for downgrading systemd and increased testing for critical system components.


2018: Python 3.7 Compatibility Problems

Start of Disruption: June 27, 2018

Duration of Disruption: Approximately 2–3 months

Python 3.7 Compatibility Issues

Description:

The release of Python 3.7 led to compatibility problems with several packages in rolling release distributions, as many libraries and applications were not yet updated to support the new version. Users faced broken dependencies and functionality until updates were made.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Updated libraries and applications for Python 3.7 compatibility.

  • Arch Community: Maintained multiple Python versions in the repositories and assisted users with virtual environments and package updates.


2019: NVIDIA Driver Updates and Kernel Compatibility Issues

Start of Disruption: September 2019

Duration of Disruption: Approximately 1 month

NVIDIA Driver and Linux Kernel Compatibility

Description:

An update to the Linux kernel in rolling release distributions led to incompatibility with existing NVIDIA drivers, causing graphical issues and system instability. Users had to wait for updated drivers from NVIDIA to resolve the problem. Ubuntu LTS, with an older kernel, did not face this issue.

Categorization: (1) The incident did not affect Ubuntu because Ubuntu conducted integration tests that prevented the update to the new version.

Measures Taken:

  • NVIDIA: Released updated drivers compatible with the new kernel.

  • Arch Community: Advised users on holding back kernel updates or using alternative drivers, and improved communication regarding proprietary driver dependencies.


2020: Mesa 20.0 Causing Crashes on Intel Graphics

Start of Disruption: March 2020

Duration of Disruption: Approximately 2–3 weeks

Mesa 20.0 Intel Graphics Issues

Description:

A faulty version of Mesa 20.0 led to crashes and graphical glitches on systems with Intel graphics in rolling release distributions like Arch Linux. Users experienced instability until a patched version was released. Ubuntu LTS was using an earlier Mesa version and remained stable.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Fixed the issues in subsequent Mesa releases.

  • Arch Community: Provided instructions for downgrading Mesa and increased testing of graphics stack updates.


2021: Krita 4.4.2 Graphics Problems on Rolling Releases

Start of Disruption: February 2021

Duration of Disruption: Approximately 2 weeks

Krita 4.4.2 Graphics Issues

Description:

Users of rolling release distributions experienced graphics problems and crashes with Krita 4.4.2 due to compatibility issues with underlying libraries. The problems were resolved in the next version. Ubuntu LTS users, on a previous Krita version, did not face these issues.

Categorization: (1) The incident did not affect Ubuntu because Ubuntu conducted integration tests that prevented the update to the new version.

Measures Taken:

  • Upstream Developers: Released Krita 4.4.3 with bug fixes.

  • Arch Community: Advised users to temporarily use the previous version and improved monitoring of application stability upon updates.


2021: MariaDB 10.5 Causing Data Structure Changes on Arch

Start of Disruption: Mid-2021

Duration of Disruption: Over 1 month

MariaDB 10.5 Upgrade Issues

Description:

The upgrade to MariaDB 10.5 in Arch Linux led to undocumented changes in database structures, causing issues with plugins and dependencies. Users faced compatibility problems until documentation improved and necessary adaptations were made. Ubuntu LTS users, on an older MariaDB version, did not experience these issues.

Categorization: (1) The incident did not affect Ubuntu because Ubuntu conducted integration tests that prevented the update to the new version.

Measures Taken:

  • Upstream Developers: Enhanced documentation and provided migration guides.

  • Arch Community: Communicated the importance of reading release notes for major updates and offered support for troubleshooting database issues.


2021: GRUB 2.06 Causing Severe Boot Problems

Start of Disruption: June 2021

Duration of Disruption: Approximately 1 week

GRUB 2.06 Boot Issues

Description:

An update to GRUB 2.06 caused severe boot problems on Arch Linux, rendering some systems unbootable. Users had to perform manual repairs to restore system functionality. Ubuntu LTS, using a stable GRUB version, was unaffected.

Categorization: (1) The incident did not affect Ubuntu because Ubuntu conducted integration tests that prevented the update to the new version.

Measures Taken:

  • Upstream Developers: Released patches to fix the boot issues.

  • Arch Community: Issued alerts to users, provided detailed recovery instructions, and reinforced the critical nature of bootloader testing.


2021: PHP 8 Upgrade and Plugin Incompatibilities

Start of Disruption: November 2020 (PHP 8.0 release), issues persisted into 2021

Duration of Disruption: Approximately 1–2 months

PHP 8 Upgrade Issues

Description:

Arch Linux adopted PHP 8 soon after its release, leading to incompatibilities with many plugins and extensions for platforms like WordPress and Drupal. Users experienced broken websites and functionality until plugins were updated for PHP 8 compatibility. Ubuntu LTS continued using PHP 7.x, avoiding these issues.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Updated plugins and extensions to support PHP 8.

  • Arch Community: Advised users to maintain PHP 7.x if necessary and provided guidance on managing multiple PHP versions.


2022: PipeWire and Audio Compatibility Issues

Start of Disruption: Early 2022

Duration of Disruption: Approximately 1 month

PipeWire Audio Issues

Description:

Rolling release distributions like Arch Linux adopted PipeWire as the default audio server, leading to compatibility issues with certain applications and hardware that were not yet fully supported. Users experienced audio dropouts and configuration difficulties. Ubuntu LTS continued using PulseAudio, avoiding these problems.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Continued improving PipeWire’s compatibility and stability.

  • Arch Community: Provided fallback options to PulseAudio and assisted users in configuring PipeWire, emphasizing community feedback to address issues.


2023: OpenSSL 3.0 Migration Issues

Start of Disruption: Early 2023

Duration of Disruption: Approximately 2 months

OpenSSL 3.0 Migration Issues

Description:

The transition to OpenSSL 3.0 in rolling release distributions caused compatibility problems with applications not yet updated to work with the new version. Users faced broken packages and security tools until applications were patched. Ubuntu LTS remained on OpenSSL 1.1, maintaining stability.

Categorization: (2) The incident did not affect Ubuntu because their release cycles lagged long enough until the problem was noticed and resolved by rolling release distributions.

Measures Taken:

  • Upstream Developers: Updated applications and libraries for OpenSSL 3.0 compatibility.

  • Arch Community: Helped users identify affected applications and provided instructions for installing compatibility libraries.