Affinity Water are the largest water-only supply company in the United Kingdom. With a history dating back to the 1880s, they supply 3.6 million people with 900 million liters of water on a daily basis. That’s roughly 360 Olympic-size swimming pools of water, every day of the year.
With such a big operation, Affinity Water rely on a vast and complex computing system. They have to store and process huge amounts of data relating to the logistics of water supply: customer information, quality control tracking, demand planning, distribution, metering, billing, and so on. All of this information has to be accessible at all times for services to function. Millions of people access and pay their bills online, and their support department is funnelled through the website, so everything has to be online and functioning at all times, without any downtime. On top of this, Affinity Water are handling large amounts of sensitive personal data, so the cybersecurity has to be flawless.
Affinity Water have approximately one-hundred Linux instances powering their service infrastructure. Some are in the cloud on AWS, and some are on-premise VMs, operating on a mix of Red Hat, Oracle, and Ubuntu.
Affinity Water’s big problem was downtime caused by server reboots. Historically, Linux maintenance has been a very manual process. Whenever a new vulnerability is discovered, a new patch comes out, and a server reboot has to be scheduled for the patch to be applied to the kernel. For Affinity Water, this process was an unnecessary administrative overhead. Server reboots needed planning way ahead of time, and the coordination consumed time and resources. The reboots usually had to happen at unsociable hours and involved lots of email scheduling and haggling between departments. Each reboot needed a mountain of documentation and management reports before it could be carried out.
Because rebooting was delayed, there was always a gap between patch release and patch application. Sometimes it could take weeks or months to organize a maintenance window. This was leaving their systems vulnerable to attackers, and potentially noncompliant with their security agreements. For a long time, Affinity Water didn’t see any other way. The Linux kernel is prone to vulnerabilities, and vulnerabilities need patching. There was no way to patch a live kernel, so the awkward reboots had to happen. And then their system admin discovered live patching – a way to keep Linux servers patched and safe, without having to shut them down. After learning about live patching, Affinity Water looked deeper, hoping to find a service that could run on all of their Linux distros. This was when they discovered KernelCare.
How did KernelCare Solve the Problem?
Affinity Water run a mix of Linux flavors: Red Hat, Oracle, Ubuntu. Unlike other live-patching services, such as KSplice, which only runs on Oracle’s own brand of Linux, KernelCare is distribution-agnostic so it can patch vulnerabilities on all platforms. This was important for Affinity Water, who needed to protect all their servers, regardless of distribution.
Once KernelCare was up and running on all of Affinity Water’s Linux instances, management’s worries about downtime and reboot cycles evaporated. All-round efficiency was immediately boosted, and the compliance teams were delighted because Affinity Water’s systems were now always up to date with security protocols. Their Linux system administrators did less weekend work and were able to enjoy more family time.
Another bonus was KernelCare’s patch roll-back facility. While Affinity Water’s system administrator was impressed with the patch turn-around time, they loved the ability to roll back patches and run more tests if they needed to, removing any concerns about whether a patch might affect performance or stability.
KernelCare freed Affinity Water from having to face the dilemma of sacrificing either security or uptime.
With rebootless, zero-interruption live patching, they could have the best of both worlds. They are secure, without any added downtime. And a bunch of staff members now had one less thing to stress about.