Got a big question on technology and security for “Byte-sized Diplomacy”. Send it through here.
How did something as simple as a software update cause global chaos and what can we learn?
The Optus and Medibank data breaches in Australia showed us the significance of data for overall cyber security. The global IT outage this month following the CrowdStrike “update” highlights how interdependent and vulnerable our digital systems really are, and why technology diplomacy – and multilateral action – are so essential.
Labelled the biggest IT failing ever, the recent chaotic incident stopped Windows systems. It downed airport, healthcare and financial systems and interrupted broadcasting as well as some emergency services. It also left millions of systems with a “blue screen of death”, requiring a relatively simple but very manual remediation process.
Digital infrastructures are just as essential as energy or water for society, but are not considered “utilities” and so the provision of their services is not standardised or regulated.
It was caused by a software update to a cloud-based product called Falcon owned by CrowdStrike. This caused a malfunction, disabling systems it interacted with, including Microsoft’s Windows products. A seemingly simple mistake – the failure to conduct effective product control on a software update – affected 8.5 million Windows devices (less than one per cent of Microsoft machines) with many cascading effects.
CrowdStrike works with 298 of the Fortune 500 companies, and one in four of these companies experienced a service disruption. Estimates suggest the outage cost Australian businesses more than $1 billion and that Fortune 500 globally, including Australian companies, lost an estimated US$5.4 billion.
All at a keystroke, as it were.
Individuals, businesses and governments are reliant on digital infrastructure and digital services that increasingly come from limited sources – mostly operated by US companies who have centralised power across the technology stack (and, increasingly, the AI value chain.)
Brendan Dowling, Ambassador for Cyber Affairs and Critical Technology, told me earlier this year that part of being in a global tech market means that:
our dependency is largely on software and hardware that's being developed outside of Australia. Finding the right way to use our levers to shape a global market and to work in concert with other countries to shape that market is super important.
Dowling highlighted risks associated with losing access to payments systems “that we use to buy coffee or to shop, because of a cyber attack”. And as the CrowdStrike outage brought home: “Imagine how much shuts down. Imagine how much of our daily lives shut down at once.”
Indeed, the Australian government has approached the issue through the idea of systems of national significance. While the protection of critical infrastructure is absolutely vital, what is often just as important is the interconnections between these pieces of infrastructure. These systems are often the most vulnerable.
Historically, infrastructure such as energy, water, telecommunications and postal services were largely run as state enterprises and regulated as utilities. Today, digital infrastructures are just as essential as energy or water for society, but are not considered “utilities” and so the provision of their services is not standardised or regulated.
Increasing resilience is essential, of course, but so too will be backup options, digital and analogue for when inevitable outages occur.
What this outage highlights is that much more work must be done between governments and industry to secure current and future digital infrastructures. Global technology collaboration is urgently required to address structural weaknesses and vulnerabilities embedded in our digital infrastructure before they are exploited by cyber actors or in conflict by nation states.
We must think about how to embed security and resilience in light of these structural flaws in internet security and resilience, where one pretty simple mistake and failure of product control can result in global chaos. This is complicated as these services are concentrated capabilities, largely operating out of the United States but impacting services and governments globally.
More work is needed by governments to map the architecture of digital infrastructure and AI capabilities, to understand how access and power are distributed in areas of key reliance. This is needed to provide a comprehensive picture of the architecture – physical and digital – that underpins AI, including the critical dependencies and vulnerabilities.
CrowdStrike rolled out a fix very quickly and has already committed to increase software testing before issuing updates in the future. It has also agreed to roll out updates gradually to prevent widespread, simultaneous failures. It’s clear there is a need for increased internal (and possibly external) governance frameworks on major – and especially monopoly – providers as well as coordination between providers.
One of the key planks in the current Australian cyber security education campaign is to install software updates quickly. To ensure confidence in software updates and systems, consistent governance frameworks and standardisation processes are important. Either a voluntary code, or standard setting and review process should be established to ensure community and business confidence.
As this and many other incidents have shown, the world is heavily reliant and interdependent on fragile digital systems. Increasing resilience is essential, of course, but so too will be backup options, digital and analogue, for when inevitable outages occur. Additionally, we’ll need more international preventative mechanisms and methods of identifying critical infrastructure impacts.