The Outage

On the 19th of July 2024, many computers running Windows crashed to pretty Blue Screens of Death. This has affected computers in almost every country and all kinds of organisations. Most of the affected computers were PCs, but servers running Windows were also savaged. This rather crippling event resulted from a bug in an update to Crowdstrike Falcon. Ironically, Crowdstrike Falcon is software meant to protect computers from cyberattacks. And yet, on this occasion, it has acted like a nasty malware.

Post-Mortem

It can be argued that surely there was a lack of testing and a bug like this one should have been spotted early before the faulty update was pushed to the customers. However, I am not interested in bashing CrowdStrike Holdings, Inc. There are bigger fish to fry.

Why was Crowdstrike Falcon Installed?

For a start, various organisations have to comply with applicable regulations, and some of them mandate that specific forms of protection against cybersecurity threats must be employed. At that point, the organisations simply have to install something, be it Crowdstrike Falcon or something similar.

Of course, I could mention that my various computers do not run a zoo of anti-malware and yet for 20+ years I had no malware ever infecting them, but that would be an unfair comparison and a rather long story.

Then, there is an idea that to have good protection one must simply throw money at the problem. A budget is allocated and spent, then boxes are ticked. Job done. In case a cybersecurity risk hits the firm, they can always say: “We did buy this protection, you cannot blame us”. Throwing money at security may also be a good way of reassuring investors… until the point where it becomes abundantly clear that it did not work.

But of course, that’s not how security works. Imagine a medieval town surrounded by a high impregnable wall, yet featuring wide open gates which cannot possibly be locked, and cargo going in and out every day. Of course, they have thrown money at the problem by building the wall, but it does not mean they cannot be successfully invaded by a dozen drunkards simply walking through the open gate. Actual security — not “security theatre” — is a hard problem, and money is not enough.

Finally, people expect that the software they are using is so vulnerable that there is no way to operate without the most intrusive sorts of cyber protection. They are mostly right. Modern software is full of bugs and vulnerabilities. They should not really be there, but they are, and they will be there because of a rotten culture which produced them and that culture is not going away.

Why was the Outage Small?

Yes, it was small, relatively speaking. It could have been much, much worse.

This is because in this particular case, to my best knowledge, the outage was caused by a bug rather than deliberately designed malicious code. Had it been the latter, there would have been no chance of recovering from that in mere days.

People Never Learn

Some years ago Edward Snowden went public with his revelations about the abuse of power and ongoing mass surveillance done by the National Security Agency and others. As of this writing, he is forced to stay in exile. Afterwards, “The Shadow Brokers” — a group associated with Russia — posted that they are in possession of the cyberweapons of “Equation Group“, an APT linked to the NSA. Oopsie.

So how did it happen? Surely, cyberweapons of the NSA are not something one can find in Walmart. You would expect them to be protected, and with extreme prejudice if necessary.

As often happens in such cases, there can be no truly verifiable answers, as those who have evidence on their hands do not live long enough to demonstrate it in public. However, it appears the leak went like this:

1. Edward Snowden mentions “Equation Group” (without much detail) as an APT linked to the NSA.
2. Meanwhile, the NSA are using Kaspersky antivirus to protect themselves from malware.
3. Kaspersky — either on their own initiative or after being asked by Russian intelligence services — added the phrase “Equation Group” to the signatures in their antivirus.
4. An update to the antivirus database happens, and the antivirus notices various files mentioning “Equation Group”.
5. Most of the files are noise, like public data about Mr. Snowden’s revelations, but some are the “Equation Group” software on NSA machines. All of that is dutifully and automatically shipped to Kaspersky for analysis.
6. And of course, “The Shadow Brokers” is a Russian APT.

If this is true at least in part — and this is not just my thinking on this — then the question is “Why in the Lord’s name have the NSA have installed Kaspersky antivirus on their computers?”. Seriously, what sort of threat exactly were they so afraid of that it was worse than having their most secret cyberweapons leaked to a state that has wanted the USA dead for around a century now? Precisely what has stopped them from at least using an antivirus made by a firm in the USA? And it’s not like Kaspersky has a spotless reputation. It is well known that Kaspersky has very close ties with FSB. In fact, in Russia, I have heard people gossiping that Kaspersky was so good at the protection against malware because half of that malware came out of their own labs. Whether that is true or not I do not know, but as they say in that part of the world: “there is no smoke without fire”.

Basic Facts

1. Every anti-malware runs with elevated privileges.
2. Every anti-malware can analyse all the data on the computer where it is installed.
3. Almost every anti-malware has enough of access to do pretty much anything.
4. Almost every anti-malware has a connection to the cloud of its vendor, and you cannot see or control what is going on through that channel.

If compromised, anti-malware is in a perfect position to cause massive damage by stealing, subverting, and damaging anything on your computer. Firewalls cannot help you as for anti-malware to work you need to approve the link between anti-malware and the vendor, which turns it into a nice exfiltration channel. Furthermore, the damage can be done in a single mighty blow rather than in a piecemeal fashion. In other words, as soon as you have installed anti-malware, you have rooted your own computer voluntarily.

Epilogue

As I have mentioned earlier, the Crowdstrike outage was not a result of malice. However, it is only a matter of time until one of the anti-malware kits is compromised by an APT and used as a worldwide cyberweapon. And then there will be weeping and gnashing of teeth.

A way to minimise the risk while still having anti-malware installed would be to install anti-malware from different vendors so that a fault in one of the systems would not take down the whole organisation. However, even in this case, the damage can be severe.

If you are not bound by the requirements of having anti-malware everywhere, you might be better off using systems which are not capable of being infected by malware in the first place, at least where it matters. Such things exist if we consider this matter from a practical rather than theoretical angle. However, you won’t be able to run Microsoft Word on them, of course.

My view is that the situation is going to get worse until it reaches a breaking point. The unwarranted complexity of software is already sky-high and rising. Complexity introduces weaknesses and bugs. Most people do not appreciate this fact and carry on as if everything is all right. Meanwhile, castles on the sand are built and sold every day. This complexity and abundance of bugs open up opportunities for malware. The malware scares people, and they buy into anti-malware, effectively creating additional backdoors. Combine that with the ridiculous level of supply chain problems — hello, “Log4Shell” — which are also impacting anti-malware, we head into the world where everything will be infected by default, no matter how much anti-malware is present. The standards will drop to a point where the presence of malware is not seen as a problem. It will be on par with death and taxes.

I am looking forward to shocking revelations that bearer tokens