It’s very difficult to be candid about a severe security incident. In compiling the CISOs’ Lessons from Security Breaches report, I’ve had the opportunity to talk to 5 CISOs who have led teams through some of the worst security breaches imaginable. These cybersecurity leaders have chosen to overcome the stigma surrounding breaches to share their insights. It’s worthwhile to listen to their stories to fully appreciate the reality of living through a breach and to heed their lessons—not just for the C-level, but for all of us.
Unfortunately, the narrative around a breach often focuses on how initial access should have been stopped. Suffering a security breach is seen as some kind of mark of shame. This can stop leaders from sharing their lessons from these breaches, and hinders the flow of information that could make the cybersecurity industry more efficient.
Security Breaches Are Inevitable
Does suffering a breach mean that you have already failed? Michael Coates, former CISO at Twitter told me “No, but that comes with a big asterisk.” It comes down to if you are sufficiently prepared when the adversary gains access. If your lack of preparation has been egregious, then yes, a security breach can lead to a severe data breach and mean failure.
On a long enough timeline, every company will suffer a security breach. As CISO Márcio Sá told me “the adversary has the advantage of a large number of distribution points they can attack while you have limited resources to defend them.” That doesn’t mean that defense is futile, but it does mean that a defense-in-depth strategy needs to consider the risks and be prepared to defend, detect, mitigate, and remediate security incidents.
In 2011, RSA was breached despite being one of the most reputable cybersecurity companies in the world, and despite having top-shelf security tools in place with world-class talent operating their defenses. Former RSA CTO Bret Hartman told me that their monitoring and visibility tools made the difference in avoiding catastrophe. RSA was able to detect the intrusion in only 5 days (the current industry average is an appalling 201 days). From there on they were able to follow the attackers’ every move across the network.
The Reality of the Breach Experience
The scale of an incident response operation is difficult to overstate. Teams gather from across the company in a physical ‘war room’ to coordinate tasks. External teams are brought in from law firms, forensic experts, government agencies, and more. “You have countries calling you” emphasized Tim Brown, CISO at SolarWinds.
Michael Coates told me that the one thing that Hollywood gets right is the image of a room full of people in hoodies, huddled over laptops in the middle of the night, surrounded by pizza boxes. There are so many ‘lines of work’ to coordinate during the day that reviewing external communications and briefing leaders happens late at night or in the early morning.
As much as one can—and should—coordinate tabletop exercises and roleplays to simulate a breach, the full experience of a breach is something that cannot be prepared for. There are countless practical lessons to be learned, covering topics such as communications, partnerships, team management, monitoring, visibility, and more.
One CISO told me that he ‘held it together’ during the breach, but afterward, he was physically, emotionally, and mentally a mess. “I cried, I cried, I cried,” he said. Another told me of the countless calls he had to make over his career to cybersecurity leaders at other companies to tell them that they were potentially breached and how grizzled cybersecurity veterans would break down, too.
The Burden of the CISO
There is still some debate over where the head of information security’s role should lie within a company: should the CISO report to the CEO, the CSO, or CIO?
What became clear from these conversations is that the information security function isn’t something that can be delegated away to one isolated team. Information security is a business risk spread over the entire organization and requires consideration when designing and performing the processes of every department. The ‘information security function’ is simply too big for one person to take full responsibility over.
There are plenty of jokes about CSO meaning “Chief Scapegoat Officer’. When a security breach happens, it’s always the head of security that takes the fall. Instead, leading a team through a security breach is a rare skill that is becoming more and more in demand. Experiencing a security breach shouldn’t be a mark of shame, but a badge of honor earned on the cyber battlefront.
While reflecting on the RSA breach of 2011, Bret Hartman told me how that was a ‘pretty big canary in the coalmine’ but more than a decade later, we still aren’t much better off than we were then. Unfortunately, the lessons to be learned from countless security incidents either aren’t being told or heard. We can only hope that more cybersecurity leaders will come forward with their lessons, that you will heed the lessons shared in the CISOs’ Lessons report, and take them to heart.