Cutting Vendor Bull: The Cyber Defense Matrix

Have you reached out to a cybersecurity vendor and been sucked into their sales meetings, only to find out later they don't ACTUALLY fit what you need? How much time have you wasted discussing a technology "Powered by artificial intelligence" only to learn its just a big "if" statement?Please, let me empower you: introducing the Cyber Defense Matrix (https://cyberdefensematrix.com), or buy the book!(Amazon affiliate link. Hey, servers cost $$$!).

Table of Contents

The Problem

Not all cybersecurity vendors suck, but come of their marketing does! One common complaint (and the main driver of the development of the matrix) is that vendor marketing seems designed to convince us that some product can solve all of your security needs. Can't we all just #behonest? Yea, I laughed too. Still, that isn't the biggest issue I see every day.One common thing I do in my day job is advise CIOs and CISOs on cybersecurity technology investment: things they need, things they don't need, and things they should plan for next quarter or year.Sometimes I have the unpleasant job of talking about how the entire budget DETECTs a threat with no thought to the need to RESPOND. Or a significant amount of money was spent PROTECTing a network with a total disregard for the crypto miner running in the server closet at a branch (yes, that was a real world detection).If you haven't gotten the purpose of the capital letters, you will momentarily. #defenseindepthIf you asked me what the most common knowledge gaps I deal with on a day-to-day, I'd list them out like:

1. Not knowing how to prioritize technology investment
2. Not knowing how to decipher the marketing buzz around a new technology (SASE, looking at you here!)
3. Not knowing what a vendor's offering actually protects, or where it fits within the business

A Solution: The Cyber Defense Matrix

Oh great... another framework? Yes and no. I certainly recommend building a security program around a framework like theCIS 18orNIST CSF, but those are frameworks with a different purpose: building a security program by illustrating controls. The Cyber Defense Matrix is quicker and serves a different purpose. It doesn't aim to dictate which type of keyboard it's appropriate to hide your password under (hint: none, sitcky note on the monitor is the power move here). It aims to assist categorizing technology by identifying theverbandnounthat technology can be described by. But first, some background.

What is it?

The Cyber Defense Matrix was created by Sounil Yu when he was the Chief Security Scientist at Bank of America. To directly take from his own words and book, his job was to meet with security startups to understand what capabilities they offer and determine whether or not these were needed within their portfolio of security controls. It is a "Mutually Exclusive and Collectively Exhaustive" (MECE) 5x5 framework mapping theNIST CSF functions(IDENTIFY, PROTECT, DETECT, RESPOND, RECOVER) with a list of different types of assets needing those functions (DEVICES, NETWORKS, APPLICATIONS, DATA, USERS).In other words, verb = NIST CSF Functions and noun = assets.At the bottom of the matrix is an illustration capturing an ideal balance of responsibility between technology and people. When possible, we should be relying on more on technology to IDENTIFY and more on people to RECOVER. and also indicating that process controls apply equally across the spectrum. Last, between PROTECT and DETECT is an imaginary line illustrating some security event, which becomes very helpful when we begin digging into vendor capability.

The Functions

To most people, theNIST CSF functionsof IDENTIFY, PROTECT, DETECT, RESPOND, RECOVER are pretty obvious. They have pretty apparent English definitions. When applying that to the matrix, things get a little more wonky. There is some needed separation from the NIST CSF functional definitions and the implementations in the matrix. I encourage you to buy the book (use the link at the top of the page!) for an exhaustive explanation. But for one example lets ask a simple question illustrated in the book: do we IDENTIFY or DETECT vulnerabilites?

Applying the boom principle: the event line

In a past professional life I spent 11 years in the Army. A common phrase during that time was "Left" or "Right" of boom. This was a differentiation of actions before or after an event on an imaginary timeline. Left of boom indicates things you could do to prepare or leading up to the event. Right of boom captures actions conducted after the event.The Cyber Defense Matrix captures this as the "event" line between PROTECT and DETECT. This helps answer the question posed a moment ago: do we IDENTIFY or DETECT vulnerabilities? According to NIST, it is a detect function:DE-CM-8:"Vulnerability scans are performed." But for ease of use in the CDM: IDENTIFY and DETECT functions have their obvious English definition, but are differentiated based on left or right of the boom, or event. In the CDM's case, it fits under IDENTIFY.There are other minor definition changes, but for simplicity of this article we will hand-wave them and continue our overview. Did I mention you should buy the book?

But what are assets?

Exactly what they sound like! By forcing us to categorize the effects of a solution on specific assets (ideally one...) we remove the temptation to believe that one vendor technology or capability covers all of our needs. The assets categories are pretty self-explanatory.

• Devices: Things in the physical space: laptops, servers, make it complicated with virtualized servers, and things dealing with the Operating System.
• Applications: Software that runs on devices, whether it's a PDF reader or your Apache web server.
• Networks: Duh.
• Data: Data in any form, stored anywhere.
• Users: The identities of the users on your infrastructure

By separating these assets out we are trying to create categories to put those pesky marketing claims. Does a firewall really protect your data? We shall see.One additional important point: you aren't limited to these specific assets. At the inaugural Cyber Defense Matrix Conference earlier in 2023, Robert Wood (CISO for Centers for Medicare and Medicaid Services) presented that they break down devices further into things like servers, laptops, and other obvious asset types because they have such a varied set of devices in the medical field. I have also used it for manufacturing businesses I've worked with to individually address the OT network. The overall point is adapt as needed, but remember this is a strategic level view.

So... how does it help?

So that's the matrix, but the real question on your mind is likely...

Now that everyone younger than 30 is sufficiently confused as to who that is, I'll tell you why YOU should.

Sharing (Experience) is Caring

Like I said before, the Cyber Defense Matrix helps you categorize vendor technology. I most frequently use it two ways:

• Mapping a company's current security tech investment to identify any obvious gaps
• Mapping a current vendor offering to cut through their marketing claims, and see how a product fits in a Company's roadmap

Both of these involve a relatively similar process, so we will build on it a bit here.There are a lot more use cases, but I can directly speak to these. Treat this as a teaser for the rest of the book.

The Practical Stuff

Now that you understand the concept, go into Powerpoint and recreate the matrix yourself for editing. It doesn't have to be pretty. We are security practicioners, not artists (apologies to the dual security/artist talents out there, I have trouble with stick figures).

1. Insert a 6x6 table
2. Label the NIST CSF functions at the top
3. Label assets on the left
4. Create new boxes for your technology 
5. Add major controls or documentation that compensates for a lack of technology

The book has many examples of technology, so I mapped some of them in the below diagram:

The Complex Technology Challenge

The main point of the matrix is to try and make each technology cover only one box. But after doing this a bunch of times I have found that visualizations across more than one box is generally the norm. The key: break down individual features of a product to make it clearer. As an example, lets take EDR (endpoint detection and response) and SIEM (security information and event management) if they are properly configured and utilized to their maximum capability. 

There are a number of protections that the better EDR agents have, which is why I subjectively categorize it across these three blocks. But this is totally subjective and is unique to the EDR technology I'm discussing with someone.A SIEM platform is really aggregating logs and providing some level of detection and alerting, but the boxes it covers would depend entirely on the logs it was ingesting.To highlight the subjective nature of categorization even further, here is a slide from the Author's RSAC 2019 presentation with a different format, but covering a lot of other technologies:[caption id="attachment_1164" align="aligncenter" width="800"]

Sounil Yu's RSAC 2019 presentation, taken from https://www.slideshare.net/sounilyu/cyber-defense-matrix-reloaded[/caption]

That being said, Sounil has thought a LOT about categorizing these things and certainly has a much larger brain than I do. So I encourage you to check out more examples of his conference presentation videos or source material.

Categorizing a Vendor

Categorizing a vendor follows the same process you did for all of your current technology, as some of the subjective discussion above:

1. Identify the specific components of a technology applied to the asset they are affecting, not where the product lives or information it uses.
2. Overlay that technology on the matrix like we did above, broken down into it's individual capabilities.
3. Don't count very small capabilities/features.  Use your judgement.
4. When complete, compare against your current technology or controls and decide if you need it.

This part can be harder than it seems as cybersecurity technology isn't particularly "open" about how it works. I hate to say it, but you may need to sit through a sales call if it is something you aren't familiar with. Or, connect with me on LinkedIn and I'll do the best I can to help.In the end, you are looking to break out the specific components of the technology. XDR? Ask how it works: "What components are in the back end that run this? Feel free to ask an engineer if you don't know, salesperson." SIEM platform? What log sources do you currently have connectors for? Who is responsible for creating detection rules, and what kinds of data can those rules analyze?"

Making Sense Of The Result

Now What?

At the end of the day, the Cyber Defense Matrix is a tool to help you make decisions. It is used as one input into your decision making process about different technology investments. You now have a visualization of your current cybersecurity technology investments. I now tend to get into the hard questions:

1. What are our critical assets to the success of our business? (OT devices in manufacturing?  ERP Servers? Production SaaS app?)
2. How does our current investment in technology and controls align with those critical assets?  Are there gaps?
3. Do we need to invest more, less, or adjust our existing investment to better protect the organization?
4. Does a new technology align to a specific niche that is hard to capture with the strategic nature of the Cyber Defense Matrix?
5. What old technology could I potentially replace with new technology?
6. If I change a technology investment, will that leave me with more gaps than I have right now?
7. Are there compensatory internal controls that I can layer to compensate for the lack of technology investment?

From there, the decision tends to make itself.

Post Article Note

I attended the Cyber Defense Matrix conference in early 2023 and was blown away by the amount of intelligence in the room, but also how each person was willing to offer help or advice. Sounil Yu came around and autographed copies of the book if you had one (or the complimentary copy you got for attending). Sounil and the rest of the people in attendance were incredibly kind during the whole event, and I am thankful for the goodwill of the security community out there!