VM Scanning software like Tenable and Qualys provide ‘good enough’ solutions, but if you’re looking for more flexibility over data presentation, interactivity, improved readability, better user adoption, and more effective prioritisation; then investing in custom dashboarding may be the solution that you need.

This post got a little longer than I thought it would. I’ve included a TL;DR at the bottom, and there’s a helpful Table of Contents to the left-hand side if you’d like to skip around!

Subscribe now

Principles of Dashboarding

There are 4 core principles I follow when creating technical dashboards:

1. Dashboards should answer at least one question in its entirety,

2. Stakeholder goals and objectives must be gathered and met,

3. Intended vulnerability outcomes should be documented and actively pursued,

4. Dashboards must incentivise regular reviews.

Dashboards should answer at least one question in its entirety

Consider the reason for creating the dashboard. What questions are being asked by the business, internal stakeholders, or yourself that you’d like to get a simple answer to? Questions could be:

• “What vulnerability should I be most concerned about?”

• “Which devices are most in need of patching?”

• “Am I meeting my compliance objectives?”

• “Which of my managers is performing the best/worst against our patching policy?”

A dashboard can answer one or many of these questions, but if an excessive amount of filtering is needed to get to the answer, you may need to consider separate dashboards that more effectively present the needed information.

Stakeholder goals and objectives must be gathered and met

I’m putting this principle above your own outcomes as I think this is the most effective way to enact change in the business. Unless you’re a one-stop security shop in your org, you’ll need other experts and owners in the business to support patching and back improvements to security culture.

Understand what these stakeholders want out of this service. Their requirements are likely to be much simpler than yours. They may not want to peruse the data with a fine comb, but instead see a simple list of “things to do,” whether that’s upgrades, patching, decommissioning, uninstalling etc. Create an environment that works for them and presents the data in the way that they want it, i.e.:

• A director may want to review their team’s overall patching performance;

• a Windows Server manager may want to see overall patch compliance across all Windows Servers;

• a Linux engineer may want a per-device breakdown of critical, exploitable vulnerabilities that have lapsed the patch policy requirements.

Understand that every team and individual has their own metrics for success, and building a platform that enables them to enact change and demonstrate success within their own area creates a positive feedback loop for security.

Intended vulnerability outcomes should be documented and actively pursued

Ask what you hope to achieve through the creation and distribution of this dashboard. What are your team’s Key Performance Indicators (KPIs)? How will this dashboard enable either your team or other stakeholders to start contributing to the broader organisational objectives surrounding security and vulnerability management? Some objectives to consider:

• Reducing number of active vulnerabilities that are not compliant to internal policy, i.e. critical vulnerabilities patched within 30 days, high within 60 days, medium within 90 days… etc.

• Removing vulnerabilities that are not compliant to accredited standard guidelines, i.e. PCI DSS, ISO27001, Cyber Essentials+ etc.

• Patch all vulnerabilities that have an exploit available, that are remotely exploitable, and are present on internet-facing devices.

Understand in your objectives that there’s a difference between security compliance and being secure. Security compliance is a requirement; good security posture is your goal. These concepts are often not related.

Dashboards must incentivise regular reviews

Dashboards should rarely be single-use. A dashboard used once is not a dashboard; it’s a report. While the interactivity of a dashboard for one-off events may be helpful, the strategic goal should always be to encourage and incentivise regular reviews of the content.

To better achieve this, you need to ensure that data is dynamic and as current as possible. Be consistent with your scan frequency. If Tenable scans occur daily, then ensure the information in your dashboard is updated daily. If the scans occur in the background every 4 hours, you better be updating that data every 4 hours, or at least setting expectations for regular updates.

If you follow the other principles listed and create something that answers key business questions, provides a solution to your stakeholders, and positively benefits your own KPIs, then regular reviews should be your indicator of success. If you are struggling to get that level of adoption, re-visit the other principles and consider where the failing is in the process.

Building a Demo Dashboard

To effectively demonstrate an example dashboard, I had to follow a few steps to ensure I wasn’t compromising security and still creating a live-like demo environment. I’ll be using Elasticsearch for data analytics and aggregation as I already have an environment ready to go; I’ll be generating dummy data with help from ChatGPT using Faker, and then building the dashboard with Kibana.

See the Appendix for the prompt used to generate the data if you’d like to follow my steps.

Vulnerability Overview Dashboard

Here’s a technical environment I generated with the following characteristics:

• 120 devices in scope with a mix of technologies between Windows 10, Windows Servers, RHEL, and some Cisco kit.

• There are 5 managers in scope with different device responsibilities (i.e. Alice manages all the Win10 environment, Bob manages RHEL, Carlos manages Windows Servers, but Diana and Evan own a mix of both).

• The internal patch policy is as follows (incredibly simplified for ease of use):

  • All Critical vulnerabilities must be patched within 30 days.

  • All High vulnerabilities must be patched within 60 days.

  • All Medium vulnerabilities must be patched within 90 days.

  • All Low vulnerabilities must be patched within 365 days.

• The vulnerability data is randomly generated and doesn’t represent any real vulnerabilities, but is instead a random distribution of made up CVEs, severities, solutions, and EPSS scores.

Below is the dashboard in full that represents that environment1:

Service Level Agreement Metrics

Use filters to draw distinctions between vulnerabilities that are still within your Service Level Agreement (SLA), and vulnerabilities that have lapsed from their required remediation date. Here’s a very simple (and not recommended) example of Criticals being fixed within 30 days, Highs within 60 days, Mediums within 90 days, and Lows within 365 days. Adjust the values to your organisation’s policy.

((vulnerability.severity: "critical" and vulnerability.published <= now-30d) or (vulnerability.severity: "high" and vulnerability.published<= now-60d) or (vulnerability.severity: "medium" and vulnerability.published <= now-90d) or (vulnerability.severity: "low" and vulnerability.published <= now-365d))

Kibana allows you to add buttons that filter on-click. Add a button that applies the non-compliance filter to the entire dashboard with one click to make it simple for stakeholders.

As any changes we make to the filter apply to the entire dashboard, the stakeholder viewing can then see which vulnerabilities have lapsed SLA and which to focus on first, either based on security improvement or vulnerability reduction, or even on a per-device basis.

Vulnerability Changes Over Time

By creating a historic data index, you can compare point-in-time data to see how data is trending.

Vulnerabilities History index takes a snapshot every 24h of the current vulnerability posture from the rolling index. This then allows point-in-time comparisons of data to see how the organisation is performing.

You can use the SLA metrics above to show changes in vulnerabilities for both the total and the non-compliant vulnerabilities:

Identify Non-Standard Operating Systems

If you are conducting wide discovery scans that fingerprint the device OS, it can be useful to have an ‘Other’ category that isn’t one of your standard operating systems. This draws attention to anything on the network that isn’t part of your strategic deployment, or that you haven’t accounted for properly in your view.

Identify High-Impact Vulnerabilities

This is a really simple way to draw out the potentially high-impact and highly exploitable vulnerabilities in your network, especially when used in conjunction with the other filters available in the dashboard. The table below organises the CVEs by their EPSS rating and allows the user to navigate to the NVD page through a context menu.

If you’re filtering by device, OS, or owner, this can help to bring attention to the most exploitable vulnerabilities in scope.

Remember that just because something is highly exploitable, doesn’t mean that you are exposed - further checks will have to be conducted and could also be built into your filters to increase accuracy.

Identify Most-Effective Solutions

It may be beneficial to patch by solution rather than vulnerability. This can call out how many devices would benefit from the fix and how many vulnerabilities would be remediated. Good for compliance and KPIs.

If you instead wanted to focus on security benefits, you can filter the data to show the Maximum EPSS score that would be remediated if the solution was applied.

The above is a great example of showing that while more vulnerabilities would be remediated by applying the first solution, there is actually a more exploitable vulnerability that would be resolved by applying the third solution, mitigating the higher risk of exploit from 53 devices. The specific CVEs are shown on the left-hand side which can then be investigated further.

Identify High Exposure Devices

Another thing I’ve used the EPSS score for here is to provide an ‘Asset Exposure Rating’. This is simply a cumulative total of all EPSS scores affecting a device. As EPSS is represented as a decimal, by adding all of these scores together you can create a really simple way to call out potentially high exposure devices. This can be most effective when applied to internet-facing infrastructure or with more granular filtering of CVSS vectors (see External Attack Surface - Priority Patching dashboard below).

More Dashboard Ideas

I use the above dashboard as a template to create other dashboards that highlight vulnerabilities and compliance issues in other business areas. Here’s a few more ideas for dashboards that might be of some use.

External Attack Surface - Priority Patching

Here’s one I use for what I call ‘Priority Patching’, where I focus on web-facing devices that are affected by exploitable vulnerabilities with a Network attack vector, using CVSS vectors to filter the data:

Filter below allows for more granular filtering against the CVSS vector and gives more control over the type of vulnerabilities that are shown that are most suited to your environment.

vuln.cvss3.vector : (*AV\:N* and (*Au\:N* or *PR\:N*))

Vulnerabilities by Software

This dashboard element looks at Software by installs, their vulnerabilities, and a heatmap of contribution to the vulnerability total.

This can help to identify software that is significantly out of date or contributing a large amount of vulnerabilities to the total; thus better informing business decisions around approved software, End-User Device Management, and patching procedures.

Compliance Standard Dashboards

This dashboard uses a custom tag for ‘PCIDSS’ that would group all of the assets that are in scope for PCI DSS compliance.

Further filters in the dashboard would enable the reported vulnerabilities to only show if they go outside of compliance of recommended PCI standards, i.e. the PCI standard typically only asks for Critical and High vulnerabilities to be remediated on the internal network by default, but Critical, High, and Medium on the external network:

vulnerability.severity : "medium" AND tags : ("PCIDSS" AND "public_host_ip")

TL;DR and Conclusion

If that was as lot of writing and you have too many vulnerabilities to fix to read in detail, here’s the short version;

1. Follow these principles:

   1. Dashboards should answer at least one question in it’s entirety,

   2. Stakeholder goals and objectives must be gathered and met,

   3. Intended vulnerability outcomes should be documented and actively pursued,

   4. Dashboards must incentivise regular reviews.

2. Create a clean, readable, dashboard that can be changed easily with simple filters for multiple stakeholder needs - use the one above as a starting point if you need some inspiration.

3. Consider iterative improvements to make the above easier. Better tagging, better ownership information, more data ingested etc. See my other post on using Elastic as a SIEM if you’d like to explore more.

I set out to create a better framework for independent vulnerability dashboarding. Searching for solutions and guidance to this problem presents you with companies that are offering you a pre-built template, but not the principles behind what makes these successful, and often not the granular control that I want to have in this area.

My personal experience has been entirely with Elasticsearch and Kibana, but the same may very well be achievable with other vendors in this space. This post is here to provide a framework rather than a tutorial for Kibana, so hopefully you can take my experience and apply it to your own environment.

Consider reading some of my other posts on vulnerability management:

Appendix - Dummy Data Generation Script & ChatGPT Prompt

from faker import Faker
import random
import uuid
import json
import os
from datetime import datetime, timedelta

fake = Faker()

# Predefined data based on user requirements
cves = [f"CVE-2024-{i:04d}" for i in range(1, 149)]  # 148 unique CVEs
solutions = ["Apply the latest security patch", "Upgrade to the latest software version",
             "Implement input validation", "Restrict user permissions"]
managers = {"Alice": "Windows 10", "Bob": "Linux", "Carlos": "Windows", "Diana": "Mixed", "Evan": "Mixed"}

# Predefined hosts with consistent IPs and OS
hosts = (
    [{"name": f"end-{i:04d}", "ip": fake.ipv4(), "os": "windows 10", "manager": "Alice"} for i in range(1, 64)] +
    [{"name": f"win-{i:04d}", "ip": fake.ipv4(),
      "os": "windows server 2019" if i % 10 < 7 else "windows server 2016", "manager": "Carlos"} for i in range(64, 88)] +
    [{"name": f"lin-{i:04d}", "ip": fake.ipv4(),
      "os": "rhel 9.4" if i % 5 != 0 else "rhel 9.3",
      "manager": random.choice(["Bob", "Diana", "Evan"])} for i in range(88, 118)] +
    [{"name": f"cisco-{i:04d}", "ip": fake.ipv4(), "os": "cisco 4000 series integrated", "manager": random.choice(["Diana", "Evan"])} for i in range(118, 121)]
)

def assign_manager(host):
    os_name = host['os']
    if os_name == "windows 10":
        return "Alice"
    elif "windows server" in os_name:
        return "Carlos"
    elif "rhel" in os_name and host["manager"] == "Bob":
        return "Bob"
    else:
        return random.choice(["Diana", "Evan"])

# Adjust the managers based on the rules
for host in hosts:
    host['manager'] = assign_manager(host)

# Severity distribution percentages
severity_distribution = {
    "critical": 0.08,
    "high": 0.30,
    "medium": 0.41,
    "low": 0.21
}

def generate_epss_score():
    probability = random.random()
    if probability < 0.02:
        return round(random.uniform(0.901, 1.000), 3)
    elif probability < 0.12:
        return round(random.uniform(0.501, 0.900), 3)
    elif probability < 0.32:
        return round(random.uniform(0.101, 0.500), 3)
    else:
        return round(random.uniform(0.001, 0.100), 3)

# Dictionary to store consistent information for each CVE
cve_info = {}

def assign_cve_info(cve):
    if cve not in cve_info:
        solution = random.choice(solutions)
        severity = random.choices(list(severity_distribution.keys()), weights=severity_distribution.values())[0]
        epss_score = generate_epss_score()
        published_date = fake.date_between(start_date="-1y", end_date="today")
        cve_info[cve] = {
            "solution": solution,
            "severity": severity,
            "epss_score": epss_score,
            "published_date": published_date
        }
    return cve_info[cve]

def generate_vulnerability_data(hosts, total_entries):
    data = []
    for _ in range(total_entries):
        host = random.choice(hosts)
        cve = random.choice(cves)
        
        # Retrieve consistent CVE info
        cve_details = assign_cve_info(cve)
        first_seen_date = fake.date_between(start_date="-100d", end_date="today")
        
        entry = {
            "vuln.cve": cve,
            "vuln.epss.score": cve_details["epss_score"],
            "vuln.solution.action": cve_details["solution"],
            "host.name": host['name'],
            "host.manager.name": host['manager'],
            "host.ip": host['ip'],
            "host.os.name": host['os'],
            "vulnerability.severity": cve_details["severity"],
            "vulnerability.published": cve_details["published_date"].isoformat(),
            "vulnerability.firstseen": first_seen_date.isoformat()
        }
        data.append(entry)
    return data

# Generate 8,567 entries
vulnerability_data = generate_vulnerability_data(hosts, 8567)

# Write the data to a JSON file formatted for Elasticsearch bulk import
with open("vulnerability_data_elasticsearch.json", "w") as json_file:
    for entry in vulnerability_data:
        # Write the index metadata line and the actual document
        json_file.write(json.dumps({"index": {}}) + "\n")
        json_file.write(json.dumps(entry) + "\n")

print("Data successfully written to vulnerability_data_elasticsearch.json")

Here is ChatGPT’s simple interpretation of the conversation to generate the dummy data:

I need to generate 8,567 records of dummy vulnerability data for testing purposes, formatted for Elasticsearch bulk import. Here are the details and requirements:

1. Host Distribution:

   • A total of 120 devices:

     • 63 Windows 10 devices, managed exclusively by Alice.

     • 24 Windows Server devices, managed by Carlos. 70% of them should be Windows Server 2019 and 30% should be Windows Server 2016.

     • 30 Linux Servers, with 80% running RHEL 9.4 and 20% running RHEL 9.3. Managed by Bob, Diana, or Evan.

     • 3 Cisco Routers, with host.os.name as "Cisco 4000 Series Integrated". Managed by either Diana or Evan.

2. Manager Assignment:

   • Alice manages only Windows 10 devices.

   • Bob manages only Linux devices.

   • Carlos manages only Windows devices (Windows Servers).

   • Diana and Evan manage a mixture of Windows and Linux.

3. Vulnerability Data:

   • 148 unique CVEs in total. Each CVE should have a consistent solution, severity, EPSS score, and vulnerability.published date.

   • Solution should be randomly assigned but remain consistent for each CVE. Possible solutions are:

     • "Apply the latest security patch"

     • "Upgrade to the latest software version"

     • "Implement input validation"

     • "Restrict user permissions"

   • Severity Distribution:

     • 8% should be critical

     • 30% should be high

     • 41% should be medium

     • The remaining should be low

   • EPSS Score Distribution:

     • Majority of vulnerabilities should be <0.1

     • 2% should be >0.9

     • 10% should be >0.5

     • 20% should be >0.1

     • Scores should have 3 decimal places (e.g., 0.001)

   • Each record should have:

     • vulnerability.published: Random date between now and 1 year ago.

     • vulnerability.firstseen: Random date between now and 100 days ago.

4. Field Names:

   • All field names should be in lowercase.

5. No Record Field:

   • The "record" field is unnecessary and should be omitted.

6. Output Format:

   • The output should be a JSON file formatted for Elasticsearch bulk import.

   • Each line should have an index action line ({"index": {}}) followed by the actual document on the next line, with no extra newlines between.

One of the biggest game-changers for me in VM was getting comfortable with using a SIEM to tailor the vulnerability management approach for the way I wanted it to work. No more playing with clunky dashboard interfaces and a moderate reduction in PivotTables and spreadsheets.

This post will cover my high-level approach to managing an effective data flow for VM using Elastic. I’ll be following this post up with a deep-dive into the dashboards I use and some principles for effective VM dashboarding. Consider subscribing if you’d like to be informed when this goes live.

Subscribe now

Why SIEM for Vulnerability Management?

First things first, why bother with this approach? There are pros and cons to this type of implementation and you have to understand both the benefits realised and the undertaking to get there.

Pros

• Combines multiple vulnerability feeds in one place, no more hopping between tools.

• Correlate vulnerability data with security event data to better feed into incident response process.

• Utilise multiple sources of data, not just reliant on what vulnerability tools will give you.

• Single Pane of Glass approach to VM that consolidates with other internal data sources

Cons

• Additional licensing costs beyond existing VM tools.

• Requires specialised knowledge for setup, configuration, and management (or the cost of hiring skilled personnel)

• Can be some time before you have a ready-to-go solution that’s appropriately scalable.

Don’t underestimate the time, knowledge, and cost required to effectively maintain a SIEM, even if just for vulnerability purposes. SaaS options like Elastic Cloud available; however, you must still be comfortable managing APIs, data indices, data normalisation, and all the other ‘fun stuff’ to get the solution off the ground.

Principles of VM in SIEMs

How can you identify if a SIEM integration is needed and, if already implemented, whether it is providing the value it could be? There are 4 key principles I follow:

1. Completeness - I have all the vulnerability data I need for all of my assets.

2. Maturity - I can use enhanced vulnerability intelligence to better filter and control my vulnerability data.

3. Fit for purpose - The solution works within the organisation’s operating parameters and provides value that directly aligns with organisational goals.

4. Scalability - The solution can grow seamlessly as our VM needs expand.

Architecture and Data Flow

I’ve included two high-level diagrams (HLDs) that represent how I matured one organisation’s vulnerability management process.

This environment has over 20,000 VM-scannable endpoints and two VM tools in place. The organisation has opted to scan server devices with Tenable and workstations with CrowdStrike. Two vulnerability scanning tools in the same environment can create inconsistency in VM approach, however it is essential to this organisation’s end user strategy and thus must be catered to.

Original Implementation and Process

In the above HLD, there are various elements that make the VM approach clunky and not fit for purpose:

• VM Team central to all processes

  • High degree of dependency.

  • Time-consuming manual processes.

  • Not enough staff to handle an organisation of this size.

• End users access platforms directly

  • Potential issues with separation of duties / RBAC / PAM - you don’t want users impacting scans or accepting their own vulnerabilities.

  • Two different platforms for vulnerability data adds complexity/barriers and reduces buy-in from stakeholders.

  • Data is presented inconsistently across platforms.

  • Filtering based on stakeholder requirements is complicated or unachievable.

• Additional vulnerability intelligence is manually integrated

  • Increases time to respond and fully consider impact of identified / emerging vulnerabilities.

  • No ability to do more effective filtering of data based on certain characteristics, i.e. ownership, service, network location, exploitability.

• Manual reporting using inefficient processes/tooling

  • Can be more difficult to present concise and valuable data to stakeholders.

  • Lack of consistency in VM Team approach to reporting, dependent on Excel competency and individual approach to data presentation / attention to detail.

  • No use of automation. Increases response time to send detail to stakeholders.

While there are some significant issues with this design, this is a relatively common, albeit immature approach to an organisation’s vulnerability management. The organisation has invested in a vulnerability scanning tool, and now they want their vulnerabilities dealt with as quickly as possible. It’s workable, but it doesn’t meet our principles.

Matured Process with SIEM

The above HLD shows the improvements made to the VM process through introduction of a SIEM.

The input feeds are split into 3 sections:

• Vulnerability Scanning - Any VM scanning performed by the organisation on its owned technology assets.

• Vulnerability Intelligence - External information sources that enhance VM capabilities by providing additional intelligence/insights.

• Proprietary Enriching Information - Internal data that adds further context or intelligence, further enriching VM capabilities.

All the data is consolidated into Elasticsearch, where it is indexed into usable fields. Kibana is then used to create user-facing dashboards and reports by leveraging this data.

Through making the changes, several problems are addressed:

• Centralisation of Elastic SIEM

  • All data is now fed into Elastic and normalised.

  • Overheads on VM Team are reduced, freeing up resource to focus on strategic improvements, vulnerability research, and threat hunting.

  • Enables consistent dashboarding.

  • Enables automated alerting.

  • Enables secure stakeholder access for strategic vulnerability response.

• Automatic data feeds and integrated intelligence

  • Both VM products now present the same data within the SIEM, creating a ‘source of truth’ for stakeholders.

  • Intelligence feeds are integrated which enriches data from VM product.

  • Greater ability to prioritise vulnerabilities to focus on material security threats.

• Internal asset information integrated

  • Accurate ownership assigned to vulnerabilities enables accountability.

  • Allows ‘gamification’ of vulnerability remediation across organisation.

  • Internal environmental information helps to determine asset criticality, and thus better prioritisation.

• Better dashboards, reporting, user access, RBAC, and PAM

  • Single Pane of Glass approach to VM.

  • Non-security stakeholders have a secure, role-based access mechanism to view their vulnerabilities.

  • Defined rules send automated alerts to stakeholders at point of data ingestion, significantly reducing ‘time to respond’ (TTR).

  • Dashboards are easily tailored to each stakeholder’s needs, from operational engineers to senior leaders.

The new process addresses some of the major issues in the original design and meets the key principles defined earlier.

1. Completeness - Solution can ingest all VM data in scope.

2. Maturity - With integrated internal and external intelligence feeds, I can better prioritise my vulnerabilities and enable my stakeholders to remediate them.

3. Fit for purpose - The organisation’s technical product strategy remains intact and the solution can handle a future shift if necessary. New approach ensures that invested resources are more efficient and provide a greater return on investment.

4. Scalability - The solution will scale for the organisation’s needs for the foreseeable future.1

In conclusion…

As someone who has worked in organisations both with and without a SIEM, I’ve become a bit of a convert to having one. There is an incredible amount of control that SIEMs give over data and presentation of that data that I think has a significant impact on culture and buy-in for vulnerability management.

VM can sometimes be a bit of a war of attrition. There are new vulnerabilities reported every day and there isn’t enough time or neurons available in my head to care about all of them. If you can give people the data that they need, in the format that they want it in, with as few barriers as possible, that clearly states ‘FIX THESE’, with intelligence to back up why… then you’re already winning half the battle. SIEMs help me to get there quickly and achieve these goals.

There is a fundamental difference between security and patching. Not all patching improves security, but security will suffer without patching. It’s your job to laser focus on the material security threats to the organisation and to invest your time into the processes and procedures that enable you to do this.

My next post will be all about dashboarding with Elastic where I’ll show off some of my creations and discuss my approach to dashboards and how I make them readable and effective. Consider subscribing to be notified when this is released!

Acknowledgements

My Elastic goblins Security Engineers who are excellent at what they do and keep the platform running (relatively) smoothly.

Clement Fouque, whose blog I saw 18 months ago that inspired a lot of work that was done internally for this process (and who’s architecture design I shamelessly repurposed for this blog). Consider following him for far more detailed technical guides for enhancing VM with Elastic and Qualys.

Elastic in general, who were very helpful with tips on setting up a test environment for this and future posts.

Extra Reading

While there’s a lot here for any person to unpack on the cultural impact, financial loss, lessons to be learned in reliability engineering etc. what really fascinated me about this was the insane theorycrafting from conspiracy nuts meticulous connection of dots from online free-thinkers who believed they had stumbled upon another example of the ‘New World Order’ keeping the common man down.

Come with me as we explore some enlightening examples of this. Feel free to skip the recap if you’re familiar with the incident.

A Brief Recap

For those not in the know, CrowdStrike is a cybersecurity company that delivers a good hunk of its services through the installation and distribution of a piece of software known as an ‘agent’ onto a device. This agent helps to manage the device and collect useful information. When installed across all your devices, you can manage them all at once without having to access each one individually. Handy!

On July 19th, as intended, an automated update was pushed to all of these agents to increase security capabilities. Unfortunately, due to a little issue in the configuration, this caused a system crash on Windows devices that we lovingly call the ‘Blue Screen of Death’ (BSoD). Not handy!

Even worse, because the update was present on the system every time it tried to boot, it was constantly being forced to blue screen and could never actually enter into any useful state where it could be remotely managed. The solution? Access each device individually to boot into safe mode and remove the agent or force an update. Somewhat easy on one device. Not easy if you have hundreds or thousands, all geographically separated.

Thus created, one of the greatest technical sh*tstorms of the modern era.

The Conspiracy

With any technical incident that gets widespread attention, and specifically one that had a noticeable impact on the non-techie folk, it doesn’t take long for people to start talking and wondering why. Standing too long in an airline security queue will do that to you. Smashing two neurons together at incredible speed has led people to see ‘Windows’ and ‘outage’ and identified the culprit as none other than Bill Gates.

But who is Bill Gates, really? The average person may tell you they are:

• The founder and former CEO of Microsoft

• Philanthropic multi-billionaire

• Co-Founder of charitable venture The Bill & Melinda Gates Foundation

• Iconic standing-position chair jumper

But on another undefined end of the spectrum, Bill Gates is a far more sinister character who is known for:

• Funding and planning COVID-19 to sell vaccines that would cause death, infertility, and/or contain devices to track you - codenamed ‘The Great Reset’

• Being a prolific purveyor of Satanism

• and also has a kinda’ cliché agenda to block out the sun.

And of course, most recently on July 19th, 2024; humanity received a grim reminder of how much control Bill Gates still has over the world’s stability.

Please play this song for a moment.

…

…

…

Thanks

Except, he doesn’t. At least not in the case of the CrowdStrike outage for as far as I can be sure. One nice thing about a conspiracy being spread in my field of expertise is that I can at least say I understand the subject matter closely enough to know why the conspiracy is rubbish. If the multitude of root cause analyses available aren’t enough to convince you, there probably isn’t a way to convince you short of any conspiracy touting media outlet you do trust retracting their story and correcting the record.

…And since I haven’t seen any pigs soaring through the sky today, I hope you enjoy this selection of stable geniuses.

The Conspiracists

Warning that there is some colourful language used below, left in its entirety for historical accuracy. There were some threats of violence that were omitted.

I know using Facebook is cheating, but sometimes you have to stare into the abyss to see what stares back. Other popular haunting grounds of Bill Gates-related conspiracies are Twitter and Reddit that are never short of exciting ways to tie Bill to some predicted catastrophe.

Conclusion

Thankfully the Internet is by majority in agreement that what happened with CrowdStrike wasn’t a psyop conducted by an evil billionaire. However, outside of IT outages, there’s still a hugely alarming amount of misinformation, conspiracy theories, and calls to violence against perceived threats to the World as they know it.

Misinformation is even easier to purport with the advent and availability of LLMs (Large Language Models, i.e. AI) that can really make bystanders and fence-sitters believe something is more widely accepted than it is.

In a future post, I’ll be exploring the “Dead Internet Theory” and whether or not there’s any truth or ‘loss of ground’ to the claim that the Internet is now mainly made up of bots and machine-generated content.

Subscribe now

If you’re looking down an insurmountable pile of vulnerabilities from a vulnerability scanner, it may feel like you’ll never get the engineering time, manpower, or buy-in to even begin to scale up remediation efforts. Trying to become compliant to that accreditation/certification you need (ISO27001, PCI DSS, CyberEssentials+), or whatever policy someone mashed together 5 years ago must feel like an impossible task... How can you begin to resolve this?

Let’s first start with some hard truths:

• There are on average 2,400 new vulnerabilities published every month, and in 2023, there were 29,066 vulnerabilities published. As of only Oct 2024, there are already 29,398 (VulnCheck, 2024).

• In most large technical environments, there is not enough time/resource to patch everything consistently.

• Effort patching the ‘wrong’ vulnerabilities damages security culture, rapport with engineering teams, and often doesn’t do anything to benefit organisational security.

So we have an ever-increasing influx of vulnerabilities, and apparently, sometimes we can patch and it might not even provide any benefits? ‘Fraid so. Of course, ignoring the potential update crashing your system or any other negative effects of applying a patch, your system won’t be any less secure, but you may be harming long-term performance in this area by not understanding how best to utilise your resources. If you’re wondering where to go from here, then here’s 5 of my essential tips to creating a rock-solid foundation for vulnerability management.

1 - Understand and prioritise your assets

Asset management. That’s right, it’s time to pull your socks up an- Wait, no, come back — I know it’s step 1 everywhere and you still don’t have a good solution for it, but hear me out. Asset management, from a vulnerability management perspective, doesn’t have to be the 100% perfect dream I know you wish it was. In fact, it’s our job to take a stab at fixing it. Utilise your subject matter experts internally and map whatever way you can. Take worksheets, CSVs, JSON files, notepad files if you have to. Start understanding what you have, what it does, and where it sits. Your goal here is to understand what’s important.

If you’ve got web servers, load balancers and other web apps that are tempting every BugBounty registered member and scriptkiddie with a Kali Linux install, put these in DEFCON 1. If it’s old, out of support, and the cornerstone of your entire business operation (see image), keep a real close eye on how someone might reach/break it. If your end user devices are in the hands of people you barely trust to button up their shirts correctly, consider what they’re allowed to install on their machines and the urgency at which it would need to be resolved to prevent disaster.

2 - Think inside of CVSS

If you haven’t heard it yet, prioritising vulnerability fixes solely by CVSS score is bad. That doesn’t mean CVSS itself is bad, but the way you interpret the data and use it to make decisions has to be able to respond to the needs of the business. For example, your business might be in providing essential utilities to customers. You don’t store anything of any incredibly high value, but your availability is what matters most. Any blip in service may impact customer’s lives. Now let’s compare two imaginary vulnerabilities:

Vulnerability 1
CVSS Score: 9.1 (Critical)
CVSS Vector: AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:N

Vulnerability 2
CVSS Score: 7.5 (High)
CVSS Vector: AV:N/AC:L/PR:N/UI:N/S:U/C:N/I:N/A:H

You’d think from just looking at the scores and criticality that the first one is obviously worse, I mean look at it, it’s precisely 1.6 worse! In fact, the only difference between the two is that I swapped the impact metrics. Vulnerability 1 affects confidentiality and integrity, whilst Vulnerability 2 affects only availability. Both are easily exploitable over the internet, with no user interaction or privileges required. Both are a PR nightmare, but the 2nd may (in this imaginary business’ context) genuinely put people’s lives at risk.

Understand the metrics that make up CVSS scores and apply the logic to your assets and your business functions.

3 - Think beyond CVSS scores — enrich your data

CVSS is not the only open standard for scoring and assessing vulnerabilities and CVSS is quite limited in the scope of what information it’s capable of giving you. Here’s two FREE ways to improve:

EPSS - Exploit Prediction Scoring System — EPSS assesses a probability score for each CVE between 0 and 1 - the higher the score, the greater the probability that a vulnerability will be exploited. If you set a threshold to only see vulnerabilities with an EPSS score over 10%, you’ve successfully deprioritised 98%~ of vulnerabilities. It’s definitely worth reading the detail about their model and understanding how it can be effectively used to prioritise your data.

CISA KEV — CISA ‘Known Exploited Vulnerabilities’ is the bible for checking whether or not a vulnerability has been exploited in the wild. This helps businesses’ to prioritise their focus on vulnerabilities that can materially manifest as a result of action taken by a malicious threat actor. As of writing, there are 1,137 entries in the Known Exploited catalogue which is a monumentally smaller number to manage than the total vulnerabilities reported. Only 43 of the 22,673 vulnerabilities released this year are ‘Known Exploited’. If you’re concerned about security rather than patch compliance, this data should not be ignored.

There are other paid solutions in this space that are offering heightened levels of analysis and proprietary scoring/prioritisation but these often are not cheap and will not typically have your environmental context in mind. There is value in these solutions, but you may find quicker and easier wins at low maturity levels by implementing the data feeds above into your vulnerability analysis.

4 - Get buy-in from the people that matter

Here’s where you’re going to have to put your laser sights on the people that are your ‘do-ers’ and ‘enablers’. Your ‘do-ers’ might not be quite as important if you’re responsible for both vulnerability management and patch deployment, but it’s still very hard to mature if you haven’t got the right people along for the ride.

You’ve first and foremost got to identify the people that get stuff done. People responsible for your Windows/Linux servers, the team that looks after your network equipment, the people managing your web servers, your VMWare engineers (if you can get them to stop cursing at Broadcom for 5 minutes)… All these people will help you to move your mountains, but you have to be willing to work with them and understand that their priorities will differ to yours. As much as some people may argue; security is a blocker. You can definitely make the process less painful, but any time spent on security is time not spent improving and iterating. That doesn’t mean it’s not essential, but often these team’s metrics are not ranking security patching as equal to their cost-benefiting deliverables.

Where you may face pushback from these teams that they simply can’t fit in the patching, that’s when you’ve got to get buy-in from your ‘enablers’. These are your senior leaders, your CISOs, directors, CTOs, CEOs etc. These stakeholders need to understand why we carve out development time to focus on security. If you aren’t getting the traction you need, then it’s your enablers who need to give that wheel some traction. Explain the risks, explain the potential ramifications, and arm them with as much information as is needed. They will ultimately have their own risk appetite for what they’re willing to tolerate - as long as you keep a log of the concerns raised and evidence that you have raised it, that’s then the business’ decision how they respond. Sleep easy knowing you’ve done all you can.

5 - Improve intelligence and reduce notification time

In the security world, we’re often assessed against how quickly we can detect, and how quickly we can respond. In vulnerability management, there isn’t always an event to detect, but there are constant shifts in the security world. Companies are getting breached, vulnerabilities are being identified, exploits are being written which are then mobilised by malicious threat actors. It’s imperative we are notified of these events as soon as possible so that we can assess the potential damage that may have already taken place, or may inevitably take place if not acted upon.

The main goal here is to bring down the notification time so that the response time can be reduced. The sooner something is in front of you to quantify the risk, the sooner the business can be informed of actions they should take. To reduce the notification time, consider:

1. Set up RSS feeds for good sources w/ instant notifications (Webhooks / email alerts)

   • Consider allinfosecnews.com, and platforms like Inoreader, Newsblur, or Feedly.

2. Subscribe to reputable security orgs that provide regular newsletters and notifications

   • CISA, SANS, and RiskyBiz (my favourite)

3. Utilise social media monitoring tools

   • I’m told HootSuite is good, X Pro (formerly TweetDeck) may still be useful despite Twitter being mostly hot garbage.

In conclusion…

If that was too much snark and not enough concise helpful info, I will never apologise, but I will deliver:

1. Categorise assets, assess their criticality, and prioritise their importance. Create your own asset database if you have to.

2. Understand CVSS vectors well, and consider which vectors are the biggest risk to which assets.

3. Enrich your vulnerability data with free and open sources, like EPSS and CISA KEV.

4. Get buy-in from your key stakeholders and become the driving force in fixing culture around patching and security.

5. Implement good security feeds that notify you quickly so that you can respond to threats imminently.

This is most useful for vulnerability managers just starting out in this area to ensure they have a strong foundation, but hopefully everyone can take something from this. Once the foundation is set, there’s lots of directions to go to further increase maturity, efficiency, and overall security.