Build a Security Metrics Program to Drive Maturity

Good metrics come from good goals.

ANALYST PERSPECTIVE

Metrics are a maturity driver.

"Metrics programs tend to fall into two groups: non-existent and unhelpful.

The reason so many security professionals struggle to develop a meaningful metrics program is because they are unsure of what to measure or why.

The truth is, for metrics to be useful, they need to be tied to something you care about – a state you are trying to achieve. In other words, some kind of goal. Used this way, metrics act as the scoreboard, letting you know if you’re making progress towards your goals, and thus, boosting your overall maturity."

– Logan Rohde, Research Analyst, Security Practice Info-Tech Research Group

Executive summary

Situation

• Many security leaders put off adding metrics to their program because they don't know where to start or how to assess what is worth measuring.

Complication

• Sometimes, this uncertainty causes the belief that their security programs are not mature enough for metrics to be worthwhile.
• Because metrics can become very technical and precise, it's easy to think they're inherently complicated (not true).

Resolution

• A metric, really, is just a measure of success against a given goal. Gradually, programs will achieve their goals and set new, more specific goals, and with them comes more specific metrics.
• It is not necessary to jump into highly technical metrics right away. A lot can be gained from metrics that track behaviors.
• A metrics program can be very simple and still effectively demonstrate the value of security to the organization. The key is to link your metrics to the goals or objectives the security team is pursuing, even if they are simple implementation plans (e.g. percentage of departments that have received security training).

Info-Tech Insight

1. Metrics lead to maturity, not vice versa
   • Tracking metrics helps you assess progress and regress in your security program. This helps you quantify the maturity gains you’ve made and continue to make informed strategic decisions.
2. The best metrics are tied to goals
   • Tying your metrics to goals ensures that you are collecting metrics for a specific purpose rather than just to watch the numbers change.

Our understanding of the problem

This Research is Designed For:

• CISO

This Research Will Help You:

• Understand the value of metrics.
• Right-size a metrics program based on your organization’s maturity and risk profile.
• Tie metrics to goals to create meaningful KPIs.
• Develop strategies to effectively communicate the right metrics to stakeholders.

This Research Will Also Assist:

• CIO
• Security Manager
• Business Professionals

This Research Will Help Them:

• Become informed on the metrics that matter to them.
• Understand that investment in security is an investment in the business.
• Feel confident in the progress of the organization’s security strategy.

Info-Tech’s framework integrates several best practices to create a best-of-breed security framework

Information Security Framework

Governance

• Context and Leadership
  • Information Security Charter
  • Information Security Organizational Structure
  • Culture and Awareness
• Evaluation and Direction
  • Security Risk Management
  • Security Policies
  • Security Strategy and Communication
• Compliance, Audit, and Review
  • Security Compliance Management
  • External Security Audit
  • Internal Security Audit
  • Management Review of Security

Management

• Prevention
  • Identity Security
    • Identity and Access Management
  • Data Security
    • Hardware Asset Management
    • Data Security & Privacy
  • Infrastructure Security
    • Network Security
    • Endpoint Security
    • Malicious Code
    • Application Security
    • Vulnerability Management
    • Cryptography Management
    • Physical Security
    • Cloud Security
  • HR Security
    • HR Security
  • Change and Support
    • Configuration and Change Management
    • Vendor Management
• Detection
  • Security Threat Detection
  • Log and Event Management
• Response and Recovery
  • Security Incident Management
  • Information Security in BCM
  • Security eDiscovery and Forensics
  • Backup and Recovery
• Measurement
  • Metrics Program
  • Continuous Improvement

Metrics help to improve security-business alignment

While business leaders are now taking a greater interest in cybersecurity, alignment between the two groups still has room for improvement.

Key statistics show that just...

5% of public companies feel very confident that they are properly secured against a cyberattack.

41% of boards take on cybersecurity directly rather than allocating it to another body (e.g. audit committee).

19% of private companies do not discuss cybersecurity with the board.

(ISACA, 2018)

Info-Tech Insight

Metrics help to level the playing field

Poor alignment between security and the business often stems from difficulties with explaining how security objectives support business goals, which is ultimately a communication problem.

However, metrics help to facilitate these conversations, as long as the metrics are expressed in practical, relatable terms.

Security metrics benefit the business

Executives get just as much out of management metrics as the people running them.

1. Metrics assuage executives’ fears
   • Metrics help executives (and security leaders) feel more at ease with where the company is security-wise. Metrics help identify areas for improvement and gaps in the organization’s security posture that can be filled. A good metrics program will help identify deficiencies in most areas, even outside the security program, helping to identify what work needs to be done to reduce risk and increase the security posture of the organization.
2. Metrics answer executives’ questions
   • Numbers either help ease confusion or signify other areas for improvement. Offering quantifiable evidence, in a language that the business can understand, offers better understanding and insight into the information security program. Metrics also help educate on types of threats, staff needed for security, and budget needs to decrease risk based on management’s threat tolerance. Metrics help make an organization more transparent, prepared, and knowledgeable.
3. Metrics help to continually prove security’s worth
   • Traditionally, the security team has had to fight for a seat at the executive table, with little to no way to communicate with the business. However, the new trend is that the security team is now being invited before they have even asked to join. This trend allows the security team to better communicate on the organization’s security posture, describe threats and vulnerabilities, present a “plan of action,” and get a pulse on the organization’s risk tolerance.

Common myths make security metrics seem challenging

Security professionals have the perception that metrics programs are difficult to create. However, this attitude usually stems from one of the following myths. In reality, security metrics are much simpler than they seem at first, and they usually help resolve existing challenges rather than create new ones.

Tie your metrics to goals to make them worthwhile

SMART metrics are really SMART goals.

Specific

Measurable

Achievable

Realistic

Timebound

Achievable: What is an achievable metric?

When we say that a metric is “achievable,” we imply that it is tied to a goal of some kind – the thing we want to achieve.

How do we set a goal?

1. Determine what outcome you are trying to achieve.
   • This can be small or large (e.g. I want to determine what existing systems can provide metrics, or I want a 90% pass rate on our monthly phishing tests).
2. Decide what indicates that you’ve achieved your goal.
   • At what point would you be satisfied with the progress made on the initiative(s) you’re working on? What conditions would indicate victory for you and allow you to move on to another goal?
3. Develop a key performance indicator (KPI) to measure progress towards that goal.
   • Now that you’ve defined what you’re trying to achieve, find a way to indicate progress in relative or relational terms (e.g. percentage change from last quarter, percentage of implementation completed, ratio of programs in place to those still needing implementation).

Info-Tech’s security metrics methodology is repeatable and iterative to help boost maturity

Security Metric Lifecycle

Start:

Review current state and decide on priorities.

Set a SMART goal for improvement.

Develop an appropriate KPI.

Use KPI to monitor program improvement.

Present metrics to the board.

Revise metrics if necessary.

Metrics go hand in hand with your security strategy

A security strategy is ultimately a large goal-setting exercise. You begin by determining your current maturity and how mature you need to be across all areas of information security, i.e. completing a gap analysis.

As such, linking your metrics program to your security strategy is a great way to get your metrics program up and running – but it’s not the only way.

Check out the following Info-Tech resource to get started today:

Build an Information Security Strategy

The value of security metrics goes beyond simply increasing security

This blueprint applies to you whether you need to develop a metrics program from scratch or optimize and update your current strategy.

Value of engaging in security metrics:

• Increased visibility into your operations.
• Improved accountability.
• Better communication with executives as a result of having hard evidence of security performance.
• Improved security posture through better understanding of what is working and what isn’t within the security program.

Value of Info-Tech’s security metrics blueprint:

• Doesn’t overwhelm you and allows you to focus on determining the metrics you need to worry about now without pressuring you to do it all at once.
• Helps you develop a growth plan as your organization and metrics program mature, so you continue to optimize.
• Creates effective communication. Prepares you to present the metrics that truly matter to executives rather than confusing them with unnecessary data. Pay attention to metric accuracy and reproducibility. No management wants inconsistent reporting.

Impact

Short term: Streamline your program. Based on your organization’s specific requirements and risk profile, figure out which metrics are best for now while also planning for future metrics as your organization matures.

Long term: Once the program is in place, improvements will come with increased visibility into operations. Investments in security will be encouraged when more evidence is available to executives, contributing to overall improved security posture. Potential opportunities for eventual cost savings also exist as there is more informed security spending and fewer incidents.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

“Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful.”

Guided Implementation

“Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.”

Workshop

“We need to hit the ground running and get this project kicked-off immediately. Our team has the ability to take this over once we get a framework and strategy in place.”

Consulting

“Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.”

Diagnostics and consistent frameworks used throughout all four options

Link Security Metrics to Goals to Boost Maturity – Project Overview

Workshop overview

Contact your account representative or email Workshops@InfoTech.com for more information.

Phase 1

Link Security Metrics to Goals to Boost Maturity

Phase 1

1.1 Review current state and set your goals

1.2 Develop KPIs and prioritize your goals

1.3 Implement and monitor KPIs

This phase will walk you through the following activities:

• Current state assessment
• Setting SMART goals
• KPI development
• Goals prioritization
• KPI implementation

This phase involves the following participants:

• Security Team

Outcomes of this phase

• Goals-based KPIs
• Security Metrics Determination and Tracking Tool

Phase 1 outline

Complete these steps on your own or call us to complete a guided implementation. A guided implementation is a series of two to three advisory calls that help you execute each phase of a project. They are included in most advisory memberships.

Guided Implementation 1: Link Security Metrics to Goals to Boost Maturity

Proposed Time to Completion: 2-4 weeks

Step 1.1: Setting Goals

Start with an analyst kick-off call:

• Determine current and target maturity for various security programs.
• Develop SMART Goals.

Then complete these activities…

• CMMI Assessment

Step 1.2 – 1.3: KPI Development

Review findings with analyst:

• Prioritize goals
• Develop KPIs to track progress on goals
• Track associated metrics

Then complete these activities…

• KPI Development

With these tools & templates:

• KPI Development Worksheet
• Security Metrics Determination and Tracking Tool

Phase 1 Results & Insights:

• Basic Metrics program

1.1 Review current state and set your goals

120 minutes

Let’s put the security program under the microscope.

Before program improvement can take place, it is necessary to look at where things are at presently (in terms of maturity) and where we need to get them to.

In other words, we need to perform a security program gap analysis.

Info-Tech Best Practice

The most thorough way of performing this gap analysis is by completing Info-Tech’s Build an Information Security Strategy blueprint, as it will provide you with a prioritized list of initiatives to boost your security program maturity.

Completing an abbreviated gap analysis...

• Security Areas
• Network Security
• Endpoint Security
• Vulnerability Management
• Identity Access Management
• Incident Management
• Training & Awareness
• Compliance, Audit, & Review
• Risk Management
• Business Alignment & Governance
• Data Security

1. Using the CMMI scale on the next slide, assess your maturity level across the security areas to the left, giving your program a score from 1-5. Record your assessment on a whiteboard.
2. Zone in on your areas of greatest concern and choose 3 to 5 areas to prioritize for improvement.
3. Set a SMART goal for improvement, using the criteria on goals slides.

Use the CMMI scale to contextualize your current maturity

Use the Capability Maturity Model Integration (CMMI) scale below to help you understand your current level of maturity across the various areas of your security program.

1. Initial
   • Incident can be managed. Outcomes are unpredictable due to lack of a standard operating procedure.
2. Repeatable
   • Process in place, but not formally implemented or consistently applied. Outcomes improve but still lack predictability.
3. Defined
   • Process is formalized and consistently applied. Outcomes become more predictable, due to consistent handling procedure.
4. Managed
   • Process shows signs of maturity and can be tracked via metrics. Moving towards a predictive approach to incident management.
5. Optimizing
   • Process reaches a fully reliable level, though improvements still possible. Regularity allows for process to be automated.

(Adapted from the “CMMI Institute Maturity Model”)