Improve Incident and Problem Management

Rise above firefighter mode with structured incident management to enable effective problem management

EXECUTIVE BRIEF

Analyst Perspective

Keep it simple. Good data and consistent processes will help you break out of firefighter mode.

Incident management teams often find themselves too busy to create the knowledgebase (KB) articles or track the incident data that will save them time in the future. It becomes a vicious cycle that keeps them constantly in firefighter mode.

The key to breaking this cycle is to keep it simple as you seek to implement better structure and processes and right-size your approach. For example, avoid complex categorization schemes, and start with KB articles for known recurring incidents. Don’t jump to automation before you have the processes and resources to support it.

Similarly, when it comes to problem management, keep it simple by starting with Sev 1 tickets and recurring incidents that are obvious candidates for problem management. Support problem management with a consistent, structured approach that enables you to prioritize your limited resources.

As you build momentum with quick wins and better structure, improved incident management will drive more effective problem management and reduce future incidents as the incident-problem lifecycle comes full circle.

Frank Trovato

Research Director, Infrastructure and Operations

Info-Tech Research Group

Executive Summary

Your Challenge

Establish a consistent incident management process to better categorize, prioritize, and resolve incidents.

Enable faster resolution time through well-defined escalation protocols.

Prevent incidents from happening in the first place by identifying and resolving the underlying root cause via problem management.

Leverage event management to predict problems before potential incidents occur.

Common Obstacles

IT managers have conflicting accountabilities. It can be difficult to set aside time for preventing incidents (i.e., problem management) when staff are already busy resolving existing incidents and working on projects.

Resolving incidents quickly boosts confidence in IT, but recurring incidents erodes confidence, as does the need to use cumbersome workarounds.

Info-Tech’s Approach

Implement structured incident management to drive efficiency (e.g., effective use of categorization to drive appropriate ticket routing), and build out a knowledgebase to expedite future incident response.

On the problem management side, acknowledge that you have limited time for this, so start with obvious problems (e.g., recurring incidents) and then expand from there as problem management starts to reduce incident volume.

Info-Tech Insight

Effective problem management drives business value by preventing incidents, but it starts with good incident management that produces the data needed to identify problems that are driving recurring and related incidents. Specifically, logical categorization and resolution codes drive effective trend analysis to identify problems, and documenting troubleshooting, resolution details, and known errors provides a solid starting point for root cause analysis via problem management.

Common challenges to incident management success

Organizations that struggle with incident management (IM) are typically faced with these barriers:

Unresolved issues

• Tickets are not created for all incidents.
• Tickets are lost or escalated to the wrong technicians.
• Poor data impedes root-cause analysis of incidents.

Low productivity

• Lack of cross-training and knowledge sharing.
• Time is wasted troubleshooting recurring issues.
• Reports unavailable due to lack of data and poor categorization.

Poor planning

• Lack of data for effective trend analysis leads to poor demand planning.
• Lack of data leads to lost opportunities for templating and automation.

Expedite incident resolution with better data and focused documentation

*Category and resolution can also be updated at Closure if needed.

The Info-Tech difference:

• Start by analyzing your existing tickets. This translates theoretical goals and challenges into your reality.
• Identify specific issues that get in the way of better incident data and processes. Are there quick wins available?
• Define action items with a realistic time frame — short, medium, and long-term — to improve processes right-sized for your organization.

Leverage improved incident data to move from reactive to proactive mode with improved problem management

The Info-Tech difference:

• Problem management is a planned activity but with limited resources, so prioritizing your efforts is crucial.
• Start with known issues – critical incidents and recurring incidents identified through effective incident classification. This drives tangible business benefits that justify time spent on problem management.
• Not every problem has a viable or practical permanent resolution. A workaround that expedites resolving future occurrences of the incident can make more business sense, depending on the time and money needed for a permanent resolution.

STOP: Ensure you have foundational Service Desk operations before continuing

This blueprint will help you improve existing incident management processes, and then build on that foundation to implement or improve problem management.

If you need more foundational improvements to your Service Desk operations, we recommend starting with the blueprint Standardize the Service Desk.

Specifically, the following are pre-requisites for this blueprint:

• There is a formal process for submitting tickets or reporting issues (e.g., Service Desk email address, phone number, or portal).
• Ticket intake process separates incidents from service requests.
• Tier 1 roles are defined that manage ticket intake and provide first-call resolutions for low-complexity issues (e.g., forgot my password).

Info-Tech’s methodology to improve incident and problem management

Insight summary

Shift-left starts with a good knowledge base

A good knowledge base expedites incident resolution and supports “shift-left” (e.g., enabling Tier 1 to solve incidents that would otherwise escalate to Tier 2 or 3).

Every incident is potentially an opportunity to document a solution, troubleshoot steps, or establish relevant operational documentation needed solve the incident.

If you capture this information only in the ticket or your own personal repository, you limit the ability to shift left and expedite future incident resolution.

Don’t reinvent your processes because of a critical incident

All hands on deck doesn’t mean abandoning processes. Instead, supplement your existing incident management processes to maintain structure to your response. For example:

• Alert senior IT leads in case they’re needed but follow existing processes to triage and assign the incident to the right SMEs.
• Notify affected users as usual but add appropriate updates to senior leadership.
• Leverage existing incident response collaboration methods – e.g., use the same MS Teams channel you normally use to collaborate on incidents; if necessary, set up a separate channel for leadership updates.

Apply structure to problem management to find value

Time must be allocated to problem management to get the long-term benefits. It’s not going to be driven by the urgency of an outage, but rather the foresight to predict and prevent future incidents.

Effective problem management follows a structured process to get the most out of the time allocated to this proactive effort. This includes appropriate prioritization, a root cause analysis methodology, and a decision point on whether to adopt a workaround or continue to pursue a permanent solution.

If problem management is ad-hoc or “when I have time,” something else will always take precedence.

Blueprint deliverables

Each step of this blueprint is accompanied by supporting deliverables to help you accomplish your goals:

Incident Knowledgebase Article Examples

Use the examples as a guide for your KB article templates.

Incident, Critical Incident, and Problem Workflows

Workflows are critical to communication process expectations and driving consistent execution.

Incident Status Updates and Incident Report Templates

Modify our examples to suit your requirements.

Incident and Problem Management Project Roadmap

Identify, prioritize, and present initiatives to improve incident and problem management.

Key deliverable:

SOPs for Incident Management and Problem Management

Clarify process and role expectations to improve consistency, efficiency, and effectiveness.

Blueprint benefits

IT Benefits

• Documented incident management processes clarify expectations for Tier 1, 2, and 3 roles and drive consistent process execution.
• Capturing good incident data makes it easier to identify and resolve problems.
• Similarly, promoting knowledgebase development as part of your core process (e.g., identifying KB opportunities as part of resolving a ticket) not only expedites future incident resolution but also provides input to problem management to resolve the underlying root cause.

Business Benefits

• Quicker incident resolution through better process (e.g., routing tickets to the correct SMEs) and leveraging KB articles.
• Preventing recurring incidents by resolving the root cause.
• Predicting and preventing future incidents through proactive problem management.

Info-Tech offers various levels of support to best suit your needs

DIY Toolkit

“Our team has already made this critical project a priority, and we have the time and capability, but some guidance along the way would be helpful.”

Guided Implementation

“Our team knows that we need to fix a process, but we need assistance to determine where to focus. Some check-ins along the way would help keep us on track.”

Workshop

“We need to hit the ground running and get this project kicked off immediately. Our team has the ability to take this over once we get a framework and strategy in place.”

Consulting

“Our team does not have the time or the knowledge to take this project on. We need assistance through the entirety of this project.”

Diagnostics and consistent frameworks used throughout all four options

Guided Implementation

What does a typical GI on this topic look like?

A Guided Implementation (GI) is a series

of calls with an Info-Tech analyst to help implement our best practices in your organization.

A typical GI is between eight and 12 calls over the course of four to six months.

Phase 1: Optimize Ticket Intake and Routing

Phase 1

Optimize Ticket Intake and Routing

Phase 2

Standardize and Streamline Incident Response

Phase 3

Establish Effective Problem Management

Phase 4

Implement Improvements

This phase will walk you through the following steps:

1. Review the incident lifecycle and your current challenges.
2. Improve how you identify, log, and categorize incidents.
3. Define a ticket prioritization scheme.
4. Drive more efficient ticket intake.

Improve Incident and Problem Management

Step 1.1

Review the incident lifecycle and your current challenges

Activities

1.1.1 Identify challenges with your existing incident management processes

This step will guide you through the following content and activities:

• Establish a common understanding of the incident lifecycle.
• Identify challenges with your existing incident management processes.

This step involves the following participants:

• Incident Management Team (e.g., Tier 1 and 2 roles and the Incident Manager)
• Tier 3 reps from relevant IT teams (e.g., network, servers, security, apps, etc.)

Outcomes of this step

• Level-setting across the team regarding incident lifecycle stages, based on ITIL.
• High-level challenges identified with your existing incident management processes.

Blueprint pre-step: Gather your data to relate this blueprint to your reality

Before you begin this project, gather the data from your existing ticketing system.

You will use this data as you work through this blueprint to help you make decisions on what the target state of your incident management program looks like.

You will need:

1. An export of your existing ticket categorization scheme; raw data is better so you can easily manipulate the data as you are analyzing it.
2. Each person on the project team will pull ten tickets from each priority level you are currently using (10 sev1, 10 sev2, 10 sev3).
3. A snapshot of the incident ticket interface to be able to quickly reference existing fields and functionality.

This image will help remind you to search through your own ticket data to help guide your decisions during the design phase of incident management.

Establish a common understanding of the incident lifecycle

1) Detection: User reporting an issue, event triggering an alert, and so on. Conduct initial triage/discovery. Confirm it’s an incident (for service requests, follow a separate process).

2) Registration: Create/update the ticket based on initial triage (e.g., incident details) or monitoring system that generated the alert (e.g., relevant system).

3) Classification: Categorize, prioritize, and conduct initial investigation (e.g., check KB for known errors). Escalate or re-assign if necessary.

4) Diagnosis: Additional investigation if solution not already identified. Peer discussion, check KB, and/or consult vendor. Escalate or re-assign if necessary.

5) Resolution: Apply solution (permanent fix or workaround) to restore service. If applicable, submit a change request to move the fix into production.

6) Closure: Finalize ticket details, including status (Closed). Provide final update to affected users. Identify if a KB is needed to expedite future troubleshooting or incident resolution.

Note: Ideally, steps 1 to 3 are executed by Tier 1 staff so that Tier 2 and 3 are included only when an issue needs to be escalated. This drives lower-cost resolution and frees time for Tier 2 and 3 to focus on project work, more-complex incidents, and problem management. Ticket updates occur throughout and are finalized as needed at Closure.

1.1.1 Identify challenges with your existing incident management processes

1-3 hours

Materials

• Whiteboard or flip chart

Participants

• Incident Management Team (e.g., Tier 1 and 2 roles and the Incident Manager)
• Tier 3 reps from relevant IT teams (e.g., network, servers, security, apps, etc.)

1. As a group, outline the challenges or weaknesses you have in each step of the incident lifecycle. Separate the challenges into people, process, and technology for a wholistic view.
2. Record those challenges for reference purposes. Phase 4 will include creating a project roadmap to address gaps and improve processes.
   • Below are examples of common challenges to consider:
     • Are incidents resolved at the appropriate Tier? Are Tier 2 and 3 resolving incidents that could be solved at a lower Tier?
     • Any challenges with identifying ticket type, category, or severity level? Is it clear where to route tickets (e.g., based on category)?
     • Is it clear when to escalate tickets? Is Tier 1 gathering enough information before escalating?
     • Is ticket data updated appropriately by Tier 2 or 3 staff?
     • Is there appropriate documentation available to support ticket troubleshooting (e.g., system information, relevant KB articles, etc.)?
     • Any common complaints from users or executives (e.g., slow response, ticket status is unclear)?

Step 1.2

Improve how you identify, log, and categorize incidents

Activities

1.2.1 Review and update your categorization scheme

1.2.2 Define resolution codes to further improve reporting

This step will guide you through the following content and activities:

• Separate incidents from service requests
• Understand categorization best practices
• Review/update categorization scheme
• Define resolution codes to further improve reporting

This step involves the following participants:

• Incident Management Team (e.g., Tier 1 and 2 roles and the Incident Manager)
• Tier 3 reps from relevant IT teams (e.g., network, servers, security, apps, etc.)

Outcomes of this step

• Improved ticket categorization scheme and resolution codes.
• Knowledge sharing between teams regarding proper use of categories and resolution codes.