![ApptioCareers-Employee[1]](data:image/svg+xml;base64,PHN2ZyB4bWxucz0iaHR0cDovL3d3dy53My5vcmcvMjAwMC9zdmciIHZpZXdCb3g9IjAgMCAyMjUgMjc1IiB3aWR0aD0iMjI1IiBoZWlnaHQ9IjI3NSIgZGF0YS11PSJodHRwcyUzQSUyRiUyRnd3dy5hcHB0aW8uY29tJTJGd3AtY29udGVudCUyRnVwbG9hZHMlMkZlbGVtZW50b3IlMkZ0aHVtYnMlMkZBcHB0aW9DYXJlZXJzLUVtcGxveWVlMS1xZzVuenB5MHdoaWJtbzZvdmJqNGM3MHh0aTUzdmo1OHM0endwajh0dHMucG5nIiBkYXRhLXc9IjIyNSIgZGF0YS1oPSIyNzUiIGRhdGEtYmlwPSIiPjwvc3ZnPg== "ApptioCareers-Employee[1]")
IBM ApptioWhat are IT Metrics?
IT metrics are quantifiable measurements that help IT leaders efficiently manage the business of IT. Traditionally operational, today’s IT metrics also help align IT investment to business strategy, customer experience, and cloud optimization. IT metrics help CIOs determine the value of technology and build confidence in IT performance.
CIOs increase the impact and strategic role of IT by taking charge of the IT value conversation. They use performance measures to focus decision, improve relationships and find new sources of value from IT. Metrics complement relationship building, helping IT leaders evolve into strategic partnership with business executives.
Defining, measuring, and reviewing IT metrics helps IT leaders build a foundation for conversations about the value technology provides to the business. To help drive these conversations, the TBM Council—a nonprofit professional organization dedicated to helping CIOs manage the business of IT—has established a set of metrics for evaluating IT investments, performance, and delivery.
IT Metrics vs. IT KPIs
A KPI (shorthand for “key performance indicator”) is a metric used to evaluate whether or not an organization is meeting its objectives. By definition, not all metrics can be “key” so KPIs are a select grouping of metrics deemed essential to meeting business objectives. These indicators help teams focus on areas that generate the most value and have the greatest impact on business outcomes.
Within IT, KPIs are very effective for answering the following questions: are we making investments in the right places, are we getting the results we expect to see, and are our customers satisfied with the value they’ve received?
Lagging and leading KPIs
KPIs are commonly described as lagging and leading. A lagging indicator looks backwards, measuring performance after the fact. Lagging KPIs capture what’s been done in the past and provide context for improvements.
In contrast, leading indicators look forward, anticipating changes and highlighting trends as they begin to take shape. Leading KPIs enable teams to quickly address challenges as they arise, before they negatively impact business outcomes.
The difference between a KPI and a metric can be narrow, but think about it this way: a metric might track how many employees were hired against a specific goal. A KPI would measure how that gain in employees is resolving problems, boosting sales, or driving innovation.
IT metrics support KPIs by tracking cost, performance, and output for IT processes. To be effective, they are often compared against established benchmarks, which provide context for the KPI’s value. Used this way, IT metrics help determine gaps between current and desired performance, track progress over time, and demonstrate how process improvements impact performance.
Why Are IT Metrics Important?
There’s an old saying that you can’t manage what you can’t measure. That may be true, but it’s also true that there are spreadsheets full of dollar signs, percentages, and bolded numbers—“measurements”—that don’t necessarily make management easy. Measurements are only as good as the story they tell.
Good IT metrics establish a fact-based methodology for measuring your progress toward business goals, leveraging data to tell a story. It’s easy to capture IT’s annual spend, but comparing that data point to IT spend year over year makes the data meaningful. The first is simply a measurement. The second is a metric that tells a story.
When IT metrics are relevant, they guide decision making that helps companies achieve strategic goals, focusing the IT team around true drivers of business performance. This has never been more important, as IT teams are asked to balance increasing business demand with static or decreasing budgets. IT metrics support discipline and objectivity for measuring technology’s impact on both.
Relevant and impactful metrics will:
- focus employees around company priorities
- communicate data in business terms
- improve decision-making
- drive performance
- evolve as the company matures
Who Uses IT Metrics and KPIs?
IT leaders in a variety of roles and responsibilities have much to gain from continuous review of IT metrics and KPIs. Here are a few examples of how IT metrics can be used to manage cost, performance, innovation, and business value:
- Office of the CIO
- IT Finance
- Infrastructure & Operations
- Applications & Services
- Business Relationship Manager
Is IT investing the right amount of spend to support growth and innovation in the business? To answer this question, the CIO might follow a metric that calculates and tracks the % of total IT spend attributed to activities that run the business in its current state, projects that support growth in the current business, and initiatives that transform the business with new products or new markets. They might also look at the % of discretionary spend going to these same categories.
Does IT have the financial agility needed to keep up with the changes to the business demand, strategy, or markets? An IT Finance Analyst will look at the variable spend against total IT spend to answer this question. This will help them understand where the balance of fixed assets and variable resources can be adjusted to better align costs with business demand. Is the IT cost structure heavily fixed? Where can IT leverage vendor resources & outsourced services, including the cloud, in a variable, on-demand manner?
Where can I trim costs and re-invest the savings to pay off technical debt or invest in new capabilities requested by the business? To identify candidates for optimization, an I&O leader might consistently review volumes and total costs associated with data center, compute, storage, and network capacity. By dividing total costs by volume, unit costs can be assessed for efficiency and contrasted with benchmarks or alternative suppliers.
Does IT investment & prioritization match where the business is headed? Where do we have duplication or under-use, and which applications should we retire? One way to answer these questions is to create a portfolio investment comparison across application run and build costs. This metric could look at the total costs of infrastructure applications and service, business capabilities, and/or external customer-facing services, including everything from hardware & software to labor & outside services. It could further divide the spend by volume to calculate unit costs that reveal the relative efficiencies among multiple alternatives.
How can we help the business understand the connection between their consumption behavior and the cost of IT in order to reduce those costs year over year? A business relationship manager can look at IT spend for each business unit, broken out by service, to answer this question. This metric would reveal the volume consumed and associated unit costs.
How Do I Track IT Metrics?
Data is central to tracking IT metrics and is the backbone of any program. Great metrics require identifying the right sources, integrating appropriate source data columns, and aligning formats.
What kind of data is needed?
Many IT teams start with 3–5 data sources from various disciplines in the organization, including:
- Finance: general ledger, fixed asset register, chart of accounts, budget
- Technology: servers, network, cloud provider, storage, CMDB
- Portfolios: applications, projects, labor, vendors
- Service: tickets, service catalog
How should costs be categorized?
Taxonomy helps align corporate data with IT costs. For this reason, the TBM Council has endorsed the industry standard taxonomy used in the Apptio TBM Unified Model™ (ATUM™). ATUM standardizes a cost model IT leaders can use to manage their technology business.
ATUM’s taxonomy aligns costs to common terminology, automates categorization via mapping rules, and customizes or extends categories where needed. It also specifies a standard model for routing costs with pre-defined allocation rules—apportioning costs to consuming elements and weighting them with operational data.
Once costs have been calculated from various data sources, organized into standard categories, and mapped to metrics, dashboards allow CIOs to measure and manage them at a glance. Dashboards give IT and business leaders an efficient, automated way to capture and analyze metrics central to managing cost and performance on a weekly, or even daily, basis.
Ideally, dashboards are designed to evolve as circumstances change. An example of evolving metrics might be this:
Let’s say in the beginning, it’s enough for IT to simply have a snapshot of the % of projects that start on time. This metric is built on a collection of very basic “yes or no” data points. A logical next step is to understand the % of those projects completed on time. This, in turn, leads to a metric that demonstrates the % of projects that not only completed on time, but also completed on budget and met the expected delivery scope.
This team needs a dashboard that provides a centralized view of the metric and enables customization that mirrors the evolution of the objective.
Apptio’s CIO dashboard—a feature available within IBM Apptio Costing—uses a standard set of third-party metrics ratified by the TBM Council and vetted by leading enterprise CIOs and industry analysts. The view provides an executive level summary of key cost metrics by business units, applications, infrastructure, financial variance, and public cloud.
When Are IT Metrics Used?
IT metrics can only do their job when IT decision makers pay attention to them. Establishing a regular cadence of IT metrics reviews for different audiences and aspects of IT performance is important. This helps the team derive relevant insights and take appropriate action in a timely manner.
Monthly leadership reviews
There’s a direct benefit to CIOs gathering their direct reports on a monthly basis and reviewing the performance of the organization. This internal-facing update keeps everyone on the same page. The agenda may include an overview of what’s happened in IT over the course of the month, how various KPIs are trending, and whether or not corrective actions are needed. It may also include a summary of variances between budgeted and actual costs, and a look ahead at the impact on forecasted spend. (At Apptio, we’ve developed a best practice approach to this meeting called the “IT Leadership TBM Review” or ITLTR.)
Quarterly business reviews
On a quarterly basis, CIOs and Business Relationship Managers should conduct reviews with the business unit partners who consume services from IT. This outward-facing conversation promotes alignment between teams. The agenda may focus on the value IT is delivering to the business, how the business’s consumption choices influence their IT costs, and where IT investments might need adjustment to align to future business priorities. These reviews are crucial for providing the transparency that builds trust with partners and encourages them to view IT as a preferred partner for technology services.
Annual planning process
During the latter half of each fiscal year, most businesses execute a strategic planning process to identify objectives, strategies, tactics, and investments for the following year. IT metrics are crucial to this process because they help IT decision-makers understand how their plans for the current year match up to reality and they provide a basis against which to plan adjustments for the new year. Armed with this knowledge, IT leaders can weigh strategic options and make wise investment decisions with confidence.
What IT Metrics and KPIs Should I Be Tracking?
There are thousands of metrics that shed light on IT cost, performance, and output. But more is not better. In fact, more can be overwhelming and unproductive. Instead, a top-down approach to metric development ensures IT leaders focus only on the data that informs key business decisions. Zeroing in on the essentials will ensure IT teams are better positioned to understand and communicate impacts on specific outcomes. When Apptio surveyed CIOs and asked them to prioritize an optimal set of metrics for running IT as a business, the following categories emerged.
Fundamental Financial Metrics
Financial metrics help IT leaders steward technology expenditures and investments and are critical to managing the business of IT. These metrics help manage the financial health of the department and may reveal levers for cost reduction, better resource allocation, and increased accountability.
- 1. IT Spend vs. Plan
- 2. Application & Service Total Cost
- 3. %IT Spend on Cloud
Metric 1: IT Spend vs. Plan (for both OpEx & CapEx budgets)
This metric answers the questions:
- Are you spending what you expected to spend?
- Which areas of spend are over/under plan?
How to use it:
- Detect and correct anomalies before they become crises
- Drive a culture of accountability within your IT organization
Cadence:
- Monthly + quarterly
Example:
Looking at IT spend vs. plan is important for managing adherence to the budget throughout the year and addressing anomalies before they are no longer recoverable.
For example: Company A finds outside consultants are taking longer than expected to complete a project—consuming more budget than allocated. With this metric and a monthly review in place, the CIO and the IT Finance team are able to detect the variance and identify the cause, enabling them to address the issue before it gets out of hand. Furthermore, as this IT team looks forward in the year, they’re able to use this metric to analyze scenarios for delaying or reprioritizing other initiatives or projects to compensate and stay within the annual budget.
Metric 2: Application and Service Total Cost
This metric answers the questions:
- What does each IT offering really cost to deliver?
- How is spend distributed across the app & service portfolio?
How to use it:
- Uncover the “long tail” of application run costs
- Align IT spending with business priorities
Cadence:
- Monthly + quarterly
Example:
The focus of this TCO metric should be on accurately reflecting the total costs of an application or service, including hidden costs such as indirect labor and soft costs (like training, travel, etc.). Correctly accumulating all of these costs enables accurate comparison to both other applications in the portfolio and to the value achieved from that spend.
Let’s look at Company B, which has grown through acquisition and has hundreds or thousands of applications in use. There’s a misperception throughout the organization that because some applications were developed years ago, they are “free.” But surprising results emerge when these apps are viewed through the lens of the TCO metric. Some of them have unallocated support costs, while others overlap with other applications performing the same function, resulting in duplicate spend. Once the business understands true costs, it’s easier to retire expensive applications.
Metric 3: %IT Spend on Cloud
This metric answers the questions:
- Are we meeting our cloud-first goals?
- Is an increase in cloud spend matched with a cut in on-premises IT
How to use it:
- Prioritize cloud migration
- Validate quick migration wins to validate the business case for cloud-first strategy
Cadence:
- Monthly + quarterly
Example:
Company A wants to validate its cloud-first strategy. It has a significant on-premises IT footprint. Its many legacy applications must be refactored, re-platformed, or re-purchased. IT operations are prioritizing and costing out the development work. Still, technology leaders must communicate cloud adoption success through financial metrics understandable to the C-suite and other business leaders.
By tracking %IT spend on cloud, technology leaders identify infrastructure and applications proving to be resistant to migration or retirement over time.
Delivery Metrics
Delivery metrics track the effectiveness of project execution and the ongoing delivery of business-facing services. These metrics may include satisfaction surveys, budget and completion measurements, and data that influences labor and resourcing strategies.
- 4. Product Lead Time
- 5. Business Value Delivered by Portfolio per Quarter
Metric 4: Product Lead Time
This metric answers the questions:
- How long does it take from an initial request to the delivery of a specific product or feature?
How to use it:
- Predict delivered capabilities (cost-saving or revenue-producing)
- Set and manage expectations for those who may be waiting for the proposed value to be delivered
Cadence:
- Program increments (PI)
Tip:
Organizations adopt Agile to increase speed to market, meet customer demand, and improve productivity, but adopting Agile alone does not deliver these outcomes. The methodology must be twinned with metrics to measure success. Agile metrics monitor productivity across the software development lifecycle, but product portfolios are ultimately judged for the time between the request for delivering a product and actual delivery. An IT strategic plan must include ongoing reviews of product lead times.
Metric 5: Business Value Delivered by Portfolio per Quarter
This metric answers the questions:
- How accurate are our estimates of delivered value?
- Which resources should we reallocate to provide better estimates?
How to use it:
- Build trust between product managers and the business with accurate estimates of delivered value.
- Identify & correct value slippage before it gets out of hand
Cadence:
- Monthly + quarterly
Tip:
Agile teams cannot meet business expectations with inaccurate estimates of delivered business value. Product managers require a trend-line of business value delivered by portfolio per quarter to improve estimation accuracy and flag resource constraints if business needs outstrip a team’s productive hours. Measuring and monitoring business value output enables product managers to meet delivery commitments. An IT strategic plan is at risk when leadership cannot review business value delivered by portfolio per quarter.
Innovation & Agility Metrics
Innovation and agility metrics focus on prioritizing and driving investments to change or transform the business. The data behind these metrics enable portfolio views of IT costs, project investments focused on changing the business, and technology investments in emerging capabilities like cloud.
- 6. % of IT Investment on Run, Grow, & Transform
- 7. % Project Spend on Customer Facing Initiatives
Metric 6: % of IT Investment on Run, Grow, Transform the Business
This metric answers the questions:
- How are we prioritizing our IT budget on run vs grow initiatives?
- Are we investing enough in technology innovation?
How to use it:
- Shift spend from Run to Grow and Transform investments
- Align IT spend with strategic business priorities
Cadence:
- Monthly + quarterly
Example:
This metric can be eye-opening for executives because it brings the balance of IT spend to light. Imagine three distinctly different companies. At Company A, IT doesn’t play a significant strategic role. Approximately 85% of the IT budget “keeps the lights on” and only 15% is invested to help the organization grow. At Company B, 60% of the budget is spent on running the business and 40% is spent on initiatives designed to grow the business. Finally, Company C spends 50% of the IT budget on running the business, 30% on growing the business, and another 20% on transforming the business.
The % of IT Investment metric helps each company understand how they are investing against business objectives and provides a benchmark they can use to shift the focus, as needed.
Metric 7: % of Project Spend on Customer-Facing Initiatives
This metric answers the questions:
- Are we investing enough on projects that impact the business?
- Are we prioritizing the customer experience high enough?
How to use it:
- Ensure that IT makes a relevant impact on external customers
- Detect & curtail internal-facing “science projects”
Cadence:
- Monthly + quarterly
Example:
At Company X, “make it easier for them to do business with us” is a new refrain coming from the board and executive suites. As a result, the CEO has asked the business to figure out applications and technologies that lower barriers for the organization’s customers.
IT leaders can use an analytic approach to demonstrate project focus and highlight gaps in spending. Metrics like this one highlight where technology is being applied to improve the customer experience and where it is meeting business goals. This data helps CIOs manage priorities, balance back-office vs. customer-facing spend, and report results back to the business.
Business Value Metrics
Business value metrics help CIOs demonstrate how technology investments are (or aren’t) impacting business outcomes. These metrics analyze value by tracking the impact of incremental IT investments against business capabilities, business objectives, and revenue.
- 8. IT Spend by Business Unit
- 9. Customer Satisfaction Scores
- 10. % of IT Investment by Business Initiative
Metric 8: IT Spend by Business Unit
This metric answers the questions:
- How does BU consumption impact IT costs?
- What is the relative IT cost of each BU’s consumption?
How to use it:
- Shape demand by showing BUs how consumption drives costs
- Identify outliers to uncover over/under spend
Cadence:
- Monthly + quarterly
CIO Reflection:
Transparency around business demand for IT changes the way the business thinks about IT. Whether you distribute control of IT spend among business units or centralize all IT spend under the CIO, metrics that break IT spend down by business unit can be enlightening. That’s because this metric is often a starting point for productive discussions with business units about value, demand, and the alignment of IT investments. When business partners realize they are accountable for their consumption, the impact can be significant.
Says Eileen Baines, CIO at CoBank, “It’s difficult to be business-aligned if you’re not able to show how IT spend and IT costs are aligning with business priorities. My team and I wouldn’t have been able to do that without the transparency that Apptio provides.”
Metric 9: Customer Satisfaction Scores for Business-Facing Services
This metric answers the questions:
- Is IT serving business units well enough?
- What is the perception of IT among internal customers?
How to use it:
- Identify & correct problems with perception of IT by the business
- Drive a culture of continuous quality improvement in IT
Cadence:
- Annually
Tip:
Customer satisfaction scores can be collected in a variety of ways: interviews, surveys, email, and feedback buttons are all tried and tested methods for understanding customer satisfaction. According to Forrester, “…the simplest and most customer-focused way to collect this data is the short form: a feedback email sent immediately after request fulfillment or a feedback button connected through the Internet and mobile apps. Then, once or twice a year, a CSAT survey based on 15 to 30 questions can provide more detailed information.”
Analysts also suggest increasing the level of management engagement and defining respondent incentives to ensure maximum response rates.
Metric 10: % of IT Investment by Business Initiative
This metric answers the questions:
- Do IT projects line up with business priorities?
- Are we investing enough in our strategic initiatives?
How to use it:
- Align IT spend with business strategy
- Identify projects that do not contribute to business objectives
Cadence:
- Annually
Example:
Business initiatives require technology investment. If Company A is focused on improving patient outcomes, building mobile capabilities, and expanding geographically, this metric helps IT leaders ensure IT is doing its part to support these initiatives by spending on the right things. Use this metric to clearly articulate alignment with business strategy and demonstrate the investment necessary to accomplish business goals.
Biggest Challenges With Tracking IT Metrics
Bad data
“My data isn’t ready” is a common excuse for not embracing IT metrics or KPIs. Validity, integrity, consistency, relevancy—these are all concerns when setting up IT metrics that will drive analysis and decision-making. But just as the cost of bad data can be daunting, the savings and value realized from having even just a few accurate metrics in place can be a reason to get started.
Data will never improve in a vacuum. Just like muscles need exercise to grow strong, data must be put to use in order for it to improve. So, don’t wait for your data to be perfect. Instead, use your data to make it perfect.
Misaligned terminology
Getting IT, finance, and the business on the same page with a common taxonomy of IT functions is crucial to creating IT metrics that tie back to business goals. One of the biggest challenges today is that IT financial metrics are often coming from the general ledger to the CIO. To drive accountability, it’s important that the IT team is able to translate cost data into language both IT and the business can understand and use to motivate better performance. Ideally, IT is using this data to make better decisions about investments and working with the business to communicate value more effectively.
Fear of transparency
IT leaders often know that they have some “skeletons in the closet”—inefficient areas requiring better stewardship—that they are reluctant to bring to light. The reality of the situation is generally two-fold. First, the situation is not as bad as they think. IT leaders make reasonable decisions on sparse data every day. Second, there are skeletons in everyone’s IT closet. Identifying weaknesses and correcting them is good for business.
Analysis lag time
Data freshness itself can be an important metric. If the IT team is loading the GL monthly (or more frequently) and looking at their costs, the CIO is far more likely to catch problems before they escalate. Unfortunately, what often happens is that metrics are assessed when the reconciliation with finance occurs, when it’s already too late to address issues that develop quickly.
In today’s cloud-enabled world, this is a critical issue. Cloud spend can get out of hand in a matter of days. If you’re waiting a whole quarter for finance to reconcile cost, there’s a high likelihood of surprise. CIOs may find themselves wondering, “Why didn’t somebody tell me I budgeted $10,000 but ended up spending $200,000 on the cloud this quarter?”
Labor/time intensive
It takes labor and time to set up good metrics, especially if you don’t have access to software built specifically for calculating accurate metrics about IT cost and value. Because most IT leaders struggle to calculate and report the right IT metrics using general purpose tools like spreadsheets, corporate finance systems, and business intelligence systems, they find it requires costly labor and time to get it even close to right. Often, the result is a brittle model that is not nimble enough to keep up with requests for new analyses or changes to the business.
Apptio can help. Learn more about our SaaS-based analytics and data applications that enable IT leaders to analyze, optimize, plan, and benchmark their technology investments.
Setting IT Strategic Planning KPIs
Empower your tech team to make informed decisions about IT spending by shifting to a process that enables the strategic plan and delivers a significant competitive advantage.
IT strategic plans rely on key performance indicators (KPIs) that provide the flexibility to support the business strategy, show financial basics, innovation, and delivery. Having metrics in place allows you to measure the success of your IT strategic plan.
