The Complex Costs of AI: Investments, Funding, and ROI Tracking

Eugene Khvostov, our Chief Product Officer, oversees the full suite of Apptio’s products and leads teams in product management, user experience/design, and product marketing. With over a decade of experience in end-to-end technical product development across industry verticals, Eugene takes a strategic, customer-first approach to product vision, roadmap decisions, and new product investments.

Artificial intelligence (AI) has taken the world by storm, dominating headlines with advances in large language models (LLMs), generative AI (genAI), and soaring GPU sales. AI is a growing phenomenon with fantastic potential for businesses, presenting myriad ways to optimize, enhance, and improve workstreams, from creative endeavors to operational efficiencies.

In the realm of technology cost management, there’s significant opportunity. GenAI is a boon for skill development and enhancing emerging business practice areas, such as IT financial management (ITFM) and FinOps. As a force multiplier, AI can not only automate time-consuming, manual processes, but it can also accelerate time-to-insight by acting as an interface for the vast volumes of data generated by enterprises every day. Imagine a world where deriving insights from enormous datasets didn’t require technical knowledge, but that natural language could be used instead.

At Apptio, we’re enthusiastic about AI. For our latest release we’ve built an AI-powered assistant on IBM’s watsonx platform that provides a way for ITFM and FinOps practitioners to quickly access domain expertise on those practice areas and access self-service support 24/7. AI agents like this are an incredibly effective application of genAI, helping to streamline user experience, accelerate problem resolution, and make products easier to use.

With all the potential use cases for AI, the market is at a fever pitch—and the dollar signs prove it. Capital Group analysts estimate capital expenditures totaling $189 billon for 2024, and Goldman Sachs projects that $1 trillion dollars will get poured into AI initiatives over the coming years. For now, investors have had a strong appetite for AI’s potential, but the money can’t flow forever without producing results. There are valid and growing concerns about the timeline for seeing a return on these unprecedented investments. For this reason, smart companies are buckling down on the financials behind their AI endeavors.

The Resource-Intensive Nature of AI

AI isn’t just algorithms comprised of thousands of lines of code—there are numerous resources needed to power and fuel this emerging technology. Here’s where the bulk of the costs start to add up:

• Data: AI—particularly generative models—requires vast datasets for training, which incur significant costs for cleansing, labeling, and storage. With global data creation pegged at 180 zettabytes by 2025, managing the growing volume of data will be a significant expense.
• Compute power: Cutting edge AI requires substantial hardware resources, driving up costs for powerful GPUs, high-performance computing (HPC) systems, and scalable cloud infrastructure. While NVIDIA and cloud service providers have benefited from this demand, the costs for customers add up quickly.
• Cloud services: Many companies don’t have the resources or capabilities to rely solely on on-prem computing infrastructure, so they leverage cloud providers for their flexibility, scalability, and economies of scale. However, with the vast compute and storage resources required by AI, expenses can quickly soar—especially during resource-intensive model training periods.
• Skilled labor resources: The competition for top AI talent has driven salaries higher, particularly for data scientists, engineers, and AI infrastructure experts. These roles are critical for development, but AI also requires specialized labor for governance, security, and ongoing maintenance, too. Moreover, as an emerging field, AI requires continuous learning and upskilling, further adding to costs.
• Energy: AI consumes a great deal of energy, particularly during model training. In fact, ChatGPT queries require close to 10 times as much electricity as a search on Google—and Goldman Sachs Research estimates a 160% growth in AI power demands by 2030. With rising power demands, businesses are under increasing scrutiny for their sustainability efforts as AI projects continue to consume more energy.

The above are just a handful of the resources required to run AI initiatives. There are also massive costs attached to cyber-security, regulatory compliance, licensing, intellectual property (IP), and legal costs.

Where is the money coming from?

While companies are spending billions to ramp up AI initiatives, they are under pressure to not just find, but also to justify funding, ensuring they don’t compromise existing operations or derail other critical projects. Unless companies seek funding from external financing or venture capital (which, of course, will have to be paid back—often amplifying ROI timeline pressure), companies need to carve money from existing program budgets. This requires a thoughtful consideration of tradeoffs. How do you strategically cut from other areas of technology investment without negatively impacting the business?

To allow room for these innovation initiatives to take flight, savvy companies are doing a deep dive on cost efficiency across the enterprise. Many of the initial AI applications are centered on driving efficiency and cost savings, enabling businesses to reduce operational spend. By focusing on AI’s potential to streamline run-the-business expenses, companies can generate savings that can then be reinvested into grow-the-business investments, like AI.

The Coming Reckoning: Expectations of Timelines for AI ROI

While it’s undeniable that AI is poised to revolutionize businesses and deliver meaningful value, there are still two significant variables causing investors and boardrooms significant concern: how and when?

The market expressed evidence of this wariness a few months back when Google’s parent company Alphabet had an unexpected stock price dip—even after beating earnings. This was attributed to concerns around the dramatic surge in infrastructure spending related to AI endeavors, with Google spending an average of $145 million each day in the quarter. Microsoft, Meta, and other magnificent seven companies have similarly ramped infrastructure spending on AI projects. And while the likes of Microsoft have generated $5 billion in sales for their genAI offerings, it’s just a fraction of their total revenue of $245 billion. Even Sequoia Capital posted a blog asking, “Where is all the revenue?” Going on to note “a big gap between the revenue expectations implied by the AI infrastructure build-out, and actual revenue growth in the AI ecosystem, which is also a proxy for end-user value.”

Tracking the ROI and impact of AI is as challenging as any other growth initiative—but essential given the outsized cost. In fact, Gartner reports that the top barrier to implementing AI is estimating and demonstrating AI value. Balancing the potential efficiency gains and long-term cost savings from AI is highly reminiscent of the early days of cloud, which promised economic benefits of scalability, reliability, better performance, and greater efficiency. However, cloud wasn’t a panacea for tech infrastructure woes, and costs could easily balloon without proper checks and balances to control spending, sometimes negating the benefits.

Without the right financial management strategies, even the most promising AI investments may not meet expectations and have similar pitfalls to cloud. Managing the full lifecycle of AI projects—from infrastructure costs to value realization, and understanding the fully landed cost of building and maintaining AI applications—will be critical for businesses seeking to justify these large capital outlays.

Challenges in AI Resource Management

AI initiatives present a new level of complexity for cost management because of the number of resources involved, the rapidly evolving capabilities of AI technologies, and the unpredictable consumption-based cost models. The dynamic nature of the AI field—with constantly changing applications, tools, and services—adds layers of uncertainty that affect planning. Costs can mount quickly and unexpectedly when pursuing these initiatives, making it difficult to predict and control expenses effectively.

It’s essential to ensure resources like cloud and on-prem infrastructure, applications, specialized talent, data management, etc. are properly optimized and allocated. Additionally, AI initiatives must be evaluated to ensure they’re actually delivering value. Only then can you know when to scale—or whether you need to pivot.

Again, like the early days of cloud, it’s easy for AI initiatives to generate sudden and unpredictable costs. Since the technology is still nascent and developing, it is challenging to model future costs as cloud providers and specialized hardware vendors continue to roll out new solutions tailored to AI workloads, adding further complexity to cost management.

Laying the Groundwork for Success: Effective AI Cost Management

The journey to effective AI initiatives requires a structured approach to technology cost management across the enterprise and begins with gathering all technology related expenditures. Imagine how much more efficient companies would have been with their cloud investments if they’d had a FinOps practice in place on day one?

By bringing cost data into one holistic view of all tech-related investments, companies can begin to interpret and derive meaningful insights. This is where companies can start to identify inefficiencies and optimization opportunities. It also leads to more informed decisions, a flywheel of continuous improvement, and enhances agility in reacting to changes or opportunities.

Apptio’s portfolio of technology business management (TBM) solutions can pull in data from across the enterprise to improve performance, optimize costs, and drive greater impact from on-prem, cloud, and labor expenditures. Through Apptio’s intelligent automation and AI-aware layers in our costing allocation models, our solutions provide a way to effectively manage the exponential complexity of AI-related spend, helping you get from simple cost-tracking to accelerated value delivery.

Preparing for the AI-Driven Future

As AI spending reaches unprecedented levels, successful companies will be those that invest pragmatically, prioritize efficient cost management, and prove ROI early. By adopting a structured approach, tracking costs meticulously, and staying agile, businesses can maximize their returns while minimizing risk.

Building rigor into TBM practice areas—including ITFM, FinOps, and enterprise Agile planning—will not only improve efficiency, but better support an AI-driven future as this technology begins to weave its way into all corners of the enterprise. Visit our Innovation Hub to learn more about how we’re continuing to support cost management across your entire technology footprint while helping you drive innovation.