An AI-skilled software development workforce is delivering business benefits to a Group CIO member of our community. Using AI to develop software and the resulting increased pace in the software development lifecycle requires a new business operating model, new tools, and a laser focus on compute costs.

Having retrained their software developers to be skilled AI users, the CIO, who has previously led technology in the financial data, insurance, and logistics sectors, has modified the operating model of a technology provider, which had to change as the firm was generating 95% of its code using AI. As a result, its developers are now largely orchestrators. They are the technology industry equivalent of pilots. Airline pilots extensively use the autopilot function of their aircraft, but as the adage of the airline sector goes, they are a vital and cost-effective element of the business in times of necessity. Pilots are the human in the loop as the aircraft autonomously goes about the business of flying.

A pods model has been adopted to ensure there is one senior engineer in each pod, focused on each outcome, and in every function of the business. The CIO says:

We can see the opportunity to build more functional tools in Human Resources (HR), Sales, and Marketing.

The roles across the business have also changed. The CIO says product management, product engineering, and prompt engineering are well-defined roles across the business, so the investment in AI skills they have made has resulted in better outcomes across the business:

We have evidence that you get very different outputs from different types of prompts. The quality of output varies considerably based on the quality of the input. No surprises there.

As AI changes the systems of our businesses, CIOs will need to ensure their people have the skills to embrace AI and create good-quality prompts, which in turn create good-quality business outcomes.

The CIO is now considering a Center of Excellence (CoE), which will go around the business and its functional teams to continually sharpen the use of AI and feed back into the new operating model; they add:

There will always be a human in the loop to make sure outcomes are healthy and affordable. Where I see us investing more time is building out frameworks, so you don’t have to revisit work when you want to scale.

Asked by a peer during our community virtual meet-up if AI agents will appear in their organizational chart, the CIO said that talking AI avatars are already within teams' lists.

As businesses increase their AI adoption and pace of delivery, CIOs are seeing the demand to understand the complexity and costs of the technology estate increase. Cloud FinOps financial management practices and, as we have already reported, observability tools will increase in importance. But this is not a future position; CIOs need to be using these now, as they reveal:

For the token costs, the market is changing so rapidly. It is not just inputs and outputs; it is also about the caching. Engineers need to see what they are consuming in terms of tokens; otherwise you may get some nasty surprises.

In the virtual event, an example was shared by a community member of an engineer burning through 55,000 tokens and creating a bill of $100,000, not something a CFO or CEO will welcome.

The CIO is using Langfuse for AI FinOps and adding that to the total business management (TBM) methodology for cost management; they add:

We are doing that right down to the user. It is measuring not just the build costs, but also the consumption. If AI FinOps is not in place, costs could just run away, as it is very easy to spin up a lot of tokens.

With the AI providers regularly changing their costing models, FinOps methods will provide CIOs with the ability to embrace AI cost-effectively and sustainably.  The CIO’s team has developed a token usage control plan that aligns AI activity to costs, while the firm is also working with Palo Alto Networks to build a real-time observability platform.

With software development times tumbling from 18 months to four, the CIO’s team has not only increased the number of applications it offers its customers and the market, but also developed a range of bespoke applications that have reduced business costs.

For his own team, they have developed an analytics engine to provide the CIO with insight into how many software engineers are using the AI platforms, what prompts they are writing, and the quality of the code those prompts create.

The Workato and N8N workflow automation tools have been decommissioned as, using Claude from Anthropic and Devin AI software development tools, internal equivalents have been developed. For N8N, itself a company and technology born of the recent surge in AI, this shows the precariousness of the enterprise AI sector. The CIO adds:

Using Claude, we evaluated N8N, and then Devin wrote a new tool in two weeks. So we have the exact capabilities of N8N with zero license costs.  I see the ability to switch off a lot of third-party tools.

The Redgate SQL database monitoring tool followed the same fate. This has saved the CIO half a million in licensing costs, they add:

I am looking at every opportunity internally, from third-party tools, infrastructure, and software estate management for how I can build my own capability.

The CIO doesn’t avoid the issue that their firm is itself a provider of software and could be similarly disrupted by like-minded CIOs. But they say it is therefore important for technology providers to use every tool available to them to increase the value of their technology to their peers.

Our community has revealed that the SaaSpocalypse is overblown as a phenomenon. Admittedly, not all CIOs - few in fact - have a development team of the size and ability that this CIO has at their disposal, but vendors would be wise to heed this story.

No matter the scale of software development CIOs and CTOs embark on, as this story outlines, the need to adopt models like FinOps and observability tools will be vital to ensure the budget remains sustainable.