AI Cost, Risk and ROI: Three Questions Nobody Asks

Everyone shows you the demo. Nobody shows you the invoice.

Most organisations we work with in Perth now have something running. Meeting summaries. Draft-first document generation. A search tool pointed at the intranet. Somewhere on a roadmap there is a slide with the word "agentic" on it, and a date next to it that nobody has stress-tested.

What far fewer have is a clear answer to three fairly ordinary questions.

They are not hostile questions. They are the questions a competent business analyst asks about any new capability before the organisation commits to it - and they are being skipped, consistently, because AI arrives with a demo attached and demos are persuasive.

Question 1: What does it actually cost to run?

Why the usual budgeting reflex breaks

Traditional enterprise software has a licence. The number is known, annual, and sits in someone's budget line.

AI is metered. Cost scales with consumption, and consumption is the exact thing every adoption plan is designed to increase. The bill arrives after the behaviour, not before it.

That reversal breaks the normal budgeting reflex. Pilot funding covers experimentation, so nobody models the unit economics at production volume until production volume arrives.

The questions that matter

• What does one transaction cost in inference - one processed claim, one summarised case file, one drafted response?
• What does that number look like at ten times current volume?
• Is there a spend cap, and when it is reached, does the service degrade gracefully or does a user hit a wall mid-task?
• Is there anything the organisation would already have scaled if it were cheaper?

That last one is the useful question. It exposes where cost is quietly acting as an architectural constraint while everyone is still describing the decision as strategic.

A capability whose running cost cannot be forecast is not yet a production capability. It is an experiment with a good interface.

Question 2: Are we choosing problems where being wrong does not cost much?

What a low-consequence portfolio does not tell you

Look honestly at the pilot portfolio. Internal audiences. Low consequence. A human reviewing every output before it goes anywhere.

That is rational risk management and no organisation should be criticised for starting there. But it has an effect worth naming: a portfolio built entirely from low-consequence problems never tests the hard part. It tells you the model produces plausible output. It tells you nothing about what happens when that output reaches a customer, a payment, or a regulator.

Governance, or the appearance of it

The question is not "do you have governance". Everyone has governance. The questions are sharper than that:

• Where is the line on what a model is allowed to touch, and who drew it?
• Has that line moved in the last twelve months?
• Which controls have ever actually stopped something?

A review step that has never been used to reject anything is not a control. It is a signature. Distinguishing real governance from governance theatre is one of the more valuable things a delivery team can do, and it is uncomfortable enough that it usually does not happen without someone being asked directly.

Question 3: Has anyone proved the return? Actually proved it.

What the MIT figure does and does not show

MIT's NANDA initiative published The GenAI Divide: State of AI in Business in 2025, drawing on interviews with business leaders, an employee survey, and analysis of 300 public deployments. The headline finding was that roughly 95 per cent of generative AI pilots delivered no measurable impact on profit and loss.

The methodology has been contested, and fairly - the definition of success was narrow, and a pilot that improves individual productivity without a P&L line is not automatically a failure. But the argument over the number is less interesting than what sits underneath it. The reported failure was not model quality. It was integration and measurement.

Which lands close to home. In most organisations the reason ROI cannot be proved is not that value is absent. It is that nobody captured the baseline before the pilot started, so there is nothing to measure against afterwards.

Why "time saved" is the weakest claim available

Time saved becomes money only when the freed capacity is redeployed to work that was not previously being done, or a cost is genuinely removed. If neither happened, the organisation has bought a more pleasant working experience.

That may well be worth paying for. It is not a return, and presenting it as one damages the credibility of the next business case.

Worth asking: what is the number, who owns it, and would you defend it to a board?

Why this is business analysis work

None of the above is a technical problem. Every one of these questions is a requirements, benefits, and decision-rights problem - which puts them squarely in BA territory.

What that looks like in practice

• Treating running cost as a non-functional requirement with a modelled unit cost, not a line item discovered in month four.
• Capturing benefit baselines before the pilot, not reconstructing them after the fact to satisfy a steering committee.
• Documenting explicit decision rights - where a model may act unsupervised, where a human must confirm, and who owns each boundary.
• Specifying failure modes. What happens when a wrong answer reaches a customer, who detects it, and how it is corrected.

The analyst who can hold a room to those questions is worth considerably more on an AI program than the one who can write a better prompt. That is the shift worth preparing for.

Ask them yourself, on Thursday

We are putting exactly these questions to Billy Martin, CIO at HIF Australia, at the next Perth BA Tribe event. Billy previously led HBF's transformation program as General Manager of Transformation Delivery, including an AI claims automation initiative. Muthu Murugesan, co-founder of Lane8 Consulting, is hosting.

It is a fireside chat rather than a keynote, and about 25 minutes is reserved for questions from the floor. Health data, an active regulator, and real member impact - the constraints are real, which is the only setting in which answers to these questions mean anything.

Thursday 20 August 2026, 5:30pm - 7:15pm
Level 5, 143 St Georges Terrace, Perth
Free to attend. Light refreshments provided.

Register: perth.ba-tribe.org

Perth BA Tribe is proudly sponsored by Lane8 Consulting, Perth's Business Analysis specialist consultancy.