Brought to you by...goClose
Want all of the AI Skills that OnlyCFO Builds?
They will all be on goClose so go sign up! We just uploaded the Skills OnlyCFO built to manage the cap table (create the grant list for board approval, reconcile equity grants, monitor activity, and 409A reviewer). They are in the the Resources section
goClose offers these resources and a month-end close tool for free.
AI can do incredible things that will benefit humanity (cure diseases, solve science’s hardest problems, create abundance, etc). But for businesses, it boils down to one question…And it’s on every CFO’s mind:
What’s the return on our AI spend?
A popular quote about marketing spend (that most marketers have grown to hate) is that half of their spend is wasted, but you don’t know which half. I think a similar story exists for AI spend today…
CFOs don’t want to suffocate innovation with AI but they also need to make sure the return on AI spend is worth the cost. Measuring that return can be hard tho…
There are two ways to increase the return on tokens:
Lower the cost of tokens
Create more value from the tokens
Lower Your AI Costs!
There is currently a lot of focus on #1. Understandably so…It’s much easier to measure and there is a lot of opportunity for “tokenmining”. Just look at Claude or ChatGPT’s revenue growth rate to see the massive opportunity here. I think an uncomfortable amount of their current revenue would be eliminated if all companies had good model routers and were utilizing open-weight models where it made sense.
*this doesn’t mean Anthropic and OpenAI are cooked (as the kids say) because the TAM is massive, but there is a lot of cost-cutting opportunity right now…
Two of the biggest opportunities for “tokenmining” (ie cutting AI model costs while not impacting quality) are:
Shifting some usage to open-weight models
Using a model router to use the least expensive model required for the task
Model routing has become a very hot product. Many companies are adding a model routing product (Ramp announced theirs this week). Stripe just announced that it is acquiring one of the leaders in the space (OpenRouter) for ~$8B, which is ~6x their Series B valuation of $1.3B from just 3 months earlier. And it represents a ~54x ARR multiple (~$150M ARR at ~$8B)!
TLDR: Cutting AI token spend is the most measurable thing to do to show improvement in return on tokens. There is plenty of opportunity there so everyone should go do it.
Are We Creating Value?
This is the question that not enough people are asking yet. But good CFOs will be pushing on this and making sure we aren’t just cutting costs, but we are also driving real value.
It’s a lot squishier to measure though…
Each department needs to set goals of what AI spend is enabling them to do and why it matters (how does it align to company objectives). Examples might include:
Incremental revenue
Realized labor savings. Not just hours theoretically saved
Cycle-time or speed improvement
Error/risk reduction
Are people creating actual value or…are they building AI slop apps just because they can? I have seen too many of these apps forced upon me - they add no value and suck up everyone’s time.
And tracking spend appropriately on the P&L is an important starting point for driving higher return on tokens…
AI Costs on the P&L
Cost of Revenue & Gross Margins
When AI is embedded in your product, that portion of the costs is also included in COGS.
AI costs that may get included in COGS:
API spend with AI models (Claude, ChatGPT, etc)
Model hosting when you self-host (not calling an API but using AWS, Azure, etc GPU compute)
Vector database costs
Orchestration and observability
While AI gross margins are up a lot in the last couple of years (from 45% in 2025 to 53% expected in 2026 and 59% in 2027), they are still significantly lower than typical software gross margins. But with the growing popularity of open-weight models and model routing, we may see these accelerate even faster for the application layer.
But it’s important to track gross margins of all products (especially AI products) separately because they can have vastly different costs and gross margin profiles. The best companies at managing this can track individual features and watch AI costs trend over time. Then they know where to focus their cost-cutting efforts or where their pricing model is broken.
Create a new line just for the AI costs running through COGS. You, the broader exec team, and the board will want to pay close attention to COGS from AI so tell your accounting team to create a separate line for it and track it by product.
How to Allocate Your AI Model Bill
Don’t treat G&A as an expense dumping ground with your AI spend. Yes, dumping everything in G&A makes all your other metrics look better but it’s incorrect.
AI spend for unpaid customer POCs/trials should go to sales & marketing. Make sure your accounting team isn’t putting it in COGS or R&D
Can I allocate AI model spend based on headcount?
Yes…but only if you are paying a flat fee per seat (ie not paying based on consumption). Then a simple headcount allocation like all your seat-based tools makes sense.
How should I allocate usage-based AI spend?
Based on the usage…duh. The only good approach now is to do direct allocations based on specific department usage. Finance teams need to segment the different types of AI costs by department. Anything else will result in materially incorrect financials and unit economics. Once you have true cost allocations then you can better determine the “return on tokens”.
Footnotes:
Subscribe and forward this newsletter to your team.
Check out goClose to get OnlyCFO’s downloadable Skills