Tokenmaxxing is over. That’s because it never measured what really counts to see ROI from AI

Hello and welcome to Eye on AI. It’s Jeremy here, filling in for Sharon who is on vacation. In this edition…CNN sues Perplexity…IBM and RedHat form $5 billion bug patching project…Snowflake signs a $6 billion deal with AWS…and the White House gives U.S. intelligence agencies $9 billion to build their own AI chip cluster.

Just a few weeks ago, it seemed that ‘tokenmaxxing’ was all the rage inside many companies. The idea was: if you wanted to find out which employees were being most innovative in deploying AI agents, you should track their token usage. (Tokens are the units of data that AI models process; a token is equivalent to about a word-and-a-half of English language text.) The more tokens expended, the more productive that employee’s AI agents were, or at least, the more AI-forward and innovative that employee was trying to be. That was the idea anyway. Meta, Amazon, OpenAI, and many other companies even established formal or informal leaderboards of token usage and encouraged engineers and developers to compete to see who could use the most tokens in a given period of time.

Of course, Goodhart’s Law still holds (it posits that any measure that becomes a target,...