I just heard a good passage.
Any thing that is worth managing, it must be measure before being able to control it.
But sometimes, they just pick a metric thats "easy" to measure instead of picking a metric that actually measure what you want to reduce/improve/increase.
In theory, you would need to measure net gain in productivity with AI to determine the effectiveness or no of that tool.
But it is not easy to measure that, as it is a new thing, there isnt a consensus about the "best" way to use it, etc.
So, managers go with the "easiest" metric to measure AI usage: just measure tokens used. It MUST be related to performance, right?
Definitively no, it just made people to consume pointless tokens just for some bonus on "productivity".
But with the bills exploding, now the managers are reconsidering this metric.
My company, so far, doesnt have any metric to measure AI usage. And we are going to a very restricting policy to use tokens from external models in Copilot (somehow, we have house-hosted models, weaker but "free" to use)