Plato's Dashboards(ferd.ca) |
Plato's Dashboards(ferd.ca) |
They're mostly there to serve as the source of truth, whether or not a specific manager is getting a bonus (and by how much).
Generally, the best metrics are "based in reality" (and usually revolve around inflow and outflow of money) -- but those are hard to fudge, without playing accounting games, and even harder to meet (it takes actual skill, rather than being able to pass the buck endlessly for a couple of years, before moving onto another org); so you may find it common for the managerial horde to pick "bullshit metrics" to cover their own asses (doubly so, if the metrics are decided by committee).
There's no reason for practical metrics to exist in most large corporations -- the incentives just aren't aligned. Bonuses are decided by metrics (and anyone who has a modicum of intuition, will always pick the metric that can be most gamed for himself, and his underlings). Also, if we're being real with ourselves, there's very little that managers "control," so picking a "real" metric would just breed an environment that selects for luck (or lies).
In my opinion, the only useful metric is profit generated. Everything else is just an emotional safety blanket for uncertainty.
This too can be a toxic metric. Only focusing on what looks good in the quarterly report encourages a type of short-sightedness that has been the undoing of many businesses.
We know by observing private equity that the easiest way to hack this metric is by doing layoffs until there is imminent threat of the business vanishing.
The corp is just a vehicle for the goals of all its passengers. Most of them are only on-board for a couple of stops, and then off to the next vehicle. To voluntarily forgo getting off at your stop, without receiving something of equal or greater value, seems awfully foolish.
The project I've been on for the last couple of years has had dashboards and monitors and we as engineers have been striving to keep the system humming and if any issues trigger alarms, we jump on them.
More recently there's been a push to handle alarms more quickly (get those response times down!). I'm not sure why since I don't know if we've really had user complaints about any of this (not that our systems are perfect, but the user-facing issues aren't due to us not responding quickly enough to alarms). It never made sense to me that these numbers were of utmost importance, and important enough to visit in biweekly status meetings.
Your post now makes me see more clearly what I had a hunch: somebody decided to take on this mission of getting response times down as an action item and by god everyone in their org is going to meet that mission.
Anyway, the product we were working on got cancelled recently, which funnily enough has nothing to do with alarm response time or any of our engineering dashboards and more to do with the "money in/money out" metric. Fancy that.
What about making a quality product because you have a passion for it? Nope, make a broken MVP and move on to the next customer.
If I can't maintain a metric until I can tie it to revenue, then I guess I don't get cpu usage on our database. Or disk usage. Or response time for anything that only employees touch. Bad data and errors are fine as long as no one complains. Actually, if we produce a shitty product long enough everyone will get used to it and stop complaining. If we stop measuring error rates we can stop wasting time on fixing things right? Tests and code refactors just get in the way of delivering that MVP. Rewards are doled out for fixing an outage, and preventing them is a cost-center to be eliminated.
On the flip side, now we've got KPIs for our ability to push things through the corporate bureaucracy. I didn't become a programmer so I can waste time estimating how long it'll take to produce a design doc that'll be useless by the time I get to implementation, just so a bureaucrat can decide if some deeply technical problem they don't understand should be fixed.
There has to be a better way.
Whatever nobility or higher morality finds within it, it's just an exception that slipped through the cracks.
Idealism towards the external world will always be met with defeat (you simply do not have the power to fight more than one battle, with one person).
The only thing left is to serve as a role model, standing tall and never wavering, for your values and ideals---so that those around you may follow in your footsteps, if they so wish.
Perhaps you could start by refusing to play the KPI game? What is anyone really going to do, besides through vacuous diatribes about being a "team player," and such nonsense. The only power these corporate constructs have is that which is surrendered to it.
During the first Internet boom, when telephone modems and $20/month all-you-can-eat ISPs were the hot thing, Lucent made the best point-of-presence device, the Ascend Max: very nearly everything you needed to get access for a town in one box. Lucent discovered that new ISPs had little startup cash but generated strong monthly recurring revenue, so Lucent financed the Max for the ISPs, with the highly in-demand Max itself as the collateral. After all, if an ISP collapsed, the used Max could be sold again at nearly full price.
Lucent didn't count on the entire ISP market consolidating.
I'm not sure this is the right framing of the problem. "Guidance" is too nebulous and leaves us with the same problem of interpretation.
Advice from an old tracker: You want to find someone, use your eyes.
Getting good questions also often involves cleaning up sources of noise in the charts. It's too easy for someone with an axe to grind to jump to conclusions that this is a repeat of a problem we had before (ie, this is Team X's fault... again) or that lack of improvement means failure to act. It is not impossible that I fix one regression at the same time someone else causes another, or changes the denominator by the same magnitude as the fix.
I think this is essential reading if anyone's interested on a product-focused tangent for the topic: https://articles.uie.com/net-promoter-score-considered-harmf...
How likely are people to promote the thing?
The two are indeed very similar, and often used interchangeably, but there's an important difference in their phrasing.
Goodhart's law states that: "When a measure becomes a target, it ceases to be a good measure."
Campbell's law states that: "The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."
In other words, the former focuses on the fact that, while metrics sound good in theory, in practice they cannot be relied on; whereas the latter focuses on the fact that enforcing metrics in an attempt to control a social process tends to backfire.
Or, in Trump language:
Goodhart = "Sounds good, doesn't work"
Campbell = "Worst trade deal ever"
Thus, the original article title might be a bit misleading. It is not necessarily about businesses running based upon illusions.
Commerce is vulgar, but as long as you're making (b|m)illions off me could you let me do my thing? Stop taking things that work for tech folk, misapplying them to business concerns without any intellectual honestly and deciding they don't work for anyone. Metrics work. The problem you people have with them is that they are a branch of statistics and if there's one thing business folks love to do it's lie with statistics. It's all fun and games until you start lying to yourself and start seeing the negative consequences.
How do you propose I escape the KPI game? It's how I'm evaluated by people who pay me. It's become how you gain influence in an organization. I can't escape to another place, because they all operate this way now. More and more I'm seeing "tech" leaders, big and small, that are much better at playing the game than holding any kind of technical excellence.
I'm not looking for some idealized world. At this point, I'm looking to have some fun while I get taken advantage of.
Then, I stopped worrying and loved the grift.
It's just a game to play.
I shelter my intellectual "honesty" and idealism within my side-projects.
For business? I play to win.
It's the only way I've found to stop myself from becoming a jaded hollow husk.
Escaping the KPI game? There are organizations that don't self-fellate with KPIs (rare, or impossible, as it may seem). I've found that the "corporate bullshit" curve follows the Dunning Kruger curve (with fat tails). You have the vast bulk of bullshit in the center, the average SME/Enterprise, where everyone is trying to peddle bullshit faster than their competition. On one end you have the "high-performance," "we're making obscene amounts of money, and we want to make even more" a la buyside high finance (note: there is a lot of "playing the game," but the game isn't as petty, because people have real shit to do, with a lot of money at stake). On the other end you have the "we're growing. We're making money, and we're busy getting more features out the door; we don't have time for this bullshit that pretends to be working" start-up type where everyone knows how much everyone else is "outputting." There's no room to coast; there's no layers upon layers of bureaucracy and bullshit metrics to disguise your actual worth to the org.
If neither of those two fit your temperament, then perhaps you should find a research outfit? There will still be games to play, however.
If you generally view most people as incompetent, and that you're constantly having to bear the brunt of the terrible decisions of those above you: you should start your own firm (or find more competent people to work with).