Ask HN: Is Your Company Turning GenAI Usage into a KPI?

6 points by hibernator149 2 years ago | 4 comments

The GenAI introduction at the companies I know of has worked like this:

- Management repeats marketing claim of 10-30% performance improvement without question

- An AI advocacy team is formed from mostly junior devs that joined within the last year

- Criticism that GenAI can not help with certain tasks is met with "You are using it wrong"

So far I was just annoyed, not surprised. But now upper management is openly discussing how they can collect usage metrics for individual developers. I have so little faith left that I can not imagine how this could lead to any positive outcomes.

Do you have any personal experience with individual GenAI usage metrics? Will this just blow over?

agile-gift0262 2 years ago |

"<anytool> usage metric" as a target in itself sounds awful. I would not like to work in a place like that.

Regarding GenAI in the company I work for, its adoption has been explored several times at different levels in different parts. I was tasked with trialing out GitHub Copilot, for example. But the company hasn't decided to go ahead and pay for any GenAI service.

Free-tiers are sporadically used for some very low-risk, unimportant, employee-supervised/reviewed use cases. But it hasn't proved to provide as much value as what they cost, even if currently paid GenAI services are still heavily subsidised (most if not all companies behind haven't reached profitability in their GenAI ventures) and the company is very cautious of what data is provided to these services.

al_borland 2 years ago |

Sounds like where I work. They started pushing GenAI hard in every town hall, even before we actually had access. There was a lot of mixed messaging, which was confusing.

Then we had global town halls where some kid with 5 months at the company is presenting in front of thousands of people, because he installed the Copilot VS Code plugin and used it for a week.

There is a dashboard I have access to (though I’m not sure if I’m supposed to) that has all kind of metrics on Copilot usage, such as how often suggestions are accepted, broken down by team/manager.

I assume the hyper focus on it will fade once it becomes a normal part of doing work and something new and shiny comes along to shift focus.

giantg2 2 years ago | |

"some kid with 5 months at the company is presenting in front of thousands of people, because he installed the Copilot VS Code plugin and used it for a week."

Yep, we have goals for specific percentages of devs to be using it.

repelsteeltje 2 years ago |

That sounds horrible.

Makes me reminisce Office Space story line of the waitress being encourage by her management to "express herself more", "maybe you need to wear a bunch more buttons?!".

Seems to me that once this gets pushed as KPIs your company has surpassed the point of reason and its going to be very difficult to reclaim any ground where GenAI might have been able to make any sense.

Good luck, hang in there...