They Said It Would Cost $54M. We Said "No Thanks."(nateglubish.substack.com) |
They Said It Would Cost $54M. We Said "No Thanks."(nateglubish.substack.com) |
> The team’s approach was straightforward. Build working software fast. Put it in front of real users early. Collect feedback. Fix things quickly. Release updates every two weeks.
> That’s a 95% cost reduction. Both systems instead of one. Delivered faster. With 643 users already on the platform
This is a proven solution. These parts, the non-AI management ones, are proven to work in all sorts of places. Gov.uk is another example.
However, there's one massive problem with this: it doesn't involve the free market and it doesn't make any money for corporations to feed back to politicians in campaign donation kickbacks. It even involves respecting civil servants - maybe even paying them market wages! These parts are so heretical that most governments would choose the solution that 10X more expensive and also doesn't work, every single time.
Modern accounting (for some good reasons) counts different costs in different buckets. If you invest in a system, the cost should be spread out over all the years the system is in use. However if you investing long term we want to be sure it is worth the investment - there is a point where something is not worth the cost. For things that are only for this year we can easially understand if the cost was worth it, but if we need to spread it out over many years it is much harder. If we can set a cost today we can have a debate on if it is worth the investment.
However designing something is not an fixed cost process. I cannot say if it will take me 1 week or another year to get the current bug I'm working on fixed, but if it takes more than a month my boss will say it isn't worth the cost of fixing it. (and then we get to sunk costs: what if I put in 1 month and am only 1 day from the fix, should I quit now?)
When you put a design contract out for bids they have to go big. They have to deliver for $54M even if things are harder. When someone says the real cost is likely to be $100M they really mean that once you have the $54M version you will realize what you wanted wasn't what you needed, and there is $46M in extras to turn what you specified into what you really need.
This is the wrong question. But one often asked because of the sunk cost fallacy.
The real question is: what if I am only 1 day from the fix, should I quit now?
Yes - and they also have to deliver for $54m if things are much easier.
The tender process often imposes an overhead of several million $ per bid, which has to get rolled back into the margin on the projects.
It also gets very silly, like one public one that company I work won was literally just iterating the company that currently hosted their infra infrastructure, so the previous company automatically checks all the boxes and any new one have to fit that, and it went to silly degrees like describing clockspeed of CPUs that have to run it. We just gave them older servers that happened to have higher clocking CPUs so the requirement backfired badly on them.
And that is not even neccesarily malice on their party, they just didn't wanted to go thru PITA of migrating everywhere just because some pencil pusher required looking for vendors every few years "in case it will be cheaper".
I genuinely don't understand this trend of slapping AI slop on top of your article. I've even seen good articles do it.
What you have in civil government is a lot of people and a lot of time -- turning that into inputs for acceptance on the new codebase is super smart -- and using only their expertise (legacy system screen caps), but relying on the AI to do all the tech spec work feels super smart.
Ah, that's what the AI ingredient is.
Seems reasonable, the kind of drudge work that gets avoided because nobody wants to do it. Requirements-capture what the existing system does. This often fails in the real world because it's done at some distance: either writing down what they think the system does, or want it to do, or getting political interference to pretend the process is something other than it is, but ignoring the actual working on the ground process.
That's kind of amazing. Alberta has a conservative govt so I am surprised "in-house" got the pass over "outside company". It is good to see fiscal conservatism over 'govt-bad' conservatism. Hats off to the deputy minister et al. for approving this.
Using Google Gemini to generate requirements/spec document from video is amazing. I wonder what the prompt looked like and if there was custom support to help process the videos.
I feel this has more importance than they think. Outside consultants would not have had this domain knowledge and would have spent months learning it. And then would have had to fix their mistakes because they misunderstood something (billed to the province, naturally)
It really does make us punch above our weight. We can ask important questions and identify critical issues early in development of new things.
When?
Today.
Minutes.
Four years.
$54 million.
Collect feedback.
Delivered faster.
Not days.
Not weeks.
It's free.
...
No $19 million in upfront costs.
They're now doing meaningful work.
Let me put that in context.
That's a 95% cost reduction.
But think about what that represents.
And we can show you how.In brief, the story isn't AI as an accelerant. It's someone finally letting loose their internal knowledge to build to spec rather than buy.
This is the problem in a nutshell. Those firms are structured to extract money from their customers, not to produce useful work. The fact that anyone is signing contracts with them any more blows my mind.
> what if a small team of public servants, equipped with modern AI development tools, built the replacement systems themselves?
Next: bridges and brain surgery.
On the other, procurement is so broken, that if their inhouse team is only marginally better, it's a win.
Anecdata: while i was in a tiny tiny software company, we got an in at a large auto manufacturer. They said they had been trying to get someone to do that job for like 2 years.
The job was of the 'two people 3 months' magnitude. The procurement system was also of the 'two people 3 months' magnitude so we simply gave up.
In the article's case, they could have done this even before coding assistants. It would have cost the estimated 5 million instead of 850k, but that's still 10x less than the 54 million.
As flawed as this new approach might turn out to be, the traditional approach may (or may not) have an even worse probability of success.
There is no reality on this planet where this project costs $54M and 4 years. Good on the folks in the ministry to notice this and not blindly follow implement.
This assumes that small in-house teams are inherently effective/efficient, which is not necessarily true.
In this sense, the difference between proven engineering leads (as the article states/assumes) leading a small team versus AI is that the latter is entirely under their control, which minimizes the risk.
So AI vs. small teams is about controlling/guaranteeing effectiveness/efficiency.
Smaller team that's closer aligned to business goals can be far more agile and efficient than bouncing between middle managers of 2 separate corporate structures
Anyone else closed the article immediately after seeing the low-taste, sloppy image at the top?
How do you call this aesthetic? "Futuristic vomit"? AKA "Generate image of: code blocks, AI-brain image, diagram, smiling guy and bunch of other crap. Make it look cool and futuristic, make no mistakes"?
Super disappointed to see most of the comments just complaining about AI and not engaging with the contents of the article.
Authors do it because it supposedly leads to better engagement, shows up bigger on social media, and breaks up the text. But generally, unless the visual content meaningfully adds to the text content, users will largely ignore it.
Wasn't there anything relevant available? Screenshots of the new tools in a before/after collage perhaps?
The other option: cost plus bids (that is you pay their actual costs plus a fixed amount of profit) is even easier to game because you can add costs in so many ways. That is why most large bids are fixed cost despite that downside - they are much harder to hide extra costs in.
There is no alternative to carefully watching your suppliers. Some are more honest than others (and you should blacklist the worst).
Hence: inhouse.