Enterprise Software Projects Killed the Software Developer(javahippie.net) |
Enterprise Software Projects Killed the Software Developer(javahippie.net) |
I'll take a software developer who writes and structures code so change requests and code are written in a way that the DSL is the same across the organization. This makes changes easy. Clever people should be writing libraries or doing research.
Don't kid yourself, you are either the guy who builds the building and its easy because its greenfield, or you are doing remodeling and the hard part is making the upgrade fit in the building and not look like shit.
That said, there is a type of cleverness that can be brought to bear on Enterprise systems that, for example, take a simple, unidirectional data flow into account - something that is rather abstract, but which can and will thwart lots of complexity down the line.
The problem is then you get some enterprise architects that go crazy with the design space.
The PHP script kid basically writes a linear program (tons of duplication) with no crazinesses. Yes, it's "low quality", but if you do a few function out refactorings you've got something very easy to work with.
I just wish there was a standard template - access check, runtime complexity at code comment at top of function (ie, O(1), O(n), O(n^2)) some reasonable comments, error handling, done. Throw in some unit tests if desired.
I disagree that is the fix. I've seen horrible code that has extremely extensive documentation, and nice code with minimal documentation. I know which I would rather work with.
Documentation has its place, particularly for high level concepts and shared data and interfaces, and tricky, exceptional, or surprising things in the code itself. But it's a poor crutch for bad design or code.
In return I ask a bit more time so I won't self-couple my code with introspection magic or make it undebuggable with weird dynamic design patterns nor will I make wormholes to so that A can communicate with B.
Deal?
to me, more often than not, it means simple
Dev time ends up taking longer and longer, wasting more and more money.
Its funny you mention greenfield being easy in the same sentence - because IMO starting over is often the better option than the shit bolting but no one wants to go down that route because they need it next week. A whole bunch of small shit bolting projects every week instead of sitting down and cleverly engineering better broader solutions.
I think this more modern approach has some important pitfalls which we're uncovering now and getting hurt by them.
At the end of the day, though, the point of treating internal products like products and not projects is accurate. Every good IT shop I was a part of landed at this answer, even if we got there through different experiences.
When I first read the Agile Manifesto - around '99, I think - it seemed clear to me that this was a great leap forward in software design, but that it was clearly implied that this couldn't be used in a "fixed deadline" environment. I really wish they had actually called that out in the manifesto itself and made that more obvious.
Businesses want to reduce risk. That means reducing the probability of expensive surprises.
That means they'd rather spend 10% more on inefficient code running in production than to risk their 10x developer quitting and nobody else being able to understand the codebase, costing the company a lot more.
A lot of developers and engineers just want to have their space to tinker, to grow their skills and their mind. As a bonus, under this arrangement they'd get paid for it!
That's not how the real world works. Work follows the money, not the passion, and that's why work generally sucks.
E.g.: Photographers don't get to spend all day doing art photography, they largely have to do wedding photos to pay the bills.
If you don't want your hobby to suck, don't do it as a job (or accept that work sucks and do stuff on the side for your own enjoyment if you have the energy for it).
My best advice for any engineer is to take active interest in the business' needs and wants and consider those to be on a higher pedestal than implementation details.
> Personally, I have never seen SCRUM or any agile approach working in a project setup ever.
they work great. Just their goal isn't successful delivery of the project (on that aspect they fail spectacularly). The goal of SCRUM/Agile/Lean - ie. what they are designed for - is extremely low latency and high observability by the management (and thus the management just loves it, total micromanagement under the guise of team freedom). That all comes naturally at a great cost of throughput. I.e. the "watched pot" situation. The project direction is changed very fast, there is a lot of activity, the bees are overly busy, the management always know and able to report the current progress state, while the project is hardly really moving toward the actually successful state.
We use a similar framework, but we create custom views when we need performant retrieval, or the hydration of a big object graph is being done and not needed (like a grid).
>If the exact language, framework, libraries are prescribed in every detail, developers sometimes need to bend these tools to solve their requirements instead of using the right tool for their problem. If the layered architecture is to be zealously followed, 50% of your code will be the mappers between the layers.
We use automapper, which does exactly what it says. We manually map one-offs. In fact, that's the basic philosophy of this design. Have the framework build everything from the entities, then one-off what you need.
Our design is generic so every table has an entity, then we use a generic service layer with your normal CRUD + search functions, then our controllers are auto-generated using the same thing. We do custom work only for one-off stuff that either is more complex than CRUD, or requires higher performance. It's cut our development time significantly, since for normal crud work, it's auto-generated based on the entity itself. You create the entity and dto's and the repository, service layer and controller are all auto-constructed using generic code. If you need something special, you create a custom controller/service. We tend to leave the repositories generic. Note this is just for the WebAPIs, front end is a different monster.
If this interests you, then this old conversation might also interest you:
One terrible thing about Enterprises is the way their finance team leads product decisions. In order to maximize their profit, they announce a product will be ready by X date, and estimate the cost leading up to it. If you don't hit those numbers, it affects a lot of other numbers, especially if you're publicly traded. And that's before the wild promises Sales makes. They are preternaturally addicted to arbitrary deliverables.
Essentially, their products are projects, and the customers are incidental to the whole thing. There is no estimate for customer happiness in the business case for a new product team. If you build it on time and under budget, everything else (getting a customer to use it and like it) is taken for granted.
I led a product finance team at a SaaS company (ServiceNow, Atlassian, Okta, etc. tier) and all of our models and analyses are product driven including a/b testing and surveys with customers.
I would add that a/b and surveys are not good enough to identify customer pain and solve the problems they most want solved. There's a raft of feedback mechanisms that most Enterprises ignore because their products are so complicated that nobody wants to sit with the users and find new methods for continuous improvement. Agile/Lean/DevOps/SRE constantly emphasize quality and immediate halting of product work until bugs are fixed, yet no Enterprise I have ever heard of does this (even for reliability - one of the core metrics of any online product!)
Sure, you'd have a Caramack here and there, but if left to their own devices, most software would end up being spaghetti code. Especially when it involves a steady stream of changes over time.
Ok, so there is a tradeoff. It does not mean that the Enterprise approach is wrong in general. Maybe that standardization is worth it? How could one quantify that?
As an architect I don't mind alternative approaches and variances from a stated architecture if they can be justified.
But mostly it’s annoying since we spend hours or days going around in circles until we arrive at the obvious solution (which the engineering team proposed at the start).
That sounds very true. I am always joking that I could probably "architect" our whole Fortune 500 company in a few days.
In sense an architect should be mainly coordinator and communicator. Find alignment and make sure everybody does their part. Detect problems and get everybody back on the same page.
(Which would then seem like the most dysfunctional example of institutionalized playing to the metrics that our field has ever invented.)
in other words: you can stand up a web site in less than an hour that will handle enormous loads across the globe, but not if you pause to read all the library code that goes into it.
fortunately, humans are pretty good a having faith in things
I follow a tutorial and if I’m not understanding how the guy got to knowing that the property we should use in that scenario was X, I think I’m not understanding anything and that I won’t be able to build anything, so I halt trying to figure out, but I’m so inexperienced that I can’t understand the docs anyway.
What I should be doing is putting that code in my notes and moving on having faith in that code.
I was thinking of opening an Ask HN about “how much of your code do you actually fully understand vs how much is just copied from somewhere “
You can probably make website faster if you learn 3rd party tools for that... but then you won't be in the position of Paul Graham in Viaweb, where a small team run circles around competition, being that much more productive. I suspect Apple's success is largely because Woz knew ins and outs of the system.
Your important here word is "anymore". Maybe we're at the wave of complexity which will wane. I don't actually think it will last.
Great interview.
For a coding example, in Javascript you have things like arrow functions that make the code more concise. But that is also harder to read for someone who hasn't picked up on them yet (which may have been the case when they were first introduced).
"Two plus Three times Five" as a sentence reads and is naturally processed from left to right for a result of " is twenty-five".
But to anyone who knows math (precedence), "2 + 3 * 5 = 17"
We can have an argument over ternary operators and new users and I'm going to argue that they're part of the common vernacular of computer science and everyone needs to sit down and learn them.
But nested ternaries three levels deep are horrendous. Stick with one level and simple expressions and keep it readable (assuming the background of just understanding the operator). Don't overuse it.
Similarly lambda functions are part of the vernacular, everyone is going to need to learn that necessary amount of complexity. But at some point nested lambda functions containing lambda functions are going to get difficult to read and reason about.
Then you start coding for the lowest common denominator.
While some ppl from my team have huge issues understanding it, the consultant read it ONCE and knew everything. He was simply a good dev that had a pleasure working for other big shops that wrote the books I read and lead those workshops I attended to.
So its like always a matter of „Do I hire cheap HR resource or an expert?” question.
Nothing more, nothing less. The issue with writing smart code is that there is not a lot of smart devs. Thats why we always derotate to lowest denominator.
Often many people like to attach mathematics and science to Software Engineering to signal elitism, but truthfully this profession is a lot closer to writing. Write clearly, first and foremost, and above all else.
$outsourced_bodyshop_ressource_0443434 needs to be able to read the news too.
The problem in these discussions always comes back to "others" aren't a monolithic entity. Different people find different things to be readable vs not. Different people will even use the exact same phrases (e.g. "as concise as possible without being opaque!") but still mean different things due to having different interpretations of key words in those phrases.
I don't care for your saved strokes. Make it apparent.
In time doesn't help when the majority of your devs are junior which is a common case in large internal dev shops.
I think your point might be the missing piece. Agile might not mix too well with fixed timeframes or fixed budgets, but rather needs an environment of continuous development where the requirements have room to drive the project within wide enough budgetary boundaries.
This to me feels sort of a natural way to build things. If we need something we build it, otherwise we don't. And those "needs" might pop up at any time, by external (customer requests etc.) or internal (new techical requirements become apparent as the project is being developed) events.
My experiences of agile development have been from companies without deadlines and are generally positive.
The reason is that enterprise implementations of "agile" are often the opposite of agile: waterfall with no frequent releases nor customer feedback, but lip service to agile practices and middle management ceremonies.
If you go through the agile manifesto and compare, these implementations violates all the points listed.
That's the point.
Agile is difficult to do exactly right. When done wrong my perception is that is produces far more dysfunction and stress at an individual level than alternative project management methodologies.
If you've heard the fitness saying, "the best workout plan is the one you stick to," my feelings about project management are pretty similar. Agile may be great, but if your organization isn't capable of sticking to its central ideas then it's just not going to work. In that situation you're better off picking a methodology that's less efficient but easier to implement (like waterfall).
There are also some fundamental challenges presented by the methodology that are genuinely difficult to deal with. These tend to be ignored and then they manifest as dysfunction elsewhere in the system.
What it doesn't work with is environments with projects and budgets that are quarterly or annually assigned. That's where abominations like "scaled agile" have arisen.
Scaled Agile (TM) and Scrum and all the rest of the ceremony are like ORMs are to SQL database, an attempt to correct the impedance mismatch between the way software teams work and the way companies work.
If you have a "this feature needs to be delivered by X date" type of corporate culture, then you have to make a commitment, and because dates are almost always tight, you need to be as efficient as possible to achieve that goal.
So basically you have the software teams wanting to work in two week sprints, and you have execs needing new thing for contract launch in X weeks, and guess who wins? It's not the software developer, so the agile becomes toothless, because you're not learning, or iterating. You're just pushing against a gantt chart the whole way a sprint at a time, and might not even be able to release beta versions and get feedback, because that takes up valuable time.
I'm not as cynical as OP, but there is a really big push/pull that happens, and it can turn into a toxic environment if forces too far outside the product/development cycle dictate priorities (like sales, or execs).
B: "When was the last time you met with a customer?"
A: "Never"
"Customer collaboration over contract negotiation" used to mean something.
I hate it, but I can totally see how it would work really well for keeping your architecture simple.
- formatting built in. There’s one way. No preferences needed
- language constructs are dead simple. There’s one way to use them, and they’re verbose. No cleverness encouraged
- patterns are straight forward Read the standard library code if stuck
Every Go project resembles every Go project. It’s great.
Under waterfall, a document is written by an analyst that describes a problem to be fixed. Six months later, the task is picked up but legal regulations or market conditions or even the rest of the software has changed. However, the Waterfall requirements are the requirements and that's what gets implemented. Best-case scenario is that the business analyst that wrote the original spec is still employed and has been updating the requirements as conditions change.
What we've done at successful agile shops I've been a part of is quarterly planning that collects at a high-level the current requests/needs of the business, prioritizes them, they get a t-shirt size to determine if all of the requests/needs can be met during the quarter and they fall off by priority until you're left with a manageable high-level plan of work for the quarter. Then those high-level requests are decomposed into epics and stories which are specced out and estimated, etc.
It works very well but it requires that people work collaboratively from the executives and business stake-holders to the technical leadership and individual engineers.
ER doctors may provide urgent surgery or medical intervention that's life saving and usually that costs a lot more than say long term chemotherapy or say HIV management with a specialist. Both are life saving, it just turns out that one conveniently has a longer time horizon which makes it easier to juggle to reduce costs while the other requires full attention and makes it difficult to juggle clients.
You would hope management doing budget cuts would adjust budget cuts based on value provided. This is arguably difficult to quantify in many cases but it shouldn't be quantified by how many, likely artificial, deadlines a group has and how busy they look. That's just silly.
I’m not trying to be dismissive, but I think this actually well illustrates the central tension between engineers who are more interested in the business problem and ones who are more interested in solving technology problems. I know that when you get to a later stage as a company you need both kinds of engineers, but at an earlier stage company you have to ensure all of your engineers are of the former kind and not the latter kind or you will probably not succeed.
We actively avoid the technology-focused kind, because we know they will not be able to adapt to the work we do. To be honest, we write a ton of CRUD apps, but anyone who would come and say "let's write a compiler" is guaranteed to get funny looks. Even if you are extremely talented and can deliver to the same pace as we traditionally do, you will probably fail to consider one or two "little" things that would turn out to be fundamental requirements! Nobody is impressed by half-working cleverly written software.
In fact it seems like it's quite the opposite: people who think in terms of loops, conditionals, and objects seem to be happy writing repetitive CRUD APIs all day. People who think in terms of business logic want to write business logic. You don't need to write your own compiler (it's weird that this became the thing I supposedly suggested), but you do need to develop good abstractions and data structures that fit your domain to get your code looking more and more like just a spec in the domain language.
The "don't reinvent the wheel" people who want their programmers to just bang out repetitive API code all day aren't any closer to the business logic than I am. They just don't understand what makes good software, and they don't realize that the wheels that are available to them out of the box actually suck for their task and always need some modification.
I learned a few years back that lawyers will give you a bunch of "ands" and "ors" in an expression without any concept of precedence. That doesn't necessarily mean left to right works, either.
I've noticed that the most successful business people I know are good at written communication. Much like programming often the better communicators write less to achieve more.
Eh, I think the news landscape would be improved if there was a news source providing it in poetic form. I wouldn't want everyone to do it that way.
"Source control? I don't know, this git command line is a little bit too much. Let's just use zip files and email the source"
100% is fully understood. Every single line. A very small percentage is copied from somewhere (usually adapted in some way), but even then: I understand every single line. And I expect the same of everyone on my team. If I'm doing a code review and someone says "I don't know; I found this code online" it does not go over well. I might have them delete all the pasted code and re-write it with me. If they keep pasting in code they don't understand, I may remove them from my projects.
I'm obviously a lot less strict about understanding libraries we import (although it's still important to develop a strong understanding of what they do, and their limitations, over time). The main difference is that someone else is supporting that code, while we have to support code that we pasted from "somewhere online".
I’m learning spring because my company is Java heavy and I’m hoping to do a lateral move there. The whole thing is an arbitrary rule set that the library writers agreed upon. Yes there are many concepts of OOP involved but I watch tutorials and the guy says “you need to write auth.userDetailsService and I can only think “yeah, great but how did you get to that conclusion? Where is the hidden recipe book I’m not seeing? How was I supposed to figure that out if I wasn’t following this tutorial?”
Correct, the libraries and compilers I use are not "my code". If there is a bug in some library method, I simply don't call that method and do something else instead. This may sometimes mean throwing the library away. Sometimes I will submit a bug report, if the process isn't awful (but it usually is).
I guess I don't understand why any of this is surprising to you?
For vanilla code I get it. Vanilla Java I can totally understand where things came from and why. It’s the libraries that get me. How am I supposed to know that the library expects X? Is a very common question I have.
I understand it at the end but that’s because someone’s given me the answer. But I couldn’t derive that by deduction if I wanted to because I didn’t even know that was a thing.
And all the guys giving the tutorials seem to be running off their memory but they don’t tell you how they figured it out
I am a self taught developer, and I did one a couple months back and it really opened my eyes to a lot of stuff.
Look for the one on Coursera, it's free and really good. Just actually do the assignments. It's a lot of work but really worth it.
These are polemic examples, but there is a line where sticking to the lowest common knowledge will hurt your best developers by either sucking the joy out of their work or by forcing write them a lot of boilerplate code. And I've heard of teams, where the whole team would ban features of a new standard of the language, because they've written code for 20 years without them before.
Well, we can draw the line based on who is going to maintain the code after we are gone. If we know that the organization is only going to hire rockstar ninja experts, then we can make code that is readable for rockstar ninja experts. If we know that the organization is going to hire mediocre developers, then we need to make code that is understandable by mediocre developers. If you think mediocre developers can't understand streams, then don't use streams. (Personally I think your examples of interfaces, exception handling, and streams, are all things that have their time and place, so it would be fine to use them when needed, and just expect that developers after you will know/learn that stuff.)
There's a secret third option: The organization could just hire good devs.
The worst is when they start talking about future this and that. Trouble!
And if you dig into all that code usually the designer didn't understand those tricks and patterns as well as they thought they did and were trying to teach themselves along the way.
You can often throw away 3/4 of the code, throw away 3/4 of the "requirements" which aren't being used and wind up with something that is actually simple and elegant and solves the problem, but doesn't attempt to construct anything as grandiose. Of course YAGNI is a whole lot easier in hindsight when you can really look over the course of years and see that nobody ever needed it so it never should have been built and throw it out.
But what I think the author meant is that the incomprehensible abstraction soup is at least in part a result of the "enterprise development" method. If you are solving a particular problem for a particular application you tend to solve the problem in a straightforward way as possible (if you're any good at least). But if you an "enterprise architect" you are trying to create architectures that both solve no problem in particular and all problems at once. So you tend to get complex abstractions that are both complicated to work with and don't cleanly deal with any particular implementation.
etc
Linear long functions are victims of bullying. Sure, it is a balance act, but I take bad linear code over deeply nested code any day. When trying to figure out how code works I can't keep much depth in my head, unless it is some tree walk on a data tree.
The person who knows all the "engineering" - you have this giant mutating pile of impossible to fellow (but maybe 5% faster) code.
The person who just codes enterprise style, long but done and easier to follow and edit.
Duplication is much better than the wrong abstraction
But for example, if go just beefed up the standard library (especially around data structures - give us everything - and common functions there) I think they could have pushed a bit longer in terms of holding off on generics.
(You must not actually remove any of them. Enterprise jokers will want all of it.)
- security (the most forgotten until it bites you)
- GUI flow complexity (client requirement you cannot touch)
- distributed state management with logical consistency guarantees (horizontal scaling)
- ability to quickly adapt for new features
- schema upgrade capability (see above)
- advanced database queries
- event reactivity and/or scheduling (logging, security, notifications)
None of the above is quite actually given to you on a silver platter by any one library really, it a combination thereof, and they're rather hard to grow bottom-up organically.
That's rarely up to the individual developer to decide. If you cook a complex bowl of abstraction soup, knowing that mediocre developers will maintain it after you, that's on your conscience.
So using the opportunity to learn from a better dev is a great objective that positively will impact not only you but also the company you work for.
On the other hand if developers stagnate and dont improve -> this is a red flag for company growth. And each corporation wants to grow. Such dev is just a bad hire and will tank the company.
Dont you think that someone before or after hiring of a real expert dev should check whether you really need that ?
They can always fire someone for being „too good” or like you say „decreasing own standards”.
Thankfully those experts sooner or later leave those medicore companies but they are not responsible for the mess that was left - its always the management that did not create clear expectations towards hires…
For a lot of corporations, making the jump to software is effectively starting a new company within the company.
> To me it sounds like going bankrupt is bad, so we as developers should strive to prevent our employers from going bankrupt, no?
Bad hires tank companies.
> So if you work for a startup that you know is going to hire mediocre developers after you, surely you should strive to leave behind the kind of codebase that doesn't bankrupt the company after you leave?
That's a red flag right here. It means sell your RSU as soon as you can.
We weren't talking about bad hires, we were talking about mediocre hires. It seems like you expect all companies to higher above-average developers, even though that is mathematically impossible.
No, but that's what I expect from the companies that will succeed.