This is bad advice. It's like saying "go into a bar and start picking up fights".

If some part of the software has problems, runs slow or has bugs but nobody is complaining, then there's no problem. Why waste time improving it?

Almost 100% of the time when you solve a problem you just create new problems of different kind in turn.

Be lazy. The less code you write the better off you are.

Perhaps some software development techniques would work though...

> The main value in software is not the code produced, but the knowledge accumulated by the people who produced it.

Those people go on to work on other things or for other organization. So, while that statement might have some truth to it, it's still the case that the code has to be useful, robust, and able to impart knowledge to those who read it (and the documentation).

> Start as Simple as Possible

That's a solid suggestion to many (most?) software projects; but - if your goal is to write something comprehensive and flexible, you may need to replace it with:

"Start by simplifying your implementation objectives as much as possible"

and it's even sometimes the case that you want to sort of do the opposite, i.e.

"Start as complex as possible, leading you to immediately avoid the complex specifics in favor of a powerful generalization, which is simpler".

When faced with a standard solution, use a standard component if you can. If you can't use a standard component, build a standard component. Keep your components simple, well-understood, and easy to maintain.

...While I do agree that "project-management" is important, I think the tools we are using today are really underpowered to deal with complexity/human-error - Which is the bigger problem IMO.

The problem is most CEOs see the binary as the asset, not the knowledge gained. I've tried to explain this concept to multiple startup CEOs, who hire outside development firms, for which it rarely works out for them.

Or the management techniques considered “traditional” are overlooking a century of iterative development outside of software. See Deming.

This site is an empty page without JS.

This is also the real problem with vendor lock-in.

You are more often locked in by the knowledge of your employees than by your tech stack.

What is the definition of "best engineers"? Those with extensive experience? those who follow design patterns and coding standards religiously? those who solve algorithms on a whiteboard? I would like to see if there is a definition for this.

I would say build the right culture (collaborative, always learning from mistakes and revise decisions and no blame or pointing fingers).

You can get a bunch of great coders/engineers _who follow code standards, break down codes to zillions of functions/methods ... etc_ but will fail to work together and conflicts will raise quickly.

> The project owners start out wanting to build a specific solution and never explicitly identify the problem they are trying to solve. ...

At this point, it looks like the article will reveal specific techniques for problem identification. Instead, it wraps this nugget in a lasagna of other stuff (hiring good developers, software reuse, the value of iteration), without explicitly keeping the main idea in the spotlight at all times.

Take the first sentences in the section "Reusing Software Lets You Build Good Things Quickly":

> Software is easy to copy. At a mechanical level, lines of code can literally be copied and pasted onto another computer. ...

By the time the author has finished talking about open source and cloud computing, it's easy to have forgotten the promise the article seemed to make: teaching you how to identify the problem to be solved.

The section returns to this idea in the last paragraph, but by then it's too little too late:

> You cannot make technological progress if all your time is spent on rebuilding existing technology. Software engineering is about building automated systems, and one of the first things that gets automated away is routine software engineering work. The point is to understand what the right systems to reuse are, how to customise them to fit your unique requirements, and fixing novel problems discovered along the way.

I would re-write this section by starting with a sentence that clearly states the goal - something like:

"Paradoxically, identifying a software problem will require your team to write software. But the software you write early will be quite different than the software you put into production. Your first software iteration will be a guess, more or less, designed to elicit feedback from your target audience and will deliberately built in great haste. Later iterations will solve the real problem you uncover and will emphasize quality. Still, you cannot make technical progress, particularly at the crucial fact-gathering stage, if all your time is spent on rebuilding existing technology. Fortunately, there are two powerful sources of prefabricated software you can draw from: open source and cloud computing."

The remainder of the section would then give specific examples, and skip the weirdly simpleminded introductory talk.

More problematically, though, the article lacks an overview of the process the author will be teaching. Its lack makes the remaining discussion even harder to follow. I'll admit to guessing the author's intent for the section above.

Unfortunately, the entire article is structured so as to prevent the main message ("find the problem first") from getting through. As a result, the reader is left without any specific action to take today. S/he might feel good after having read the article, but won't be able to turn the author's clear experience with the topic into something that prevents more bad software from entering the world.

Why?

Because there is no formal definition for what is bad or good software. Nobody knows exactly why software gets bad or why software gets good or what it even exactly is... It's like predicting the weather. The interacting variables form a movement so complex that it is somewhat impossible to predict with 100% accuracy.

What you're reading from this guy is the classic anecdotal post of design opinions that you literally can get from thousands of other websites. I'm seriously tired of reading this stuff year over year rehashing the same BS over and over again, yet still seeing most software inevitably become bloated and harder to work with over time.

What I want to see is a formal theory of software design and by formal I mean mathematically formal. A axiomatic theory that tells me definitively the consequences of a certain design. An algorithm that when applied to a formal model produces a better model.

We have ways to formally prove a program 100% correct negating the need for unit tests, but do we have a formal theory on how to modularize code and design things so that they are future proof and remain flexible and understandable to future programmers? No we don't. Can we develop such a theory? I think it's possible.

If someone suggests you focus on the 20% of customers who make 80% of your revenue, and you run the numbers and find a 75-25 distribution, should you call the person making the suggestion an idiot?

In my experience, developer-side evaluation has a very low impact (I was about to write: zero) on the perceived and actual goodness of the software itself. Which is tied mostly to factors such as user experience, fit to the problem it was designed for and to the organization(s) it is going to live in (user experience again). These properties do not strike me as amenable to algorithmic improvement, no more than "pleasant body lines and world class interiors" in the original car analogy. But they are a (big) part of good software design, besides being the 'raison d'etre' of the darned thing to begin with.

But let's forget cars, as hard as it is. Few months ago HN was running the story about developing software in Oracle. Now, Oracle may be by now a little soft around the edges, but I think that most would agree that it has been setting the standard for (R)DBMS for decades. Success may not on itself be the tell-all measure of software goodness, but the number of businesses that have been willing to stake the survival of their data on Oracle is surely a measure of its perceived goodness (as that other elusive factor - hipness - tends not to be paramount in the DBMSs business).

The development side story, taken as face value, was pure horror (https://news.ycombinator.com/item?id=18442941). Everything in it spoke bad, outdated, rotting design. The place must be teeming with ideas on how to improve just about everything in that environment. And yet if that came to be, maybe by some nifty edge pruning algorithm, it would do nothing to improved the goodness-to-the-world measure of the software, not until the internals' improvement translated to observables in the user base experience.That type of improvements will still require vaste amount of non-algorithmic design and, in the meantime, a very concrete risk will be run of deteriorating the overall user experience (because ehi, snafus will happen).

This (internals are just a small part of the story) is one of the reasons why so many reimplementations I have seen failed ("ehi, let's rewrite this piece of shit and make it awesome") and the reason because everyone resists the move from IPV4 to IPV6. I could think of many more examples.

You should seek to demonstrate instead that you're making software that is more malleable, has less bugs, is easier for new hires to understand, is easy to add new features, etc.

(PDF available here: http://pespmc1.vub.ac.be/ASHBBOOK.html )

Cybernetics might be the "missing link" for what you're talking about.

It might be a team culture or company culture issue, and even radical changes in the management are not enough to fix it.

I mean, the point of short quotes is to be memorable and get future listeners to hunt for the reason behind them. "The sun will rise tomorrow" may also be meaningless for some people on its own.

Nothing wrong with elaborating on this subject again via a blog post, I was just pointing out to the commenter who's never heard this expressed before that it has a long history, that's all.

Are you referring to something like Knuth's Literate Programming (en.m.wikipedia.org/wiki/Literate_programming)? As a non-professional who's learning to develop on the side, something that follows more of a natural language approach appeals to me, as sometimes I have a few months between working on my project, and comments on my source code help me not to forget why I do certain things in the code. However, I'm not doing Literate Programming, just python with comments.

2) In my experience, this basically never happens.

Your comment encapsulates a lot of what I have come to call "Scrumbutt." It's Scrum, but. And while I have no idea if it's intended on your part, the sentiment is a fantastic way for a Scrum consultant--only some shade thrown; I've been a "DevOps consultant" before, after all--to come in and pull from deep in their Scrumbutt something to the effect of "you're doing it wrong, Scrum has not failed, you have failed Scrum."

Within epsilon of nobody does Scrum "as prescribed"--because the amount of responsibility that must be undertaken at all levels is virtually impossible to get full buy-in on--and as such the boil on our collective behind that it is persists because criticism is immediately bedeviled by Scotsmen of unknown provenance.

Back to the question - what do you do about poor knowledge transfer in a project? I think a moderate de-emphasis on thinking of the user story text and the additional info like acceptance criteria etc. as self-sufficient documentation and adding more emphasis on that close relationship between developer, user, and maybe a tester, can help fill in big knowledge gaps.

And probably he was the best of what followed as well, so this literacy thing didn't go as well, where power figures were concerned...

So I think "working code/application/program" is when it does what it is supposed to do. Including not crashing.

Me too, but I still feel that saying "documentation quickly becomes outdated" and refusing to write any, is not that different from saying "software quickly becomes full of bugs" and refusing to write unit tests. Yes, if you believe that something is doomed, and therefore you refuse to even try, it becomes a self-fulfilling prophecy.

Yes, documentation quickly becomes outdated, if no one updates it. Duh. If a person creates/modifies a part of code, they should also create/modify the corresponding documentation accordingly. (And the person reviewing the code should also review the docs.) If you don't do it, then yes, obviously, the documentation becomes outdated. Did you expect it to update magically by itself?

If you believe that documentation is useless in principle, go ahead and don't write it. Then you won't have to maintain it. Also, make sure to include memory tests to your interview process. If you believe that documentation is useful, write it, and maintain it. But if you have a documentation that you never update, you get the worst of both worlds.

At the very least, the document should capture high-level (again this is relative term) design, possibly an architecture diagram of major interacting functional units. The success measure should be the relative ability to build a mental model of the system by looking at this document for any newbie.

That design document would be a start, and most likely not "quickly outdated".

My personal beef with the agile camp is precisely this: when they let go of documentation, they don't do the design doc as well, and all that remains of the system is thousands of incoherent stories and huge amount of code.

Another thing that helps is to write good commit messages giving the business context for a change. When code is reviewed, the commit messages should be reviewed as well. If they don't agree then that's a problem.

I like being partially or fully nude in the home, occasionally chewing gum, and having the right to freely criticize or endorse ideology on its own merits; even if it sometimes sucks to see dirty black spots on the sidewalk, or to hear people making weak arguments just to upset eachother.

Certainly a weird cosmopolitan fascist (u|dys)topia.

And, if you are on the wrong side, it is very "effective" at ruining your life.

Most of us would take a bit less "effective" in order to avoid that, thanks.

But in this case I think 11 Messi's would win everything there is to win in football for a decade straight.

Effective just means it achieves the intended outcome, it's not a value judgment on the goodness of that intention.

That sounds like the happy path for dealing with the Canadian government. Well, except the months part.

If you want great devs, you're going to have to invest in junior devs, and you're going to have to expect them to be learning at work. This is also why the best use of your senior devs is as mentors to your less experienced ones.

    Documentation is essential. How things work is an important thing to 
    document. Ideally it should be in version control and be generated 
    from the code, because then it's less likely to go out of date. 

Generally, this falls into two categories.

1. Hacks/kludges to get around bugs in hardware, external services, or included libraries. These manifest in code as incomprehensible, ugly bits of code that are difficult to distinguish from code that is simply "sloppy" or uninformed. More importantly, they represent hard-won knowledge. It often takes many programmer-hours to discover that knowledge, and therefore many dollars. Why throw it away? (Tip: include the version of the dependency in the comment, ie)

    # work around bug in libfoo 2.3, see blahblahblah.com/issues/libfoo/48987 for info
    # should go away once we can upgrade to libfoo 3..
    if error_code == 42 reset_buffer()

2. Business logic. This too is difficult/impossible to discern from looking at code. Often, one's git commit history is sufficient. But there are any number of scenarios where version control history can become divorced from the code, or require a fair bit of git/hg/svn/whatever spelunking to access. And this of course becomes increasingly onerous as a module grows. If there are 200 lines of code in a given module, it is a significant time investment to go git spelunking for the origins of all 200 lines of code. Some concise internal documentation in the form of code comments can save an order of magnitude or two of effort.

    It still has problems (What do you do when the code and the 
    documentation disagree? Which is correct?), but they're not as 
    severe as the problems that arise when there is no documentation at all.

In the first place, only a true maniac would intentionally update

    # no sales tax in Kerplakistan on Mondays
    return nil if country_code==56 and day_of_week==1
 

I agree with this 100%. However, to be useful it needs to hit the right level of crudity. For most projects, a short (<10 pages) description of goals, design principles, architecture and an overview of interfaces is sufficient.

It is best when this exists as a standalone document which is a required reading for any new developer. After this they can look at module descriptions, function docs, code, etc. and understand how to make sense of it and how to add their code without breaking general principles of the project.

> Ideally it should be in version control and be generated from the code, because then it's less likely to go out of date.

With this, I have some beef. In my experience the best documentation is the one that complements the code. Usually this means a short description by a human that explains what this chunk of code does and assumptions or limitations (e.g., "tested only for points A and B in troposphere") and IME most useful information is not derivable automatically. Auto-generated docs are very useful, but cannot replace clean explanations written by a human. My 2c.

Interestingly, this has been a big point of discussion in the Dota 2 playerbase. Dota 2 is one of the most complex games ever created and it rapidly changes on the order of days or weeks. At one point, the in-game descriptions of spells were months or years out of date because they were being updated manually. After much hue and cry from the community, the developers finally made the tooltips get generated from the same code that determined the spells' effects. Things are a bit better now.

There is still a quite a bit of ways to go though, in terms of generating documentation for all the other mechanics in the game, which are crucial for gaining competency in the game, but which are only available due to third-party community efforts (often via people reading the game's codebase to understand subtleties), instead of being available inside the game.

Not always - when you want to document the requirements (in whatever format), having them be separate from the code is often a plus. The code might implement the requirements incorrectly, so being able to recognise that is important.

I find this very similar to writing tests that are separate from your implementation. In fact, Cucumber/BDD tests try to make product requirements executable to validate the software has been written correctly to meet the requirements.

I never got documentation about the thought processes, the iterations, the design meeting, the considerations, etc. Which is way, way more important to understanding a system in context than knowing "convertLinear" takes 2 unsigned ints.

That doesn't sound too bad from a dev point of view, better than the opposite - half arsed specifications with no thought given to the important details. Though I can imagine a lot depends on what exactly you are trying to build.

> Ideally it should be in version control and be generated from the code ..

May I ask if you have suggestions for tooling to capture the high level documentation. We use javadoc a little, but it seems best for lower level reference. Also for diagrams, like sequence diagram and/or state machines, how do you capture this?

Thanks.

I worked on an enterprisey line of business app that assigned sales leads to salespeople.

The algorithm to do this was a multi-step process that was (1) rather complex (2) constantly being tweaked (3) very successful (4) contained a number of weighting factors that were utterly arbitrary even to veterans of this app.

It was full of many `if country_code==KERPLAKISTAN && day_of_week==MONDAY` -style weighting factors. Each represented some hard-won experience And when I say "hard-won" I mean "expensive" -- generating leads is expensive business.

We had a strong culture of informative commit messages, but this file had hundreds if not thousands of commits over the years.

It was the kind of code that resisted serious refactoring or a more streamlined design because it was a recipient of frequent change requests.

A few human-readable comments here and there went a loooong way toward taming the insanity and allowing that module to be worked on by developers besides the original author.

Knowing the why for many of these rules made it much easier to work with, and also allowed developers to be educated about the business itself.

[NB 30 isn't an exaggeration - I think the vast team who wrote it were paid by the abstraction or something].