On Coding, Ego and Attention

On Coding, Ego and Attention(josebrowne.com)

549 points by digitalmaster 5 years ago | 170 comments

camgunz 5 years ago |

I get and agree with what the author is saying here, but I also think a big part of this is that in software engineering, so much of what we do is ephemeral. If you're a carpenter you'll know if you're good or not. You'll be able to do stuff like frame a house, replace a door, etc. And then when someone asks you how long it will take to frame a house, how much it will cost, what supplies/staff you need and so on, you'll be able to say.

We've been doing CRUD in our industry for decades. How can we not just say "this is how you do CRUD, we're done w/ that now". We've been doing data serialization for decades now. How can we not just say "this is how you serialize"?

There are communities where this is the case. Why have we abandoned them? Why have we abandoned that knowledge and experience to reimplement things in language X or using platform Y?

We might not like to hear it, but my guess is it's a culture problem. They say the way to get ahead at Google is to build a new successful product. Is that the same thing we're doing? It's easier to get ahead by building a new Z framework than to become a core committer on X framework from 10 years ago? Are most X frameworks run by toxic communities? Is there something specific about software that means tenured projects become less and less useful/maintainable/understandable over time?

There's something in here that's specific to SWE. I don't know exactly what it is but, I think we should figure it out.

themacguffinman 5 years ago | |

CRUD and data serialization is an incredibly broad and diverse field. It's like saying "we've been moving our hands and feet for centuries, how can we not just say: this is how we move our hands and feet?"

Software has so many more possibilities to explore than carpentry which is constrained by our current physical technology. It's far better to encourage engineers to explore these diverse possibilities than to encourage conformity and allegiance to some singular path that everyone is supposed to agree on and work towards. You would simply miss a lot of different innovations by grinding away on the same path. Communities that do so just stagnate.

brainless 5 years ago | | |

This is my way of thinking: 99.99999% of applications out there still will store their CRUD into a standard relational DB and run on standard operating system with standard protocols.

Sure you can create a lot of fuss all around it, but I feel we create a lot of fuss because of ego, because we want to be perceived that we came up with new ways.

The reason to not conform is ego. Software is perhaps the cheapest ego boosting tool ever created.

saghm 5 years ago | |

Humans have been constructing houses and doing maintenance on them for a few thousand years, but we've only been writing software for a few decades. We certainly didn't reach out current process for framing houses in the first few decades of carpentry.

That being said, I assume that the first few decades of carpentry didn't undergo as many changes as software has in its first few decades. My theory is that software changes so quickly because it can be bootstrapped. When framing a house, you can learn from the process so that you can make the next frame better by changing the process, but the output of that process (the house frame) doesn't directly affect the next time you attempt it. On the other hand, you can write software that invents an entirely new process for editing software (e.g. a compiler or interpreter), which then you can use to write software that might not have been possible before. You can then repeat this process with the new software, creating yet another paradigm for writing software, and so on. More generally, when a process produces a tool that can then be used for to do the process in a new way, the process will be able to evolve much more quickly than if updating the process can only be updated with the output from other processes.

DavidVoid 5 years ago | |

> They say the way to get ahead at Google is to build a new successful product. Is that the same thing we're doing? It's easier to get ahead by building a new Z framework than to become a core committer on X framework from 10 years ago?

A Kurt Vonnegut quote comes to mind:

"Another flaw in the human character is that everybody wants to build and nobody wants to do maintenance."

seangrogg 5 years ago | | |

I think the reality is that some people are actually just fine doing the maintenance - but they're unlikely to boost their career/paycheck by doing so comparable to what they'd have gotten from making a new thing instead. And that's an issue.

I'd love to go back to old code with the benefit of deeper domain knowledge and greater understanding of my tools and be able to make products even better. However, it's hard to square that against making +20% earnings by helping build a new chat app.

closeparen 5 years ago | | |

Aside from cleaning and lubrication, a lot of "doing maintenance" is still throwing away old material and bringing in new. Just being selective about exactly which part is at the end of its duty cycle.

People talk about it like there's something wrong when, at any given time, a few microservices are being rewritten. But I would expect that for a sufficiently large machine, on any given day a few parts are being replaced.

scruple 5 years ago | | |

Yes, but... Job security in this industry boils down to little more than evolve or die.

kortilla 5 years ago | |

If someone wants a website that lists their company hours and has a contact form, that’s pretty much a known amount of hours for an experienced web dev. That’s about the equivalent of asking a carpenter to put in a door.

If someone wants a custom built order and inventory management system, that’s like asking a carpenter to build a custom 4 story house from some napkin sketches.

The whole reason computers are valuable is because they automate away all of the rote, repeated, predictable stuff. The unpredictable part of SWE is not comparable to carpentry, it’s more easily compared to architecture/engineering where the problem statements are vague and most of the job is getting agreements on what the thing will actually be. The carpentry part of programming is mostly predictable.

TeMPOraL 5 years ago | | |

Agreed. Or in case someone brings up the good ol' "civil engineering" analogy - programming isn't like constructing a bridge. Constructing a bridge is what compilers do. Programming is the design and engineering that results in a blueprint. And our occupation is unique in that the construction part is so cheap, we can design iteratively, instead of actually thinking about what we're doing.

pingyong 5 years ago | |

>There's something in here that's specific to SWE. I don't know exactly what it is but, I think we should figure it out.

It's changing requirements. When you build a house, people don't come in 6 months later and ask you if you could make one small change by placing some jet engines on the walls so the house can fly somewhere else during the summer. It's just a small change, right?

The problem is that in code, it often is a small change. Or at least, it is possible to make one quick adjustment to satisfy this new use-case. But often, these small changes are also a hack which doesn't fit into the previous overall design and which would've been implemented in a completely different way had the requirement been designed for in the first place. Now, one of these "small changes" don't tend to kill the product, but years or even decades do. That's why refactoring exists in software engineering, but not really in home building. Well, in some sense it does exist by renovating. But nobody thinks it's a good idea to completely renovate a house 25 times around an architecture that just doesn't work anymore for what it's being used for.

If you build a piece of software for exactly one well specified use case and only use it for that, it'll probably run really well forever. But (almost) nobody does that.

colordrops 5 years ago | |

The differences between carpentry and software engineering is that the problem space in carpentry is much smaller and pretty much static over time. It's rare for carpentry tools to get an order of magnitude more powerful over the course of a decade, and for 100 people to work on the same carpentry project.

rumanator 5 years ago | | |

> The differences between carpentry and software engineering is that the problem space in carpentry is much smaller and pretty much static over time.

Anyone is free to compare software development with any engineering field, which typically have to solve large problems.

Thus if you feel carpentry is not a good comparison them look into civil engineering.

And no, the key factor is not the 'power' of software tools. The key factor is stuff like processes and standardization.

Sometimes it feels like software developers struggle or even completely oppose adopting and establishing processes and standards of doing things. Hell, the role of software architect is still in this very day and age a source of controversy, as is documenting past and future work.

We're talking about a field that officially adopted winging it as a best practice, and devised a way to pull all stakeholders into its vortex of natural consequences as a way to dilute accountability. The field of software developme t managed to pull it off with such mastery that even the document where the approach is specified is not a hard set of rules but a vague "manifesto" that totally shields their proponents from any responsibility of its practice delivering poor results.

If an entire field struggles with the development and adoption of tried and true solutions to recurrent problems then it isn't a surprise that basic problems like gathering requirements and planning is something that is still the bane of a whole domain.

fingerlocks 5 years ago | | |

The availability of inexpensive CNC machines and 3D printers are carpentry tools that have certainly bolstered productivity in the last 10 years. Probably not “order of magnitude more powerful”, whatever that means, but as one very successful carpenter friend put it: “I don’t even fuck around with table saws anymore”.

By contrast, I’m still writing code more or less the same way I was 10 years ago, with mostly the same tools, and have not seen “order of magnitude” level of anything contributing to my productivity.

l0b0 5 years ago | | |

Basically, this and other comments show that the analogy completely breaks down. The scales, changes in scales, and degrees of freedom are just utterly different from anything physical humans build.

harikb 5 years ago | |

> It's easier to get ahead by building a new Z framework than to become a core committer on X framework from 10 years ago?

Sometimes, yes.

This particular angle is explained in the article

>> This is ego distraction in action. Self comparison determining effort. If we feel like we’re ahead we continue to put in the effort. If we feel like we’re not, we determine it’s not worth the effort.

The reason people would prefer working on a newer project/framework/whatever is that there is a higher chance they might be able to contribute meaningful code / support. I am admitting to that, and I am sure many have similar thoughts. It is purely guided on where one thinks success is achievable.

Also keep in mind - progress is being made. Python is clearly more productive that Perl. Or Django vs CGI/FastCGI. So 15 years ago, if that were my two choices for two projects, I would have taken the path of Python. Not just because it was new & shiny then.

Fast forward a decade, Go is clearly more productive than many things that came before. Kafka is clearly easier to manage than home-grown queues via databases and flat files. So why should I stick to old process?

The problem I feel is lack of arriving at any standards for anything basic. We have 10 message queues, but limited interoperability. We have 50 popular databases, but no easy migration. We don't even have universal support for Parque in all languages even though it has been around for a while. When can I grep a parque file? Something as simple as Azure blobstore and Amazon S3 can be linked together without arcane and inefficient copying.

megameter 5 years ago | | |

One of the biggest difficulties of ego in software comes from the difficulty of finding "the ground".

New languages are popular. Why are they popular? "Because they are better." But in every other domain of software we also say "The best technology doesn't always win." Why would languages be any different? What if Go is, in fact, Worse is Better? And if it's a Worse is Better, then what is the Right Thing?

Ultimately, I think most programmers, given enough experience, eventually settle on a style and propel the style through the language, not the other way around. And to that end, there can always be new languages so long as there are styles of coding that remain unaddressed.

But this is counterbalanced by the assumption of a rationalist project existing: that code is made to be shared, and to be shared, it must be standardized.

If one looks at the hardware/software ecosystem, it is not rationalist in the slightest, though. It is a merciless field of battle where vendors manuever against each other to define, capture, control, and diminish standards. The small ones seek to carry a new standard; the large ones absorb the standard into their empire.

Software bloat is a result of this: everything must go through a compatibility layer, several times, to do anything. Nobody understands the systems they use. With each wave of fresh grads, another set of careers is launched, and they join in on the game and add more to the pile.

In that light, rational standards do not exist. They are simply the manifest ego of "us and them", and therefore are mostly a detriment for all the reasons that ego is a detriment.

There exist several examples of excellent feature scaling from small codebases: VPRI STEPS, various Forth, Lisp, and Smalltalk systems, project Oberon, and microkernels such as Minix. The quality they all share is an indifference to standards: they are sometimes used where convenient, but are not an object of boasting.

Therefore I currently believe that developers should think of standards as reference material, not ends in themselves - that is, you use one if you can't come up with a better way of getting the result.

AlexCoventry 5 years ago | | |

Did you mean Apache Parquet? If not, what is Parque?

arkadiytehgraet 5 years ago | |

From my perspective, the root cause of this problem lies in lack of one common measure for code quality, correctness and usability or even programming in itself.

Say, there are two approaches for a problem - how do we decide which one we go with? In the last 10 years I have not seen a single case where the decision was made based on something other than subjective opinions of a person or a group of people. "Past experience", "this is how it's done here", "this is the only way I can do" and countless other reasons - all of those are subjective and cannot be used for objective comparison of approaches.

You could say, "days to implement" or "money spent" is such metric - but then, there are no reliable ways to mathematically calculate this measure for any code you plan to write and then prove it in advance.

To put it another way - there is no standard unit of code/system correctness, by which we could have measured what we are actually doing or plan to do. Until one emerges, we are bound to continuously reimplement same things over and over again, justifying it by nothing else than our projections, prejudices and boundless ego.

napicella 5 years ago | |

I agree it's a culture problem - we developers can't agree on anything even when someone else already went to the trouble of defining a standard. I also think there is another component which is inherently related to the software engineering profession: technology move fast and some things are indeed worth adopting because they are beneficial in the long run, even if it means reinventing the wheel or having to re-learn something from scratch. But understanding which is which it's not that simple. Every time I start a new project in the team, we need to learn a new way to deploy, to instrument the code for metrics, a new integration test framework, the new features of the the CI/CD pipeline which replaced the old ones, maybe a new framework or even a new language. This is even before writing any meaningful code. How much of the new stuff is an improvement, rather than just a slightly different flavor of the old stuff?

TeMPOraL 5 years ago | |

> Is there something specific about software that means tenured projects become less and less useful/maintainable/understandable over time?

Complexity. Understanding a legacy codebase is pretty much a small-scale research project. You need to gain domain knowledge, become familiar with the team, get acquainted with the codebase and its history, before you'll be able to reliably tell bad code from clever solutions to tough problems. The longer a codebase is developed, the more is there to learn and retain in your head. It very quickly becomes just too much, which means onboarding people takes a lot of time, and day-to-day development also involves being extra careful, or creating obscure bugs - both of which make the project take longer.

Yes and no. Not every one of us plays the office politics. Some of us code because we like it. The yardstick then is one of personal growth, the ability to comprehend and build increasingly complex, powerful and beautiful systems, or automate mundane things faster and faster.

But, regardless of the "core drives", one thing is true: building a system from scratch is a much faster way to learn about the problem domain than trying to understand someone else's system and maybe contributing a patch somewhere. We learn by doing. That's why there's so many half-baked libraries for everything out there. Yes, there is ego involved - particularly for people who go out of their way to make their half-baked libraries seem production ready - but a big part of the picture is still that programmers learn by writing code and building systems.

(The difference from most other professions is that people there can build stuff xor share stuff - not both at the same time.)

derangedHorse 5 years ago | | |

I disagree that the cause of the problem is complexity stemming from size, and propose that the real issue is the industry's poor history of efficient documentation. Processes to efficiently create and read documents that describe large systems are rarely in place at most of the places I've seen. That's probably the biggest barrier to contributing code to old framework Y. It's just easier to develop framework Z. I agree with you that some things can be designed to reduce complexity, but ironically whenever something like this happens, someone from an older product will glean ideas of the new one and port some of those concepts over (potentially further proving that the big problem is the lack of resources for understanding)

skybrian 5 years ago | |

Technology changes and user expectations change, and we need to adapt.

And it's not my area, but this seems to be true in construction as well? The building codes change, and available materials and components change, as do their relative prices. Maybe not as fast, but fast enough to make older books out of date.

mdorazio 5 years ago | | |

Have to disagree with most of this. Technology changes and user expectations change, but there’s a missing link here to show that either of these really necessitates Yet Another Language/Framework, launching Yet Another Product/Service, or rebuilding things from the ground up. It’s a bit like a homeowner wanting an updated kitchen and a contractor telling them they need a whole new house for it to work, when really the contractor just prefers building flashy new houses for their portfolio over doing renovations on a budget.

Also, side note: with respect to carpentry, books from 50+ years ago on wood working techniques, framing, joinery, etc. are perfectly relevant today. And many of my grandfather’s tools are still in use in my workshop.

oarabbus_ 5 years ago | |

>We've been doing CRUD in our industry for decades. How can we not just say "this is how you do CRUD, we're done w/ that now"

As an analyst, can you explain this bit?

I keep hearing things like "that's not actually a software development job, just CRUD", "we're done with doing CRUD" etc. But it seems like between the application and the DBA all the CRUD is taken care of, wouldn't the developer just work on the application itself? And isn't saying "we don't do CRUD anymore" somewhat akin to saying "we don't do [+-*/] anymore"? How can you have persistent data without CRUD? I must be missing a piece of the puzzle in this discussion.

skybrian 5 years ago | | |

It's a reductive, dismissive way of thinking, like saying that everything is ones and zeros, or that we're just copying protobufs around.

The data that we manipulate has business meaning and there are consequences for the users that arise from how we model things. Consider the genre of articles like "Falsehoods Programmers Believe About Names" [1]. There is ridiculous complexity here, for those willing to see it, but some people get tired of it.

[1] https://www.kalzumeus.com/2010/06/17/falsehoods-programmers-...

bradstewart 5 years ago | | |

Not OP, but my take: When people talk about "CRUD" in the way you describe, they're usually talking about one of two separate (but related) things.

The "it's not _actual_ development" framing is usually directed at applications which "only" allow users to perform basic actions on some data, basically UIs for manipulating a database. It is absolutely real development (in my view), but less sexy than AI/ML, big data, etc, etc.

You are correct that every application (with some sort of data persistence) needs CRUD. But how CRUD is implemented, for better or for worse, depends on the requirements of the application storing the data. For (most) relational databases, the low-level "how do I CRUD" is well defined: standard SQL queries. But if I use NoSQL, or flat files, or something; it changes.

The definition of CRUD also varies depending on the layer of abstraction within an application or the perspective of the user/developer. For example: from a DBA's perspective, CRUD is SQL queries. From a UI, CRUD might be a JSON API or GraphQL endpoint. From a server-side application, CRUD might be a specific ORM library.

thruflo22 5 years ago | | |

Yeah, CRUD is a solved problem but you still have to do it.

Mapping state to the database is to web dev what applying paint to the canvas is to painting. It’s how you do it that counts. Saying otherwise is overly reductionist.

Frameworks exist that abstract CRUD away. But you end up sacrificing UX and / or flexibility.

pydry 5 years ago | | |

CRUD is looked down upon because it's time consuming and repetitive when you do it with poorly designed tools and because it's the most common role.

I think it's mostly a class thing though. Test automation is similarly looked down upon even though it is often much harder to do right than regular coding.

There is a definite pecking order when it comes to programmer roles and it's not necessarily related to difficulty (although it correlates very strongly with pay).

peterkos 5 years ago | |

I remember reading an article that studied junior and senior devs and discovered that there was no way to get better at debugging. No matter how much experience someone had, their ability to problem solve was about the same.

I think that might have to do with this complexity, but also: software has so many ways of doing something, even within the same language -- and that gets permuted across, say, five different languages (Python, Rust, PhP...). It's impossible to say the "right" way to do it because there are multiple ways to achieve a valid result that's readable, AND there is a margin for disagreement between what is "readable".

Arelius 5 years ago | | |

I feel this needs better context, because besides not being able to prove a negative, debugging is so much beyond only the essential ability to "problem solve". And as an anecdote, I've certainly gotten significantly better at debugging with my experience among many aspects. For instance the ability to recognize a somewhat common bug based on it's symptoms is something that at least within a certain context, improves with experience and is at least to some degree "getting better at debugging"

travisjungroth 5 years ago | | |

I’d love a link to that article if you can find it. I wasn’t able to on my own.

I was just thinking today about how to teach someone to be better at debugging.

thrwaway2892 5 years ago | |

Well written article, well written response. Sometimes us humans think that perceived improvement of conditions = improved conditions. This is false, but as a business guy calling the shots, my goal is to do the thing on paper in front of me by the deadline whatever the costs. Combine that with a developer's creativity and you get a new framework.

artsyca 5 years ago | |

I'd venture it's not so much the technique behind the individual layers but the understanding of the need for all the layers and their interactions and the best practices in given situations.

We're prone to tediously repeat the same conversations over and over and take the cosmetic approach rather than the fundamentals-first way of doing things.

forgotmyp77 5 years ago | |

i think it's all there if you commit to using tech more than 10 years old.

tantalor 5 years ago | |

this is how you serialize: https://developers.google.com/protocol-buffers

abvdasker 5 years ago |

In my experience, the best way to insulate yourself from the pangs of imposter syndrome and unproductive self-doubt is with experience of past successes.

Early in my software engineering career I would constantly and painfully wonder if I was actually capable of fixing a certain bug or solving a new or difficult problem. But then after working hard on a solution, 99% of the time it would work out. After going through this process of debilitating self-doubt and eventual success over the course of years, it has become much more manageable.

I still sometimes panic when initially faced with a very difficult programming problem, but I can put those fears to rest much more easily by saying, "ok, I've solved hard problems before. I may not know how to solve this particular problem yet, but I feel confident that I will be able to figure it out just like I did in the past with difficult problems X, Y and Z."

At the risk of sounding pedantic, part of leaving the beginner phase — and the true value of experience — is developing a kind of armor against those feelings of inadequacy (of course you don't want this to go too far into feelings of overconfidence or an inability to reflect when things do go wrong).

I also think it's the responsibility of more senior engineers to recognize when a more junior teammate might be having those self-doubts and be empathetic while helping them build up their own successes.

halfcreative 5 years ago |

I feel like for me, reading and writing articles like this are a major source of distractions. I often find myself reading articles like this, lessons of self-improvement and tips of motivation and ways to be a better programmer, instead of doing what actually makes me a better programmer (actually programming). To extend upon the article, I feel like one of the easiest distractions from self improvement is constantly reading about self improvement. Not to say that the lessons in these articles are a sham, but that there's a point where the idea of and dream of improving yourself becomes a dangerously stealthy distraction.

lostmyoldone 5 years ago |

Big tired after an intense day, and I haven't really sat down and digested the article, but it seems close enough that I think my - admittedly primitive - tip can be of relevance.

If you get stuck, you tell yourself in whatever way you want, and honestly, some version of the following: "I don't understand this thing that is happening, but I know there is a cause. It does not happen without cause."

Honestly, it's a bit odd, and I don't know if that's the best way to express it in English. Nevertheless, several people have some back to me and told me that it has helped them.

My initial inspiration, and hypothesis is that the simple acknowledgement that I don't understand the problem, and that the problem still - despite my lack of understanding - still follow the laws of cause and effect, somehow temporarily halts our brains tendency to protect our ego at almost any cost, logic be damned.

I started trying this out after puzzling about why it's unreasonably common to figure out the answer to something only moments after you get up from your desk to go ask someone else for help, even when you might have worked with it for hours. It had to have a reason, although I don't know exactly what it is!

ak217 5 years ago |

I love the introspection and positive attitude in this post.

One thing I would add is that intrinsic motivation seems to be framed and activated very differently depending on one's personality. I find myself performing best when I'm on the edge of failure, trying to catch up to the high performers, and when recalling past times when I overcame failure or adversity. Comparing myself to the group described as demotivating in the post is the best motivator for me. And then there are little tweaks to one's environment (for me it's coffee, exercise, occasional travel, specific movies and music) that I find end up making an enormous difference in motivation, focus, and overall mental state. I suspect this has a lot to do with personal physiology and the environment in which you grew up.

With that caveat, the post is incredibly thoughtful and helpful, and I really enjoyed reading it.

ericmcer 5 years ago |

The part about problems being either fun challenges or a nightmare really resonated with me. That's why pair programming is so important in my opinion, if you work too long in a silo the magic, fun, craft whatever of programming fades away. Just watching a coworker code or talking through problems with one can bring back that spirit of fun challenges though. Engineer morale is super important.

geoffbp 5 years ago | |

Agreed, sometimes a second pair of eyes or rubber duck programming does wonders

digitalmaster 5 years ago | |

100% Agree.

rubicon33 5 years ago |

I read this, and I have also read "The Practicing Mind" which is exactly what this post is about. My issue isn't in understanding the premise. It all makes sense, and I get a sense of "ah hah!" every time I read it (I've read it twice).

The issue for me is that I really struggle to turn this theory in to effective practice. Each time after reading "The Practicing Mind" I have tried to cognitively remind myself whenever I was frustrated, to stop and look at the problem as a beginner would, to drop my ego, etc.

The problem is that it would sort of help, temporarily. I'd find myself a little bit better at getting a solid day of work done, but not dramatically better. After a week or so, I'd forget to even do the exercises, and I'd be back to struggling.

What honestly helps more than anything, the "magic bullet" really is pharmacology (aderall). For me, it somehow calms me down. I don't feel more energy, I feel tranquil, and able to let defeat roll off my shoulders.

Sadly, taking aderall is not a sustainable solution. Amphetamine is a neurotoxin which raises blood pressure. Not to mention, I don't like being "tranquil" for anything other than my work. I like my 'normal' state of semi-uncontrolled energy, which is great for exercising and video games. I'd like to be able to turn this feeling on or off, and taking a medication doesn't allow for this.

So I tend to see saw between three states... 1) Struggling at work, barely getting by, quality of life sucks. 2) On medication, happy at work, feeling productive and peaceful, but desire to get off medication 3) Off medication, using "Beginners Mind" but find my ability to implement it in a way that is strongly effective, absent.

pistachiopro 5 years ago |

This post did not reach the conclusion I was expecting based on the title. For me, I think I've largely experienced the opposite relationship between ego and my programming productivity.

Learning to program as a kid was probably one of the most exciting developments in my life up to that point, and I expect that's true for many people on this forum. I originally attributed this to programming's usefulness, and the mathematical beauty of watching all the pieces fall into place when solving a problem. And those were surely both important motivators, but, looking back, the primary motivator was the pure power trip of it. Programming is extremely powerful (software is eating the world, after all), and I could immediately sense that, and that power was the biggest high I got from it.

Throughout my teens and twenties, I didn't really consider this, and just followed the high, and it led me to develop skills and a successful career as a programmer. For me, it was a positive feedback loop, where the more I put into programming, the better I got, and the bigger the ego boost. Unfortunately, though unsurprisingly, it got to a point where my inflated ego started getting in the way of my personal relationships, and even my self perception. I considered myself a great programmer, but not a very good person. I became quite self-loathing for many years, but I've noticed that's healed up after moving away from programming as a primary job responsibility, and my personal relationships have benefited, too.

I still love programming for the beauty of it, and I still dive into little personal programming projects a few times a year. Part of me wishes I did so more often, but I'm held back because the only way I've found to get through a project of any duration longer than a few days is to basically develop delusions of grandeur about it. Programming is fun and beautiful, but very hard, too, and somehow without the promise of the conference talk, or the influential git repo coming out of it, there's just too much friction. So, more often than not, these days, I simply don't bother. I guess with my current middle-aged testosterone levels, I'd rather keep my family and friends than be king of the world.

(That said, if anyone out there finds this relatable, but has been able to push through and develop a healthier, less ego-reliant, relationship with programming, I'd live to hear about it!)

undergrowth54 5 years ago |

> My ego creates a tight bond between my work and my identity. Linking my self worth to how well I do my job. This then creates the need to track my performance. To keep score. Spinning up mental processes that consume valuable resources which make staying on task very difficult.

I think this part is definitely true for me...

I have let "notice when you are confused" and "understand the impact of your work" and "make sure you are building the right thing" and "make sure you know stakeholder needs" get kinda etched into my identity. I keep wanting to _understand_ the systems I work with and I keep getting distracted by noticing problems with its UX or implications to business process.

I can turn that voice off with deliberate effort, but I don't know how to get it to stay off.

Does anyone else have any methods for more permanently-silencing UX-worries and just cranking out code?

felideon 5 years ago | |

No. I was unable to silence those thoughts and became a product manager, and am now a UX practitioner with strong opinions on software dev which I also can’t seem to silence (speaking of ego).

digitalmaster 5 years ago | |

To be clear, i'm not suggesting you stop thinking about things like "understanding the impact of your work". In fact that's exactly what I think you should be thinking about. That's the "the" type question. It's only when you start to think thoughts like "Am I doing a good job at understanding the impact" that we get away from the goal. "I" am at the center of that question, instead of the "the impact of your work"

visarga 5 years ago |

I am not convinced by the article. For one, I don't think beginners mind is necessarily superior to expert mind. After all, why would society value the expert more than the novice? Has anyone actually checked to see if experts don't have an even greater openness towards learning and novelty?

About giving up on projects and how the ego plays into that, I don't think in such black and white - giving up = bad, persevering = good. Sometimes you need to give up in order to find a better approach. There are reasons why this instinct is present in our species (something to do with the exploration exploitation trade-off). We can't paint over it with self help advice.

Comparing yourself to others is bad? Why? It's an evolutionary advantage to learn from the experiences of others. By doing comparisons you can calibrate your values. Competition is a great motivator. Having a row model can be fast way towards improvement. Comparison between peers is like a second order metric, first order metrics relying only on self.

The advice about not comparing yourself to others is useful only in a limited setting - where you devalue your accomplishments and have nothing to gain from it. But when comparison motivates you to improve, then it's actually not bad. Also when comparison prompts you to take action and avoid a crisis you could be spared a lot of suffering. Comparison can act like an alarm. Another function of comparison is to make groups more cohesive - if they form a common culture they can function better - so aligning oneself to the group can be beneficial for all.

cloogshicer 5 years ago |

Pretty interesting article. If this spoke to you, I recommend reading "The drama of the gifted child" by Alice Miller.

Don't be put off by the title, it's a wonderful read no matter if you think you're gifted or not.

slindz 5 years ago | |

Counter point:

I read that book this year.

It definitely contributed towards resolving some unresolved childhood trauma, and I'm grateful for it, but it was no walk in the park.

cybernabjo 5 years ago | | |

Why is that a counter point?

poindontcare 5 years ago |

This is so critical and so easy to forget. Every day is an effort to remind oneself of these basic truths.

“Whenever distress or displeasure arises in your mind, remind yourself, “This is only my interpretation, not reality itself.” Then ask whether it falls within or outside your sphere of power. And, if it is beyond your power to control, let it go.” ― Epictetus

nsilvestri 5 years ago |

I needed to read this. Just today I realized today that my ego was distracting me from learning and being effective at my job. I am a new dev, <1 yr experience, and I've often found myself not reaching out for help when I need it. For no good reason, of course, but for fear of outing myself as not knowing something I perceive others might view as basic.

I need to teach myself to take the ego hit and that in the long term it'll pay off more than independently struggling on a problem.

duderific 5 years ago | |

I'm pretty senior (over 10 years), and I see this a lot with my younger colleagues. They will struggle for hours and hours before reaching out for help, which is obviously counterproductive.

I try to counter this by reaching out myself when I need help, which is pretty regularly. I'm hoping my younger colleagues will see that even the old fart needs help sometimes and isn't afraid to admit that he doesn't know something.

moreaccountspls 5 years ago |

"This is ego distraction. It’s about putting off uncertainty till later to buy temporary relief. To protect our ego from a perceived threat. Things like hard problems, the possibility of failing publicly, or negative feedback all become threats when they’re linked to my identity."

I disagree with this [at least for me personally]. Being a good programmer is just straight up not something that's part of my identity. There's millions of people who are much, much better at it then me, and that's totally fine.

The reason that it's hard to work on hard problems is because they're hard! Sometimes, programming can be a really difficult or a slog of an activity. It's same with mastering any skill. Learning to play guitar, becoming an Olympic athlete, whatever. You can't Zen Buddhism your way out of the fact that you're going to spending years and years practicing until your fingers bleed, or until you're completely exhausted, etc.

jugg1es 5 years ago |

This is an interesting article because it made me consider how I think personally as a highly productive programmer but also a leader at my company. Even if I didn't touch the code, I am personally responsible for it considering that I am the lead and have - or should have -reviewed it before it went out. If you go into the assumption that you are accountable no matter what, it isn't really a big deal. If a subordinate f'd up, then you are still accountable because you should have caught it.

Thinking this way is very liberating because it means everything is your fault, but.. you are human and humans make mistakes.. so that means that this is a learning experience. If your mindset is that we are constantly learning, no mistake can ever really touch your ego.

mtalantikite 5 years ago |

A great book where the last quote in this post pulls from is Suzuki’s “Zen Mind, Beginner’s Mind”. [1]

I often find the solution to many of my problems is to go back and practice from this mindset. I coincidentally went back and reread selections from this book a couple weeks ago, as I found my ego had been creeping into many facets of my life recently and I needed to go back and be reminded to practice with this mindset.

[1] https://en.wikipedia.org/wiki/Zen_Mind,_Beginner's_Mind

xd_dino 5 years ago |

Wonderful writing. I get the same feeling about the ego thing. What makes me wonder is there are people who would like to write about ego and coding in great detail. I admire. Truly resonates with me.

from what I see others comments in HN. "Points and Counterpoints" Every article or idea doesn't work for everyone as we are a complex cocktail of ideas and impressions. if some idea resonates with you, you have found your type of the idea. so enjoy it else don't resist the idea wait for next one that might work or not. thank you for sharing in any case.

helsinki 5 years ago |

This is really good. Not much else to say here, other than ‘thank you’.

linux_devil 5 years ago | |

Yes , quiet good introspection, really interesting article.

alexashka 5 years ago |

There is a predictable pattern to these types of posts and these types of life experiences.

You have a problem, you discover a potential solution and it seems to be working. You get excited and you try to make sense of it and you want to tell everyone because it is such a game changer! Then some time passes by, the emotions fade and you arrive at a new perspective - that your life hasn't changed that much or if it has, now you have a new set of challenges and a new potential solution will come your way sooner or later.

It never ends, unless you at one point recognize that it never ends and cease wanting to be better and wanting to understand it all so much. It's not that you purposely cease trying or wanting, it's that you relax the wanting, because you know the problems will be there tomorrow and the day after, no matter how much you try.

That's when 'it's the journey, not the destination' finally sinks in and life takes on a new quality :)

For some people, it happens when they are reminded of death and the inevitability of it all, for some when they've burned through their health enough that they can't do it anymore, for others it just occurs to them one day - I can't keep up with this bigger, better, faster, stronger culture and frankly, I don't want to, either.

Vysero 5 years ago |

I can relate to the parallels between programming, and athletics. I like to consider myself an athlete, and if there is one thing I have learned about sports it's that the process of falling from the top to the middle of the pack can happen in a week despite that it can take years to get from the middle of the pack to the top.

So when the author was talking about how being an expert is really just a matter of become a great student I was quickly reminded of my golf game. Where I often time find myself with the lowest handicap I have ever had without actually feeling like I am improving. I shave a stroke one day. Then another.. and another, and before I know it I am a 3 instead of a 10.

That being said, I wouldn't say I have an ego problem in coding myself because tbh I have always felt like a bit of an imposter. I think my imposter syndrome has actually ended up being a good thing over time in my career as a programmer. It seems to have kept me grounded, and as the author suggested is a good thing, it seems to have kept me in the forever a student mentality.

joallard 5 years ago |

The author has overfitted for his own psychology. There are a lot of assumptions in there. My brain doesn't work like that at all.

barrenko 5 years ago | |

Reluctantly agree, as I fit the author profile.

elite1percenter 5 years ago |

> Top students convincing everyone else to stop trying. Or, great engineers convincing the rest to stop trying.

This happens in a more literal sense as well. Since we compare ourselves to each other, it makes sense that more experienced engineers love feeling like they're ahead and beyond newer engineers, and it bleeds into behavior.

For example, engineers love asking candidates obscure questions in interviews - as if not knowing a specific JVM perk makes the candidate less of an engineer.

Let's also not forget the severe elitist attitude some engineers have when interacting with others. It's almost like everyone else is trash. For all the work he's done, which I admire, Torvalds was seriously toxic to interact with.

It all ties back to us wanting to compare ourselves to each other.

cgs 5 years ago |

> My ego creates a tight bond between my work and my identity. Linking my self worth to how well I do my job.

Same experience here, and probably for a lot of devs. It took a good 10 years or so to break out of this. If I couldn't get something working that I expected to work, I silently took it out on myself. Must not be good/smart enough. And so I'd beat my head against the wall. Once I started letting broken code or an unfixed bug wait until the next day and saw that the sky didn't fall, and that I could eventually fix the issue, I started to trust myself more, and just let things be. Some days, everything works. Some days, nothing works. Your build fails and you spend hours updating some obscure library, and you don't get feature X done that day. It's really your decision whether or not you let this effect how you feel.

ozim 5 years ago | |

I think people get wrong is mostly is part about judging "how well I do my job". They come up with insane expectations for themselves.

mylons 5 years ago |

this is a great blog. i don't think it would have really made sense or resonated with me if i hadn't been in therapy the last year or so almost exclusively working on this. which is what i want to really endorse if you're someone curious or in pain. there's ways to work on it.

eikenberry 5 years ago |

Did anyone else find the "Relative Distance" section seeming wrong. Not the harvard-vs-random school thing but the conclusion they draw, that you perform relative to your standing. How are students aware of their positions? Do some schools post rankings or something? When I was in school you're standing/grades were private and no one knew another students standing (aside from the obvious, people failing out and such).

If I was guessing at a cause, I'd put it more on the teachers/school. That they have it in their mind that there should be a range of performance in the students and they enforce that idea. Eg. if everyone gets an A in their course they start making it harder until they get the distribution they expect.

tppiotrowski 5 years ago | |

Many universities grade on a curve. X% get an A, Y% get a B, etc. If you are getting C’s in your classes you can infer that you somewhere in the middle 30%, A’s and you’re likely in the top 30%. You are correct in saying that you don’t know who is the best and worst in your class but all you need to know is where you stand for Relative Distance to work.

Artlav 5 years ago | |

Seems wrong to me too, but for a different reason - i would expect a better school to not only attract smarter students, but to also be harder. So it should balance out to the same curve.

noir_lord 5 years ago |

There is a perspective shift that comes (usually but not always) with age I think.

When I was younger I got into computer programming, for the first ten years (1987-1997) I thought I was hot shit because I could do things with computers that no-one else I knew could even understand (with the exception of a family friend who was a programmer in aerospace) then I ran into other programmers on-line and realised that there where other much better programmers in the domains I was interested in (strangely I never got into programming games, I always liked utilities and 'productive' stuff).

So I doubled down and resolved to be the best programmer I 'knew' again except this time I knew hundreds or over the years thousands of programmers an impossible treadmill.

Sometime in my late 20's/early 30's (so ~2007-2008) I realised that not only wasn't I ever going to be the best programmer I knew, I really didn't know much about programming in the general sense if you look at the whole field (no-one does really except the odd person) so I re-framed it, I was going to be a better programmer than the me of a year before and focus on the other skills I'd let languish over the years what I'd often derided as 'soft' skills (I don't think I was ever an arse-hole but I was the guy who'd sit in the corner muttering with the headphones blasting thrash metal).

In the end what I realised was that after all this, I like programming, I like providing value and when it comes to work the best thing I can get is feedback from a user whose life I've improved by making whatever I've touched that little bit better.

If I can do that then it was a good day.

The freedom from all this is I learnt to play again, if I'm interested in functional programming I'll go poke at that for a bit, if I'm interested in algorithms I'll go poke around over there - free from the the self-imposed need to compete I get to satisfy my own curiosity and nurture the devs on the team I run.

With 7 billion people on the planet it's statistically unlikely you are ever going to be the best and even if you are it's likely in only one dimension.

I noticed that the programmers I normally really admire are all older than me and seem to be excited/happy about technology and wondered how they kept that enthusiasm for so long in an industry where so many seem miserable and I think I can hazard a guess now.

Oh and because the universe loves a punchline, I have a dev on my team now who is determined to prove himself the best programmer, never says a word and listens to thrash metal all day while muttering, he's talented so I'm curious to see how he figures it out.

noisy_boy 5 years ago |

I found this quote from the article quite motivating:

> “I cannot say this too strongly: Do not compare yourselves to others. Be true to who you are, and continue to learn with all your might.” ― Daisaku Ikeda, Discussions on Youth

HellDunkel 5 years ago |

Self-improvement piece once again. Why are people so obsessed with this? Imposter symdrom seems to be a thing I really dont get. It never really bothered me if i actually caused the bug when looking for the root cause- ok maybe sometimes you worry if you messed up just before the release but thats pretty much it. Maybe realizing that you dont need to be a great programmer to make a great piece of software helps.

razzimatazz 5 years ago | |

Imposter syndrome for me is kept in check if I keep within my 'quota' of mistakes. It doesn't bother me if I am the author behind a bug (Normally every bug is an opportunity to understand the software we've built better), but if consecutive bugs come back to my work, then I notice it. And I notice it in others too, and then I want to see them putting effort into improving their skill, etc..

HellDunkel 5 years ago | | |

Quota.. is that something like a percentage of bugs you feel responsible for? That again would be comparing youself to others- not saying its a bad thing unless its bringing you down. Dont really know what im talking about here as i have never suffered from that. Insecurity yes, imposture sydrom no.

submeta 5 years ago |

I have my best coding sessions after I start the day with fifteen minutes to half an hour of mindfulness meditation. That calms my mind, makes me „less anxious“, lets me go into new directions without feeling the urge to walk away, procrastinate, do something else.

vzidex 5 years ago |

Incredible post! I've been struggling with anxiety, depression, and a lack of motivation at work for the past year or so, and slowly coagulating towards the feeling that I needed to focus on my work.

This article feels like it gives me the how - thank you.

nick-garfield 5 years ago |

This was an amazing post!

100% spot on that the external distractors are easier to manage than the internal ones. A buzzing phone, tempting social media websites, and loud rooms all tend to be relatively easy problems to fix. As for internal distractors, I feel like telling a personal story after reading this.. There are two internal distractors I've recently noticed myself struggling with:

1) A busy mind.

I often find my brain meandering on ideas or conversations completely unrelated to the work I'm trying to do. Daydreaming, imaginary arguments, and unnecessary tangents all tend to creep in (esp in the afternoon for some reason). I'm glad this post touched on Zen Buddhism and the beginner's mind. At risk of proselytizing, I have to say the best way I've found to manage a busy mind is through meditation. Consciously setting aside 10-15 minutes everyday to practice letting go of thoughts has helped build a (tiny) mental muscle which I can sometimes use to bring my focus back on the things in front of me.

2) Alcohol.

This is a bit of an external distractor, but also an internal one. In college, I was able to stay up all night drinking and coding. No longer! I find it amazing how insanely less productive I am even after a single glass of beer. I now get tired shortly afterwards and have immense difficulty focusing. Perhaps as the article mentions, the alcohol is wrapping up my ego in the task at hand. I don't have a drinking problem, but I now solve this by consciously deciding how to spend my next couple hours. "Am I going to grab a drink and take an extended break (perhaps for the rest of the day)? Or am I going to grab a water/tea and continue working?" Gone are the days when I could reliably reach the Ballmer peak (https://xkcd.com/323/).

kubanczyk 5 years ago |

So, one part of my brain is an adult another part is a child acting from beneath the consciousness.

Then most of these articles usually say the recipe is to have the adult very closely observe the child and hit it with a stick any time it wanders of the "desired" path.

No.

How do you expect it to work between a real parent and a real child? I think it would fail miserably and annoy both sides. I'd suggest coming up with some nasty parental tricks instead. Also adjust them after some time, because they do tend to stop working.

curvinator 5 years ago |

There is an interesting parallel drawn by a famous theologian: sin is self curved in on itself.

bilekas 5 years ago |

>It’s clear to me now that it's not about what I know, but rather how I think that's different on these days.

yup

_7fvc 5 years ago |

I think software ecosystems are currently infested by corporate interests. I feel kinda funny for saying this. It sounds like some kinds of left-wing politics. But software has become increasingly consolidated. There're 5-10 companies in the US that drive most of its development. They even picked up the most popular open source projects. When you're part of an ecosystem, your objective is to work the system. Making software has lower priority.

It's extremely hard to make money being an independent software developer. There's a lot of noise and money in the market. It's hard to compete with marketing from big companies. You have to work for a corporation or a startup with funding. You have to be part of an ecosystem. When you are accepted to a program like YC, you win an entry to an ecosystem.

Can you be a software artisan nowadays? Can a small team develop and sell software without having an ecosystem behind it? I've seen some examples of this, like Ruby on Rails, 37signals. But they are rare exceptions.

I'm currently working on an open source project. Let's see how long I can be independent for. Check out my project :)

https://github.com/vidalab/vida

carapace 5 years ago |

It might seem like a tangent by I would recommend studying (self-)hypnosis.

beardedetim 5 years ago |

Glad to see Jose writing again! Great post as always from them.