Ask HN: What skills are worth learning long-term against AI?

33 points by candrewlee14 2 years ago | 35 comments

I'm graduating this semester with my MS in CS and I'm beginning what will likely be 45+ years of career work. With the pace of AI/ML developments in the past 5 years, what kinds of skills are most worth investing in for long-term benefit/stability?

Everyone and their dog has been told in the last decade that learning to code is important. I've grown up hearing that sentiment consistently and I'm sure it contributed a lot to me falling in love with this field.

However, coding (at least in its current form) seems like one of the most at-risk forms of work in terms of AI automation. I love writing code and I think I'm good at it, but I'm not so sure I'll be able to compete long-term with AI. I'm sure there'll be a few "John Henry"s, but many devs will probably change. Keep in mind, I'm thinking on the scale of length of a whole career and not saying this is happening already. However, with LLM context lengths increasing and special hardware made for ML inference (like groq), it feels like we're not far out from repo-level AI coding. At this rate of development, I could imagine a decade of progress might transform many aspects of day-to-day work for software development.

With all that being said, what advice would you give to someone just starting their career who wants to avoid shorter-term local maxima? I've heard lots of mixed reviews on PhDs and advanced degrees compared to working in industry, but should that calculus change with AI? Thank you!

mindcrime 2 years ago |

In 45 years? I predict the skills you'll need then will include:

foraging, scavenging, hacking, repurposing old junk into shelter and transportation, hydroponics, stealing electricity, small scale farming, digging, ammo reloading, evading AI, creating improvised explosive devices, smuggling, ICE breaking, soldering, welding, firearms maintenance, broadcasting pirate radio and television, etc.

LabMechanic 2 years ago | |

You forgot, fire making, water filtration (essential) and fishing (protein, fat, vitamins). How can you do all the stuff without energy? Are you running on sunlight, or do you do nuclear fission inside your body?

tmaly 2 years ago | |

I tend to like the view of AI maximalist. All the post apocalyptic stuff is not great for your mood day to day.

cm2012 2 years ago | |

Either that or just know how to enjoy yourself, since our benevolent AI God has got life covered.

hnthrowaway0328 2 years ago | |

I kinda agree given the developing situation.

damir 2 years ago | |

Reads like a sceene from terminator movie...

mindcrime 2 years ago | | |

In truth, I'm partly (maybe even mostly?) joking. I'm generally an optimistic in regards to technology.

And yet...

I have my doubts, and some concerns about what the future holds with the way some things are going. A tinge of concern that a cyberpunk dystopia might just be in our future has started to creep in around the edges.

solardev 2 years ago | | |

Fun fact: those movies are just easter eggs thrown into our timeline by future ChatGPT

paulpauper 2 years ago |

Writing has proven to be surprisingly resistant to Ai. Or re-writing people's AI-generated papers to sound more human or readable. AI-generated writing has a certain unmistakable clunkiness that tends to give it away.

candrewlee14 2 years ago | |

Totally agree, it’s the textual equivalent of the visual uncanny valley. Communication in general is obviously not gonna go away. Certain kinds of corporate copy-writing where the tone matches the LLM is probably not as resistant. It’ll be interesting to see to what extent that uncanny effect will be reduced over the next decade.

pillefitz 2 years ago | |

Give it another year at maximum. Claude already sounds a lot less robotic than GPT4, and I heard of some smaller models sounding quite natural.

aristofun 2 years ago |

It may sound banal but the more i live the more i see it’s true.

Learning basics never gets old.

Philosophy, logic, seeking truth, foundations of math, of basic nature laws etc.

Whatever skills gonna be needed in future- you’ll adapt, if you’re smart and flexible.

And only seeking the truth keeps you in shape, exploring basics helps you get smarter.

LabMechanic 2 years ago | |

Who knows what the future will bring (will we ever get AGI)?

Needless to say, fundamentals, that is, math (geometry, algebra, and calculus) and physics (mechanics, electromagnetism, thermodynamics).

Some basic algorithms/techniques such as backpropagation, triangle ray tracing and rasterization, and some fundamental awareness that you can compute anything given 5-6 instructions (Turing completeness). Knowing how to program in a language like Python might suffice.

Then there are skills that are essential to maintain your life, such as fire making, fishing, and water filtration. You need to get energy someway.

See also Maslow's pyramid.

chadash 2 years ago |

Building something the right way doesn’t matter if you don’t build the right thing. Requirements are never fully specified and it is your job to use good judgment on the ambiguous parts. Figure out what business problem you are being asked to solve and recommend solutions accordingly. “If I asked people what they wanted, they would have said a faster horse” - Henry Ford

Also, you most likely think your job is what’s in the job description. That is not true. Unless you are in a field where output is super easy to measure (e.g., professional golf) your job is to make life easier for your boss and other people above you at your company.

austin-cheney 2 years ago |

Performance, security, and written communication. In my experience CS graduates are hilariously and profoundly bad at each of those.

When it comes to performance most developers are absolutely and utterly incapable of measuring things, a bridge too far. As a result most developers simply just guess at this. The problem with guessing is that people tend to guess wrong more 80% of the time and more those 50% of those wrong guesses are off by one to three orders of magnitude.

In my experience most developers have absolutely no idea about security best practices and yet somehow believe themselves quite brilliant at it. Security in software is Dunning-Kruger at its finest, and the confidence therein is superbly comedic. Don’t lie to yourself about security. If you want to prove to the world just how much you aren’t full of shit get a CASP, CISA, or CISSP. In my experience you are typically much safer listening to the security opinions of software developers and executing the exact opposite.

Written communication is also a poor developed skill for most software developers. I have seen people insane things to avoid having to send brief emails. Your ability to articulate your thoughts in writing with precision and clarity are the best way to unintentionally prove to the world just how brilliant you are. Good written communication requires practice. Don’t be that coward that hides from this because you are only harming your own professional development.

kypro 2 years ago | |

> Performance, security, and written communication.

AIs are good at all of these. Performance & security is largely a knowledge problem which AIs are very good at. And LLMs are great at written communication.

jamil7 2 years ago | | |

I asked GPT-4 today to optimise a function for me, giving it a hint at what algorithm to use and it produced something incorrect and about 100x slower than what I was trying to optimise.

austin-cheney 2 years ago | | |

If that is true then why bother hiring expensive humans to write slow, insecure, inferior software products at all. Replace them all.

omosubi 2 years ago |

I think if you understand the fundamentals of how computers work and can debug anything to figure out what is causing a problem you will be valuable for a long time. most people either don't have the requisite knowledge to do this, or just simply lack the drive and curiosity to figure out weird bugs. also, there are senior engineers with 10+ years exp. at the company that I work at that don't bother to google or do any real debugging before asking a question in slack. don't be one of those people.

f0e4c2f7 2 years ago |

Poker. Game of skill. Looks like gambling from the outside. Rich people use it as a way to socialize while wasting money. Not dependent on any technology. Has existed for hundreds of years (Lindy).

Practice enough to win at an average table and then know at any time you can get on a plane for Vegas, Atlantic City, or another place with consistent games and make a living playing cards.

Perfect as a backup plan / hedge. Poker also teaches lots of life skills so still useful if you never really use it too.

fragmede 2 years ago |

Fundamentals don't go away. I'd recommend getting into understanding how LLMs work, from scratch, and getting into that corner of the industry. Even if the LLMs write all the code, someone still has to direct them on what to write. They're not going away, so expertise on how to train and operate them is going to be valuable for the foreseeable future. What is MoE or RAG? If I'd just graduated, I'd force myself into that corner of the industry.

usgroup 2 years ago |

You have to learn to maths and to code, that takes about a decade. It requires a lot of output from you. There’s no way around that. Even if it was all redundant due to AI , it is essential mental scaffolding for higher thought unreachable without it.

mindcrime 2 years ago | |

> essential mental scaffolding for higher thought

Yes, definitely. And as AI grows into more and more facets of intellectual output, humans will need to reach those higher thoughts to find a niche to occupy. The best thing one can do today is exactly to start building that scaffolding that will allow that to happen.

quickthrower2 2 years ago |

Coding will change but still exist. If the computer is coding for you you still need to understand what it did, review it etc. If it can automate everything to do with coding it can probably automate sales, marketing, finance and support :-).

gregjor 2 years ago |

I have almost 45 years experience in software development, so I can describe the skills that led to my own longevity.

Hardware and the tools and languages available for writing software changed and improved in many ways during my career. My phone has more compute power and memory than the mainframes I worked on in the '70s and '80s. We have more expressive and safer languages, and better tools for writing and testing code. On the other hand I still use, every day, things that date from early in my career: C and C-family languages, relational databases and SQL, Unix in the form of Linux, modern versions of the vi editor. Very little about the practice of software development changed in the last 4.5 decades -- programming books written in the '70s have the same lessons as those published last year, you just have to read PL/I in one and Javascript or Rust in another. Read anything by Brian Kernighan to understand how little software development has actually changed in 40 years, and how programmers continue to repeat the same mistakes, "reinvent the flat tire," as Alan Kay put it.

A big lesson learned way back in the history of software development: Getting the requirements right is the hard part. That hasn't changed, software projects still succeed or fail mostly due to success or failure in the requirements and specification steps. A lot of programmers don't have the experience or interest to learn the business domain or engage with management and users, so they can't write or even correctly interpret requirements. When I started my career I worked with analysts who translated business domain expertise and business requirements into actionable software development plans for the programmers to implement [1] [2]. The analyst role faded away in many companies as software development changed from an ancillary activity (supporting an accounting or logistics department, for example) to a standalone discipline producing products that businesses would adapt to. Today software often dictates requirements rather than the other way around -- if you've ever worked at a company that adopted Salesforce or SAP or Oracle you know what I mean.

Another lesson known as least as far back as Brooks[3] wrote a book about it: People and organizational issues determine the success of software projects to a greater degree (I'd say much greater) than technical skill or choice of language. Learning how to manage teams and projects and work with all of the business stakeholders will prove more durable and valuable than learning the new language of the year.

Writing code got more mechanical over the years, and bugs and performance issues and resource usage became less critical because modern hardware can handle sloppy and bloated code, and modern operating systems and programming languages have a lot more guardrails and fault tolerance. That glosses over a lot of bad code, which in my opinion led to even more sloppy programming. Since LLMs will train mostly on modern code in modern languages I think we can expect more of the same -- workable code that just reiterates what someone else already wrote. That describes what a majority of programmers do now, plugging APIs together, so LLMs can join the team.

To answer your question: I would focus on the fundamentals, which have little to do with programming languages. The fundamental technologies that have stood up for decades with no sign of going away:

- Relational databases and SQL.

- Unix-family operating systems [4].

- C, because it influenced so many later languages, and expresses the essence of programming succinctly.

- Lisp, because it offers a counterpoint to the imperative C style that came to dominate, even if almost no one writes Lisp code today.

- Algorithms and data structures, and algorithmic complexity, which you can learn from '70s-era books by Knuth and Wirth.

I'll add networking and security, things we didn't have or worry much about pre-internet, but today I see a lot of demand for actual experts and few people going that direction, preferring to join the front-end web dev React swarm because of the relatively low bar to entry.

I would also focus on learning business domains and the systems that keep businesses running. I can't get a job today because I know Javascript or Go, but I can get a job because I know a lot about enterprise-level logistics. Outside of the Silicon Valley tech meat grinder you find millions of companies solving business problems they can describe and attach actual costs and risks to, not looking for the next disruption (or grift) with a small chance of success. Businesses don't have requirements like "We need another thousand lines of Javascript by next month." They have requirements like "We need to reduce late deliveries and mistakes by 20%." So learn at least one business domain (best to learn on the job), pay attention to the whole organization and the people, make contacts and friends who aren't fellow Rust geeks, and hone your skills translating business needs into actionable software specifications.

Learn and practice speaking and writing. People who can effectively communicate have a significant advantage over those who can't. [5]

You face a tough job market, especially for people trying to enter the field. Look beyond the big tech companies. You can't ignore AI (what we're calling "AI" this decade) and the possibility it will eventually write entire working software applications, though today that's more hype and VC dreams than reality. But if that does happen I think businesses will still need actual experts who have experience and skills, not just a good memory and algorithm for spewing out code. And I know businesses will need people who can solve business problems rather than just write code.

[1] https://www.amazon.com/Structured-Design-Fundamentals-Discip... (for example)

[2] https://www.amazon.com/Reliable-software-through-composite-d... (for example)

[3] https://en.wikipedia.org/wiki/The_Mythical_Man-Month

[4] http://doc.cat-v.org/bell_labs/utah2000/

[5] https://www.dreamsongs.com/Files/PatternsOfSoftware.pdf