JavaScript's Dependency Problem

JavaScript's Dependency Problem(danthedev.com)

19 points by codecurve 4 years ago | 15 comments

skywal_l 4 years ago |

> A majority of the dependencies for modern frontend development are only used during development

This is important to note. I think we have to separate non-dev dependencies and dev dependencies. The attitude should be different. One is embedded in the product you deliver the other is only used locally to make the product. No one would say that we should start writing our C++ compiler by hand because "dependencies!".

I think this is an important strength of javascript: the tooling infrastructure. Its ability to embed everything you need into your package.json makes it so easy to start working on a project. I'm old enough to remember the weeks spent trying to get all the licenses, accounts created, software installed and configured in order to just start compiling a project.

This being said, we should also be careful about the tool we're using. If you want to use mustache with express, do not install the dependencies that link them together (mustache-express). Just write the 4 lines of code you need to adapt one to the other, so that you have a better understanding of what's going on and when something fails, you'll not be lost in a sea of unreadable JS code you didn't write.

perilunar 4 years ago |

Not Javascript's dependency problem, but NPM's dependency problem. You don't need to use NPM to build things in JS — it's perfectly feasible to manually load just the scripts you need, and avoid build steps, frameworks, and dependencies.

codecurve 4 years ago | |

Doesn't feel unreasonable to conflate the two. npm is the dominant way to handle dependencies for frontend and backend, even some of the most popular script-tag delivery CDNs are powered by npm (unpkg, skypack).

Coincidentally one of my previous posts was about a no tool approach. https://danthedev.com/web-dev-without-tools/

perilunar 4 years ago | | |

> npm is the dominant way to handle dependencies

Precisely. Which is why it's NPM's dependency problem, not JS's. Avoid NPM and no dependency problems (well, fewer anyway).

Liked your no tool article, but I laughed at this bit: "Traditionally, using other people's code has required some combination of npm and yarn to download those modules". NPM is only 12 years old (and Yarn is newer) — that's less than half the age of JS. No, 'traditionally' you just viewed the source and copied the code you wanted, or went to a project page and downloaded the script.

Madeindjs 4 years ago |

As a developer, I love JavaScript ecosystem for some of theses drawback of this package manager.

I'm be able to quickly publish a library on NPM and this is great IMO. Few years ago, I tried to do it for Maven in Java and it was a nightmare.

> We don't need two million packages. We probably don't even need two thousand.

I don't think so. There are many tools, frameworks and libraries in JS and I love that. I learn from all of them and this is exciting. I don't want an ecosystem with only JQuery to do all I want. Sometimes I may use Vue.js, sometimes Angular or React... and that's great.

codecurve 4 years ago | |

Why is it great to be able to publish a package quickly? You may be a smart programmer who only releases production quality, bug-free, vulnerability-free code, but is it a good thing that it is easy for inexperienced developers or malicious users to be able to publish packages at the same ease/speed?

No one wants an ecosystem with only jQuery, but there's a middle ground somewhere before you get to 2 million packages. Competing frontend frameworks fit comfortably within that space. I just don't want a world where there are 16 competing packages that all implement a slider in React.

There's a huge difference if you only use npm for personal projects, too. The consequences of picking unmaintained/undocumented/insecure/buggy software are much, much lower if you can afford to rewrite/throwaway in a weeks time.

wildrhythms 4 years ago |

The problem is that there is little incentive to "clean up" and move away from the bottomless trough of is-even type packages... similarly, how many front-end tools require one of the many utility packages that effectively just wrap some minor functionality around document.querySelector? Is the problem inexperienced developers? (the article talks about this as well)

I'm also curious how many HN readers and JS devs in general were, at some point, told to "not reinvent the wheel", instructed (by a professor, by a mentor...) to use libraries wherever possible, and extrapolated that out to today, where so many devs have found a way to justify requiring is-even (there are hundreds, thousands of packages like this)...

codecurve 4 years ago | |

I don't think we talk enough about the downsides of DRY.

Reinventing the wheel for the sake of reinventing the wheel (not-invented-here) is a problem, but reinventing it for the sake of learning more about wheels is a big deal.

I suppose many developers reach for libraries because they're more confident that the libraries will implement things correctly/more efficiently than they could. But if they keep reaching for libraries (instead of trying to write an `isEven` function themselves) then they never really improve either.

DrSiemer 4 years ago | | |

Most developers work under constant time pressure and usually with a fixed budget. Choices have to be made. Sometimes ugly code will creep in, because doing better is simply not worth the investment. Many projects don't have a very long lifespan anyway.

Pet project usually get a lot more attention, but rewriting existing libraries is not exactly fun and will require maintenance when made public, so that does not happen often.

eatonphil 4 years ago |

I'd like to see an algorithmic grading system that grades every repo publicly on things like: number of (transitive) dependencies, regularity of releases (explicitly not frequency since some things release reliably once a year), test coverage, and so on.

I'd like this to be public so that anyone can enter in a package and check out its score. My thought is that over time this might influence/produce a natural selection of only high quality repos.

My other idea, which is what this post is getting at, is that some group of devs band together to build a "distribution" of reliable JavaScript packages like every Linux distro does with the packages they distribute.

tlb 4 years ago | |

I like the last idea best. It's not an unreasonable amount of work -- every project basically does this on their own now. One or two dedicated people could do a great job.

acemarke 4 years ago | |

A recently announced tool called Socket is trying to do exactly that scoring approach :

https://socket.dev/

eatonphil 4 years ago | | |

I've seen some of these and my angle is separate from security. So far all the grading systems have been focused on security and not code quality, including not counting (transitive) dependencies.

It's not just better packages I want but fewer of them.

The tools like socket.dev could of course choose to incorporate stuff like this.

Also, to make permanent progress it's important the the grades all be public. Not sure if socket.dev is public.

Tknl 4 years ago | |

Some of these data points are captured by Synk advisor https://snyk.io/advisor/

qudat 4 years ago |

I have a slightly different spin on how JavaScript ended up where we are: https://erock.io/2021/03/27/my-love-letter-to-front-end-web-...