NPM: 429 Too Many Requests(github.com) |
NPM: 429 Too Many Requests(github.com) |
Regardless, can we talk about the conduct in this GitHub thread? I know every community is different but is it common to have memes and jokes posted this quickly and often in a GitHub issue? It makes it really hard to follow and discourages genuinely useful discussion of workarounds or progress.
It's certainly acceptable to blow off steam but there are times and places for that. The official issue thread is not one of them. It looked like a Reddit post more so than the official NPM repo. I have nothing against Reddit just pointing out it's a reasonable venue for this sort of commentary (or even HN).
The thread looked like folks celebrating like the power went out and they'll be going home from school for the day instead of "oh crap, this is my job and I have to fix this issue or my livelihood is at risk because I decided to have no backup plans in place for such a situation where NPM is down."
</old-man-moaning>
> Posting memes/jokes in an issue wastes the time of people who actually need to be able to read through the issue.
> Stop it. Use emoji reacts if you feel the need.
In two minutes I have 24 dislikes and 13 likes. I think the introduction of emoji reactions was a bad idea, it gamifies the issues system.
It depends on the community. I've seen many where the reactions are genuine and actually serves a purpose. From what I've seen, its almost always a popular js/node project that attracts this kind if behaviour like seen in this thread (based on my own observations).
Last example is vscode's santa hat[1]. It seems that that thread is mostly empty now but I remember there was huge meme thread going on in that issue or in reddit/hn when that thing was happening.
About all you can do is change the mindset of the community making the comments, which is hard as they're numerous.
1. GitHub presents an interface that encourages Reddit style behaviour.
2. NPM outages are essentially a meme in themselves at this point.
Both of them together create a perfect storm.
However, regarding #1... the interface allows people to post images in their comments on issues. This has a valid, useful reason - for showing screenshots of bugs, for example.
The problem is not the interface. The problem is the people using the interface.
This almost entirely explains the entirety of the javascript ecosystem and the seemingly ever increasing number of, and frequently changing popularity of, the various frontends frameworks. It seems a hype driven ecosystem, so it's not surprising it's also full of people posting memes and such.
I wonder what the reason is for this kind of behavior to exist only in the Javascript community? Could it be that a vast majority of Javascript developers are really in high school? Are there any good stats sources for it?
Actually those animated gifs might have been better than all those "me too", since a text-only post informing about the situation would have stuck out much better.
I am honestly surprised about the number of very upset people in this thread and the trashing of millennials. Yeah the ass of every workplace generational joke- most millennials have kids going to college now.
If you think millennials like memes Wait until you work with some zoomers. A lot of people screaming get off my lawn right now.
25-40?
if something goes bad at work, someone (orally) always makes a joke to defuse the situation. This is the same thing happening here.
Does it though? Sounds like an opinion.
Given that the comments in question add absolutely no value to the issue and instead insert a bunch of visual noise and fog then I'd argue it's not an opinion.
- N developers can't work for X hours
- or the company can't release new versions due to CI dependency on the registry.
- or the registry removes a package you were using
- or the existing package contents changes to something malicious
BUT you pay this price very occasionally and if you're a small shop, the cost is often negligible.
On the other hand, maintaining your own mirror has very real costs even though they can be small. One time setup, hardware, sometimes license or hosted service fee, security upgrades. When there's a sponsor maintaining the central repository, having very good uptime and offering it for free, the marginal utility of a local mirror is quite small.
- https://open-registry.dev/#get-started
WARNING: Research who runs the mirror before putting your trust in it.
How to turn on:
npm config set registry https://npm.open-registry.dev
npm config delete registryThe result was lightning-fast fetches and no rate limiting.
We needed the two-tiered system because this was Kube and occasionally we would have to rebuild/restart nodes and we didn't want to completely lose the cache when that happened.
You can easily extend this system to handle any package artifacts used in your build process: .deb's, .rpm's, etc.
> I am the engineering manager for the DDoS protection team and this morning at 11:06 UTC we tweaked a rule that affected one of our signals. The signal relates to the HTTP referer header, and we have a piece of code that looks at invalid referer headers. In this case we tweaked it to include not just "obvious garbage" but "anything that does not conform to the HTTP specification"... i.e. is the referer a URI? If not then it contributes to knowledge about bad traffic.
> So... why did this impact npmjs.org? It turns out that a lot of NPM traffic sends the referer as "install" which is invalid according to the HTTP specification. As NPM is also a heavily trafficked site this resulted in the DDoS systems picking this up and treating the traffic as a HTTP flood and determining that a rate-limit should be applied.
> When we noticed that NPM was seeing an increase in HTTP 429s (as seen on Twitter) we contacted NPM and started an internal investigation. As soon as we identified the root cause we reverted the change, which was at 13:00 UTC.
> We'll note that NPM and 1 other site use the referer for purposes outside the HTTP spec and we'll update our systems to ensure that this does not happen again. Additionally we'll improve our monitoring around changes of this nature so that we can discover impact sooner and roll back automatically.
[0] https://github.com/npm/cli/issues/836#issuecomment-587019096
Unfortunately, I am unable to resist the urge to add to the noise by complaining about people complaining.
In Travis there's some (not too obvious) caching mechanisms that in many cases avoid this and speed builds up a ton.
I wonder if we would win from a review on popularity of configuring the cache and to educate people further on its use. I'm sure other CI systems have similar capabilities.
1. Eliminate redownload of packages on every CI build 2. Reduce the amount of gigantic IO operations from unpacking the tens-of-thousands of files sitting in node_modules. 3. Better security: code checked in can be audited better if not downloaded every single CI build.
yarn’s PnP system is promising for the zero-install paradigm, but it doesn’t seem quite ready yet (so many packages don’t seem to get their dependencies right).
The thing with using web technology for distribution is that it’s easily accessible and, crucially, that it’s cachable in-line.
Is that like a local cache?
a single failure in any of those centralized systems that we don't pay for and builds fail.
It 100% is, unfortunately. I don't recall this happening in recent history, but it has been the case that 3rd party services have broken CI/CD pipelines and production pushes (e.g pip broke a few weeks ago, and their own pipeline for deploying changes was blocked by the bug).
One day this is going to happen for real and it will be because npm org decided to charge for API requests by `npm ci`.
> Investigating - We are currently investigating 403 / 429 errors received by some users.
> Monitoring - Our content delivery partner has informed us they've implemented a fix. We are monitoring. Feb 17, 13:07 UTC
Edit: And in the OP GH thread, people are also mentioning it's back up.
[a] https://blog.sonatype.com/using-nexus-3-as-your-repository-p...
Includes a built-in package manager for resource fetching, thus no need for NPM.
It really doesn't make a lot of sense to re-download tons of packages for every minor commit where you run your CI.
That’s been my experience anyway with local servers. We’ve had a lot of problems.
Edit: vendoring dependencies, on the other hand, is very reliable. But it doesn’t work well with DVCS.
https://docs.microsoft.com/en-us/azure/devops/artifacts/get-...
Whether you use expensive 'turnkey' solutions like Artifactory or keep things simple, there's just a surprising number of ways for a local mirror to go wrong, especially if you depend on it for any kind of third-party dependency compliance control.
Some repository mirrors will also become very large, which means that if you're e.g. running them in a cloud provider the bill can add up. Not really a problem on local hardware but the up-front cost of hardware can be substantial and a lot of startups have little to no in-house IT capability (e.g. the org I work with right now has reached hundreds of employees without having a single system administrator on staff, so as devops person I end up having to do the care and feeding of our recently purchased local hardware as well).
In general I think this is an important and often overlooked issue in modern tech businesses - it is amazing how many technology-centric firms like software startups get to appreciable size relying entirely on outside SaaS/PaaS providers with no real in-house IT operation. This reduces up-front and staffing cost but has a way of coming back to bite you when you hit a certain point. A conversation I've been in before, in reasonably large software outfits, is "we want actual real office phones now, but telephony-as-a-service is real expensive and the on-prem products use scary words like VLAN and QoS in their setup documentation". As someone with an IT rather than software background it's a little baffling to me how this happens, I feel like a combo sysadmin/network engineer would be an early hire. But here I am working for a company instead of running one...
However, early adopters of npm in the frontend world (back in Browserify and Require.js days) didn't like the practice (notably, because many parts of the dependencies contained node-only code, tests and scripts that were needed for building dependencies, etc.), and started putting node_modules in .gitignore. At the same time, Node people started to use other means to manage dependencies for reproducible builds: namely, private npm registries, dockerfiles, etc.
Over time both frontend and Node communities recognized the need for lockfiles, which we eventually got with Yarn and later versions of npm.
https://blog.isquaredsoftware.com/2017/07/practical-redux-pa...
We had a task every month for one developer to go manually upgrade one or two dependencies and commit the changes after testing (java libraries tend to upgrade much slower than Node).
In early days of node (circa 2011-2013) we used to do the following: 1. run `npm install --ignore-scripts` first. 2. Check the node_modules folder to source control, 3. run `npm install` again - this time without the flag 4. put all extra files generated by install scripts to .gitignore
This way the third-party code (at least, the JS-part of that code) was in the repository, and every developer / server got the version of binaries for their architecture.
It wasn't a bullet-proof, though, since: 1. The scripts could do different things anyway 2. More importantly: one could upload a new version of library to npm with the same version number.
These days, lockfiles and stricter npm publishing rules largely eliminated both issues, and updating dependencies doesn't produce 10k-line diffs in git history anymore.
This idea to redownload all packages all the time from external sources (and not even having a fallback-plan) seems completely brain-dead to me. Didn't the people learn from leftpad-gate?
And it's consistent with the rest of GH's markdown parsing for README, issues, etc.
It really clutters the conversation so when you're trying to find, e.g., the flight information, or the link to $IMPORTANT_THING, that somebody posted a few days ago you end up scrolling through screens and screens of total guff.
I wish there was a quick way to collapse messages that aren't relevant or useful, and that goes for GitHub issues as well.
[1] Apart from the fact this is distracting, and potentially a nuisance if you're working, driving, whatever, it also drains your phone's battery.
*FYI, I did not work at GrubHub, which probably fails the most at pretending to be cool and fun.
This is the opinion part.
There will always be a "cool new" language attracting newcomers and kids.
Now you have to sync it or risk running into unreproducible build failures. Also, if you update the binary dependencies on say, macOS, then you still need some x86-64 Linux to build the dependency.
Not saying it is not possible but without a proper process (e.g. a build server being the only place that updates dependencies) this is going to be painful.
I'm seeing comments on here that some of these people are acting like high school kids. Some of them probably are high school kids.
And that's fine. One of the biggest goals of Open Source in the first place is that software shouldn't only be developed and used by professionals. If you want that reality, you have to tolerate that unprofessional people are sometimes going to be part of the surrounding conversation for some projects.
In short, if the NPM maintainers hate this, they can modify community rules or remove spammy posts. And if they don't hate it, then I don't really get what the big deal is about allowing people to be unprofessional when talking about code -- other than maybe arguing that Github should have better filtering tools for readers.
1. When I sign up for updates I get spammed with notifications -- you should be able to listen only to updates from the project maintainers.
2. I can't find the official status -- again, this is a problem because the official status is buried inside a conversation thread. This could be pretty easily solved by allowing maintainers to pin comments, or by (again) allowing filtering comments by project maintainers.
3. Replies to comments get lost in the memes -- because Github issues don't support threading, and (again) there's no filter by replies or mentions.
Or...
4. Memes just make me mad because they're unprofessional -- and okay, this one is the community. But I just can't muster the energy to care about this, or to feel sympathy for the people who care about it. If we were talking about abuse, or 3rd-party unwanted advertisements, maybe I could get on board. But a gif is not a real problem.
The fact that Github has literally no searching at all on issue threads is the fault of the tool, not the community. The fact that NPM doesn't have an official status page is the fault of NPM, not the community.
Apppearly 4 important packages were unavailable in my region because they were deleted by accident. Some people were vpning into Europe, I found an Italian mirror.
Not sure if things are always that crazy but I don't have a reliable feel.
Eternal September or the September that never ended is Usenet slang for a period beginning in September 1993, the month that Internet service provider America Online began offering Usenet access to its many users, overwhelming the existing culture for online forums. -- Wikipedia
I agree that it's a valid concern.
I wonder if javascript will be in similar place in 10-20 years that PHP is today (e.g. "its not like it used to be and things a actually quite good now")? The language it self (javascript) might be there already, but is the community anywhere close yet? SQL injections used to be the thing PHP was known for in the past but not so much these days (the community is more experienced?). Meanwhile, problems with NPM and the whole packaging situation is what javascript is known for these days, but will the be so on 20 years?
"&" is a ligature for the letters "et". A traditional way to handwrite it, other than as &, is a "crossed epsilon", something like Ɛ̸, which looks more like "et". There's a Unicode character closer to this form: 🙲. I have no preference between 🙲, &, etc. and et cetera — none is common enough that I would recognize it by shape while reading prose.
Nonetheless, Tiro would approve. ⁊c.
PHP got (or is getting?) type hinting, while javascript doesn't have such thing (typescript or flow doesn't count). Javascript is event loop based, while PHP is (afaik) request->response based. These days, javascript usually is compiled for web, while PHP just sits there on the server and you can swap out the source files when you upgrade. You could go on and on.
Rather: if I'm looking for the solution for some problem, I don't want to scroll through memes, because I want to be productive, getting things done. Spam/fog in the resolution-thread obviously doesn't help with that; the opposite is the case.
With the increase in adoption of certain technologies (node etc) I have noticed a relationship with the people employed to work with them and their poor level of professionalism in the workplace with regard to their work and how they deal with other people (in the UK) and even how they dress. Similar age groups in other areas don't display these traits.
A number of these people work as contractors as well, so at least it was easy to get rid of them.
Go post memes on the LKML and see what happens
While there are many points in your post that I agree with, we really should stop worrying about how people dress.
If it's a developer sitting behind a computer screen all day long with zero customer contact whatsoever, he really shouldn't be forced to wear a certain dress code just to satisfy someone's standard of professionalism. Such feelings are in similar spirit as "woman should not wear trousers" a few decades ago - there is just no rationale behind it other than conditioning by society.
There is a limit. I'm sure you have one too if you really think about it. Is it tattoos all over the face? What about sandals? Or shorts? Or going bare-feet around the office.
I don't know... It's about people giving me money for work. I don't feel like I can go to work dressed like any given Sunday.
> Go post memes on the LKML and see what happens
I think the seriousness (almost angry tone) in certain communities also is a disservice to attract new members. If I make my first PR in some project, it's easy too feel attacked when the reply is a negative one, albeit strictly professional. If it however has a more light-hearted tone and a bit more sandwiched fun-serious-fun one feels more welcome.
There's a big difference between shooting the shit in your private communications and cluttering up places where people are looking for solutions.
As others have said, I too am not against a light-hearted tone. But when you have a legitimate problem that might be blocking you from doing your work and your only source of information is a GitHub issue then it should go without saying that the information should be dense and compressed and it should absolutely lack memes, yes.
The sheer numbers of users of NPM says that this can happen from time to time and the stability they are providing is nothing other than exceptional.
Otherwise professional conduct mainly comes down to creating an atmosphere of noncommittal distance to deal with difficult personalities. I think being subjected to it for too long should disintegrate any personality, because it is completely unnatural. Hard to imagine people longing for more of that.
"yeah, u know, i could do it better, lol".
Edit: to add that I agree with the idea there is a problem with people making a distinction between work and play contexts.
https://hackernoon.com/for-inclusive-culture-maybe-less-is-m...
I disagree entirely that we can't be social with people different as us. I also disagree that business shouldn't be social, but that's mostly an opinion.
FYI, the word you're thinking of is "cue", not "queue".
And why not? As you said: they are giving us money... for work. Not for playing dressing up.
If they expect otherwise, they should be clear about it up front and give a good reason to give up on that freedom while we work for them.
To me (and of course, it's a personal opinion) it shows that someone cares. It's like when you do your resume, you want it clean and presentable, without typos, to show that you care.
We all like good user interfaces and experiences in our programs. To me, the look is like the presentation layer of our own UX. Is it required for our work? Most of the times it is not, because our work depends on the logic layers, but it doesn't hurt to have a nice presentation layer.
And I'm not telling about going to work in a D&G suit.
EDIT: and I'm not telling that I would discriminate or think right away that someone doesn't care! I don't do that. My attitude about good looking is mono-directional; from me towards others.
They care by solving problems.
They care by being there when the team needs them.
They care by being honest, authentic and candid.
They care by making things simpler and smoother for themselves and those around them.
They care by striving to avoid office politics and back stabbing.
They care by being the team mate we all wish we had and strive to be.
They care making our common working space a place we are all eager to come to in the morning.
And I would pick a reliable, self-aware, well-rounded, face tattooed, green haired and barefoot colleague that's comfortable in their own skin any day, over any well-dressed, high maintenance, drama king / queen.
What's interesting though, is that even knowing, and thinking that, I couldn't help but be spontaneously biased towards he or she who made the so-called effort of dressing to look the part. And that's a problem.
Also, I am nowhere near being that ideal team mate. More like a work in progress >.<'
Sandals/shorts/barefeet? It's winter here but in those brief moments of summer please do that. Wearing a three piece suit means you won't be able to work as hard during the heat wave.
I think the key thing is balance, not everyone is comfortable to wear a 3-piece suit but also few people are comfortable working with people who ave poor hygiene (e.g. dont shower after the lunchtime gym)
I think as long as someone is clean and presentable (context dependent) then there shouldnt be an issue.
Work is the key word here. Is you work is about dressing yourself up? Then why makes it about this?
Do what you are paid for, working.
Anecdote: my wife is a psychologist. They are taught to observe how a patient presents him/herself at an appointment. Is her/him presentable? What about personal hygiene? Does he/she take care of him/herself? These are characteristics that say a lot, and should be taken into account when meeting someone. There is a difference between taking care of oneself and trying to appear like something else by wearing branded clothes, or to appear like something else by dressing like a 60's hippie (or whatever).
> And I would pick a reliable, self-aware, well-rounded, face tattooed, green haired and barefoot colleague
Me too. But the first 3 are characteristics you wouldn't know for a while. In the meantime you have to work the hiring with what you see. Unfortunately, it's like that.
I also believe that the worst and most dangerous people in the world walk around in expensive suits, behave correctly in public and never say a bad word.