public.ecr.aws was failing for me earlier with 5XX errors due to the AWS outage: https://news.ycombinator.com/item?id=45640754

I manage a large build system and pulling from ECR has been flaking all day

A bit like the ambiguity of search facets: if I select one facet, I get results that match, but if I add a second facet, should the results expand (OR'ing my selections) or contract (AND'ing my selections)? Presumably they should be OR'd if they belong to the same category (like selecting multiple colors, if any given result has only one color) but AND'd otherwise (like selecting a color and a size). But then a category could consist of miscellaneous features, and I want results that have every feature I've selected, which goes against the general case.

Almost all cloud providers help here by having inter-region failures as well.

There are multiple AWS services which are "global" in the sense that they are entirely hosted out of AWS East 1

According to https://news.ycombinator.com/lists it's "Most active current discussions"

I find that it better surfaces the best discussion when there are multiple threads (like in this example), and it keeps showing slightly older threads for longer when there's still discussion happening.

During the last Docker Hub outage we found Google mirrors lost all image tags after a while. Image digest references would probably work

Yea we have thought about that - I also want to remove most dependencies on externally imported actions on GitHub CI and probably just go back to simple bash scripts. Our actions are not that complicated and there is little benefit in using some external action to run ESLint than just run the command inside the action directly. Saves time and reduces dependencies - just need to get time to do that…

Thanks. I will keep an eye on your project as it looks great and something we would definitely benefit from.

P.S. Your blog could do with an rss feed ;). I will track https://github.com/spegel-org/spegel/releases.atom for now

Our images are in a private docker registry on quay.io

Hmm yea with a self hosted runner this could work. Gotta need to set the dockerd config into the VM before the runner starts I assume - unfortunately GitHub itself does not allow to change anything for the prepare stage - and it's a known issue for 2 years at least...

https://github.com/actions/runner-images/issues/1445#issueco... https://github.com/orgs/community/discussions/76636

In terms of user reports: Some users don't know what the hell is going on. This is a constant.

For instance: When there's a widespread Verizon cellular outage, sites like downdetector will show a spike in Verizon reports.

But such sites will also show a spike in AT&T and T-Mobile reports. Even though those latter networks are completely unaffected by Verizon's back-end issues, the graphs of user reports are consistently shaped the same for all 3 carriers.

This is just because some of the users doing the reporting have no clue.

So when the observation is "AWS is in outage and people are reporting issues at Google, and Microsoft," then the last two are often just factors of people being people and reporting the wrong thing.

(You're hanging out on HN, so there's very good certainty that you know what precisely what cell carrier you're using and also can discern the difference betwixt an Amazon, a Google, and a Microsoft. But lots of other people are not particularly adept at making these distinctions. It's normal and expected for some of them to be this way at all times.)

More likely the outage reports for google and microsoft are based around systems which also include aws

It’s very likely they’ve bought companies that were built on AWS and haven’t migrated to use their homegrown cloud platforms.

They might be using third party services that rely on AWS.

> as soon as people start using any vendor-specific functionality

It's also true in circumstances where things have the same name but act differently.

You'd be forgiven for believing that AWS IAM and GCP IAM are the same thing for example, but in GCP an IAM Role is simply a list of permissions that you can attach to an identity. In AWS an IAM Role is the identity itself.

Other examples; if you're coming from GCP, you'd be forgiven for thinking that Networks are regional in AWS, which will be annoying to fix later when you realise you need to create peering connections.

Oh and while default firewall rules are stateful on both, if you dive into more advanced network security, the way rules are applied and processed can have subtle differences. The inherent global nature of the GCP VPC means firewall rules, by default, apply across all regions within that VPC, which requires a different mindset than AWS where rules are scoped more tightly to the region/subnet.

There's like, hundreds of these little details.

I think there's some irony in Docker being impacted specifically, as they're one of the main tools to help achieve true multi-tenancy.

And even if you think it’s important enough to justify the expense and complexity, it’s times like this when you discover some minor utility service 1) is a critical dependency, and 2) is not multi-cloud.

Complex systems are hard.

Multi cloud is just a way to have the outages of both.

From what I've seen: I highly doubt it.

Edit to add: This might spur on a few more to start doing that, but people are quick to forget/prioritise other areas. If this keeps happening then it will change.

“Their own” can and often does mean something hosted on a major cloud provider (whether they manage it in-house or pay a vendor for their system)

Yeah, perhaps. I don't know how many folks host mirrors. Most places I've worked for didn't, though this is anecdotal.

I'll be honest and say I hadn't heard of Spegel before, and just read the landing page which says "Speed up container pulls and minimize downtime with a stateless peer-to-peer OCI registry mirror for efficient image distribution", so it isn't exactly clear you can use it for more things than container images.

What exactly does "stateless" mean in this context?

It's not entirely uncommon to only have parts of the solution open. So a license on one repo might not be the whole story and looking further would take more time than giving a good suggestion to the author.

If I have to dig through your website/documentation to find basic information we’re not getting off to a great start. It’s pretty common for open source projects to proudly proclaim they are open source from the get-go. “____ is an open source tool for ______.” Simple as that

Also it's good feedback for the developer of this solution

Good for you! This of course doesn't help in the situation where a dependency author retroactively changes the licensing state of a component, or reconfigures the project to rely on a new external dependency with differing license states (experienced both of these too!).

Having the landing page explain the motivations of the authors vis-a-vis open source goes a long way to providing the context for whatever licensing is appearing in the source repos, and helps understand what the future steer for the project is likely to be.

There are loads of ostensibly open source projects out there whose real goal is to drive sales of associated software and services, often without which the value of the opensource components is reduced, especially in the developer tooling space.

I agree it's a good thing, but I'd also agree it's not something you need/have to shove in people's faces, especially when it's literally one click away to find out (The GitHub icon in the top right takes you to the repository, and you don't even have to scroll or click anything, the sidebar shows "MIT License" for you).

There is a GitHub icon fairly prominent on the top right. Choosing to spend precious text for a fleeting would be user on it is a choice and not everyone wants to market that fact very prominently. Should everyone who writes their project in rust include that prominently as well? It seemingly markets very well and a lot of people seem to care about that as well.