AWS Web Console Down?

typicalrunt · 2019-04-25T16:30:43.000Z

https://console.aws.amazon.com/console/homeIt appears to be down, saying "Website temporarily unavailable". But it's intermittent for other employees at my company so it might be failing app services behind a load balancer.CLI access or EC2 services doesn't appear to be affected, at least from us-east-1.And, of course, the status health dashboard shows Green across the board. https://status.aws.amazon.com/UPDATEHatchedLake721 mentions that it appears to just be us-east-1 Web console that's down. Use this instead for now: https://us-west-2.console.aws.amazon.com/console/home?region=us-west-2Thanks @HatchedLake721!

whatthesmack 7 years ago |

It seems that the issue may be specific to us-east-1...

https://console.aws.amazon.com/console/home?region=us-east-1

...doesn’t work, but:

https://us-west-2.console.aws.amazon.com/console/home?region...

...works.

codecrane 7 years ago | |

Both of those are working for me. Much appreciated!

fiveoak 7 years ago |

It's annoying how often status pages for various services (when they even exist at all) show that things are working when they really aren't

EnFinlay 7 years ago | |

I want status pages to show traffic to the status page itself over the last 24 hours (or some time period). A sudden uptick in traffic but green across the board would indicate that there is an issue, but they just haven't updated the page yet.

dogecoinbase 7 years ago | | |

Yes! I have yet to convince anyplace I've been working to do this, but I do make increases in traffic volume to status pages generate non-critical alerts. Has caught out a few problems much earlier than they otherwise would have been discovered.

MadVikingGod 7 years ago | | |

Sounds like an interesting way to make developers (or ops) life hell. Just point a traffic cannon at the status page and bingo bango non-critical page.

brianwawok 7 years ago | |

If you can write a perfect automated status page - can't you basically write perfect integration tests and make sure no bad code gets deployed?

Bit of a chicken and an egg problem :)

chabreck 7 years ago | | |

Do you write every single test for your codebase? Do you have control over every line of code that gets deployed? The purpose of monitoring is to detect issues, because preventing them entirely is next to impossible.

By your logic if you can write perfect integration tests, can't you write perfect application code that doesn't need to be tested?

dahfizz 7 years ago | | |

I think you've missed the point.

OP is arguing that a perfect codebase is not possible, therefore it's a bit unfair to complain that the status pages do not work perfectly. Hence the chicken and egg problem of "if I could write a perfect status page, I would have skills such that the status page would not be necessary".

brianwawok 7 years ago | | |

There are two ways to have a status page.

1) A guy with a button. It's right, but it requires the guy to notice or get told - and remember to push the button.

2) Automation. But your automation has to somehow catch bugs that you can't even dream of. And not be triggered randomly. And somehow be able to be automated but not fixed at the source (as if you know how something will fail - why not just fix that over adding more status checks?)

jcwayne 7 years ago | | |

That may be true for small failures, but the AWS Status Page has routinely failed to report large scale outages.

loudtieblahblah 7 years ago | |

It's bc they're manually updated.

xur17 7 years ago | |

I've always wanted to build a public status page for services like Google Cloud and AWS with real tests against different services they offer. Not sure there is a good way to monetize though.

kureikain 7 years ago | | |

https://statusgator.com/ does that.

xur17 7 years ago | | |

> StatusGator monitors the service status pages of more than 410 cloud services.

It looks like they are just an aggregator for status pages. I want something that does its own testing to determine if a service is up or down.

skrebbel 7 years ago | |

I've always wondered about this strategy. I'm a developer and a business owner. I'm in AWS's core target demographic. I know three things about AWS:

    - Their products have weird names
    - Their web console has horrible UX
    - The lie about uptime and server status

I'm not looking to switch to AWS soon.

philg_jr 7 years ago | | |

I take it you have never used Azure's UI? AWS is much better. I can't comment on Google Cloud because I've never used it.

KZeillmann 7 years ago | | |

Some of the AWS UX is okay but the parameter store UX for the web console is some of the worst I've seen. It's difficult to navigate and paginate, and the search function is barely worth using. It would be amazing if they had fuzzy search on the names.

philg_jr 7 years ago | | |

I do agree that the CloudWatch console search is like this, where you need to type out the entire string from the beginning to get it to match. But for the CloudFormation and Lambda consoles, I can just type a piece of the string I'm trying to find and all matching will just show up.

gtsteve 7 years ago | | |

The UX is a bit poor and there are long standing issues like the S3 console timeout bug and the SWF console not obeying any known law of web design but serious teams don't really use the console, they use tools like Terraform, CloudFront or they roll their own automation.

I use the AWS APIs almost every day but probably log in once a week or so.

zedpm 7 years ago |

FWIW, if you're already logged in to the console, it seems to be working just fine. In fact, if I try to go to a specific service page, e.g. https://s3.console.aws.amazon.com/s3/home?region=us-east-1# instead of just browsing to the console home page, that also appears to work.

PDFormat 7 years ago | |

This way worked for me too. I was able to go straight to the Lambda console.

DonHopkins 7 years ago |

At least AWS doesn't start spamming you weeks after servers go down!

I canceled a dedicated server with IBM SoftLayer (nee ThePlanet), and a few weeks later I started receiving hourly IPAlerts about it being offline!

The server was canceled so there was nowhere in the interface for me to turn them off!

I opened a ticket, and they said other users were experiencing it too, and they though they had it fixed, and asked it I was still getting them. I was.

Their only suggestion was for me to make an email filter to ignore the IPAlerts, but what about the IPAlerts for servers I hadn't canceled that I actually want to see?

We went several rounds of this, each time they thought they had it fixed, and asked if I was still receiving them, and of course I was, like clockwork.

It's been more than a week and a half, and I'm STILL getting them!

I kept posting the raw email bodies so they could tell by the headers where it was coming from.

I even begged them to deploy one of their most powerful firewalls around the offending legacy nagios server to protect me from it, but they wouldn't do that.

I'm afraid if I cancel my other two servers and move to AWS, they's start spamming me with TWO MORE never-ending sets of IPAlerts about canceled servers!

What a passive-aggressive way of punishing long time customers for canceling their servers!

Has anybody experienced anything like this with AWS?

    Received:  from ipalert05.dllstx6.inside.theplanet.com
        by mx.softlayer.com with esmtps (TLSv1:DHE-RSA-AES256-SHA:256)
        (Exim)
        (envelope-from <ipalert@softlayer.com>)
        for xxx@xxx.com
        id 1hJhFu-0003RG-Tm; Thu, 25 Apr 2019 11:29:50 -0500
    Received: (from nagios@localhost)
        by ipalert.theplanet.com (8.13.6/8.13.6/Submit) id x3PGSB17029986
        for xxx@xxx.com; Thu, 25 Apr 2019 11:28:11 -0500 (CDT)
        (envelope-from nagios)
    Date: Thu, 25 Apr 2019 11:28:11 -0500 (CDT)
    Message-Id: <201904251628.x3PGSB17029986@ipalert.theplanet.com>
    To: xxx@xxx.com
    From: <ipalert@softlayer.com>
    Subject: PROBLEM: xxx.xxx.com

kylek 7 years ago |

I think the thing I miss most about AWS was sipping coffee while watching the #wtf peanut gallery on wednesday mornings and during events like this

an_account_name 7 years ago | |

!fq cockroaches

amazon_throw 7 years ago | | |

In the post-nuclear-holocaust wasteland where only cockroaches survive, the cockroaches will use IRC to rebuild.

rosencrantz 7 years ago |

Looks like it is not reported having a look at the status dashboard https://status.aws.amazon.com/

sgt 7 years ago | |

They follow the industry standard of hard coding their status dashboards.

mulka 7 years ago |

Individual service pages seem to still be up. So, you can go to the EC2 page for example, click on the services dropdown on the top to select the service you want. https://console.aws.amazon.com/ec2/v2/home?region=us-east-1

busterarm 7 years ago |

Can confirm that I am having this issue.

API requests/CLI work fine though.

HatchedLake721 7 years ago |

You should be able to login through https://us-west-2.console.aws.amazon.com/console/home?region...

bdcravens 7 years ago | |

And then switch to us-east-1 services?

crystite 7 years ago |

https://status.aws.amazon.com/

Shows AWS Web Console:

9:50 AM PDT We are investigating increased error rates when loading the AWS Management Console.

crystite 7 years ago | |

AWS keeps their status page up2date these days. It's probably easier to check that page than combing through posts on HN, but to each their own.

This makes me wonder... do people keep posting and looking here because they want an element of control, they don't trust or know about status.aws.amazon.com ... or why?

nerdjon 7 years ago | | |

The status page was stating all green for about 20 or so minutes after this post was made.

Not really "up2date", so this post was valuable for everyone that was seeing errors and unsure how widespread it was.

ulysses 7 years ago | | |

The date on the status page is 9:50 AM PDT, the original poster posted well before that.

It isn't historically uncommon for AWS outage information to be posted and commented on here well before the status page is updated.

TheGRS 7 years ago |

I'll just report that we had an issue with an Elastic Beanstalk deployment that seems to have resolved itself now. Not sure if its related, but simply retrying the deployment resolved it without making any changes, so it sure seems related.

crystite 7 years ago |

According to the AWS Status Page

Alternate link is available at: https://us-west-2.console.aws.amazon.com/ec2/v2/home

coffeesn0b 7 years ago |

Is this the apocalypse?

zonywhoop 7 years ago |

DynamoDB access via api in us-east-1 is down for me as well. Also, access to any region or resource through SSO is failing too.

slenk 7 years ago | |

I am having issues with SAML authentication as well

goodfight 7 years ago |

Strange, I'm experiencing this issue as well. Wonder if it's an attack or just normal failure

actionowl 7 years ago |

Also having issues, the AWS status page is oblivious...

cortesoft 7 years ago |

Same, came here to check if there was any info on it.

giffyUnicorn 7 years ago |

confirmed, us-west works for logging into the console.

jshaw3 7 years ago |

us-east-1 is down for me

giffyUnicorn 7 years ago |

well this is embarrassing.

hsk823 7 years ago |

So you can get to AWS resources going direct to the respective URLs, such as https://s3.console.aws.amazon.com/s3/ or https://console.aws.amazon.com/ec2/v2/ or even changing regions works. It seems to just affect https://console.aws.amazon.com/console/