Backblaze S3 API is having some downtime

atYevP 5 years ago |

Yev here - we're investing the S3-endpoint issue - apologies for delay in response here, it took them a bit of time to wake me up.

*Edit -> we've identified the fix and are rolling it out, you should see S3-Endpoint service resuming shortly!

mahesh_rm 5 years ago | |

Happy to know who to bother the next time it happens! :P EDIT: But really, get a simple b2s3api status page up, doesn't need to be anything fancy..

atYevP 5 years ago | | |

Well, hopefully there won't be a next time - but also, bothering me doesn't help debug or resolve the problem, our Ops/Tech team do that :D

OJFord 5 years ago | | |

I was baffled that I couldn't find a status page at all, but I assumed I was just failing to find it.

> Well, hopefully there won't be a next time

Are you seriously saying that not only is there not one, but you're against having one because you hope the status is always green?

atYevP 5 years ago | | |

No, that comment was edited after I had responded - we're not against status pages - they are on the roadmap, but unfortunately not implemented yet.

autorun 5 years ago | | |

you just need to update your pixelated loading spinner, everything else is great, thanks for so much, sorry for so little

OJFord 5 years ago | | |

Ah, glad to hear, thanks!

doctorfoo 5 years ago | |

Thank you, and I do love your service. It would be nice to see a post mortem on what went down. /goes to sleep

nhoughto 5 years ago |

Looks like Europe is down too?

https://s3.eu-central-003.backblazeb2.com/

koreanguy 5 years ago |

https://bgp.he.net/dns/backblazeb2.com#_website

This site can’t be reachedCheck if there is a typo in backblazeb.com. If spelling is correct, try running Windows Network Diagnostics. DNS_PROBE_FINISHED_NXDOMAIN

nhoughto 5 years ago |

But non s3, b2 api is up

https://f002.backblazeb2.com/file/blah

https://f003.backblazeb2.com/file/blah

akchou 5 years ago |

Today's task: starting working on moving back to aws. Storage savings cant compete with the cost of developer time and downtime even when we store a petabyte.

OJFord 5 years ago |

Yes! Glad it's not just me. And no status page. (?!)

jsiepkes 5 years ago | |

I also can't find a status page and not so much as a peep from Backblaze anywhere. I pinged them on Twitter [1] to no avail. I like Backblaze but this has been going on for 3 hours now. You can't be down for 3 hours without any communication about it...

[1] https://twitter.com/JasperSiepkes/status/1329347119774375937

OJFord 5 years ago | | |

From the support page, I get the impression nothing will happen until 9am Pacific. Tickets have a 1d response time, and chat is Pacific working day with a lunch break.

Bottom of https://www.backblaze.com/help.html

I think they're all sound asleep, blissfully unaware of the day they'll wake up to!

slice-beans 5 years ago | | |

It doesn't reflect well on their monitoring/on-call setup that everyone else was paged before them, assuming they are indeed unaware.

Perhaps the first they'll know of the problem is when they read it on HN

doctorfoo 5 years ago | | |

yup, here desperately migrating to the regular b2 API...

zhan_eg 5 years ago | |

Well, it looks like it is in the time frame of their scheduled maintenance https://www.backblaze.com/scheduled-maintenance.html -- Disregard, I got AM and PM confused :(

OJFord 5 years ago | | |

No, I saw that, but that says 3 pm not am.

Pacific time is several hours behind me in the UK and it's 11am here.

I don't think it was even Thursday yet Pacific time when this started.

jsiepkes 5 years ago | | |

Can't be because this has been going on for over 3 hours now. The timeframe is only 1 hour.

mahesh_rm 5 years ago |

Argh. I believed I was going insane. How can this be the only B2 status page?

chima3ra 5 years ago |

well, what can u expect , 4 times cheaper than AWS, 4 times less stability and professionalism.

I am now wandering if my data is safe there. I think it will be smart to keep a copy in aws glacier.

doctorfoo 5 years ago | |

Feeling kind of the same... I just completed moving my entire app and all user data from AWS. And here we are!

manojlds 5 years ago |

How prominent is usage of Backblaze and the S3 API in specific? Is HN going to be status page for any random service?