a bug, not a feature. Genderless* cyberfae & co. at your service
assigned adult by the inexorable passage of time don't use he/him or she/her pronouns for any of us without express permission
note that if we ever make you uncomfortable in any way please tell us so we know to stop. we're not always good at figuring these things out on our own
Discord post-mortem from a cloud engineer on the outside
Inferences I'm taking from what I can see about the Discord/GCP outage:
1.) Discord's database and storage is effectively single homed. If they had a hot failover it woulda failed over. If they had a cold failover its cost to fail over is probably high or else they would have.
2.) Discord's DB is intolerant of single-region networking issues. Google reports 8% packet loss in one cluster in one DC, and that knocked Discord offline.
3.) Discord may or may not have backups (they better) but the restoration time for a dataset of their size would be days since it's single-homed.
4.) Discord is reliant on Google Cloud. If they had their infrastructure striped across multiple providers, see point 1.
5.) Discord has no response process for if Google Cloud just completely soils the bed, as noted by their updates saying "All we can do is wait for GCP to fix itself"
6.) Discord has no way to gracefully handle a failure between API service nodes and their monolith DB/cache. Granted the nature of the app limits what could be done, but "Wait and hope the message doesn't redline" isn't good and just increases load in a fragile situation.
Synthesis.) A major incident could knock Discord offline for days, or permanently, depending on the reliability of their backups. Absolutely have backup methods for reaching your important contacts.
Representation for intersex ppl and cis folk w unusual hormone levels, chromosomes etc Its not just trans ladies that can grow beards or trans guys that grow boobs sex is complicated af and gender is only a thing bc we made it up
I wish people could stop conflating having trouble articulating your thoughts with no knowing things. I know what I'm talking about, I just don't know how to get you to know about it!
a bug, not a feature. Genderless* cyberfae & co. at your service
assigned adult by the inexorable passage of time don't use he/him or she/her pronouns for any of us without express permission
note that if we ever make you uncomfortable in any way please tell us so we know to stop. we're not always good at figuring these things out on our own
Joined Jun 2019
Computer Fairies is a Mastodon instance that aims to be as queer, friendly and furry as possible. We welcome all kinds of computer fairies!