OFA + AWS

69 points by dylanvee 13 years ago | 18 comments

siliconc0w 13 years ago |

Anyone know of a good tool to make these types of diagrams? Preferably automatically? I've been meaning to create something that plugs into Chef and or nmap/netstat. In my experience, any diagram/inventory/whatever that isn't automatically updated invariably drifts from reality.

danoprey 13 years ago | |

We're attempting just that at http://www.madeiracloud.com/ , would love to chat more: dan [at] mc2 [dot] io

jared314 13 years ago |

The diagram follows the style of the AWS Reference Architectures[0]. I have yet to find a published set of diagram components that compare well to that style.

Edit: Juan Domenech[1] has a small custom made component pallet in png form. And, it looks like the original diagrams are custom Adobe Illustrator drawings[2].

[0] http://aws.amazon.com/architecture/

[1] http://blog.domenech.org/2012/06/aws-diagrams-palette-v10.ht...

[2] http://blog.domenech.org/2012/05/aws-diagrams.html

mappu 13 years ago |

That's an awesome diagram and quite a few servers (but latency kills the map-overlay experience, i think a static png or svg might have been better...) Interesting that servers are are either paired autoscaling over two AZs, or statically provisioned in three AZs - or am i mis-reading the diagram? Two entire mirrors for testing and staging must have become a pretty big cost.

For others outside the US confused about what the application actually does, and what the Narwhal in the testing/staging pictures refers to, this helped; http://arstechnica.com/information-technology/2012/11/built-...

DrJ 13 years ago | |

Looks like the AutoScaling groups are applications or stores where they do not need to coordinate their actions. The 3-az deployments seems to be APIs, which I am guessing scales with each regions (to reduce data cost?) and probably brought up/down automatically with Puppet to handle post-launch configurations. (you are reading it correctly).

I would guess testing is a skeleton version of the entire deployment so the cost is minimal and just need to test new deploys and verification for tests.

Staging probably wasn't a full mirror, at best I would venture to guess they had hot swaps coming up in staging and then being switched against production via ELBs.

They mention costs a few times in articles, so I would venture to guess they did optimize around many of those corners.

vosper 13 years ago |

This is really great - I'd love to have a similar diagram for data flow and components of the systems I work on. It might not benefit engineers very much but it'd be fantastic for clients or some of the less technical people in the company.

I wonder how hard (I'm thinking hard?) it would be to have a tool for something like this, making the layout easy and revealing more and more information as you zoom in.

Even better if it was live and updating with system metrics.

chubot 13 years ago | |

Yeah it looks kind of cool, but kinda hard to read. Is this graphic based on something else?

I would like to see one where the labels are clearer. It doesn't seem like the 3D perspective adds any clarity.

danoprey 13 years ago | |

You should check out http://www.madeiracloud.com/ , we're working on something very similar to what you described.

doctorpangloss 13 years ago |

When folks from the Romney campaign complained of technical deficiencies in his get-out-the-vote operation, pundits and news outlets intimated that the complaints were scapegoating IT for bad policy.

But looking at this diagram, I'm pleased to see that engineering was a big part of the Obama campaign. It really hammers home how badly out of their league Romney's TechOps team was.

obilgic 13 years ago |

http://aws.amazon.com/solutions/case-studies/obama/

twic 13 years ago |

Is each one of those little grey cuboids an virtual machine? If so, that is rather a lot of virtual machines. I am surprised so many were necessary. What kind of load was this system handling?

harper 13 years ago |

i guess this leaked. ;)

harper 13 years ago | |

Turns out it was released. We had been working very hard to get this diagram out. Last I was involved, it was stuck with lawyers, etc.

OFA is awesome for releasing this. Hopefully next will be some of our codes and projects.

sneak 13 years ago | |

Please watch this: http://www.youtube.com/watch?v=QNsePZj_Yks

You are materially responsible for the re-election of someone who commits large-scale crime (in the form of blatantly unconstitutional surveillance of Americans under no suspicion of criminal activity).

Please reconsider your choices. You are working for the wrong side.

SkyMarshal 13 years ago | | |

>You are working for the wrong side.

Which side should he be working for that doesn't do any of that?

dylanvee 13 years ago | |

Saw it on Twitter and had to submit, y'all are pretty cool.

jlgaddis 13 years ago | |

Oops, wonder how that happened? ;)