That's an understatement. A close reading reveals that it misrepresents the position of Tufte's writing, the conclusions of the investigation board, and the report of the subsequent return-to-flight group.

And it uses "sweetened" (fabricated) figures, to boot. <https://news.ycombinator.com/item?id=30622688>

> Attempting to have slides serve both as projected visuals and as stand-alone handouts makes for bad visuals and bad documentation. Yet, this is a typical, acceptable approach. PowerPoint (or Keynote) is a tool for displaying visual information, information that helps you tell your story, make your case, or prove your point. PowerPoint is a terrible tool for making written documents, that's what word processors are for.

I think that’s on point for many companies. A lot of the terrible slides you see in meetings are actually intended as documentation after the fact, and few people recognize (or care) that this makes for a terrible presentation.

Ironically, I think Powerpoint isn’t such a bad tool for creating handouts. If the intended reader reads the document on their screen instead of printing it, a nice PDF with screen-shaped pages might actually be close to optimal.

You just have to be 100% clear whether you’re creating a document or a presentation.

[1] http://mamamusings.net/archives/2005/11/19/the_culture_of_th...

The title is not reassuring. Conservatism in engineering is essentially about creating safety margins through conservative estimation. The title is saying we need to be careful because a tile was likely penetrated. Hell, if I remember correctly they were reporting that there was known tile damage on the news before reentry, but that they didn't know the extent.

"NASA felt the engineers didn’t know what would happen but that all data pointed to there not being enough damage to put the lives of the crew in danger."

If you thought they didn't know, then ask them what they do know! It's right on the slide that flight conditions are outside of test parameters and that the mass of the projectile was much higher. How the F do you work at NASA and not understand the basic principles of mass, velocity, and energy well enough for that to stand out enough to ask questions or run your own calculations...

The reason the slide is laid out the way it is, is because it's describing the thought process and creates a deductive argument for how they got to their concern. This is a presentation for a briefing for other engineers, not a conference or sales pitch. It's supposed to be formal and contain the synopsis of technical points. Using projectors for technical briefings predates the use of PowerPoint. I see nothing wrong with the layout in that context.

Edit: why downvote without a reply? NASA has a history of blaming vendors when they screw up. This looks like another example to me. The presentation format does not have any issues given the setting and target audience.

When the shuttle design was finalized in the late 1970s they knew it had a 2-3% chance of a hull loss per launch. They were still planning to launch it 50 times a year so that would have meant losing a shuttle and crew every year!

The shuttle had hundreds of critical flaws and that 'normalization of deviance' meeting at which slides like this were shown at was a routine part of each shuttle launch. For each of these unacceptable situations they had to convince themselves that, with some mitigation (or not), they could accept it. It was inevitable that something like this was going to happen and then there would be recriminations about the details of that meeting.

Every other crewed space vehicle had an escape system to get the crew away from a failed rocket. The Challenger crew survived the explosion but were killed when the reinforced crew section hit the ocean. Similarly the Colombia astronauts were killed by a thermal protection system that was "unsafe at any speed". When the first few shuttles were launched there was a huge amount of concern about tiles breaking and coming off. Once they'd dodged the bullet a few times they assumed it was alright but it wasn't...

In the literature "normalization of deviance" has turned from a formal process used in managing dangerous technology to incidents such as: surgeon takes a crap and goes to work without washing his hands, forklift operator smokes pot and operates, etc.

One huge issue, beyond whether a rescue mission would've been possible, is whether it would've be ethical. If NASA knew that Columbia was stranded in orbit, then it would be knowingly sending a second crew up on a vehicle with the exact same potential problem, with no time to mitigate it. I'm sure a rescue crew would've volunteered despite the risks, but anyway the point is that "the slide that killed seven people" is erasing all of these questions.

That's not how I remember it being presented to the public. The official word at the time was that there were no feasible rescue options. Yes, they could have done a spacewalk to inspect the damage but if it had been bad there still wasn't anything that could have been done. I think the main problem with launching a rescue mission was the time it took NASA to get a shuttle ready for launch.

Reading word for word off a text-heavy deck in a monotone with no images or diagrams is a recipe for disaster. I tend to have my decks (back when I was doing presentations) be relatively text-lite and involve images/diagrams that back up my talking points. And I've seen image-heavy decks that really don't convey anything either.

Then they would have been fired unceremoniously and replaced with engineers that knew better than to make their bosses look bad. (who themselves would then, of course, been held responsible for apparently preventable deaths).

Stop blaming the engineers for this stuff. This is the fault of the timeline chasers.

This is not just a normal, routine presentation. This is an all hands on deck emergency and discussion. And NASA isn't just a bunch of MBA's, but rather people who have spent their entire careers immersed in this kind of stuff.

No matter what is one the slide, I expect that the audience asks detailed questions. Even if the slide has just a big thumbs up emoji, I suspect you would still get a lot of really hard questions.

Think about presentations on programming, where someone in the audience points out that the example code on the slide is incorrect/won't compile/undefined behavior

I would expect a bunch of geeks (which I think would be there at NASA) to scrutinize the slide and try to find any flaw in the logic. Especially when the lives of people they deeply care about are on the line.

If they are so cavalier about human life that they just skip the details of the slide while making literal life and death decisions, it speaks of a very deep culture rot that goes far beyond PowerPoint.

Slides make sense as a way to introduce the outline of concept to a broad audience in a way that requires much less effort than 1-on-1 discussions. They are not a means for coming to decisions. If you need to make a critical, life- and business-shaping decision off the back of a side deck, you should instead delegate the decision to someone who knows more than you.

That is part of the reason Powerpoint is everywhere. You cannot assume that people have read anything before the meeting, you cannot assume they will read during the meeting, so you need to read it out loud to have a decent chance of it being received.

I am also not thrilled accepting the use of titles and formatting as a excuse to skim 100 words. It is just the refusal to read/comprehend on a smaller scale.

https://www.edwardtufte.com/bboard/q-and-a-fetch-msg?msg_id=...

One slide and bad culture killed seven people.

Wouldn’t the foam initially be travelling as fast as the spacecraft? So it’s just the time between it’s release and hitting the wing to accelerate.

There is a simple low-effort high-information solution to this problem - have everyone vote (or bet) on the decision. This, more than anything else, will reveal whether or not you've reached understanding/consensus/alignment.

(This is not to say that the final decision should be made by voting, rather it's to gauge the level of consensus)

Alternative title: death by ears, how failing to listen and communicate killed seven people.

My own personal experience is that it's easier to be concerned with the small things (we have to have naming conventions), than with big things (are we building the right thing). I think there is a tendency to think "it'll probably work out".

I agree: it's an awful slide. But the information was there. And I can imagine that the engineers were asked to assemble that presentation in a couple days, so I'd be surprised if they could do anything better.

I agree: an outsider would not understand a word of that slide. So what? What was the audience of that presentation? Why did the audience not read the slide / documentation beforehand? Why was it not understandable by them?

Again: the point here is the stakes. Would I read such a presentation for my day-to-day work? Probably not. Would I read it until I understand every single word if there were 7 lifes at stake? OF COURSE! Would I try to understand the engineers that had to assemble a comprehensive and credible document in a matter of days, and do my part of the work? If it was my day-to-day job, probably not. If there were 7 lifes at stake? OF COURSE.

So yeah, terrible slide. Don't try to justify people not reading it, though.

https://www.amazon.com/gp/product/0596522347/ref=ppx_yo_dt_b... https://www.amazon.com/gp/product/0470632011/ref=ppx_yo_dt_b... https://www.amazon.com/gp/product/1101980168/ref=ppx_yo_dt_b...

If you only have time to go through 1 book, I would recommend Slide:ology.

I mean obviously Boeing engineers need to communicate to NASA their assessment of the situation, but they don't want to be blamed for any technical difficulties (e.g. if second shuttle would have to be launched to save the crew). So they think Columbia will probably be fine, but let's communicate our worries to NASA, but let's do that in deliberately vague and conspicuous language, in hope that NASA managers won't see the fine print.

But this '...has grown exponentially...' is just such BS. sigh I just cannot get used to this expression entering lay language.

Alternatives like Prezi exist, but are not really going to be accepted in formal presentations https://infovisu.com/assets/pubs/linder2015beyond.pdf .

If you really bring me a physical piece of paper today, I doubt I would be able to keep track of it.

We don't need Tufte and his subtle points to see this was an abominable piece of communication. More important, would be the question "is it safe to call out a bull shit slide in a corporate meeting". We hear of how Bezos or Jobs would be rude and obnoxious to their employees when something was not laid out clearly. This, on the other hand is where politeness takes us.

To say that "the slide doesn't have any issues" is laughable on the face of it. But it's immaterial; your claim is that "NASA just ignored the engineers from Boeing" rather than "NASA didn't understand the engineers from Boeing". Communication is a two party process, and believe it or not, NASA isn't actually incentivized to take risks that lead to loss of life and damages public perception of them; it's far more likely they didn't understand the stakes, and looking at the slide from that perspective, it's very easy to see why they would not have understood the stakes even if the Boeing engineers did.

It is a godawfull title. I believe that is how it reads to you but it totally reads the opposite to me. I read the title and it translates in my head as “we reviewed the test results and they suggest that the tiles are built sturdy enough to not get penetrated”. Exactly because what you say the word “conservatism” means to me that a system is designed to meet the loads plus reasonable safety margin. So if the review of test data indicates conservatism that means to me that the test found the test object roboust even with a reasonable safety margin. Otherwise i wouldn’t say that it “indicates conservatism” but that it “indicates lack of safety margin”.

> The reason the slide is laid out the way it is, is because it's describing the thought process

I agree, but that is not a good thing. People think in all kind of haphazard ways, before you communicate to others it is on you to look at your ramblings and make it orderly. The penultimate sentence is the most important one that should go first “flight conditions is significantly outside of test database”. That doesn’t mean that the tile is broken, nor does it mean that it is not broken. It means that we can’t tell from our tests.

And, for context, Edward Tufte, whose review of the slides is being referenced in this article, is the same one who misunderstood and misrepresented what the actual issue was during the briefing the night before the Challenger launch, in his paper reviewing the presentation the engineers made then.

Edit: previous HN discussions of Tufte's Challenger review here:

https://news.ycombinator.com/item?id=10989358

https://news.ycombinator.com/item?id=19034783

It looks like the bad "before" example in a presentation skills workshop. This was created by engineers working on life and death issues involving billions of dollars of hardware.

A quick and dirty re-writing of the title (and slide):

_______________

Review of Test data indicates incident is well outside of safety margins.

- Volume of ramp is 1920 cu in vs 3 cu in for test

- Once tile is penetrated SOFI can cause significant damage.

- Flight condition significantly outside of test database

_______________

Now that should get a reader's attention.

To the point of the article, I think this is the wrong takeaway, meaning the slides were not communicating effectively.

The second point illustrates this. It says the models were overpredicting the penetration. Meaning the models were conservative and the the actual penetration was likely less than what models show. They were setting the table for an optimistic outlook.

The real issue, IMO, is highlighted later in the article where there isn't sufficient fidelity in the tests to back up those claims. Tests after the incident showed the foam acted very differently at the delta-v that actually occurred.

And regarding your point about blaming contractors, the vast majority of work done by NASA is done by contractors. NASA is, to some extent, a pass-through organization that funds other organizations like Boeing, Lockheed, Honeywell, Jacobs, etc.

>If you thought they didn't know, then ask them what they do know!

This gets to the same cognitive biases that led to Challenger, EVA 23 and a host of smaller incidents nobody hears about. Data is not objectively weighed in these situations because of schedule pressure, optimism bias, etc. In this case, most launches were showing foam shedding with no issue, so it lead to a false belief that it wasn't dangerous even though it was out-of-spec. Add to that a slide that says the models are too conservative and you can see where cognitive biases may influence the decision. Lastly, most people like to think they're self-aware enough to identify these biases in real-time, but they aren't. It's also why the incident lead to a separate organization within NASA focused on safety, quality, and risk that has a segregated chain of command.

I'm not so sure how many people are familiar with the term "conservatism" as used here. I'm not. Some might be aware, those who are not aware will just skip over

I read this slide a couple of times. There's no thought order, no connection between the topics (even if we assume people are familiar with the subject) and several typos.

It is not a Powerpoint fault it is a fault of whoever wrote this.

This is an issue with information hierarchy. If this is a risk (and I can't imagine what might have been a bigger risk at that mission) it needs to be brought into attention. Not added to line 4 of slide 7 and be done with it.

I don't expect "foam strike more than 600 times bigger than test data" would go over well politically. You'd be telling the audience that these people will die while everybody watches. No manager want to be the messenger for that kind of messages.

Have you ever worked for a business with “management”?

(not the one that downvoted your comment btw)

At the Flight Readiness Review for the solid rocket boosters before launch the engineers of Morton Thiokol (the solid rocket booster manufacturer) objected to launch because of the detrimental effect of the cold temperatures on the o-rings in the solid rocket boosters. They had never launched in that cold of temperatures before and previous test data had shown erosion of the seals on previous flights. These concerns were not communicated by the Morton Thiokol management or NASA present at that flight readiness review to any of the higher level managers that got final approval on launch.

For Challenger it was a known issue and people specifically said "don't do this".

[0] https://history.nasa.gov/rogersrep/v1ch5.htm

Atlantis was already scheduled for launch 41 days after Columbia. Columbia had ~30 days of air. NASA believed that Atlantis prep could have been accelerated and launched in time to attempt a rescue.

https://arstechnica.com/science/2016/02/the-audacious-rescue...

That there was "no other feasible plan" than re-entry was presumably because the risk of re-entry was assumed to be low. If the risk was correctly estimated to be higher, then it seems likely the rescue plan would become feasible (since, what other choice would NASA have?)

Everyone here is dogging the Slide. Sure it sucks - but I DISTINCTLY remember analysis of resuce. Do a space walk and repair it; send a second shuttle (they existing crew would run out); something else.

Also, the crew had a chance to call their loved ones and "sat good bye", and the pilot was anticipating the wing melting off.

NASA knew the risk, they tried to prevent it - but the shuttle still crashed

This led to a certain realistic fatalism:

"Then [Linda Ham] delivered the sentence that would define the rest of the tragedy; a sentence that was repeated as common wisdom by almost every senior manager that I talked to over the next two weeks: ‘You know, if there was any real damage done to the wing, there is nothing we can do about it.’ As unsettling as that was, I had to agree; going back to the first shuttle flight it had been well known that there was no way to repair the heat shield in flight. Nobody, not even me, thought about a rescue mission. Why would we?" - Wayne Hale, https://waynehale.wordpress.com/2013/01/07/after-ten-years-d...

Once it became clear, during the investigation, that the foam impact had created a hole in the reinforced carbon-carbon leading edge of the wing, the board's chairman, Admiral Hal Gehman, insisted that a test should be performed to demonstrate that this was likely. As this meant destroying one of the few spare parts, and it had not been decided at this point to retire the shuttles, he was unsure whether this was worth doing, but what convinced him to go ahead was the number of engineers and managers who still doubted this could have happened, despite all the evidence.

It is never just one thing.

The whole of Wayne Hale's retrospective starts here: https://waynehale.wordpress.com/2012/08/14/after-ten-years-w...

The people making the decisions, whether they listened to the presentation or not, perhaps gave too much credence to what was on the slide. The audio portion is temporal, even if recorded. You're there in the moment listening, but it doesn't necessarily last.

But the slide is "eternal". Always there to be referenced, or passed along. The slide was going to be viewed repeatedly over a longer time frame. Biasing the memory of those who may have been at the presentation, and serving as a "single" source of truth for those who did not.

As much as PP is meant to be a visual AID, most folks are actually pretty lousy at using it that way, and PP has morphed, even if unintentionally, as an artifact of record.

I'm as guilty as the next guy wanting to skim the PP deck rather than listening to the presentation. I can skim a deck in 5m, vs sitting for 60m listening.

Similarly for technical papers. Read the intro, the summary, skip to the end, read the conclusion. Only if any of what I read is actually interesting will I dig deeper in to the paper.

It’s a wall of text. It should have had a single sentence in bold giant font:

WE HAVE NEVER TESTED AN INSULATION COLLISION AT ANYTHING CLOSE TO THIS LEVEL

"SOFI and ramp mean the same thing, whats the reader to dooooo"

"Significant is used 5 times... without explanation!"

"There are 100 words!"

Yes, we need to know how the presentation went, and we also. need to know what NASA would have alternatively done and if it was honestly considered at all or even feasible

On a side note, its crazy that a couple tiles compromise the entire vessel, but I understand that the heat would travel across the inner metal. Its still crazy to think there isn't some other kind of dissipation measure possible.

[EDIT] There's also a kind of horrendous, illegible house style some places, that's expected to be how these documents look. The US military and any big businesses that work heavily with them are infamous for this. They routinely produce some comically terrible decks and graphics. I wouldn't be surprised if that weird, seemingly-intentionally-hard-to-read style is also present in at least some parts of NASA, especially up at the administration level.

That's true, but it's also how much time the presenter had to put the presentation together. Was he already 6 weeks behind on a dozen (meaningless) deadlines when he found out he had to give a presentation tomorrow morning at 8 AM?

I've been told—very seriously—by multiple management consultants that public and private sector executives alike won't read a damn thing unless it shows up in powerpoint format, and even then you have to walk them through point by point or they'll miss most of it. This, when the documents are coming from people they're paying tons of money specifically to tell them stuff.

The company in question (you've heard of them, if you've heard of any management consulting companies at all) quite literally had an off-shored office dedicated to producing PowerPoints decks from notes overnight, while everyone on an actual engagement was sleeping. The primary tangible output of an engagement, as I understand it, is, overwhelmingly, PowerPoint decks. It's the Final Draft of the upper-end management world—apparently, you'll be dismissed and lose face if you show up with anything else, or even send something else in an email.

If this was a presentation done in high school they would have gotten a failing grade. And I think we can expect more from university educated people.

I mean, NASA is the one who called in the vendor engineers to answer questions they had. If you didn't get answers to those questions, or if the answers weren't clear enough for you, then NASA needs communicate that. You can't get the information you want if you don't ask the questions. Everyone here is blaming a slide deck (that had the information on it!) instead of asking how NASA could have ignored that information. Slide formatting is a pitiful excuse for lack of due diligence by the recipients of that briefing - the people responsible for the safety of the at-risk personnel.