Including “And. And. And. And. And.” in a Google doc causes it to crash(support.google.com) |
Including “And. And. And. And. And.” in a Google doc causes it to crash(support.google.com) |
Also. Also. Also. Also. Also.
Jimmy: What do you call yourselves?
Derek: "And And And."
Jimmy: "And And fuckin' And?"
Derek : Well, Ray's thinking of putting an exclamation mark after the second "and." Says it'd look deadly on the posters.
Jimm: Psshh...
Outspan: You don't like it? You think it should go at the end?
Jimmy: I think it should go up his arse.
Outspan: Well, we're not married to it.
> Dear Google Docs users, we are aware of the issue and working on a fix right now. Thank for surfacing this issue and sharing it with us. We will keep you posted!
> Deving
> Google Employee
The task was to "determine the cross ratio between sea and and and and and land".
Wouldn't the sentence 'I want to put a hyphen between the words Fish and And and And and Chips in my Fish-And-Chips sign' have been clearer if quotation marks had been placed before Fish, and between Fish and and, and and and And, and And and and, and and and And, and And and and, and and and Chips, as well as after Chips?
fking google.
there are good [as well as technical humurous] comments on the page.
Doesn't appear to be an issue for the android app, but that might be a cache thing.
Buffalo buffalo Buffalo buffalo buffalo buffalo Buffalo buffalo. (2)
[1] https://en.wikipedia.org/wiki/James_while_John_had_had_had_h...
[2] https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal...
> May 6, 2022 Update: We have fixed an issue in Docs related to repetitive use of the word ‘and.’ This fix should soon be in place for all customers.
However. However. However. However. However.
Why. Why. Why. Why. Why.
Edit: You guys have no sense of humor.
Which means, on a privacy standpoint, whatever you’re writing and guessing, they are absolutely processing something.
We the user are the product, apparently. This is mildly creepy to me because, I do vent on google docs sometimes. And assume only I can read it..
Top comment mentions it being grammar related, which requires processing however many written words before, to provide a possible suggestion.
Considering an “and” clause to meet all possible cases of suggestions may cause the program to crash.
(gotem)
Of course "Google" can read your doc. That has nothing to do with "you being the product" (infact it's the opposite since free Google Docs is a loss leader for their paid GSuite product).
That doesn't mean a person is reading it.
> Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize. Assume good faith
It is a joke, poking fun at our overly aggressive cyber crimes laws.
?
I read up to this point thinking you’re being overly pedantic about the specific use of a sarc mark and overly dismissive of the benefit of intent-clarifying hints in text. All the while thinking “I’m going to comment about how much I value intent-clarifying hints in text… and then I have to decide whether I want to mention I’m autistic, and prepare for all of the ways I might be misconstrued or dismissed further.”
So here we are, you’ve saved me the trouble of making that decision. I personally very much appreciate when people signal intent when their meaning can be ambiguous. It doesn’t always feel necessary for me, but it’s never once felt like it taken away from something I otherwise understood as obvious.
My take, which is much cooler than it was when I was gathering thoughts leading up to this but still mildly hot is: what harm does it do to you if someone voluntarily makes something more accessible to someone who’s not you? If you already grokked /s from a sarcastic remark, it’s a tiny bit of information you can scroll past. I understand not explicitly recognizing and endorsing how it might benefit autistic readers, but explicitly rejecting it because it might is baffling.
And it probably says more about me than the writer, but I always cringe when I see /s since it seems to be implying "Hey, in case you're a bit slow, this is sarcasm. Glad to help."
Sarcasm is by definition kind of elitist. You get it, and you're in the cool group, or you don't and you go on your way. It's a puzzle to solve. Removing all uncertainty removes its fundamental essence.
"{sarcasm here} /s {seriousness here}"
Has something to do with grammar. The document does not fail when `Show grammar suggestion` is turned off.
Each in caps 5 times with the same word with a period and space after each word and newline at the end is what I have found so far.
Can anyone find others?
Edit: added words that work found in other comments, and found more.
"Although", "Besides", and "Moreover" also trigger the behavior.
I did not expect them to weaponize it, but Skynet does as Skynet does.
Though, when I worked on Google's indexing system, some researchers were having machine learning generate regexes to run on every page in the visible web... and mis-implemented the feature to re-compile the regex to DFA (which re2 effectively lazily converts to NFA via memoization) for every single page load. The speed of the indexing system dropped in half one day, and <Edit: Name Witheld> dug into it. <Name Withheld> took the gperf graph showing the giant node for regex compilation and wrote a savage meme "Your mother doesn't work here. Optimize your own code.", and sent it out to the researchers in question and also the indexing team. Maybe 6 months earlier, I cut into the same researchers for writing and approving C++ header file changes that defined (and leaked) macros "DO" for "{" and "OD" for "}" so that they could write C++ a bit more like Bash. As I remember, the macro leak for DO caused compilation errors in SpiderMonkey, which I fixed. After fixing the breakage, I just left an extra comment on the code review "Really? Leaking DO and OD macros to avoid typing curly braces?" without emailing any lists. They were really embarrassed removed DO and OD within a couple of days, and <Name Withheld> didn't know that I had laid into them a bit 6 months earlier.
(I had implemented some very coarse-grained super-lightweight type-based data flow analysis into SpiderMonkey, which is why some of the Google headers were being included while compiling SpiderMonkey.)
Any web app pentesters here willing throw in their 2c? Could this offer insights into the way data is parsed in the backend, or might result in something more interesting than a crash?
would be funny if it were a remotely exploitable bug in an api endpoint.
famous last words. finding security relevant bugs is often a game of identifying what the original developers might have found to be not "too exciting" or places they were out of their depth and then focusing intense effort on finding their mistakes.
> Duration and the body: I thought about something I had read a while ago which said that a body, the body, is defined by duration. That a body in the present is inseparable from its previous state, that a body is linked in a continuous strand… and so on and so on… I thought about my body. It’s past. It’s present… Which made me think about the word and. And. And. And. And. And. Then.
> Now. Now. Now. Now. Now, I felt in the present like I was living always alongside a previous body. This is why I had expected to find myself in the apartment when I returned home from California.
The answer was as follows:
The landlord of the "Dog and Partridge" pub commissioned a signwriter to letter a new board outside. On looking at the work, the landlord declared that he liked the colour but would prefer more spacing between Dog and and, and and and Partridge.
But also this was from an early draft so it could be a mistake.
Let's place bets:
A) The user just let autocomplete "take it away" (not sure about this one since they were able to access the console)
B) Pen Testing?
C) Error copy and pasting?
D) Actual dialog in a sci-fi post-apocalyptic love story where a robot discovers the Turing test and attempts to set itself into an infinite loop.
The query complexity exploded, ES ran out of memory, and the index got corrupted and I don't remember why, it could not recover.
We had to re-index all the data. Lots of fun.
Lesson learned: prepare for the impossible, keep your infrastructure up to date, escape queries :)
What is twice is bad as that the worst of these, fail (at least to my eye) on two fronts.
1) Allowing SINGLE character searches. I.e No min query-length.
2) Not escaping querying - lucene(solr/es) syntax.
You can sometimes see the front-facing "html/js/api" is just a thin layer over ES/SOLR.(Which is not bad by itself) it's when you don't know the limitations or what sort of queries are x100 harder to do than orders.
Does anyone know why this bug doesn’t repro for some words other than And if this is the case?
> Obviously, this is partly intentional- Gregory Markov modelled his famous Chain after his younger brother, who would try to finish all of Gregory's sentences for him. The one way Markov could fool him would be to repeat the same word multiple times, and then say "Jinx", also I made all of this up, good luck Google Docs team
> Obviously, this is partly intentional- Gregory Markov modelled his famous Chain after his younger brother, who would try to finish all of Gregory's sentences for him. The one way Markov could fool him would be to repeat the same word multiple times, and then say "Jinx", also I made all of this up, good luck Google Docs team
– I bet you a beer you can't make a logical, grammatical sentence with five ands in a row
– I used to be a sign writer in a previous life and one of the jobs I had was to repaint the sign hanging over the door to this very pub. Except the publican was adamant that he wanted more space between the words. Where exactly I asked? In between the Pig and and, and and and Whistle he replied.
When I re-create the document from scratch, it does not crash.
When I copy the link to my non-crashing document and load it in a new tab, the crash then occurs when I edit the document in the new tab but not when I edit it in the original tab.
Thank you for the repro case!
It was very surprising that there was a way to get Clang to segfault. Should I report it somewhere?
The code is basically doing a recursive template expansion with some C++20 concept constraints. So it's not quite as simple as "And. And...", but it's similar in that certain input text causes a crash. I just have no idea whether to report it, or where.
In that case I'll spend some time to clean up the repro case and submit it. Thanks again.
Hypothesis from chatting about this with people nearby - somehow this string makes the grammar engine search space too large (that's the AI that predicts your next words) and it's running out of memory.
[1] https://www.pcmag.com/news/google-drive-flags-text-files-con...
I don't recall that the reason for that bug was ever explained. I wonder if the reason for this one will be.
Probably because it's out of paper.
Related: how is paper fed into an Apple Magic Keyboard - Hebrew?
Edit: Tried to reproduced with Arabic keyboard, plugged in now. Accidentally inverted right-to-left in brain.
dnA. dnA. dnA. dnA. dnA.
So, more seriously, what might cause this (mis)behavior?
EDIT: Ah, I had to reload the page, thank you child comments.
"And. And. And. And." caused no problems.
"And. And. And. And. And. And." also crashes (5 "And."s is a substring, so makes sense).
I cannot imagine how this bug is occurring.
That was the part that led to the apocalypse.
This was not a coincidence, because nothing is ever a coincidence.
Wouldn’t the sentence ‘I want to put a hyphen between the words Fish and And and And and Chips in my Fish-And-Chips sign’ have been clearer if quotation marks had been placed before Fish, and between Fish and and, and and and And, and And and and, and and and And, and And and and, and and and Chips, as well as after Chips?
Only with very careful intonation and appropriate pauses after a comma can someone read Gardner's above sentence aloud such that a listener can fully grok the connection between "and" and "and,", and "and," and "and", and "and" and "and", and "and" and "and", and "and" and "And,", and "And," and "and", and "and" and "And", and "And" and "and", and "and" and "and,", and "and," and "and", and "and" and "and", and "and" and "and", and "and" and "And,", and "And," and "and", and "and" and "And", and "And" and "and", and "and" and "and,", and "and," and "and", and "and" and "and", and "and" and "and".
(sentence due to Python 3).
"Far, får får får?"
"Nej, får får inte får, får får lamm."
English would be:
"Father, does sheep get sheep?"
"No, sheep does not get sheep, sheep gets lambs."
No, google translate does not make it unscathed through that sentence :-)
I always wanted to start a Yello cover band called Purple, just to perform a song called "Oh No!"
Mmmmm, Ugly... More ugly... Such a bad time... A really bad time...
https://en.wikipedia.org/wiki/Buffalo_buffalo_Buffalo_buffal...
Try saying "a Nand and an And" in a way that your students can understand you.
So, the the trick is using it as a conjunction and a noun.
If you look at the XML (change .docx to .zip) in styles.xml you see the declaration of the style "BodyText3":
<w:style w:type="paragraph" w:styleId="BodyText3"><w:name w:val="Body Text 3"/><w:basedOn w:val="Normal"/><w:semiHidden/><w:rPr><w:rFonts w:ascii="Wingdings" w:hAnsi="Wingdings"/><w:i/><w:iCs/><w:strike/><w:color w:val="FF0000"/><w:sz w:val="52"/></w:rPr></w:style>
1. Create a Word document (most likely using the Blank document template, Normal.dotm)
2. Type text of first line; press Enter; type text of second line (technically Word calls these 'paragraphs'--Shift+Enter inserts a newline within the same paragraph)
3. Place cursor on first paragraph
4. Click a Paragraph Style from the Styles ribbon section to apply it (e.g., the second one, No Spacing)
5. Right click the style; choose Modify...
6. Change the formatting (e.g., the font to Wingdings)
7. Confirm the dialog
8. Select the entire first paragraph (doesn't matter whether you include the end-of-paragraph/newline)
9. Use manual formatting to override your changes to the style so the text matches the default style, Normal (e.g. use the listbox in the ribbon to change the font back to Calibri)
Done; if you now delete the newline, the second paragraph merges with the first and takes on its style, as parent points out.
Styles are the "proper" way to format Word documents (interesting to see what fraction of users actually use them). They're like a mix of HTML tags and styles: each paragraph (div) must have exactly one Paragraph Style, and each span of text can only have one Character Style. "Manual" formatting has highest precedence, followed by Character Style, followed by Paragraph Style. The benefits are the same as in HTML: semantic correctness and easy restyling of the entire document (e.g., by applying Themes from the Design tab). This sequence of steps is a fairly good demonstration of how they're used.
Edit: clarify
This kind of thing can be easily debugged using the style inspector, "reveal formatting" which shows the formatting applied to the selected text and whether it's from paragraph formatting or direct text formatting.
RIP WordPerfect 'reveal codes'.
Skilled speakers frequently use repetitions of a word (like 'and') as an interjection[0]. It's a handy way of giving yourself a second to think without saying 'uhh' or 'umm' (which, for whatever reason, are considered 'bad' interjections), and seems to be a kind of defense against being interrupted.
[0] https://www.nbcnews.com/id/wbna42822623 (a Meet the Press transcript which contains eight "and, and"s and one "and, and, and"!)
The number required was pretty high, but you could get to it by copy pasting an already typed (In the emergeny dialer) number for a while, crashing the lockscreen until the next reset
At one such position (reasonably well-known product within the tech world), there was clear pushback not to file bugs of this nature.
I didn't stick with that position long.
That comment is from the submitter of the issue (and HN post), the poem is from Eliza Callahan (copy found here): https://durationandthebodyelizacallahan.cargo.site
The relevant excerpt: "I thought about my body. It’s past. It’s present… Which made me think about the word and. And. And. And. And. And. Then."
personally, i've happened across some pretty serious security bugs this way.
Please do. You can open an issue (Bugzilla has been deprecated) on LLVM's github repo: https://github.com/llvm/llvm-project
The message should include the URL.
Sounds like a bug that should be reported.
Edit: Got unbored. Discovered that "Firstly", "Secondly", "Thirdly", and "Fourthly" trigger the bug, but "Fifthly" and above do not.
Edit 2: The first word can actually be any of the "problem" words -- "Also. And. And. And. And." also has the issue.
> You should have seen this sign, it had 1000 ands in a row, and there was too much space between and and and and and and and and and ...
Explodes even faster.
https://stackoverflow.com/questions/592322/php-expects-t-paa...
One can like or dislike PHP for a variety of reasons, but surely being put off it for life because a non-English string is exposed to the end user is a bit much.
People are often defensive about PHP, but at the end of the day we choose our tools. I wrote a significant amount of PHP for a brief period and out of maybe 20 or so languages I've used professionally I found it by a wide margin the least pleasant to work with. Supposedly it has improved a lot, and I'm glad if so. But I have no desire to use it again, and am unlikely to ever have a reason to.
If PHP support has to handle the same stupid question about the meaning of a very unnecessary term that's confusing people and causing more support requests than anything else... then it made sense to change it.
I'm glad I read the blog post; it clears up a LOT of thoughts I had about the error... calling it childish seems off. Maybe you had a dog in the fight at some point?
No! But read the part where Rasmus Lerdorf "beams into" the conversation and his opinion is dismissed with "let's take a vote".
I mean, if you wanted a language that wasn't intentionally designed by Rasmus Lerdorf, you kinda picked the wrong one. This is what I was referring to.
For my part, I once made a bad assumption about how the Google SAX-style parser handled callbacks for zero-length XHTML start-stop tags. I presumed that <title/> would get a callback with the end-of-open-tag and start-of-close-tag pointers being equal, at the character after the close of the tag. Instead, the parser called the callback with the start-of-close-tag pointer after the start-of-close-tag pointer. (I had misinterpreted the API as passing pointers to the start and end of enclosed content.) I had test cases for un-closed <title> tags and <title></title>... but when my code hit production, the few pages (fewer than 1 in a million) that expressed an empty title as <title/> caused my code to try and construct a string with negative length and crashed that portion of the indexing system. I was right to feel very embarrassed for my oversight.
I remember the savage meme so clearly because it was quite out of the norm, and I felt bad for the guys since they were so quick to fix things even when not publicly shamed. (Only the author and reviewer got notified when I left a comment on their code review.)
Is there any science showing rude reviews improve some metric or some greater good?
There is a point where someone has to put their foot down and demand things be done properly, otherwise the inevitable consequence is a giant mess leading to disaster.
You might be used to small startup teams with responsible, experienced developers.
Out there in larger industry you get people doing absolutely crazy things that break huge, expensive systems.
There’s a difference between “oops I didn’t realise this library doesn’t scale the way I assumed it did” and “rewriting language symbols because I’m too stupid to use more than one syntax forever and ever.”
The standard you walk past is the standard you accept.
Are you saying you would walk past C code with DO…OD instead of {…}?
Would you accept that standard just to be “nice” all the time?
What I've learned since then is that, with a healthy company culture, you can give frank feedback -- even about stupid mistakes -- without it being a rebuke.
It's also important that you target the right problem. It's only human for smart people do stupid things sometimes. In the specific DO...OD example, I'd be more interested in how it got through a code review than the mistake itself. (Funny enough, early versions of the C source for Bash itself had macros like that.)
Now, if someone exhibits a pattern of ignoring good, constructive feedback, that's a problem. The folks I've seen like that both gave terrible feedback and took feedback terribly. That's a behavioral problem, and you only get so many chances to correct that before it's time for them to find other employment.
This is what management is for, too bad tech companies are too cool for that and prefer to live out lord of the flies.
Now if only we could all agree how that looks exactly
researcher types often get to work on problems that swe types find interesting, so some swes get grumbly. researcher types also tend to write pretty horrific code which adds salt to the wounds.
but there also can be a sort of envy that emanates from the research side. many are fully aware of their shortcomings and are envious of the swes ability to get things done on computers cleanly.
it often seems that there can be yearning to wear each other's hats from the two groups. if i were running a company i think i'd try to break down that wall as it would probably make a lot of people happier.
of course, the right answer here isn't a meme... it's performance regression tests in the ci suite. and maybe a little training on why customizing a programming language with macros is bad.
Both the author and the reviewer had passed C++ style certification. They knew why it was bad. They just got a little lazy and wanted to write their code in a way that felt familiar to them, and figured it was harmless. I got a bit grumpy at having to drop what I was doing right away to fix their mistake due to their laziness.
Finding some perfect compromise for those groups would probably be a superpower for whichever org pulled it off, I agree.
When you go far away from the humanities, you lose some humanity.
Bash (like Bourne shell before it) uses do/done. I think DO/OD was in an Algol (68?). I'm not sure where else?
Bash does do the reversing for if/fi and case/esac, though.
The original bournegol macros for DO and OD were a bit different:
#define DO ){
#define OD ;}
Typically you run Flume over docjoins in your project’s quota. If you screw up, all that should happen is that your Flume job is out of quota.
Also I’ve heard that Google automatically fails people who use regex in interviews, so their average engineers probably aren’t the best at it.
Hmm... I don't remember that, and I conducted over 100 interviews back when I was at Google. A blanket failure for using a regex sounds like it's missing some nuance. But, maybe that was before or after my time.
I wouldn't be surprised if they start out with some interview questions that are easy to implement as regexes, but then gradually make the question harder and more general, where a pushdown automaton is necessary for a general correct solution, and fail people who either don't get a correct answer or end up implementing a pushdown automaton by dynamically generating larger and larger regexes.
In general regexes finish instantly for normal sized inputs, but the actual O(n) might be quadratic or whatever.
> Probably…
Conclusions.
We cared because indexing latency is important. Engineers get paged and woken up in the middle of the night when indexing latency gets high. Doubling CPU usage meant doubling indexing latency. The pre-Percolator indexing system ran roughly 1,000 threads per box on roughly 1,000 Warp18 boxes. Of course, most of those threads were blocked on I/O. (Warp18 was something like 16 cores per box... I think those were the RevE Opterons that had the memory fence bug that caused the glibc semaphore implementation to not actually synchronize operations.) I personally would have written things using more async and/or non-blocking I/O, but the main reasoning in Google at the time was that massively multithreaded blocking I/O is a simpler programming model and closer to what was being taught in school those days.
Also, the indexing system's quota is huge. Someone made a mistake in their benchmark program that they thought got maximum utilization out of the Warp18 hardware. The indexing system actually got higher utilization than the hardware stress-test, resulting in cooling being under-provisioned in the Atlanta datacener (more than a decade ago, I'm sure indexing has moved now). There was a couple day stretch one hot Summer where we needed to stay on the phone with staff inside the Atlanta DC to keep track of the temperature inside the datacenter. When the temperatures started to get above something like 80 F, we needed to dial back on the number of workers in our Borg jobs.
While testing the prototype for the Percolator indexing system, we were luckily sitting right next to an engineer who planned datacenters. We luckily overheard him saying he couldn't figure out why the power usage seemed to be swinging so wildly. It was literally the power usage of a decent sized town suddenly ramping up and down. We asked him the exact times power usage was ramping up and down, and figured out it was when Frank and Daniel were starting and stopping the Percolator prototype.
My best guess is that the indexing system right before Percolator was a bit more energy efficient than Percolator, but maybe not. In any case, doubling CPU usage was a big deal. Real money, and in Atlanta, probably real carbon footprint. (The Dales was hydro, but I don't think much of Atlanta's power was renewable at that time.)
It sounds like it would be even cooler to coordinate with the local electricity suppliers to spin up and shed extra load a few minutes in advance, but I'm not sure if this would actually have a practical benefit.
> Also, the indexing system's quota is huge.
Well, it might no longer be the case, but for a number of years the indexing cluster was the largest in the entire fleet, especially when it came to memory. I don't know if you were around in the days of yl-i (I think), but it was edifying to rank it against the Top500...
Was the datacenter engineer Peter P?
I would just like to know exactly how are the engineers going to make the decisions, and the key point here being that they are in plural.
They all have different opinions, preferences, levels of ambition etc, how does this spontaneously merge into a cohesive team that pulls into the same direction?
It doesn't, in any other type of group in all of society. Every group needs a leader, so how is this spontaneously going to "just work" in software engineering?
If you don't want to have a manager, you have to have something else. Democracy? Plutocracy? Gerontocracy?
I think it's pretty obvious that just leaving it to chance is not going to be efficient or any type of healthy environment, and will probably quickly start to resemble something pretty nasty, especially because their livelihood is on the line.
if so that's silly, as it is literally 2x the work to type.
if they also included some sort of state that had to be managed, then it starts to verge towards reasonable.
Yes, "OD" was definitely just "}". DO might have been ") {" to allow
while (a < b DO
...
ODThe real answer is to just do something like
} // while
True on a US keyboard, not true on many others.
I still find this kind of preprocessor abuse horrific; I’m just responding to your specific statement.
i once had a team lead in a quasi monorepo situation tell me straight up to copy-paste some functions i was interested in from their code. they made up some fishy story about deployments. was the story true? did they even know? did anyone know? who even owned it? would they even know?
or the guy who got up in arms because i touched his team's code and didn't write the unit tests that they didn't write when they originally wrote the code.
oh, monorepos.
I had thought of the case for <title/> but basically out of laziness talked myself out of writing a separate test case for it, presuming the test cases for a zero-length title and an un-closed title covered the corner cases. (The entire document was guaranteed to be converted to valid UTF-8, perhaps with invalid character substitution characters, by that late in the Content Converter pipeline.)
So, as soon as someone asked me if I had changed the title parsing code, I was 50% sure of which corner case I had screwed up before looking at any code. It took me about 30 minutes to submit a code fix with updated test cases. I think less than 1 billion documents had been processed, resulting in less than 1,000 pages missing updates due to my bug.
I guess there is some pressure/tiredness/newness/taste/external factors driving the choices, test count, and style guide aberrations.
I think it was an issue of familiarity and comfort, not newness. The author joined Google before I did. If you're basically working with one other person, and you're rarely getting code reviews from outside your coding pair, and few other people interact with your code, it's easy to develop some bad habits and forget that your code choices have externalities. To be fair, the externalities were usually rather small.
One can never, however, be sure that someone really feels that way ("The secret to success is sincerity - once you can fake that, you've got it made!") Nevertheless, faking it involves never saying certain things, and that turns out to be hard, so the fallible approach that I aspire to is to assume sincerity unless given evidence to the contrary.
In this view, blameless postmortems are the right way to go, at least up to a point: neither embarrassment nor regret are things that can truly be imposed from outside, and definitely not sincerity.
But culture isn't a magic tool that completely neutralizes assholes, and there are assholes in _every_ organization of sufficient size, like the "[name redacted]" character in the previous post
Edit: I should point out that we were in Google NYC, as were the researchers. We had lunch with them some times. I remember the first name and face of the guy who submitted the slow code, but forget his family name and intentionally left his name out.
New Yorker to New Yorker adds a lot of context. Google's corporate culture is generally very Californian, but this happened all within the New York office between people who generally got along pretty well and knew each other decently.
In context, there was a heavy note of respect for someone's abilities and disappointment that they weren't performing at a top level. He wouldn't have been so harsh with someone who was new or was a weaker engineer, or someone who wasn't used to New York culture.
As a lifelong Californian who moved to NY, this makes sense.
Though I don't know if it's a blanket excuse... There's a reason that my friends at Goldman would agree that Google's culture is better, and a lot of it has to do with the difference between CA and NY culture. I completely dismiss the claims that the difference is just about surface-level abrasiveness, instead of noticeable differences in how unkindly people treat each other. I regularly see strangers here treat each other in ways that I didn't see in 30+ years in California.
Ie, "he's from New York" isn't quite a rebuttal to "he's being an asshole at work". The work culture is more tolerant of assholes, but that barely makes it better.
Have you tried trigraphs? (/me ducks.)
New York and Hong Kong are both very impatient places that don't suffer fools lightly. They're both brutally honest places, but with strong shared identities. Both places have lots of people from elsewhere who are sympathetic to being new to the city.
The quintessential New York story is the woman pushing a stroller (buggy, for my overseas friends) out of a subway car. A man in a hurry is passing her, looks over his shoulder, they make eye contact and silently agree he should help. He grabs the front of the stroller, they carry it up the stairs, and he sets the stroller down at the top of the stairs. They give each other a nod, and he runs off. Never a word is spoken. On the other hand, if you stop in the middle of the sidewalk to read a map or talk on the phone in New York, don't be surprised if people bump into you, perhaps on purpose.
I just left Goldman after more than a decade. I actually really preferred the corporate culture to the Google culture, but most of my time was in Hong Kong. I even knew a couple of people who worked at Goldman, went to Google, and came back to Goldman. I don't doubt that there are some assholes there, particularly in trading, but my experience in both Strats and Technology was overwhelmingly positive. In my experience, abusive MDs didn't last long, but maybe I was very lucky. If you're having a tough time, I don't want to give out full names, but some pretty unique names: Laurent, Jia, Dunstan, and C.K. all really seem to care for their folks. The heads of SRE, One Delta Strats, and Flow Strats in Hong Kong also seem to be really great managers, but Grahaeme, Nick, and Alain are pretty common names.
At Google, a friend of mine managed a couple of small teams (on a shoestring budget, using mostly interns) and completed a couple of projects ahead of time, only to get them cancelled by middle management right before launch. One of the projects even had heavy buy-in from upper management, but he got essentially zero credit for either one because they were cancelled before launching. It was incredibly frustrating for him and he left. In my experience, Goldman runs projects much better, particularly using "Projito" internal contracts that list the value proposition, deliverables, timetable, and whose budget is paying for it.
Also, despite Goldman's reputation, I saw a lot less arrogance at Goldman than at Google. Goldman has a much more healthy respect for its competitors, though with good reason. I've also seen a few people transition from Google to Goldman and take a bit of time to adjust to the idea that Google's way of doing things is best in Goldman's operating environment. At Google, I got the sense that lots of people thought that Google was better than its competitors because they were smarter. At Goldman, I got the sense that lots of people felt they were better than their competitors because they had higher expectations of themselves.
About 4 years ago, I was managing a guy right out of school who pushed a minor bug that broke real-time risk calculations for a major multinational financial institution in the middle of the European trading day, prior to the NYC open, and people were yelling over email that they were trading blind. Someone had committed an important change right after his, so a simple rollback was highly sub-optimal.
Remembering how I felt years ago, I reassured the new guy that people were yelling over email because it was important, not because they were mad at him. I told him that I thought he was the most familiar with his change and the most capable person to fix it, and that he should do his best to calm down and focus, but he should let me know if he needed help, and I would do my best to calm folks down. I told him he would probably remember that mistake the rest of his life, but nobody else was going to remember it a week later. He had the bug fix in production in under an hour.
He sent me an email from home that night worried that he had let the team down, and I reiterated that he was going to be the only one who remembered the mistake longer than a week. The post-mortem follow-up was just to reiterate to authors and reviewers the importance of corner-case tests, and nobody brought it up later.
I really only remember it because my manager sent me an email that night praising how well I handled the new guy's first big production bug.
This is outstanding advice, and very well put. I shall be borrowing it, thank you!
Edit: if you are managing someone who is new in a job, it's your job to make sure they don't push bugs to break important stuff in the first place.
Yes, I and the person who reviewed the change bear more responsibility than the new developer. Also, I say "new guy", but the person who had interned with us the Summer after "the new guy" had already joined full time at that point, so "the new guy" had been working full time with the team for at least 9 months at that point. I also remember the room where it happened, which wasn't the first room we were in, so maybe he had been with us full time more like 18 months. In any case, it was the first time when he was trying keep the weight of billions of dollars out of his head and calmly but quickly fix a bug.
That's impractical in organizations with flatter structures and general purpose communication channels.
What would you suggest, kicking everyone off of internal IRC/Slack/mailing lists/etc.?
I've definitely written more than one bug where post-mortem estimates were over $10,000 in losses.
August 20, 2013, I finished a code change (in Hong Kong) to Goldman's global algorithmic trading system and sent it out to a colleague in Europe to review. A friend of mine was a machine learning person in our Tokyo office and was in town for work, so a bunch of us had dinner and a small number of drinks. I stopped by the office on my way home to check if the change had been approved. It had, and I hesitated a bit to put it in production, because I had a couple of drinks and it was late at night. However, rationalized that I had written all of the code while awake and without a drop of alcohol, and pushed the change into production.
I woke up the next morning to read news [1] that Goldman had lost up to 100 million dollars in an automated trading problem within 1-2 hours after I pushed my change. I couldn't see how my change could possibly have caused that error, but was still a bit panicked until I reassured myself that my cell phone would have been called once a minute until I woke up if I had made a change that caused a loss of that magnitude.
I went into the office and saw that a chat window I had open with a friend in the NY office showed "presence unknown". An email sent to them bounced. So, I walked over to the derivatives (Flow) Strats desk, sat down in an empty chair next to one of my friends, and just quietly said "... so " and the name of my friend in NY. My friend on the Flow desk's eyes got wide and he said "how did you know?". I actually didn't know until the Flow Strat's reaction confirmed my guess.
My friend in New York was actually very careful, but he had been working under time pressure late at night and pushed a bug into production. He'd been more responsible than I had the night before. I got really lucky, and he got really unlucky. He's actually a really solid engineer. He caught plenty of very subtle bugs in other people's code, at least once when he hadn't been asked to review the code.
After August 20, 2013, if at all possible, I push changes into production before noon, and not on Fridays.
If memory serves the "maybe $100 million" ended up being around $28 million.
And that's the time that I could have easily caused a $28 million loss.
There was also a time I misplaced a paren and had a bad actor noticed, they could have used 60 million customer computers in a DDoS UDP traffic amplification attack. My test cases weren't matching my hand-worked-out examples, but I eventually just gave up and assumed my code was correct and put incorrect values in the message authentication code test vectors. Never roll your own crypto, especially if your test vectors aren't coming out as you expect. That was 2004.