Alan Kay and Missing Messages

Alan Kay and Missing Messages(ovid.github.io)

115 points by Ovid 7 years ago | 59 comments

jasode 7 years ago |

This blog post proposes a "data" sanity solution (alternative external oracles, or "scoring" on probabilities, etc) but that's not really what the comment by Samuel Falvo II was about.

His problem was syntax getting broken and not about "questionable data". In the context of dynamic vs static compile, he's worried about scenarios like this:

  x = customer.last_name  // worked on Friday and x=="Smith" 
  - remove field customer.last_name // Saturday refactor
  + add field customer.full_name  // Saturday refactor
  x = customer.last_name  // broken on Monday with a runtime error

With a static type system, the changes on Saturday would have told them immediately at compile time that the last_name field access by other client code was broken. (And yes, one typical answer for handling errors like that in dynamic type systems is unit tests -- but that's veering off into a different conversation.)

This essay is a "solution" to a different problem.

zeugmasyllepsis 7 years ago | |

> ...would have told them immediately at compile time that the last_name field access by other client code was broken.

What's the proposed fix in this case? Should all clients accessing that field switch to `customer.full_name`? Do you control all of the call-sites? If so, that change is pretty easy to automate in dynamic languages by marking the field as deprecated, adding the new field, and forwarding `customer.last_name` to `customer.full_name`. That way, running systems don't break but you get the new behavior. You can automate logs to track when the deprecated field is accessed and, if necessary, fix call-sites after a reasonable period of monitoring, reducing risk (assuming there's not "once-in-a-century" paths in the code that invoke the call site).

It seems reasonably likely that this is not the desired behavior - you may have external clients, and some callers may be relying on this field to actually return only a last name. In that case, deprecation is still the correct path, I think, since it's unreasonable to expect external clients to immediately fix systems relying on that field being available.

Static typing can tell clients where they need to make changes, but it's not realistic demand that they immediately update every call site, every method depending on that behavior, and every integration with their external dependencies that relied on the interaction between your code and another system. This is why we have (semantic) versioning, deprecation warnings, and support agreements for larger systems. API's should evolve slowly and gracefully over time, not shift suddenly overnight.

tene 7 years ago | | |

The proposed fix is that you don't deploy code that the type system can prove is broken, and the person performing the refactoring needs to do additional work rather than pushing out something that will fail in production.

My reading of jsode's example is in a shared codebase, where it would be reasonable for the person performing the refactoring to update all call sites, and where checking in a change that does not update all call sites to whatever branch is released to production should be unacceptable, as it will provably cause problems in production.

If the change was made in a standalone library that other teams depend on, the type-checker error would be produced when they update the library to the new refactored version, and it's the same story: prohibit deploying known-invalid code to production until you've updated all call-sites to match the changes in the library API.

If updating all call-sites immediately is implausible for whatever reason, leaving the old field in place with a deprecated annotation would be quite reasonable, but the purpose of a type checker in this context is to prevent you from failing to handle this in at least some way. As you say, responsible maintainership of an API should facilitate gradual transition; the use of a type-checker here is to prevent deploying code that will fail at runtime when someone has made a mistake, and can help facilitate responsible maintainership.

jasode 7 years ago | | |

>What's the proposed fix in this case?

For the purposes of my reply to the author, the type of fix doesn't matter and its discussion would be a distraction away from my main point: the blog author Curtis "Ovid" Poe misintepreted the comment by Samuel Falvo II and therefore wrote the wrong "solution" as a response. Ovid's health data example of obj.fetchDataFromServer() error and reconstructing data from other sources with a threshold score is not the same abstraction level as the commenter's example of obj.last_name getting broken because the plumbing API was renamed/refactored.

(My guess is that Samuel Falvo II's hostile tone makes it easy to miss that he was actually describing a syntax error and not a data interpretation error.)

Regardless of whether the fix is:

- stringsplit(full_name) to extract a last_name

- or change the client code's usage of last_name and only use full_name

- or re-architect the upstream customer object to include both last_name and full_name so there's a gradual migration

...it isn't the issue.

What the angry commenter was trying to communicate was, "tell me about the last_name refactoring causing a syntax error _immediately_ instead of letting it sit there as a ticking time bomb that will blow up and wake me up at 4am".

He doesn't necessarily have to immediately fix it, but he does want to immediately know of the error's existence before a crash at runtime.

mcguire 7 years ago | | |

So you marked the field deprecated and stated it would be removed in release "potato-wedge". The "potato-wedge" release rolls around, you delete the field, and Ted, who didn't care to pay attention to the deprecation warnings, ... does what?

In a statically typed world, Ted tries to compile his project and gets an error saying, "last_name doesn't exist". In a dynamically typed world, Ted ships his code into production (or possibly testing) and gets a call at 4:00a.m. saying it threw an exception or just died. (The testers are in Bangladesh.)

In Alan Kay's world (or at least that of the author), the code goes on robustly producing whatever results it produces in the case where no one has a last name. Better?

sundbry 7 years ago | |

This is about message passing at an abstract level, which is what Alan Kay imagined with Smalltalk. You are the one missing the point....

When you are dealing with external systems, such as a third party API, the message passing paradigm is what you are dealing with. There is no static typing for someone else's API (even if you are using their SDK).

jasode 6 years ago | | |

>This is about message passing at an abstract level, which is what Alan Kay imagined with Smalltalk.

The merits of Alan Kay's message passing are fine but it is not relevant to this case of the author writing an inappropriate solution because he misunderstood a commenter's feedback.

The "this" in your "_this_ is about message passing at an abstract level" comment seems to only refer to blog article #2 in isolation -- instead of considering how it was provoked by the author's previous misunderstanding.

>You are the one missing the point

You want to lecture me on being wrong but it seems that you are the one jumping into the middle of a conversation without realizing what communications have transpired so far. If you didn't completely read both blog articles and the user comment, you're missing the full context. Let's recap:

The blog author wrote 2 articles.

Blog article #1[1] didn't say anything about "external systems 3rd party API". The blog author himself brought up the issue of dynamic vs static in regards to "hot swapping" code without restarting the system. It was not "external systems". It included an example:

  print customers.fetch(customer_id).last_name

At the bottom[2][3] of that blog, a commenter named Samuel Falvo II asks what to do when a broken "customer.last_name" message fails because it wasn't statically checked for syntax errors. The commenter also wasn't talking about "external APIs". He was saying static compile checking helps him find errors early instead of errors later at runtime with an unexpected emergency at 4am.

Blog article #2[4] then responds with an elaborate solution to reconstruct missing data which is a totally different abstraction level from the error scenario the commenter was asking about. This 2nd article introduces an "ETL workflow" that possibly comes from servers of external partners. The 1st article did not.

My top-level comment was explaining to the blog author that he misinterpreted what the angry commenter was asking.

[1] https://ovid.github.io/articles/alan-kay-and-oo-programming....

[2] https://ovid.github.io/articles/alan-kay-and-oo-programming....

[3] screenshot of Disqus comment in case it gets deleted: https://imgur.com/a/D09ma0x

[4] https://ovid.github.io/articles/alan-kay-and-missing-message...

DonHopkins 7 years ago |

I think there's a big difference between not understanding the message, and not understanding the data (a message parameter).

When they get an unrecognized message, Objective C objects call their doesNotRecognizeSelector method, and Smalltalk objects call their doesNotUnderstand method.

And the object sending the message can first check with respondsToSelector in Objective C or respondsTo in Smalltalk, before sending the message.

But validating and sanitizing input parameters is a totally different thing than handling unknown messages, orthogonal to object oriented programming.

pron 7 years ago |

> This is the software equivalent of "Not My Problem." ... This is how you deal with unanswered messages. You think about your response instead of just letting your software crash.

A terrific 2014 paper, Simple Testing Can Prevent Most Critical Failures: An Analysis of Production Failures in Distributed Data-Intensive Systems [1] found that most catastrophic crashes in distributed systems are a result of catching exceptions and not paying thought to how to handle them.

[1]: https://www.usenix.org/system/files/conference/osdi14/osdi14... (talk: https://www.usenix.org/conference/osdi14/technical-sessions/...)

hi41 7 years ago | |

I have c programming experience but not java. I would greatly appreciate if you can tell me the where to catch an exception. When there are nested method calls I can’t tell where to handle it correctly - should I handle it in the top most function or the one where the current method was called from? Can you point me to some resources so that I can understand the correct way of handling exceptions.

pron 7 years ago | | |

As the paper and the article say, what matters is not the technical aspect of where to handle an exception, but to give some serious thought to what should be done when it occurs (as opposed to just logging it). Serious problems happen when people don't pay enough attention to that.

sundbry 7 years ago | | |

You should handle it in the most immediate context which can do something productive with it- if there's nothing you can do but swallow it, you should probably let it continue to pass up the stack. If you can handle it and log a warning and return something else, you should handle it there.

For example, if you're writing an HTTP handler for an API, you should catch any exceptions at that point in your handler and return a 500 with an appropriate response to the client, since returning 500 is something productive we can do (vs letting it pass and crashing the server).

kstenerud 7 years ago |

I can't shake the feeling that we're talking about different classes of problems here.

Data importers and sanitizers are de-rigueur when dealing with real world data, of course. You have to expect to find crazy things inside, and be expected to just cope with it. That's not a problem.

But when you're dealing with purely internal systems, the calculus is different. If you had to keep chasing down fopen(), printf(), and malloc() calls, you'd spend more time in administration than you would getting actual work done.

So if I'm understanding this correctly, we're talking about purely internal code vs code that traverses domains (much like in DDD), which require different styles of solutions.

Or am I still missing the point?

tjpnz 7 years ago |

I was asked to do something very similar at a film post production facility for credits. Essentially I was given a day and a half to reconcile spreadsheets of crew member names (compiled by department HODs) with their "credit name" on our employee database in order to create a master for the production company.

I had all kinds of issues - variations of names, mispelled names, nicknames, names with the middle name used as a surname (and vice versa) and a few lacking even that. I recall leaning heavily on tables of common names, various Python string normalisation methods as well as soundex. In the end I was left with a dozen or so names which needed follow up but it was pretty good for the ~1000 I had started with initially.

The most harrowing part of it all was attending the crew screening - it is actually possible to address issues on an end crawl (provided you catch it early enough) but I really didn't want to be the guy who screwed up the credit of someone who had just spent the previous three months in crunch to get the picture out the door.

geospeck 7 years ago |

I've watched one of Rich Hickey's talk and at some point he brings that issue on the table, what happens when the receiver is not responding to the message.

He is advocating Queues and one of his arguments is that Queues decoupling the requester from the receiver. So you don't really have to worry about things like this by using a queue.

[1] https://youtu.be/ROor6_NGIWU?t=1955

Fellshard 7 years ago | |

That's abstracting in time more than in implementation; and it's primarily useful with stateful objects that are best reasoned about as a single, synchronous timeline (which most objects are). You'll see this pattern used in, say, Actor systems, as a queue-backed inbox.

mcguire 7 years ago |

Note that "...how you might handle objects that don't respond to messages" refers specifically to what happens when you send a message (or call a method, in the parlance of most non-Smalltalk languages)

    thingy.spamford()

and the thingy doesn't implement a handler (or a method) for spamford. It has to do with this paragraph from the original, original post:

"Kay argues (correctly, IMHO), that the computer revolution hasn't happened yet because while our bodies are massively scalable meat computers, our silicon computers generally don't scale in the slightest. This isn't just because silicon is slow; it's because of things like print customers.fetch(customer_id).last_name not actually having a last_name method, throwing an exception (assuming it compiled in the first place) and programmers getting frantic late-night calls to bring the batch system back up. The only real upside is that it offers job security."

This has nothing to do with network protocols.

tyingq 7 years ago |

Having worked in healthcare IT, I'm less astonished than I should be about approximate matches and score thresholds being used to confirm "same entity" for pharma data. What could go wrong?

(Not knocking the author...the practice is unfortunately needed because providers won't provide good data)

Ovid 7 years ago | |

Sorry, I wasn't clear.

It wasn't quite like that. It was to help the pharmaceuticals work together to "pool" talented researchers who had the required training to proceed in the clinical trials. If a bunch of companies don't share this data, they drive up their costs tremendously in trying to find and recruit qualified doctors for their trials, pay to train them, and then have them conduct studies, only to find out that the doctor in question doesn't recruit any candidates themselves, or fails to report results.

If, however, they pool their resources, they can find qualified doctors who have already taken the training and are known to participate well in trials. They can then correlate that with populations who might benefit from the drugs in question and save a fortune (and possibly many lives), by reducing the cost-to-market.

The matching algorithm was actually very strong, but there was still a human step, from the pharmaceutical companies, to verify that these were the researchers they were seeking.

tyingq 7 years ago | | |

That's helpful, though I can confirm the same sort of process is used in areas where it matters more.

Ovid 7 years ago |

This is the original discussion: https://news.ycombinator.com/item?id=19968496

frou_dh 7 years ago |

There's something I don't understand about the "messaging" metaphor as opposed to the more pedestrian method calling system that most people are used to...

What about private methods? Decently written objects in the wild tend to have some private methods that they call on themselves. The highfalutin metaphor of a message goes out the window there, because it's an individual object doing something to itself, not a communication.

corysama 7 years ago | |

That's true if there is no composition going on within the object. But, once you have a class hierarchy, some mixins, some dependency injection, or some such... you are "privately" talking to parts of yourself that come from elsewhere.

frou_dh 7 years ago | | |

But when it's the exact same syntax to call a statically-dispatched method on the current object as it is to send a message to something else, it breaks the illusion of messaging being meaningfully different while writing code.

Whether some actual bonafide communication ends up being done one level removed (e.g. inside such a private method, or chained off of a field access) is immaterial because when appraising the conceptual integrity we cannot continue after seeing a fail.

(edited)

earthboundkid 7 years ago |

The article is making essentially the same argument as this one: https://dave.cheney.net/2019/01/27/eliminate-error-handling-...

dnautics 7 years ago |

am I crazy about this? You should have (at least two) type of message guarantees - "don't care" and "must respond", kind of like "udp" and "tcp". Obviously other messaging systems have other types of guarantees, but these seem like they are a reasonable basic.

If your agent that send a "must respond" message doesn't get a response within some customizable time, then it itself fails loudly.

ngcc_hk 7 years ago |

Repeat ... coupling and strict error processing kill token ring and let Ethernet win. So is sna and internet.

Cross system require much flexibility and to allow for error.

simion314 7 years ago |

I personally did not understand the solution proposed here, I would like to see an actual solution to the problem that was raised.

zeveb 7 years ago | |

The idea is that rather than try/catch (which tries to do something and can report low-level failures but cannot resume them) one uses something more like Lisp's condition system. In a condition system, signalling an error preserves the stack, and it means for example that a low-level loop can detect an error (say, a poorly-formatted CSV cell), signal that to higher-level code, the high-level code can make a decision (e.g. to replace the contents of the cell or skip the cell or send an email to the CEO with the contents of the cell and wait for his reply) and the low-level loop can continue processing as if there had never been a problem at all. There's a really great explanation in Practical Common Lisp:

http://gigamonkeys.com/book/beyond-exception-handling-condit...

In the context of the example, I can easily imagine how the system might have processed most rows without issue, called out to a scoring system for most exceptional rows, and raised a few rows to human attention when even the scoring system couldn't figure it out.

pjc50 7 years ago | | |

This sounds like something that could be introduced into a lot of modern functional-ish VM based languages, in the form of restartable exceptions, it just .. hasn't.

simion314 7 years ago | | |

Is this lie instead of throwin errors/exceptions you raise Error events ?

pjc50 7 years ago |

Right, so this is basically the Go approach: explicit error checking on everything. No exceptions, because exceptions are a weird sort of non-local control flow, and can escape and take down your program as a whole.

The "if you don't know the answer, find an approximation from another route" is .. situational. In this case it's exactly what the customer wants. In other cases (finance, security, aerospace) it could be a disaster waiting to happen.

I worked on a point-of-sale system where it was a matter of design philosophy that any inconsistency should be an immediate crash-and-reboot; since it also operated as a save-everywhere and audit-everywhere system, if you did manage to crash it then within 30 seconds you'd be back at the screen just before the last button press with no data loss other than the button press that caused the crash. I believe this crash-and-recover approach is very Erlang: https://ninenines.eu/articles/dont-let-it-crash/

Thinking of exceptions and message validation also makes me think of "in band" versus "out of band" signalling and errors. Exceptions are "out of band" - outside the normal function return process. Internet communication is all "in band" and the whole approach of separate control and data planes has almost entirely gone away, apart from SIP and (nearly dead) FTP.

theoh 7 years ago | |

Sum types arguably permit a form of out-of-band return value. A function that returns a sum type has the choice of returning the usual type (an integer value, say) or an error value of some other type.

IMO this counts as a kind of out-of-band arrangement because it doesn't involve using one of the normal return values to signal error.

https://en.wikipedia.org/wiki/Semipredicate_problem

chucksmash 7 years ago | | |

But are you actually returning either an int or an error? You're always returning a Result type which may contain either an int or an error.

lostmyoldone 7 years ago | |

Go has panics, which is essentially exceptions with a different name. Failure to handle all panics will lead to process termination, as with unhandled exceptions.

zeugmasyllepsis 7 years ago | | |

To be fair, their general use is discouraged unless you want the process to terminate. Not that you can't do that in other languages, but the culture in Go seems to prefer returning error codes.