Clojure 1.13 adds support for checked keys

Clojure 1.13 adds support for checked keys(clojure.org)

225 points by FelipeCortez 58 days ago | 47 comments

slifin 54 days ago |

This is a case I never really thought about - if the key is missing today you'll get nil as the value and since Clojure is a nil punning language it usually does sensible behaviour in your program

I know this sounds unreliable but in practise I like a language that defaults to pragmatic code paths so I don't have to stay up at night imagining a million code paths

This adds a throwing codepath which is quite drastic so I'm glad people don't build this into programs everywhere - I'd be nice to hear what the team imagine as the use case for this

Normally for correctness I'd like to see specs at the boundaries for programs and different test suites for internal behaviours

kevincox 54 days ago | |

The problem in my experience is that while nil is a perfectly reasonable default 9/10 times that 1/10 happens often enough and causes major problems that it is worth taking the extra few seconds to write it explicitly in the code to acknowledge that case and that you have checked that it is fine in the 9/10 cases or handle it in the 1/10 case.

I have seen multiple major production outages in Golang code because people accidentally read a non-existent map key and used the default value. As a funny bonus in one of those cases we were stumped when debugging because this code had tests, but the tests were also reading the default values out of the map and asserting that "" was in fact a valid textproto (it always is!) so silently testing nothing.

So even if defaults are useful 9/10 times that 1/10 is so painful and expensive that it isn't worth it in my experience. The time spent responding to, debugging and fixing those outages far, far outweighed the time saved by the convenient default values in the 9/10 times.

jbritton 54 days ago | | |

I suspect there should be some data model guideline that says if keys can be missing then values can’t be nil, or if values can be nil then keys must be present. In the famous saying there are two hard things: naming, cache invalidation, off by one errors, I think one more to add is handling missing data.

slifin 54 days ago | | |

defaults are a different thing to nil punning

So nil will have had special consideration in Clojure core functions

That doesn't mean it doesn't crash either it will absolutely be unhappy with nils in your math

nil is usually unexpected in test outputs or at the very least an unhappy path

In terms of debugging that's why I can't quit this language flowstorm let's me visually step through what happened line by line, backwards, forwards, programmatically - whatever

Languages should be competing against each other by their best time travel debugger it just completely removes the need for guesswork

TacticalCoder 54 days ago | |

> ... if the key is missing today you'll get nil as the value and since Clojure is a nil punning language it usually does sensible behaviour in your program

And that is still doable AIUI: they're optional checked keys. The doc describing them makes the distinction between required and non-required keys.

Arguably we already had those: I religiously use spec'ed maps in my Clojure since a great many years (and Clojure spec is still in alpha, but "alpha" in Clojure land basically means: "more stable and less likely to change than any feature in any other language" and I'm only slightly exaggerating here).

In my case I use good old defn-spec (form Orchestra but YMMV) instead of defn. And my maps are (partially) spec'ed, using spec'ed keys (as well as any other non-spec'ed key I feel like using). Sure it's only runtime checks but it's really great.

You get to both have the extensibility (you can for example add keys that don't exist yet later on without changing any of your specs) and you can specify which keys are required.

For there is such a thing as maps where you know that this and that key must always be there.

I don't think it's an issue to have optional checked keys. Especially not when you can mix both required and non-required keys in the same map.

jimbokun 54 days ago | |

This is really the kind of thing you want to fail at compile time which isn’t real possible in a dynamic language like Clojure.

xoxolian 54 days ago | | |

Unless you use Typed Clojure, which is a library. https://github.com/typedclojure/typedclojure

I haven't used it, so I don't know its tradeoffs; but its docs say its types exist at compile time: https://github.com/clojure/core.typed/wiki/User-Guide

slifin 54 days ago | | |

Well it is possible - you can add a user macro that calls into clj-kondo (or anything actually) to check your codebase on compile

It just doesn't make much sense to do - most modern developers will be running static analysers through LSP or their editor (knowingly or not) continuously on code change so as to see those errors quicker than re-compiling the program

xoxolian 54 days ago | |

For me: documentation at the "front door" of an interface, especially in that long moment before you decide to add a spec or Malli schema.

bjconlan 54 days ago | |

Yeah I'm with you, this feels like a case for assertions & not a new core feature. Perhaps I'm missing something because it was promoted by Michael Fogus and has been ratified by those who would know (Alex I'm assuming). To me it doesn't pass the test of necessity as something needed at the core but at least it feels somewhat ideomatic.

lukaszkorecki 53 days ago | | |

For me it solves a real issue, a lot of the code that I work on looks like this:

  (defn foo [{:keys [a b c] :or {a 1}]
    {:pre [(some? b) (some? c]}
     .....

having `:keys!` will automatically remove the need for `:pre`

sokoloff 54 days ago | |

> if the key is missing today you'll get nil as the value

You can add a third parameter to override the nil if detecting the missing key matters.

(You almost surely know this, but not all HN commenters will.)

  user> (:bar {:foo 1})
  nil
  user> (:bar {:foo 1} :missing)
  :missing

geokon 53 days ago |

One thing to note that's maybe less obvious is that you can destructure some keys with the check and others without. This makes the function interface a bit self-documenting. At a glance you see that the username is required and other parts are maybe not.

    (defn my-function
      [{:keys! [username]
        :keys  [firstname
               lastname]}]
      (do-stuff username
                firstname
                lastname))

A minor downside is that now it seems `nil` is even more overloaded b/c you can explicitly pass in a nil and give it a special meaning. This generally cascades in to messyness (better to have a special key like `:missing-username`).

Feels like throwing an error on nil would have been better/simpler? But I'm sure there's an angle I've not considered

hk__2 54 days ago |

Some explanations from https://clojure.atlassian.net/browse/CLJ-2961:

> Clojure’s idiomatic use of maps has proven valuable, but missing required keys, misspelled keys, and invalid values can lead to failures that do not connect to the actual source of the problem (e.g. NPEs) making diagnosis difficult. At the same time, Clojure lacks a simple inline mechanism for functions to document and check the keys they require and accept. Existing tools either separate those expectations from the function itself or couple data shape and data provision.

moomin 54 days ago |

This is actually great, and I predict that fans of nil-punning will rapidly discover the joys of actually having errors trigger where the error was introduced rather than propagating through the program.

Any news on ClojureScript gaining the feature?

swannodette 54 days ago | |

Working on it :)

embedding-shape 54 days ago | | |

Amazing! What are the most interesting areas for ClojureScript in the future, if you don't mind me asking for some casual semi-serious prediction?

Thanks for everything you've done for Clojure and ClojureScript, I'd surely have dropped programming as a whole if I didn't discover Clojure and ClojureScript at the time I did.

akkad33 54 days ago | |

What is nil-punning?

undershirt 54 days ago | | |

allowing functions to treat nil arguments as empty versions of their expected types

fnordsensei 54 days ago | |

Well, kind of. This is the kind of problem that you might throw schemas (eg. Malli) at pre 1.13.

It’ll be nice to have it at hand in the base language though.

ndr 54 days ago |

Is it only me or this sounds a bit counter to clojure philosophy?

summarybot 54 days ago | |

As a Clojurist the standard pattern for ensuring keys-are-set before doing-something is not-as-elegant-as-this. Clojure is full of macros that do useful things :) Simplifying oft-used patterns into compact representations is very on-brand. Plus, you need this like, all the time.

This will eliminate two whole classes of errors: 1) where keys are supplied a value at an undesired nesting-level. 2) where keys are not-yet-set for some other reason.

For the many programmers who have to write in checks and verifications themselves for this, this saves quite a bit of time, removing the interruption from coding and restoring the flow of getting logic-to-symbol.

rads 54 days ago | |

The maps are still open to new keys even if some keys are checked. I think that fits in with how clojure.spec and Malli work already, but in a lighter syntax.

codemonkey-zeta 54 days ago | | |

The maps haven't changed at all, this is a feature at destructuring sites.

exabrial 54 days ago |

I love the idea of clojure and perfect immutability, but holy crap I cannot grok the syntax. My C-trained brain explodes.

wry_discontent 54 days ago | |

You get over it really quickly once you start actually using it. I find it basically impossible to read Clojure outside the editor in any meaningful sense.

barrenko 54 days ago | |

It's like a light saber instead of a machine gun. Let the bullets come to you.

iLemming 54 days ago | |

Lisp is tricky. Pretty much every programmer for whom it's not their very first PL hates it initially, but then there's a time, a threshold after which no other language feels more readable than Lisp.

Using structural editing idioms and the REPL, usually makes the process less vexing.

zshrdlu 53 days ago | | |

I think for some people it's an aesthetic knee-jerk reaction and the fact that it's "unusual" for a programming language. It's not like their brains "can't grok" -- it's just syntax after all. People don't tend to have that reaction to xml for some reason. It's data, so it gets a pass. But then this is code-as-data and they have to break that barrier conceptually and then get past the aesthetic knee-jerk reaction.

erichocean 54 days ago | |

It took about two weeks for me to be able to read it.

Might as well have been Russian.

Now it's as natural as any other language.

drob518 54 days ago | |

I went through a similar phase decades ago with Common Lisp. It takes a week or two. Now, it’s quite a natural syntax and I see the parens as a huge benefit. I like Clojure syntax even more than CL and Scheme because of the map and vector literals.

beders 54 days ago | |

It'll take a while but now other programming languages look alien to me. Once you've adopted s-expressions it is hard to go back.

temporallobe 54 days ago |

We just updated one of our projects to 1.12.5, but I might push for 1.13 as this could be very useful, although an alpha version might raise questions.

pgt 54 days ago |

This is helpful, because practically many functions in the wild have assert-like checks at the top of the function, e.g.

`(if-not key1 (throw (Exception ...))`

...and pre-conditions, e.g. `:pre [condition1 condition2]` do not run when `assert` is off.

thom 54 days ago |

Ah yes, the missing seventeenth way to validate function parameters.

IceDane 54 days ago |

Slowly but surely dynamic programming proponents discover the value of statically verifiable correctness. Who'd have thought?

lastofus 54 days ago | |

Snarkiness aside, this is hardly a static type check. This appears to be a runtime argument check, somewhat akin to the following python:

  def foo(*, a, b): return a+b

which errors out at runtime if `a` or `b` are omitted, despite being keyword arguments which are usually optional.

beders 54 days ago | |

Oh we know. That's why static types are a la carte in Clojure. You can have them if you really really want them.