K: War on Raze

79 points by chrispsn 4 years ago | 22 comments

mlochbaum 4 years ago |

Hm, just wrote up an explanation[0] of BQN's high-rank searching, which is the same concept used in J and APL and turns into rank-sensitive search in K. The issue it addresses is that in array programming you'd like to search for each of an array of elements all at once, but this is hard to distinguish from just searching for the array itself. So you look at the entries of the array you're searching to see what rank they have, and from that determine how to split up the other argument. But BQN/J/APL and K are different in that the APL group uses multidimensional arrays, giving it a concept of rank (number of dimensions) that's separate from array nesting. K uses only 1-dimensional lists, and it considers the rank to be the level of nesting. With respect to rank-sensitive search this is kind of precarious. You might think of a structure as a list of lists, but will K interpret it as a matrix? So it's understandable you'd want to avoid it in K.

This difference in array models is also part of the reason why Dyalog's ⍸ does a "deep where". In APL it's not considered "deep" at all, because the argument is a numeric array; it would be called "high rank". There's still a problem with this version though, because the high-rank version of ⍸ needs each index[1] to be a list, but the normal version (when the argument is 1-dimensional) uses one number for an index. The results of ⍸ for different rank arguments are actually inconsistent. In BQN, I decided Where should just be restricted to the 1-dimensional case. I wrote a little more about the kinds of array indices at [2]. It's worth noting that K has this problem a lot less than APL, because most of the time you will only index into a single dimension. But with constructs like deep where these problems appear in K as well, and you'll end up with the same inconsistency if you try to combine the two versions.

[0] https://mlochbaum.github.io/BQN/doc/search.html#higher-ranks (in the code examples, "high"≍"rank" couples rows to give the 2x4 shape seen below, and similarly 3‿2⥊4 creates a 3x2 array)

[1] https://aplwiki.com/wiki/Index

[2] https://mlochbaum.github.io/BQN/doc/indices.html

pault 4 years ago | |

This is my first encounter with K, and it looks absolutely bonkers. The amount of logic you can express in a few characters is mind boggling. However, it makes my brain hurt. Is this a paradigm that becomes intuitive once you learn it, or is it like Ruby in the sense that it’s so flexible that every developer uses their own custom DSL and onboarding is like solving an infinite series of brain teasers?

mlochbaum 4 years ago | | |

It becomes intuitive. APL primitives are particular instantiations of clean high-level concepts. While a C programmer would think "I'll make a sparse encoding of this boolean array" a K programmer would just write Where. Having a rigorous definition for a pattern like this is a nice guide for the intuition, but it's possible to get in a situation where what you want to do and what the language provides don't line up. And there are domains like graph algorithms that largely don't fit into an array style. For these you'd have to fall back to writing scalar-style code.

There's not much of a custom DSL problem, because APL/K emphasize using the primitives that are built in, and user-defined functions will have names not symbols. Some codebases will be a challenge to read because these languages tend to attract programmers who write things their own way, and for K in particular many users—the inventor Arthur Whitney most of all—don't like to write long explanations of how their code works. The OP, while well explained, is pretty advanced material. Knowing a lot about array programming makes the high-level picture clear but I would say many of the code examples would take some effort for even a good array programmer to understand.

icsa 4 years ago | | |

Once you look at the verbs as symbols for algorithms, it will make much more sense. Most languages make you create algorithms. k code is the composition of algorithms.

RodgerTheGreat 4 years ago | | |

It's unfamiliar, but highly learnable. The problem solving style can then be applied elsewhere to great effect.

If anything, K is the opposite of a language where everyone disappears into DSLs- you can write expressive code without wrapping it in custom abstractions. An anti-lisp, if you will. It's not uncommon for K programmers playing with a puzzle to independently arrive at character-for-character identical solutions.

bear8642 4 years ago | | |

from my small experience with K and its links with APL would imagine that it becomes intuitive once learnt.

Don't know DSL ability, though aware someone wrote expanders from symbols to words, - imagine DSL doable but forth, lisp macros spring to mind first in the DSL space

chrispsn 4 years ago | |

> The results of ⍸ for different rank arguments are actually inconsistent.

Yep. For this reason deep where ideally wouldn't live on '&'. It should be able to absorb any structure for subsequent use with deep indexing etc, regardless of rank. Perhaps another reason to make 'explode' an adverb...?

RyanHamilton 4 years ago |

The authors tweets describe the kind of manical pinnacle of short perfection that can only be achieved after 5 x 13 hour days of solid coding to produce an epiphany that it can all be replaced with 1 line or in the case of K, 10 characters: https://twitter.com/kcodetweets I love it.

marrowgari 4 years ago |

Love seeing the random k/q code mentions here. All too often I feel as though I'm coding in a black box. This shines a little light letting me know others are out there too. Nice work.

pault 4 years ago | |

What industry do you work in? Is it a common stack there?

smabie 4 years ago | | |

Main industry for kdb+/q is finance, specifically sell-side or very large buy-side firms.

I wouldn't call it "common", but it's pretty well known.

kdb+/q devs certainly have good job security, heh.

neolog 4 years ago | |

check out the Arraycast podcast

1vuio0pswjnm7 4 years ago |

Is there an ngn k equivalent of "hcount" (-7!x)?

rak1507 4 years ago | |

Size of a file in bytes? Nothing built in (afaik) but pretty easy to define yourself. hcount:#1::

1vuio0pswjnm7 4 years ago | | |

Why did Arthur make this a builtin.

The beauty of ngn k is I can compile it on NetBSD and with musl on Linux.