Fuzz me wrong – How QuickCheck destroyed my favourite theory(thma.github.io) |
Fuzz me wrong – How QuickCheck destroyed my favourite theory(thma.github.io) |
I'd recommend it if you don't understand much of the article.
That said, for anything that fits in memory, it's so easy for a parallel map to preserve order that it probably should anyhow, and if it doesn't fit in memory it's still not that hard and probably a better default. (If you've got some sort of map with wildly variable compute times per element, then you could starve the thing that finally emitting them in order if you hit a very expensive particular element, and you could potentially have gotten better throughput if the rest of the system could just keep working around it and ignoring order. But I'd still expect order-preservation to be the default such that I'd have to tell my parallel map that it's OK to ignore order.)
Nit: I think this would read better if the author replaced “personal theory” with “hypothesis”.
1. Parallel [efficient] MapReduce requires a commutative monoid for deterministic results
2. A naive implements of parallel MapReduce in haskell using par doesn’t because it will only do reduce operations in a certain order. Imagine if the map for every second element took 0s and the others took 10s. In normal mapreduce, you would reduce together the fast half of the results as they come in but in this haskell implementation you would need to sit on them and wait until the other adjacent elements come in. In the article’s very naive implementation, you can’t even reduce together the first two elements together until you’ve reduced everything after them (modulo some weirdness around lazy evaluation)
I think if one thinks of mapreduce as an operation on sets not lists, it should be obvious that the monoid must be commutative.
I've used property tests the exact same way many times. In complex algorithms assumptions can be hard to think through, but a property testing tool can find counterexamples shockingly quickly.
If anything, you could think of it as an operation on multisets, but not sets. Sets don't have repeated elements.
If your reduce is slow for some inputs, its better to have those inputs near each other, as all consecutive values before and after can be reduced during that time.
And if every step is somewhat constant time, I don't think reordering (that's basically the same as combining random elements instead of consecutive ones) the input does anything.
If you find a text in German and you don't understand a word, because you have never studied German, do you think you are too stupid to speak German?
This is sort of like PG's Blub paradox: the reader opens an article, sees a bunch of seemingly bizarre syntax and novel terminology, can't easily map it to something he/she knows from C/Java/Python/Javascript, and panics.
For some reason people don't assume the same about natural languages. No Western reader will take a peek at a webpage written in Japanese and cry "I can't understand how people speak this language". (Not in this day and age, at least).
not all things require the same amount of effort to be understood at an adequate level.
> If you find a text in German and you don't understand a word, because you have never studied German, do you think you are too stupid to speak German?
there are definitely people that are able to look at a german text without having ever studied german before and make sense of it in a rough way
1. Read a Haskell book and/or tutorials online, some of which are free.
2. Try writing simple software and reading other people's code. Ask for help when you get stuck. The Haskell community tends to be friendly and helpful.
3. Write your own non-toy code, hit real world problems and solve them.
The path is very similar to other languages, except that because Haskell doesn't share the same lineage than C/C++/Java/Javascript, you won't be able to "wing it" by skipping the steps above. That's likely the source of your confusion: you can't as easily draw analogies to the languages you're already familiar with.
Note that learning Haskell from exploratory articles which are meant for an audience already familiar with the language is not the best way to go about it. This is like trying to learn Java from an article describing the subtleties of the garbage collector in some interesting corner cases: it'll just confuse you and it won't have any practical lessons for the novice.
People can and do write actual production Haskell code, so it can't be that hard.
There's plenty of Haskell out there that is easier to pick up on without looking at what seems to be a jumble of letters that imparts deep meaning.
frobnicateM :: M a b => b -> a -> f (b a) a
I don't have that feeling when using other FP languages like Lisp or OCaml.
Sometimes, Haskell seems to be more mathematics than programming.
I found this interesting paper on the subject of non-commutative reducers: "Nondeterminism in MapReduce Considered Harmful?" https://www.microsoft.com/en-us/research/wp-content/uploads/...
On my latest project I'm writing Haskell in a largely Elm style, and it's glorious.
Sure, by knowing both English and Swedish, German has sufficiently little "uniqueness" left to it that I can often get a rough idea of a German text despite never studying it. But how is this different from reading Haskell when you've previously worked in Scala and Rust?
Documentation is powerful stuff, especially when it's machine-checked the way Haskell type signatures are.
You can follow them to their deep dark places - but they have travelled far, and finding them in their tunnels is a journey of many moons. To the Haskell programmer, us surface dwellers are all confused - missing the essential True Names of programming. Don’t despair - you can travel where you will; but know that discovering the hidden truths will humble and age the best of us. Why? I know not. Perhaps realising we’re small and stupid is the cost of staring into the abyss of Math.
It turns out that when you do that, all sorts of stuff that simply doesn't fly in other languages "just works", particularly around representing things using structures otherwise rarely seen outside of mathematical logic and category theory. This is all very powerful and elegant, but it means the landscape is unfamiliar, confusing, and intimidating to programmers who don't have that background or the inclination to learn about it.
Hence all the monad tutorials — not because monads are particularly special, but because beginners encounter them quickly due to IO, so they need some sort of explanation (it's like how if you teach a beginning programmer Java, there's loads of ceremony to explain around classes and public and so on just to do "hello world")... whereas you can get away with using haskell for longer without learning about, say, functors and monoids — though it somewhat misses the point of the language if you do so.
In Prolog there is another kind of revelation about inputs and outputs: There's no distinction between inputs and outputs and they aren't even necessarily ordered in time any more.
Currying is a mechanism of representing all functions with two or more arguments using functions of only one argument; with currying we can "bootstrap" functions of higher arities in a substrate like a lambda calculus that has only one argument functions.
This arrangement of implicit partial application is a syntactic sugar.
Syntactic sugar for partial application is not partial application itself. Manual partial application is just as "functional". Functions are being applied, and passed around as values.
Implicit partial application can be obtained with Lisp macros, but that can never be fully general for at least two reasons.
One is that Lisps support something very useful, namely variadic functions, and functions with optional arguments. Variadic functions mean that you don't need ugly shims like "zipwith3" in a Lisp.
However, implicit partial application also requires static typing. In order to know that f is a three argument function being called with two arguments, and therefore reduced to a partially applied one-argument function, our syntactic sugar processor needs complete arity information about f.
In a Lisp, f can be redefined at run-time from having three arguments to having four.
In principle, in spite of that, all function calls could be treated as partial applications in a very generic way, but it would be horribly, horribly inefficient --- and the diagnostics when things go wrong would be pretty terrible.
And yes currying is not the same thing but it's common in informal settings to say "Haskell has automatic currying" and everybody knows you're talking about automatic partial application.
Lisp can of course do partial application and currying but not automatically; as you say you'd have to write special macros and sacrifice variadic functions to make it work (and I have done so). And it's really not that necessary in Lisp because we have parentheses; Haskell is the only example I've yet found of a Lisp-like language that has successfully removed most of the parentheses, and it succeeded because of its implicit partial application. But it had to sacrifice variadic functions.
When I first started using Haskell I lamented not having macros, and then I realized that many of my use cases for macros in Common Lisp weren't necessary in Haskell because of its laziness.
CL is still my daily-use language. But Haskell expanded how I think about programming almost as much as Lisp did all those years ago; that happened because Haskell took the functional metaphor to an extreme. I now write better Lisp programs because of my exposure to Haskell.
Different people find different things confusing.
It's most pronounced with lisp and ml-style languages, but I've also seen Java lifers bounce off of things as innocuous as Python's list comprehensions.
I get a little bit wistful when I see people bouncing off of exotic languages. Where's the delight in a chance to discover something new, and the excitement of a new thing to wrap one's mind around? It makes me wonder if some magic has been lost, or isn't being passed along like it should be.
As mumblemumble says, I think this is due to a blinkered understanding of programming, combined with the mistaken belief that if you know one programming language it's pretty straightforward to learn any other language.
If someone knows Java, Python, and JavaScript, and their idea of exotic languages are Rust and Go, then they have no appreciation of how diverse programming languages really are. Lisp, Haskell, Forth, Prolog, and assembly, will strike them as completely alien, which should be fine except they're under the impression that it's meant to be easy to learn another language.
Well not speak it, but reading it... yes. Japanese writing is like a prank that I would not believe is real... and I still doubt it.
This is a really smart guy with a gift for languages struggling with it: https://www.youtube.com/watch?v=bcdYKxHT8kY (NativLang 7m10s)
...and yet children can do it. Use that as a lesson.
Children? No, it takes them up to middle school to get somewhat proficient with Kanji knowledge for daily life, but they can still not read a lot of Kanji at that point.
Trust me you can learn this too on the same time span and semi-full immersion if you started now.
I analogize Blub more to Legal Latin: you're using an uncommon reference point to explain the material that could be explained perfectly well without it. The example I'd give is call/cc; I'm sure I could give a thorough description of how call/cc works, and leave a lot of programmers in the dust by doing so. Or I could explain how the yield operator works, and most of those some programmers would suddenly understand what I'm talking about. But they're basically the same thing (the only real difference being that yield continuation magic can only operate within its generator function, whereas call/cc works across function boundaries).
Should Haskell practitioners cater to C-like developers? If so, why bother with Haskell at all? This is one step removed from saying "just write an improved C and forget about Haskell", and in fact there have been attempts at it! It's just that they aren't Haskell.
Is it too much to ask of people trying to understand Haskell code to learn about it first?
It adds to the perception of "I need to understand masters level maths to use Haskell".
What about words like "operator", "function", "mapping"? Those are all from math too.
Besides, learning what a "monoid" is is not harder than learning what a for-loop is or a hash map. They aren't natural concepts, but they are easy to learn by programmers.
To compare it to spoken communication, there's a threshold where very formal queen's English from an extremely eloquent person makes them sound like an obnoxious showoff rather than fancy and serious. I feel like posts about Haskell often touch that threshold without a good reason. But that's my very subjective opinion.
We don't, that's the point. We abstract it above numbers, so that we can find that lists and natural numbers and many other things share the same "shape" when looked at from certain perspectives.
It's not about showing off, but about finding generalizations -- finding the "shape" of things -- that allow us to use the same abstractions over them.
And to talk about these shapes, we must use the appropriate vocabulary. It's shorter and more precise. Just like you say "function" instead of explaining what it means in lots of words every time you mean function.