“Mostly functional” programming does not work(queue.acm.org) |
“Mostly functional” programming does not work(queue.acm.org) |
Nearly all useful, reusable software today is written in a mostly or entirely imperative language. This is despite the fact that functional programming has been around for at least 20-30 years. So, my basic question is, if functional programming is so much better, why isn't more software written in a functional language? Or put another way, why are there so many blog posts promoting functional programming when it clearly hasn't produced results.
I think the reason for this is that pure FP replaces classic programming with puzzle solving. It satisfies math-oriented minds by forcing usage of this math-oriented layer. But it looks like software isn't math and you often need to work with a computer in a more direct way.
In some ways the reason it isn't used a lot is for historic reasons as much as anything else. Functional programming, for a long time, was seen (often rightly so) as more computationally expensive than the same imperative code for most tasks. For a long time, that meant that you weren't going to use functional programming: when every single byte and cycle matters, you're going to go with assembly or whatever is closest to it/translates really well to it. (e.x. C)
We now live in a world with abundant memory and excessive unused clock cycles. If I remember correctly, there's also been a good bit of research and work in the last 20-30 years The historic reasons for not using functional programming are gone for many (but definitely not all; people still hand roll assembly, too) of the original use cases. However, you then have 30+ years of programmers who were largely trained in and almost exclusively used imperative languages over the course of their career. That kind of market share is hard to compete with (because it's similar to market share; if everyone is using imperative languages, you need to be familiar with them too if you want to stay in most parts of the industry), and is self perpetuating (if you want to work in CS, you're going to need to know imperative languages for most jobs).
I'm also fairly certain there's a good number of useful, reuseable programs that're written in functional languages, but I'm in an airport and nothing immediately comes to mind.
Learning when it's useful to use state, and restricting side effects to those parts of the program that require it, is a good skill to have. Now, pure functional languages may have had a slow adoption, but they are a good way to learn the paradigm that can be translated to other environments.
Also, if you think FP hasn't produced results you haven't been paying attention, or you don't program neither web applications nor database APIs. Declarative libraries like jQuery selectors and LINQ have spread like fire, and any programmer of complex asynchronous applications should take a look at Reactive Programming (which represent state changes in a functional style with the Streams and Futures abstractions) and Agent-based Models (which encapsulates them through message passing).
I like to think that the whole discussion is straw-man. The useful, reusable software can be created in any language. But when every project is a gamble, and most projects starts out as imperative in orientation, then the chance of those bubbling to the top is greater. Good modularity of software happens on a higher level than picking the programming language of choice.
Probably the statement "> 50% of large telecommunications companies have done something with Erlang at some point in time" is correct. I guess you could even make it a 100% considering that parts of routers are often programmed with FP.
However, most stuff is done in imperative languages. ;) Routing and Connection handling is certainly nice using FP, just consider the amazing Hot Code Swapping features in Erlang. On the other hand, writing APIs, connecting those etc. is much more convenient in imperative languages.
Pure functional programming is programming with mathematical functions. ... Calling a function with the same arguments will return the same result every time.
That leaves a whole lot of other functions, the non-mathematical ones, the real-life ones where you read files or I/O or user input out of the "Pure" definition, if I understand it correctly that is. And by that logic, means '"Pure functional" programming does work, but only if...'. Or, in other words: '"Pure functional" programming does NOT work in real-life software'
Instead of
readFile(file_name)
readFile(file_name,complete_description_of_the_universe)
Granted, a full complete_description_of_the_universe is rather space consuming. I don't even think it would fit on a floppy. Maybe a Zip drive?
Thankfully, you don't need the COMPLETE description. You only need the parts that your computer can access. There's no reason to store the amount of gas in the bowels of every elephant in India, unless you actually have an elephant gas sensor. So our function is really
readFile(file_name,complete_description_of_everything_the_computer_measure)
Since, by definition, the computer can measure the value of complete_description_of_everything_the_computer_measure, there's no reason to store it in memory at all. We just measure it when it's needed. Granted, it's a little slow to read the hard drive, versus having the entire drive stored in memory, but premature optimization and all that.
It's still completely functional, though. We still pass the function two parameters. The first parameter is a set of bytes in memory with a EBCDIC representation of the name of the file. The second parameter is a physical computer that can perform the measurements. As long as you pass the exact same computer to the readFile function, you'll get the same result every time.
Same thing for writeFile. We pass in a physical computer and the function returns to us a new, physical computer that's similar to the one that we passed in, except that this computer has some different magnetic spin arrangements on its hard drive and its system clock is reading a different value.
Granted, carrying all of these physical computers around gets tiring, plus people start giving you guff about conservation of mass or whatever whenever you make a new computer. Thankfully, the functional compiler guys have found ways to optimize the code so that you can destroy the unneeded computers very rapidly. In fact, it's faster than the human eye can detect. Without careful measurement, you'd think that it was just one computer the whole time.
http://chris-taylor.github.io/blog/2013/02/09/io-is-not-a-si...
You're right in thinking that software with no side effects at all wouldn't be very useful.
Functional programming might well produce better software, for some criteria of "better", but these quality-based criteria are dominated by the "is it cheaper and more effective than paying a person to do this" criterion employed in industry. This isn't necessarily a bad thing - if even the "worst" imperative software is value-creating, then there's a lot of scope for superior functional software to create even more value, there just isn't a lot of pressure to do this within industry because imperative programs are good enough for most use cases.
There are other factors, such as path-dependency issues relating to developer skill-sets and toolchains, which give an advantage to imperative programming too. It seems likely to me that functional languages will close at least part of this gap, and perhaps Erik Meijer is underestimating the importance of "mostly functional" programming as a stepping-stone to "totally functional". Dijkstra was wrong when he claimed that learning BASIC was a mental mutilation preventing a person from ever understanding programming, and by the same token I suspect that we'll see plenty of people becoming good functional programmers who first encountered FP in passing callback handlers around in JavaScript.
As for functional programming, all of the recent languages (Rust, D, Go, Clojure, Scala, etc.) have functional features such as lambda functions. I don't think it's a stretch to say lack of support for functional programming is not an option for new languages.
As a couple of examples of software written in a functional language, consider Twitter and (ironically) Hacker News.
For several reasons, the hardware required to run Lisp code was too expensive vs what was required to run C, Pascal, Algol and their descendants.
Business don't change programming languages, just because. Unless you are doing greefield projects, there is a whole eco-system from tooling, building, trainings, developers, legacy code that needs to be taken into consideration.
By the time computers were getting cheap enough to handle FP at reasonable price, the OOP finally managed to get into the industry at the enterprise level. As such it got the place that could have been taken by FP.
OOP goes back to Simula and Smalltalk, which happen to also be Lisp inspired.
As for not producing results, many train schedule systems run on Lisp.
Probably swift will be a significant cause of exposure for it.
However, I don't think a total functional language is the best, but a mix (like swift, julia).
Because (1) honesty the “industry” programming is in the stagnation. Otherwise the boring Java♯'d run on Martin-Löf type theory; (2) it may require more brain; (3) we all learn imperative programming first.
(And of course this reasoning is ignoring one aspect: we (mostly) don't want to write bug-free software. Who would pay for support?)
Placing onerous restrictions on what someone is permitted to do in order to satisfy some formal abstract programming model - that is the thing that really doesn't work too well.
This makes arbitrary programming arbitrarily difficult: many simple concepts are completely prohibited. Fanciful convoluted ways that don't violate the formalism have to be fabricated...because we can't violate our arbitrary formalism! No! Not in the name of instructing a computer to do something. /me adjusts his english headmaster cap.
Data.ByteString is basically a pointer to a byte array, and it's one of the most common datastructures in Haskell.
Even more importantly, the optimizations the Haskell VM is allowed to do because of the immutability and purity constraints means that even naieve and trivial solutions using Data.ByteString on general I/O problems like webservers perform in the best of class amongst native competitors like Nginx.
edit: And the majority of the web is ran consistently and reliably? You mean apart from it being threatened by huge security vulnerabilities almost every month for the past 20 years? With so much insecure systems still connected that DDOS attacks are a day to day concern for system administrators?
I was really bashing the single-paradigm purist approach in general. It's much too rigid and difficult for anyone who isn't a card carrying mensa member.
I'm absolutely all behind the idea of looking at what the future of programming languages is going to be, and how we're going to cope with diverse, highly parallel systems, and how we can reduce errors, bugs and failures. But it will be a while until we figure that out.
Not only are monads not the only solution to doing I/O in a pure programming language (stream based I/O or functional reactive being another), they need not such a mathematical explanation.
An I/O monad is a simple class modelling a box. There's two functions for it, one puts something that's not in a box in a box. The other function allows you to give it a function that is applied over the function in the box, without taking it out of the box (this is called a 'bind' operation).
Note that there's no function to take the value out of the box.
The trick is that the only thing that knows how to get the value out of the box is the VM itself. So its task is to at the end of your function, execute whatever function created your monad, and then execute all functions you gave it using the 'bind' functions sequentially over the returned value (which might include unwrapping more i/o monads).
Since every bind function relies on the previous value of the monad, it can naturally only be executed sequentially.
Note that this explanation does not explain monads either in full or very accurately, it's just one of the patterns that monads are used in, and hopefully gives you an idea about how it is that monads are used to enforce encapsulation and sequential execution of effectful functions.
http://stackoverflow.com/questions/5892653/whats-so-bad-abou...
Some classes of functional languages had lenient evaluation, evaluation is still lazy, but all expressions will be evaluated at some point in the program. You cannot do the infinity tricks, but you can still write the nice recursive functions, those that are used to show off the benefit of laziness.
The Haskell type of laziness bites you in the ass when trying to do parallel evaluation, which is why you need the rpar/rseq constructs to avoid opening the nasty trapdoors that regular evaluation will not hit.
Even besides I/O and parallelism the laziness can bite you. Mainly, it can make it hard to predict memory/processing use, unless you are intimately acquainted with the innards of Haskell.
That exact phrase was said to Erik Meijer by Leslie Lamport when Erik interviewed Leslie. [1]
It seems clear (if you take Leslie's word as gospel) that there is room for 'Mostly Functional'. That is, accept that your program goes through a series of states, but use pure functions to calculate those states.
However, as an imperative programmer who's found Haskell over the past few years (thanks, in a large part to Erik Meijer), I happen to think that Haskell has it just about right.
[1] http://channel9.msdn.com/Shows/Going+Deep/E2E-Erik-Meijer-an...
Declarative sublanguages such as LINQ work precisely because of this "contract" of being mostly-functional, even if the work behind the scenes (database accesses etc.) certainly changes the state of the world.
- lazy evaluation with side effects - replacement optimisations based off of referential transparency when there are side effects - general side-effectiness
This is basically saying that functional programming doesn't work when it isn't functional... but we can still have functional "chunks", and monads _are_ an effect system.
I don't really understanding what he's trying to prove, of course purely functional semantics fall apart when side effects are introduced.
using(var file = File.Open(path)) { _someField = file; }
..and then use _someField in another method.
It's not so much a problem with understanding closures, but of the limitations of IDisposable.
I must say that the article has a completely different tone than when you'd talk to him personally, though he likes to talk in extremes and only afterwards talks about what is practical yet not in a diminished importance kind of way. I interpret the article just as a big indicator to a fundamental problem how we can express stuff in language and their accompanying downsides.
I'll try to illustrate my point:
In programming we often have to do something more than once. In terms of mental complexity, I'd rank our options like so, from easy to hard:
Loop -> Recursion -> Fixed Point Iteration
We can do the same thing with all of them. Recursion in some cases can lead to a stack overflow and any fixed point iteration has the disadvantage of, well, being f-ing hard to understand.
A beginner can get a `while` or a `for` loop in seconds. They're very, very intuitive. Once you figure out solving problems in software often requires doing things repeatedly, loops open up a lot of power.
Recursion is more difficult once you throw in conditionals and return values. They're harder to reason about. Loops are simple. So why bother with recursion? I learned recursion because it exists and I love learning. Could I have written just as much software without it? Yes. Would it have been just as functionally correct, maintainable, easy to read? Yes.
I know very little about fixed point iteration. Why? Well, because I have loops and if I'm feeling jaunty, recursion.
Having said that, I recently came across this article about the Y combinator:
http://matt.might.net/articles/implementation-of-recursive-f...
I recommend that article for everyone. But the most important part is the section "Exploiting the Y combinator".
> This formulation still has exponential complexity, but we can change it to linear time just by changing the fixed-point combinator. The memoizing Y combinator, Ymem, keeps a cache of computed results, and returns the pre-computed result if available:
The author is talking about his algorithm for calculating Fibonacci numbers. While the naive recursive approach is limited due to its computation complexity, by caching intermediate results we can short-circuit the calculations made.
> The end result is that the 100th Fibonacci number is computed instantly, whereas the naive version (or the version using the ordinary Y combinator) would take well beyond the estimated lifetime of the universe.
I don't know enough about what's being done here to say that this optimization couldn't have been made with a loop (I believe Turing equivalence proves that it could have been) but at least I get to see some of the magic that fixed point iteration gives us.
Haskell, as a functional language, offers a lot (purity! type safety!) but the community hasn't really shown how these things are valuable. Type safety is great, but is Haskell materially better than Objective-C (my primary language) in that regard? Pure functions are trivial to test, but in a language with a reasonable amount of type safety, how many individual functions are getting tested anyway? Of course I can write pure functions in an imperative language and often do. And I can do so without painfully abstract concepts like the dreaded monad if I want to do something as pedestrian as generate a random number.
The reality is writing software for most people is about the platform they're targeting, available libraries, ease of use and techniques for dealing with long-term complexity. I have yet to find a reason to use Haskell even though I would love to -- more accurately, I have yet to find a situation where I could justify using Haskell.
Despite this being akin to heresy, I much prefer to implement, then formalise later. I find it easier to "get the job done" that way.
Edit: I see there are quite a few. Please tell me about one :)
Haskell actually models the real universe better than any other languages - in the real world, we have the concept of now - some state of the current universe until an event elapses, when we have a new now, and we have a concept of the past, but in fact, this past does not exist anywhere - it was transient - we can't go back and modify this old universe to cause a change in the future, unless we introduce theories of parallel universes. the only thing that ever exists is the now, as you might learn from a Buddhist.
A Haskell function which does IO is just an event which causes a change from an old now in the universe to a current now, and the old one, the past, no longer exists.
It sounds like you may be conflating pass-by-reference with passing references.
If you mean implementation: which ones? In the context of immutable and often even persistent data structures, copying them doesn't make a whole lot of sense.