New View of Microservices(github.com) |
New View of Microservices(github.com) |
Communication between nodes is transparent on the language level, its all message passing between processes.
There is a approachable language with great tooling on top of the Erlang VM [1] and a great web framework [2].
[1] http://elixir-lang.org [2] http://www.phoenixframework.org
Plus it's easier to learn it than it seems first time when you see it. Most of our Erlang-developers came from other languages, but they learned basics very quick: 1-2 days until the first commit on production.
The only one thing, it's not so easy to debug it in the beginning. But it's a question of practice, I think.
[1] http://learnyousomeerlang.com/introduction#what-is-erlang
I thought we all agreed CORBA style location transparency was a bad idea?
https://en.wikipedia.org/wiki/Common_Object_Request_Broker_A...
In other words, the problem in the past was, we designed it as if it would be used locally, then we naively made it remote. But now, we're talking about designing it for remote use, but then using it locally.
Interesting.
Things like distribution are a tradeoff, an optimization : you should never split a monolithic application into 2 communicating parts until you have good reason to do so.
A monolithic system executing a given function is pretty much guaranteed to be simpler than a distributed system doing the same function.
A small amount of microservices is easier on the guy doing ops work for the system, because they can be individually replaced, upgraded or changed (IF the developers put in the constant amounts of efforts needed to make that possible. Every release should be able to deal with past versions of everything it talks to). These guys are also usually the ones doing load balancing and the like, so bonus points for being able to spread them over the network.
Of course, numbers of microservices always grow, and grow, and grow some more. Soon it's not easier anymore on the ops guy, and becomes a constant drag on progress for everyone.
Keep it really simple, until it breaks too often.
Now that I have my old fogey hat on anyway, I might as well state the 'old wine in new bags' feeling I get with articles like this. Encapsulation and separation of concerns is good you say? But inconvenient because there is overhead? So infrastructure to manage dependencies and deployment of sets of loosely coupled components is useful? Whodathoughtthat!
Didn't Vonnegut have a line about how it affects you to see the same mistakes repeat over and over?
http://hook.io uses this approach see: http://echo.hook.io/source https://github.com/bigcompany/hook.io
At StackHut (www.stackhut.com) we're working on something similar, a micro-services based system that's provides a scalable, typed, JSON interface into JS/Python classes wrapped up in containers.
Can't wait to see where this microservices-based path ends up...
People interested in this should also check out JAWS: The Server-Less Framework Powered By AWS Lambda.
JAWS V1 is coming out in a few weeks and it is insanely awesome.
http://fmontesi.blogspot.com/2015/06/non-distributed-microse...
Very buggy especially when clients disconnect/reconnect. But, it's good starting point if you are interested.
Note that there's always a catch in going from a monolithic application to one with a more distributed nature. Distribution of data requires data synchronization and results in integration challenges, because of the unreliability and higher latency of networks.
Lastly, Martin Fowler c.s. wrote some nice articles on microservices: http://martinfowler.com/tags/microservices.html. That will help as well.
https://blog.codeship.com/exploring-microservices-architectu...
* Front end: Perhaps a basic Node.js or Rails app. It's UI only; all data handling is done through APIs to the backends.
* Admin: Completely separate app. Same here, UI only.
* Post backend. This is a data store for articles. It has APIs for creating, updating, deleting articles. It also has a permission system that connects users to the groups that they are part of. This could be a separate microservice, but it's natural to keep it close to the data (also for performance reasons, since every post lookup needs to check if the calling user is allowed to see it).
* Login backend. This has a user database and can do things like accept logins via other services such as Google and Twitter, via OAuth.
* Miscellaneus support services. For example, say you want to support upload images: Make it a microservice that calls into ImageMagick and stores stuff in S3. Or let's say you want to support importing WordPress exports (WXR archives). Make it a service that parses and uploads asynchronously to the post backend.
I have built this as a monolith, and as a microservice system. The microservice platform is much more flexible and clean.
It also allows us to reuse every part for other apps — and this is in my mind the number one benefit that microservices gives you. We've had several new projects completely unrelated to what we've built before, and we have simply reused the microservices we had already created, only swapping out the new front end.
Making reusable microservices is an interesting challenge, since core concepts (things like users and data objects) need to be generalized, and mechanisms built in so that you can make specific features that don't burden the microservices with too many conflicting concerns. For example, let's say that your CMS's public site wants users to be able to "like" individual articles. Instead of building this into the article store, create a microservice which abstracts the concept of "liking". Or let's say you want users to sign up for a newsletter version of the site that sends them a digest every week; again, don't build it in, but make a microservice that abstracts this into "subscriptions", with a registry of emails and what users are subscribing to, and so on. To generate well-formatted emails, simply let the front end provide an API for rendering templates, which this "subscription" microservice can invoke to do the actual rendering. Generalizing it this way makes it easy to then create new types of subscriptions, such as SMS messaging.
Another important point is that every microservice has to be multitenant; a single microservice can host multiple unrelated datasets, and they need to be partitioned in a way that allows the APIs to not comingle the data by accident. It's not rocket science, but it takes some planning and practice to make the APIs feel natural.
I guess what bugs me is not the mind changing (I do that all the time myself). It's that those without a history lesson behave as if this is a bloody revolution, and they have to pick sides. And the people who think switching solutions will magically fix all of our problems.
Pass by reference became de rigeur for high performance systems when we agreed that the message passing overhead represented the plurality of computation time.
But pass by reference is shared state, and as we started trying to build systems with threading and for multiple cores the ugly truth about memory coherence on modern processors came out. It was pretty bad. Java and later C had to come up with some pretty odd looking compromises to get safety without throwing away all of the speed advantages.
We haven't measured the actual time. I would say the time learning Elixir has not been a hurdle or anyway limiting factor to the projects we are working on.
We basically switched as a company from writing servers in C#/nodejs/python to just Elixir. Did this gradually during end of last year and this spring.
Though I must mention that all of our developers were already familiar with multiple languages. And since we have multiple projects going on we can transfer knowledge easily. So YMMV.
I love the Elixir syntax but worry it'd be like Scala: a better way for people who already know Java, rather than a way to use BEAM / OTP etc for people used to modern languages.
There are a handful of rules to remember, if that, and it's reasonably straight-forward. module:func becomes :module.func, the erlang func probably wants char lists rather than strings, atoms are lower-case and should be changed to :atom, vars are upper case and should probably be lowered.
Learning the Elixir language is easy, thinking functionally less so (for me at least).
Agree with rossj, there're some rules to remember and then it's not that hard.
Learning the procedural aspects of Erlang or Elixir is very easy. In that sense I agree with the previous responses. Erlang gets a lot of bad rap for the syntax. But, it is really no more difficult that any other language to learn. The syntax can easily be learned in a day. Elixir's syntax can be learned even more quickly because it looks a lot like Ruby or Python. Thus, you can certainly start write scripts in Erlang or Elixir relatively quickly. My guess is that the average, experienced developer could start writing basic scripts in an hour or two. However, those scripts would be entirely procedural in nature, unless they are already very familiar with functional programming and pattern matching.
The first hurdle in Erlang/Elixir is the functional aspects of the language and understanding the power of pattern matching. Both of these require rethinking how you build a program. But, both of these are also incredibly, incredibly worthwhile and powerful. Elixir in particular is a great medium for better understanding these concepts because it is a very approachable language. Getting over this hurdle did not happen over night for me. I rarely used recursion in Ruby or Javascript (there is no tail call optimization). On the other hand, in functional languages recursive functions are the norm. I learned to really love them with Erlang/Elixir and now it frustrates me when I can't use them elsewhere because it has become much easier for me to reason about recursive functions than loops. Overall, functional programming and pattern matching are amazing things to learn but really grokking them won't happen over night.
The next hurdle with Erlang/Elixir is understanding OTP. This is THE incredibly powerful aspect of the Erlang VM. However, it is also not an easy subject matter to approach unless you already have a strong background in distributed systems. I did not. Hence, it took me a while to compose programs that actually used the Erlang VM for what it was built for, OTP. Thus, this portion of the Erlang/Elixir learning curve felt long and slow. I was not used to spawning new processes on a whim and passing messages back and forth. Initially I would send messages only one way without realizing that the appropriate pattern is to view it in a standard server-client relationship where there should always be a request and a response. Eventually I started viewing all the little applications running inside my larger program as self-contained capsules/servers that communicated among each other. However, it took me weeks of really intense study to get comfortable with. But, once you get comfortable with OTP you have a lot of built-in power (people rarely mention Mnesia, DTS, ETS gen_server but they are awesome).
Overall, my experience with Erlang/Elixir has shown me one big thing: pick the best tool for the job and there are a lot of jobs that you don't need OTP for. When you need to build Whatsapp, you should pick Erlang/Elixir. When you want to build a simple web app that only has a couple of end points and much of the dynamism is handled on the front-end, use Go, Node, or my personal favorite Clojure. Erlang/Elixir shine in the context for which they were built. But, they are not built for everything. And, if you only want the procedural aspects of the languages, there is no reason to choose them over Go, Node, Clojure, or Scala...all of which have solid approaches to concurrency.
And what about a monolithic system executing thousands of different functions?
If your app is a simple todo list, sure, go monolithic (but you should still delegate storage to some other tool). If it is much more complex than a todo list, then it is very likely that you have many packages containing many modules developped and maintained by many programmers. And if one glitch on one side of the monolith affects a remote module and functionalities depending on it, or even breaks down the whole monolith, then you should go microservices. Or split your code base. Or do whatever is needed so the thing can still move forwards, if slowly, without breaking apart at each little step.
See , that's the problem with you people advocating micro-services , RIGHT HERE. Your statement is ridiculous and make it hard for people like me to take micro service advocates seriously.
Seriously though, all of the problems breaking the system up will be blamed on people who already left the company, whether they deserve it or not, and the patterns that keep you in a monolith will continue on forever.
The only solution I've seen sort of work for this is to make a single binary, but create modules with extremely clear boundaries. Even so far as compiling them separately if the language is statically typed. But even that can go sideways the moment someone builds caching into the system.
I've come to appreciate why external caches are so popular, but for my money I prefer caching at the HTTP layer. You have to solve all the same problems, but you get one level of cache for free, it's easier for QA to understand, and it puts cache eviction in the hands of mature pieces of code that all implement the same spec (which might account for my previous point).
If you honestly believe you can't write robust & scalable monoliths at all, that is an extreme position that is hard to square with many peoples experience.