AWS Lambda(aws.amazon.com) |
AWS Lambda(aws.amazon.com) |
I feel that GAE is a much more generalized solution for building/deploying applications, e.g. it's very integrated with BigTable and has caching/cron/queuing support. To a point you do not have to worry about the infrastructure, but write any GAE app at scale and soon you'll run into questions of caching some content and optimizing DB writes.
Whereas Lambda, at least right now, is a more specific use case: respond to an event by running some code. It's too early to tell whether managing Lambda functions will require fine-tuning the "advanced settings" (in all honesty it probably will) but it seems like it's much simpler.
Scheduled Tasks
AWS Lambda functions can be triggered by external event timers, so functions can be run during regularly scheduled maintenance times or non-peak hours. For example, you can trigger an AWS Lambda function to perform nightly archive cleanups during non-busy hours.
Now, how to deal with node packages that require libraries not on the default amazon linux ami?
Similar single event -> function service
Incidentally I almost had a heart attack when I saw the Lambda announcement. :P
[1]: We also provide the ability to respond to other kinds of events as does Lambda.
edit: oh also delete the generated file once accessed using lambda by either delete on get notification or just a lambda scheduled event.
You could "wrap" a Jenkins fleet running on EC2 with Lambda functions to set up on a build request event and tear down on a build finish event, and theoretically this would reduce the amount of resources you consume if your builds are few and far between. Think of GitHub webhooks triggering Travis CI as an analogy.
Whether or not that would actually be useful is debatable.
The way I see it is pairing it with the container service (ECS) and spawning containers in response to events, possibly spawning EC2 instances if you need more computer.
It looks like Lambdas can also be set to execute at a scheduled time, so you could make a delayed Lambda that would shut down any EC2 instances you spawned as well.
Sorry, where were you getting servers in 1999 that existed for fractions of a second?
If thats possible Lambda would be like PiCloud but without Python, and will stick around (hopefully).
I also put together the Netflix use cases in the keynote so if you have any questions I’ll try to answer them!
I think it would be really awesome to make applications that do not have any dedicated instances associated with them, but are entirely based on the various AWS platforms.
For some things, it seems like Lambda could be a drop-in replacement, whereas others, it would not quite fit the bill. Do you think it will be possible to build a viable a "server-free" application with Lambda and the other services (S3, DynamoDB)? Or is it probably going to be limited to the types of use cases listed on the landing page[1]?
Of course, it would be much more viable to have some sort of server running (for example I doubt Netflix has replaced their entire backend with Lambda functions), but a great majority of workflows dealing with stateless data manipulation and events can now be streamlined and even replaced.
Oh what I would have given to have this capability a year ago. :)
This looks like it's more geared towards enabling you to automate / orchestrate your AWS resources, that run customer-facing code.
What this does it make it a lot easier for smaller companies to do it without having to invest in all the infrastructure.
So I think yes, it will be revolutionary in the sense that it will get a lot more people using what I think is the future direction of compute and application building.
IMO what's really going to be revolutionary is what we end up doing with it, but we're going to have to wait a bit to find that out :)
https://www.youtube.com/watch?v=ere8VTJDqao
https://www.youtube.com/watch?v=u1MrZkGFpuE
TL;DR version:
It's a little convoluted, but that seems to be the way they're going for now.
(figured people would want to know this, and you have to scroll a ways to find out, so)
The jokes ppl make are starting to get a bit more real: https://www.destroyallsoftware.com/talks/the-birth-and-death...
But kudos for Amazon for furthering the datacenter-is-the-computer approach. It is simply the right thing to do.
Also, CS hardly owns the greek letter lambda, it like many greek letters has been used for many products and features over the years. Delta is an airline, but they're doing OK, among others. I am just curious what your issue with this is, and if you can come up with some concrete scenarios where this could really confuse people.
It's a bit amusing to note that they could have built Lambda on top of their own previous Elastic Beanstalk Docker offering instead, but chose to build ECS presumably for cost reasons (which everyone who has evaluated Elastic Beanstalk's Docker support will understand.)
I guess I would have expected others to describe this the same way ("replaces your distributed task queue"), but since I'm not seeing that description I wonder if I've misunderstood.
lambda_signal(Model, signal_name, lambda_fn_name)
When that signal fired on the model, the model would get serialized to JSON and sent to the lambda function. It'd be fun to whip this up if no one beats me to the punch.
lambda_signal(User, 'post_save', 'sync_to_mailchimp')
So I'd love to have a reactive task system where I didn't have to worry about the resource usage. Think how much precious developer time I've wasted just managing resource expenditures. Much of my coding this week has been just trying to optimize for this stupid constraint. How many processors do I have, how much disk io, is the memory getting close to swapping ?
I for one welcome our new cloud overlords.
Will definitely be interesting to see how Lambda actually works.
Try out Blockspring, you can play around with your usecase now (and no vendor lock-in).
Having reductions and composition would be really cool, but from what I could glean from the demos each function is containerized. Stateless and idempotent like lambda functions, but right now chaining more than one together requires an intermediary event.
With Lambda, the resource consuming the queue is managed for me - that is huge. Also, the pay by time model is perfect for this - instead of managing when to stop/start resource intensive instances, I don't even have to think about the problem. I only get charged what I used, which can be crucial during growth stages of a business or prototype.
The big penalty is the vendor lock-in, but this tips the scales for me for certain things.
Are you referring to kinesis as the queue here? Sqs/sns are not (yet?) supported as event generators for lambda.
I realize they are starting with node.js, but I wonder how this will work? It sounds like they plan to support arbitrary dependencies. Will you upload a Docker container with your necessary dependencies? They talk about milliseconds until your code is running, and containers may be (?) slower than that. Or am I hoping for too much.
We do it at http://wym.io/ by putting our runtime library in the environment that the code runs in and keeping the runtimes around, keeping the environment clean along the way.
We're working towards doing all that (and more than) Lambda does in a deploy-your-own fashion...
The ability to pay only for the fractions of seconds actually used, and the ability to scale quickly without provisioning (or over-provisioning) EC2 instances, is awfully attractive.
Plus, Amazon has priced this pretty aggressively — i.e., it looks shockingly cheap.
Oh, and FWIW, when has AWS raised a price? My servers have gotten cheaper every year I've had them with AWS (3).
Anyone know of any similar mechanisms for the OpenStack world, or more generally for any cloud infrastructure other than AWS?
I HAVE THE POWER
Similar, yes. We're excited that developers are seeing the power of the task/worker/lambda being the scalable unit of work rather than server or VM or even container.
Haven't used Lambda much yet but off hand we see many IronWorker advantages:
- Supports all languages - Easily scale to thousands of parallel workers - Integrated logging, error reporting, alerts, auto-retries on failure - Integration with IronMQ our queue service to achieve stream processing and pipelining - A great dashboard to manage your workers/tasks
And many innovations/announcements coming soon that'll make using IronWorker a great choice for developers both on AWS and inside the enterprise firewall.
IronWorker has a few benefits such as being able to run any docker container and react to a HTTP Post (webhook) directly, but overall very similar to Lambda.
And of course, it supports more than just Node.js.
From the FAQ :
"At launch AWS Lambda supports code written in Node.js (JavaScript). Your code can include existing Node.js libraries, even native ones."
so it seems that you can avoid lock-in if most of your javascript code is not too coupled to Amazon's API
And they claim on the detail page:
"With AWS Lambda, there are no new languages, tools, or frameworks to learn. You can use any third party library, even native ones. At launch, AWS Lambda will support Node.js code, with support for other languages coming in the future."
Anyway, libcloud seems to be leading the way in this kind of standardization, so that's good news. There's also providers that have drop in AWS-compatible APIs for some services. Then there is stuff like apache mesos for creating that infrastructure, which is less relevant to smaller companies.
But I suspect that the last thing Amazon wants is to turn into a commodity, so it's in their interest to release as many differentiators with non-standard APIs as they can.
It seems that this system puts together all events generated by Amazon services to trigger new events. This is usually not that complicated to code, but the catch is to make these (usually) scripts/programs highly available, which Amazon provides.
But I think it is simply named after the functional language feature. The core concept of this service is function.
Amazon has a LOT of services, and the better the naming scheme they use, the easier it will be to remember what they each are. I'm genuinely happy that they used a name that will make it trivial to map the function (heh) it serves in my head.
Just like how you can get specific increases on your spot bids so long as you acknowledge that if you get a spot spike, and your personal limit is far greater than the spike -- you'll be charged your rate.
A quick back-of-the-envelope test on my machine for both node and python suggest similar (around 100 ms) start up times for the bare environments.
Do you just mean that you'd prefer to use node for smaller jobs anyway?
so maybe lambda its not suitable for things that are less than a few seconds.
I could write python with no django and try to structure things so they don't need database. but then I would have to take input from and put output data back to a queue of some sort.
>> A quick back-of-the-envelope test
and actually python on mine was even faster than node when both are doing nothing. so my assumption was wrong (but its based on mostly using django and all my node stuff is small servers and tools)
» time node noop.js node noop.js 0.04s user 0.01s system 98% cpu 0.045 total
» time python noop.py python noop.py 0.01s user 0.01s system 90% cpu 0.026 total
but importing significant numbers of libraries would be the real test.
the main benefits to lambda would still be:
- scaling for high demand - no cost for idle time
and let's not worry about the micro pennies in startup times
I published some explorations of that environment here: http://alestic.com/2014/11/aws-lambda-environment
Let me know in the comments on that article what other commands you'd like to see output for.
Starting with S3 fills a gap as there is AFAIK otherwise no way to start processing based on changes to S3, without writing a rather wasteful program to list the resources, maintain state, and hunt for differences.
I wonder if there's any good way to generate that timer expiry event...
As someone who does use AWS a lot and lambda functions, then yes, overloading the term is dumb. I'll also probably end up using Lambda functions.
Naming the product something that closely resembles its functionality seems like a great idea for a product. It's still Amazon Lambda, and I think it's very unlikely that people will simply refer to it as "Lambda." If CS students find themselves hopelessly confused by this product name, they are going to have a real hard time tying their own shoes.
If you search for Y Combinator, luckily, https://en.wikipedia.org/wiki/Y_combinator still shows up on the first page of results, I expect the same will be true for Lambda, so I can confidently say we won't lose a generation of CS students to this faux pas.
And, as I've noticed in the past, searching for "y combinator" + any language name will often keep useful results away until you finally block this site. I'm not sure if this is still comprehensively true, but it seemed so to me at one point. So there is a very real impact on search results that can happen.
If you tried to sell ecommerce software under the name Amazon, you would get a cease-and-desist very quickly.