AWS Lambda now supports up to 10 GB Ephemeral Storage

zozbot234 4 years ago |

If it's ephemeral to a function, it's not "storage". It's just core memory that may or may not be backed by physical "storage" media, and is organized as a tmpfs for the user's convenience.

10000truths 4 years ago | |

One man’s swap-backed tmpfs is another man’s page-cache-backed filesystem. /s

DevKoala 4 years ago |

I remember when I was breaking my head against the wall to fit puppeteer inside a lambda. Good times.

otterley 4 years ago | |

What was the use case?

DevKoala 4 years ago | | |

I was working for a programmatic advertising demand side platform. We had customers that submitted all sorts of ads to us and the lambda was in charge of rendering the ad and performing some validations on it. We accomplished it in the end using layers. It was in 2017, but it feels like a lifetime ago.

nunez 4 years ago |

this is fantastic for anyone wanting to run Chrome inside of Lambda (one of the biggest challenges is getting Chrome to fit)

matt-attack 4 years ago | |

What's the use case? I see a sibling commenter mentioned puppeteer which I'm aware is essentially Chrome. Are you too referring to running puppeteer in Lambda? If so, what is the purpose?

mulhoon 4 years ago | | |

Puppeteer-core can already fit in a lambda around 60mb and can be used for web scraping, taking screenshots, or performance testing.

atonse 4 years ago | | |

How’s the overhead? Are you launching full chrome every lambda invocation?

mulhoon 4 years ago | | |

In one lambda I can load chrome, visit a website, take a screenshot and save it to S3 in around 2s with around 280mb memory used. You could keep chrome open and reuse it if available, but you can't count on it with lambda being stateless.

I use this Chromium binary https://www.npmjs.com/package/chrome-aws-lambda

vdm 4 years ago | | |

> lambda being stateless

/tmp is ephemeral in the sense that it dies with the Lambda instance, but it is also persistent between multiple serial invocations landing on that instance, even if the instance is frozen and thawed between some of those invocations.

The process memory state on the instance (set up during the Init phase) is also available between invocations.

https://docs.aws.amazon.com/lambda/latest/dg/runtimes-contex...

nunez 4 years ago | | |

this package requires using JS for your stack. your options are much slimmer for other languages.

nunez 4 years ago | | |

Does that work outside of JS?

unixhero 4 years ago | | |

Chrome headless

bl4ckm0r3 4 years ago | | |

scalable pdf rendering

giorgioz 4 years ago |

This is also useful for deploying a big static website with AWS CDK which currently uses a Lambda and had the limit of 512mb (it was already possible to pass it by using an attached storage but this makes it easier).