Ask HN: What are the alternatives of hosting Apache Spark? I truly love what databricks is doing, but their pricing model is unpredictable. Are there any other hosting companies that provide a fixed price? |
Ask HN: What are the alternatives of hosting Apache Spark? I truly love what databricks is doing, but their pricing model is unpredictable. Are there any other hosting companies that provide a fixed price? |
Depending on your use case, you might spend many times less than you would using regular VMs. Many instances that are several dollars an hour on AWS can be used for a fraction of the price.
Its also fairly easy to automate the region selection and bid (on AWS that is, not sure about gcloud).
If you need streaming, obviously this might not be the way to go.
At a previous job, we just built our own ec2 image that ran spark in standalone mode for ephemeral spark clusters, and it was wonderful and cheap. And the clusters launched very fast compared to EMR.
Do you have more details on this?
Disclosure: I work on this service
[0]: https://aws.amazon.com/emr/
Disclosure: I work here.