Etched Is Making the Biggest Bet in AI

Etched Is Making the Biggest Bet in AI(etched.com)

45 points by CompuIves 2 years ago | 17 comments

How will these companies replace the software? Don't they need something similar to CUDA?

arjvik 2 years ago | |

(I work at Etched.) You need something as complex as CUDA only to support general-purpose programmability; Sohu is built for one thing and one thing only: transformers. So while we certainly need a software stack to harness the chip, it’s much easier to do so, and even easier then to adapt existing LLM serving tools (vLLM, etc.) to use this stack.

samspenc 2 years ago | | |

> Sohu is built for one thing and one thing only: transformers

Thanks for clarifying this. Could you clarify whether your chip supports the transformer architecture in general, or only specific models for e.g. Llama 70B? In case of the latter, would your ASIC have to be reprogrammed for each model?

CommieBobDole 2 years ago |

"In five years, AI models became smarter than humans on most standardized tests."

This seems to be a novel definition of "smarter" - one could also argue that the printed answer key for a standardized test is smarter than most humans.

ilaksh 2 years ago |

Wow.. I wonder how this compares to groq?

I made a request to access their developer cloud. Anyone have any idea when they start processing those requests and how many slots they might have?

Havoc 2 years ago | |

There are a couple now. Groq , sambanova and tenstorrent come to mind.

The more the merrier. If we do need power plants worth of juice then it may as well be dedicated custom hardware to optimise this to the max

ilaksh 2 years ago | | |

Thanks for telling me about Sambanova. Too bad they don't have an API. The speed is absolutely incredible. They actually have a demo where it just keeps re-running the chat completion as you are typing your query.

seany62 2 years ago | |

A coworker of mine recently got access to Groq. Not sure how many slots there are but we have a pretty strict rate limit for what thats worth

ilaksh 2 years ago | | |

I have access to grow, was asking about sambanova

samspenc 2 years ago | |

Was curious about this as well, seems like the closest comparison today is to Groq, wondering how it holds up.

jeffs4271 2 years ago |

Etched has an interesting approach, I am not convinced in their public info because it is vague. Could be great or could be bullshit. Well certainly some marketing.

candiddevmike 2 years ago |

> Specialized chips are inevitable

I mean, are they? Seems like the industry would prefer these things to become commodities, especially if it helps with portability and reproducibility.

jononor 2 years ago | |

The specialized chips may very well become commoditized over time. It is the way things go if the market is large and competitive enough - and that there is sufficient competitive advantage over the existing types of devices (GPUs in this case). For a rather recent example, see Bitcoin mining ASICs.

Of course GPUs (and CPUs, and pre-transformer/LLM type NPU/TPUs) are also going to respond to better suit LLM workloads. So we may see a convergence in capabilities.

What is for sure is that the LLM chip architecture is not yet settled. Currently available chips are mostly designed pre LLM craze, with slight adaptations. And have large areas of potential improvement, at least 10-100x (maybe 1000x) in power efficiency. And power efficiency is a key element (in addition to up from hardware investment). Which is the opportunity this startup (and others) have identified.

osnium123 2 years ago | |

Given that Moore’s law is slowing down, given that circuit design is getting more automated and given the rise of chiplet architecture, I expect that you will have more specialized chips.