Ask HN: Will transformer based LLMs hit an improvement ceiling?

1 points by jaguar75 324 days ago | 2 comments

A fundamental question I have is whether the current transformer based architectures are powerful enough to allow scaling towards AGI or are there ceilings here that are becoming more evident?

Though folks from the leading LLM companies are in the forefront of this, it’s hard to take their word due to an obvious conflict of interest. And it’s unclear if academia has more theoretical knowledge here as it seems these commercial companies are also leading the research work here.

eimrine 324 days ago |

What do you expect from AGI? What aspects of LLM have impressed you the most?

jaguar75 323 days ago | |

AGI is a vague term but a naive definition is smarter than any human in any fields/domain.

Clearly current LLMs are impressive in document/code generation but I’m having a hard time extrapolating that to AGI (assuming that it’s still reliant on current transformers arch).