Does the current GenAI path lead to AGI?

5 points by jaguar75 1 year ago | 2 comments

I struggle with this question. It’s rationally hard to see a token predicting machine (no matter how good) lead to reasoning and understanding of our world. Surely feels like a different approach is needed which may not extrapolate from the current rapid success of Transformer based GenAI approaches.

Would love to hear thoughts from the HN community.

dtagames 1 year ago |

This is a philosophical question more than a technical one. We already have LLMs that can produce output that appears to be intelligent and is close enough to be used in many contexts where "intelligent" output would be helpful. So the technical question is solved. An essay called The Bitter Lesson[0] suggests that with enough nodes we can capture every kind of context.

The philosophical question hinges on whether or not transformer-type behavior actually is intelligence. Some suggest that the human brain actually operates this way, and that belief is the basis for all the vocabulary and context in machine learning. Others believe (philosophically) that intelligence is more than the sum of these node parts and signals.

Will medical science be able to answer the question? Are humans able to escape the observation paradox enough to find out how their own intelligence operates, when such information could only be produced through intelligence itself?

Personally, I think intelligence is more than node firings but less than mystical, spiritual soup. While we don't know how it really works, we can push LLMs to be close enough to "intelligence" to serve many useful purposes.

[0] http://www.incompleteideas.net/IncIdeas/BitterLesson.html

jaguar75 1 year ago | |

Thanks for the reference. Yes, it boils down to what is intelligence. I guess true AGI is more than pure intelligence and also involves a sense of “self” or ego and such.