Ask HN: Whats your intuition on AGI breakthrough?

6 points by filup 51 days ago | 2 comments

Fast won't be necessary, and we won't reach AGI with how LLM's work.

We will need a complete reconstruction of the architecture. We need a DNA based system. A system that can potentially hold models billions of times larger.

I think the breakthrough is going to happen when we start taking a different approach like watching the bird fly and creating the wing and airplane.

The clock speeds will be low, but every experience should update the entire model of what reality is.

What's everyone's intuition on what the major breakthrough will be?

JohnFen 50 days ago |

I honestly don't know what anybody means when they say "AGI". There are too many incompatible definitions floating around for it to be a useful term.

If the term means "a human-like intelligence", then I don't think it's not going to happen within my lifetime. Probably not ever, but "forever is a very long time".

mindcrime 51 days ago |

My first intuition is that we don't have a (generally accepted) definition of AGI that's rigorous enough to really have this conversation. I've heard people make convincing arguments in favor of:

* We won't have AGI, ever.

* We won't have AGI for another 50 years

* We'll have AGI in 2029

* We already have AGI

* We've had AGI since 2021

and so on. So clearly there's a broad spectrum of definitions at work in all of this.

My personal take? I wouldn't call anything we have today "AGI" just yet. And I generally think that just making bigger and bigger LLM's probably isn't the path. OR, I might say that "making bigger and bigger LLM's" could theoretically get to AGI, but if that does happen, the result will be so computationally inefficient that it would still be worth researching other (better) ways to get there. As an aside, I guess it's implicit in what I just said, but to say it explicitly: I believe there are almost certainly multiple different ways to get to AGI.

With all of that said, I think there's probably a lot of value in leveraging well known and understood algorithms that run efficiently (on CPU, maybe GPU, maybe on analog computer, whatever) in combination with neural networks (whether using Transformer architecture or something different). I find myself drawn to looking into neuro-symbolic techniques, and looking at ways to reuse some older ideas like stuff inspired by Minsky's "Society of Mind" or the old "Blackboard Architecture" approach. Throw in the idea of using (possibly) ensembles of SLM's, LLM's, maybe some "MLM's (Medium Language Models, if we can talk about such a thing), in conjunction with ideas from classical planning, FOL inference, etc., and collaboration the aforementioned SoM / Blackboard kind of approaches and I think you might get somewhere. I expect there's a place for some evolutionary techniques in there as well.

Over the years, I've developed a belief that the single biggest problem with all of this is the lack of shared representation (of "knowledge" or whatever you want to call it) that can easily be shared across these different modalities. Like, interfacing FOL inference and an LLM isn't obviously straightforward, at least not in an efficient way. You could hard force "natural language" to be the closest thing to a shared representation and translate in and out of NL and things like N3, Common Logic, Ontolingua, KQML, UNL, etc., but that's still leaning awfully hard into making the LLM do a lot of the heavy lifting.

Anyway, sorry for the long-winded ramble there.