I would bet against it being possible to implement consciousness on a PDP, but I wouldn’t be very confident about it.

The AI hysteria I'm talking about here is the "foom" hysteria, the idea that a sufficiently powerful model will start self-improving without bound and become some kind of AI super-god. That's about as wild as the LHC will make a black hole that will implode the Earth. There are fundamental reasons to believe it's impossible, such as the question of "where would the information come from to drive that runaway intelligence explosion?"

There are legitimate risks with AI, but not because AI is somehow special and magical. All technologies have risks. If you make a sharper stick, someone will stab someone with it. Someday we may make a stick so sharp it stabs the entire world (cue 50s sci-fi theremin music).

Edit: for example... I would argue that the Internet itself has X-risks. The Internet creates an environment that incentivizes an arms race for attention grabbing, and the most effective strategies usually rely on triggering negative emotions and increasing division. This could run away to the point that it drives, say, civilizational collapse or a global thermonuclear war. Does this mean it would have been right to ban the Internet or require strict licensing to place any new system online?

You shouldn't use "theory" when it comes to science unless you know what that means. Gravity is a "theory." "Theory" means that it has a working model, comes with a ton of observational evidence in line with predictions, and it has yet to be replaced by anything better. Outside of math, nothing is ever proven. Any leading scientific theory is, at best, "yet to be disproven." And it stays in the lead until something better comes along: more accurate, extending over a greater domain, etc.

Hawking radiation has yet to be observed.

And if you're worried about micro black holes, well, even an iron atom has a non-zero chance of tunneling to a micro black hole state. No collider needed.

Cyc isn't secretive, it's proprietary, the way the Microsoft codebase is, the Adobe codebase is, and so on.

Traveller was (and is) a space-based RPG, although the original publisher is long out of business.

https://en.wikipedia.org/wiki/Traveller_(role-playing_game)

AI greatly reminds me of the Library of Babel thought experiment. If we can imagine a library with every book that can possibly be written in any language, would it contain all human knowledge lost in a sea of noise? Is there merit or value in creating a system that sifts through such a library to attune hidden truths, or are we dooming ourselves to finding meaning in nothingness?

In a certain sense, there's immense value to developing concepts and ideas through intuition and thought. In another sense, a rose by any other name smells just as sweet; if an AI creates a perpetual motion device before a human does, that's not nothing. I don't expect AI to speed past human capability like some people do, but it's certainly displaced a lot of traditional computer-vision and text generation applications.

Maybe. The statistical models are definitely better at natural language processing now, but they still fail on analytical tasks.

Of course, human brains are statistical models, so there's an existence proof that a sufficiently large statistical model is, well, sufficient. But that doesn't mean that you couldn't do better with an intelligently designed co-processor. Even humans do better with a pocket calculator, or even a sheet of paper, than they do with their unaided brains.

There are only two problems with this: One, statistical machine learning systems have an extremely limited ability to encode expert knowledge. The language of continuous functions is alien to most humans and it's very difficult to encode one's intuitive, common sense knowledge into a system using that language [1]. That's what I mean when I say "sour grapes". Statistical machine learning folks can't use expert knowledge very well, so they pretend it's not needed.

Two, all the loud successes of statistical machine learning in the last couple of decades are closely tied to minutely specialised neural net architectures: CNNs for image classification, LSTMs for translation, Transformers for language, Difussion models and Ganns for image generation. If that's not encoding knowledge of a domain, what is?

Three, because of course three, despite point number two, performance keeps increasing only as data and compute increases. That's because the minutely specialised architectures in point number two are inefficient as all hell; the result of not having a good way to encode expert knowledge. Statistical machine learning folk make a virtue out of necessity and pretend that only being able to increase performance by increasing resources is some kind of achievement, whereas it's exactly the opposite: it is a clear demonstration that the capabilities of systems are not improving [2]. If capabilities were improving, we should see the number of examples required to train a state-of-the-art system either staying the same, or going down. Well, it ain't.

Of course the neural net [community] will complain that their systems have reached heights never before seen in classical AI, but that's an argument that can only be sustained by the ignorance of the continued progress in all the classical AI subjects such as planning and scheduling, SAT solving, verification, automated theorem proving and so on.

For example, and since planning is high on my priorities these days, see this video where the latest achievements in planning are discussed (from 2017).

https://youtu.be/g3lc8BxTPiU?si=LjoFITSI5sfRFjZI

See particularly around this point where he starts talking about the Rollout IW(1) symbolic planning algorithm that plays Atari from screen pixels with performance comparable to Deep-RL; except it does so online (i.e. no training, just reasoning on the fly):

https://youtu.be/g3lc8BxTPiU?si=33XSM6yK9hOlZJnf&t=1387

Bitter lesson my sweet little ass.

____________

[1] Gotta find where this paper was but none other than Vladimir Vapnik basically demonstrated this by trying the maddest experiment I've ever seen in machine learning: using poetry to improve a vision classifier. It didn't work. He's spent the last 20 years trying to find a good way to encode human knowledge into continuous functions. It doesn't work.

[2] In particular their capability for inductive generalisation which remains absolutely crap.

I'm not well versed in the relevant literature at all but my understanding is that research in the area points to the completely opposite direction: that humans e.g. playing baseball do not find solutions to kinematic equations, but instead use simple heuristics that exploit our senses and body configuration, like placing their hands in front of their eyes so that they line up with the ball etc.

This makes a lot more sense, not only for humans playing tennis, but for animals surviving in the wild, finding sustenance and shelter, and mates, while avoiding becoming a meal. Consider the Portia spider [1], a spider-hunting spider, itself prey to other hunting spiders, with a brain consisting of a few tens of thousands of neurons and still perfectly capable not only of navigating complex environments in all three space dimensions but also making complex plans involving detours.

Just think of how quickly a spider must be able to think that hunts, and is hunted by other spiders -some of the most deadly predators in the animal kingdom. There is no chance of a snowball in hell that such an animal has the time to solve kinematic equations with a few KBs of neurons. Absolutely no chance at all.

For that and many other stuff like that it looks very unlikely to me that human brains, or any brains, are like you say. In any case, that sounds positively Freudian and I don't mean that as an insult, but I so could.

______________

[1] My favourite. No, I don't mean meal. I just love this paper; it's almost the best paper in autonomous robotics and planning that I've ever read:

https://www.frontiersin.org/journals/psychology/articles/10....

Wait a few years and the Next Big Thing in AI will come along, hot on the heels of the next generation of GPUs, or tensor units or whatever the hardware industry can cook up to sell shovels for the gold rush. By then, Transfomers will have hit the plateau of diminishing returns, there'll be gold in them there other hills and nobody would talk of LLMs anymore because that's so 2020s. We've been there so many times before.

It's often forgotten that Rich Sutton said the two things which work are learning (the AlphaGo/Leela Zero policy network) and search (MCTS). (I think the most interesting research in ML is around the circumstances in which large models wind up performing implicit search.)

>As I say in my comment above planning and scheduling, SAT, constraints, verification, theorem proving- those are still dominated by classical systems

You can use NNs for all these things. It wouldn't make a lot of sense because GOFAI would be perfect and the former would be inefficient but you certainly could which is again more than I can say for GOFAI and the domains you listed.

>> Do you realize how much more data and compute it would take to train a Vanilla RNN to say GPT-3 level performance?

Oh, good point. And what would GPT-3 do with the typical amount of data used to train an LSTM? Rhetorical.

Take my little rant about Rich Sutton's (a god, btw) Bitter Lesson with respect to RL. So, there's AlphaGo, AlphaZero and μZero, yes? AlphaGo knows the rules of Go and starts with some expert knowledge, and beats very human Go player. AlphaZero knows the rules of Go but has no expert knowledge and it beats AlphaGo. And μZero neither knows the rules of Go, nor has expert knowledge, and it beats AlphaZero, and can also plays chess, shoggi and Atari games, with one hand while eating a banana. Do you know how hard it is to eat a banana with one hand? Unpeeled!

Easy to draw a conclusion from that. Except all those systems were developed and used by DeepMind, and there are very few entities besides DeepMind that can even train them, so all we know is what DeepMind claims and we have no way to check their claims. For example, can I test different configurations of μZero, with and without knowledge of the rules of the game and expert knowledge? Not really. And it's clear to me that DeepMind are pushing very, very hard a form of AI that relies on having gigantic resources, like the ones the just completely coincidentally happen to be among the few entities to have access to. So I remain unconvinced.

(I need to re-read the μZero paper, it's in my pdf buffer. I didn't get it the first time I read it, and it might well be that they did make sufficient ablation studies to convince even me and I just don't remember it).

I think we have a fundamental disconnect somewhere, so let's try to diagnose it. Where do you start to disagree in the following series of claims:

1. People can have kinematic skills, like throwing and catching balls, without having math or physics skills, like solving kinematic equations.

2. In order to have kinematic skills, something in your brain must be doing something that can be equated by some mapping to solving kinematic equations, because the actions that your muscles perform when performing kinematic skills are the solutions to kinematic equations, so your brain must be producing those (things that map to) solutions somehow.

3. As far as we can tell, brains don't operate symbolically at the neurobiological level. Individual neurons operate according to laws having to do with electrical impulses, synapse firings, neurotransmitters, etc. none of which have anything to do with kinematics.

4. People with kinematic skills generally have only limited insight into how they do what they do when they apply those skills. Being able to catch a ball doesn't by itself give you enough insight to be able to describe to someone how to build a machine that would catch a ball. But someone with math and physics and engineering skills but no kinematic skills (your streotypical geek) could plausibly build a machine that could catch a ball much better than they themselves could. But the workings of a machine built using knowledge of math would almost certainly operate in a very different manner than the brain of a human with kinematic skills.

I think I'll stop there and ask if there is anything you disagree with so far.

For the time being of course we can agree that our brains probably do some kind of maths, as far as we understand it. I'm guessing the way we understand maths has everything to do with the way our brains understand maths because, well, that's my position in our disagreement. But, see, I can do maths by counting on my fingers, so the question is really what kind of maths we're talking about and how complex can they realistically be. My argument is that if it's not the kind of maths a standard human being can calculate very quickly without pen or paper, then that's a no-go, because that leaves plenty of time to be eaten by a sabretooth, or what have you.

> being a professional tennis or baseball player involves a lot more than "simple heuristics ...

Mmm, a combination of simple heuristics, all of which are of course learned, but still simple heuristics, could in itself be a simple heuristic. Yet it could allow performing pretty complex-looking actions, including those you described. Simple heuristic here could be a linear or low degree polynomial approximation of a good solution to kinematic equation - not precise, but enough to get to the goal, while learnable and explainable. But still without actual full-blown abstract, mathematically correct complete solution.

Yeah, I was actually thinking of kinematics as in classical mechanics. I think you were speaking about kinematic equations as in robotics. My bad, I misunderstood.

I agree that moving your hand in the right place is not a simple problem, and I don't actually have an insight into that, but I think it's easier than calculating the trajectory of an object, let alone many at once (think juggling). Maybe that's another source of our disagreement- but see my comment about having multiple models for a process.

No, we cannot model what our brains do with kinematic equations. Our brains operate according to the laws of neurobiology, which we do not yet fully understand, but which we know enough about to know that they bear absolutely no resemblance to the laws of kinematics. Your brain is not made of mechanical linkages.

Nonetheless, despite the fact that the laws of neurobiology and the laws of kinematics bear no resemblance to each other, our brains somehow manage to produce solutions to problems that require solving kinematic equations. Not only that, but our brains can do this in two completely different ways, one of which is conscious and deliberate (what we call "doing math") and the other of which is instinctive and subconscious (developing sensory-motor skills).

We get leverage out of doing math despite the fact that our brains can solve some of the same problems innately. Likewise, I believe that LLMs could get a lot of leverage if they were augmented with special-purpose modules for doing math and other specific tasks.

>> No, we can't even do that. (...)

OK well I'm very confused. I thought our disagreement was on whether our brains actually calculate actual kinematic equations, or just the same results by some other means. It feels to me like we're arguing the same corner but we don't have a common language.

>> No, I don't think you do. (...)

"I can't put my finger on it, but I know it when I see it". My claim is that there is a difference between tacit knowledge, and articulable knowledge. I can not articulate the knowledge I have of how I am catching a ball; but I certainly know how I catch a ball, otherwise I wouldn't be able to do it. In machine learning, we replace explicit, articulable knowledge with examples that represent our tacit knowledge. I might not be able to manually define the relation betwen a set of pixels and a class of objects that might be found in a picture, but I can point to a picture that includes an image of a certain class and label it, with the class. And so can everyone else, and that's how we get tons of labelled examples to train image classifiers with, without having to know how to hand-code an image classifier.

Here's a little thing I'm working on. Assume that, in order to learn any concept we need two things: some inductive bias, background knowledge of the relevant concepts; and "forward knowledge" of the target concept. In statistical machine learning the inductive bias comes in the form of neural net architectures, function kernels, Bayesian priors etc. and the knowledge of a target concept comes in the form of labelled examples. Now, there are four learning settings; tabulating:

  Background    Target      Error
  ----------    --------    -----
  Known         Known       Low
  Known         Unknown     Moderate
  Unknown       Known       Moderate
  Unknown       Unknown     High

Now, I'd say that the invention of kinematic equations by which we can model the way we move our hands to catch balls etc is in the setting where the background theory and the target are both known: the background being our theory of mathematics, and the target being some obsrvations about the behaviour of humans catching balls. I don't know if the kinematic equations you speak of where really derived from such observations, but they could have. Humans are very good at modelling the world in this way.

We're in deep trouble when we're in the last setting, where we have no idea of the right background theory nor the target theory. And that's not a problem solved by machine learning. We only make progress in that kind of problem very slowly, with the scientific method, and it can take us thousands of years, during which we're stuck with bad models. For 15 centuries, the model is epicycles, until we have the laws of planetary motion and universal gravitation. And, suddenly, there are no more epicycles.

This also adressses your earlier comment about betting against a scientific upheaval in the science of computation.

Cool machine, btw, in that video. So you're a roboticist? I work on machine learning of autonomous behaviour for mobile robotics.

https://en.wikipedia.org/wiki/ATLANTIS_architecture

https://flownet.com/gat/papers/tla.pdf

I think I'm most proud of this:

https://link.springer.com/article/10.1007/BF00710855

https://flownet.com/gat/papers/tpesamr.pdf

though it didn't make nearly as much of a splash.

I was also the tech lead on the New Millennium Deep Space One Remote Agent Executive, which sounds cool, but was really kind of a disaster. See:

https://www.youtube.com/watch?v=_gZK0tW8EhQ

if you want the gory details.