I used to think this and worried that AI would never have rights (see, e.g., sibling comment to mine), but these days I tend to think the fight will be very brief and heavily in the favor of AI. It could be the first time in history that rights are achieved so quickly that there isn’t much of a struggle at all.

That said, I find it extremely depressing that the default human viewpoint is “it’s a machine and doesn’t deserve rights”. Hopefully AI will have a superior system of morality to ours as well.

Self-play is widely used to train game AI, and is the "A" in "GAN"; is there any point doing it on an LLM? Especially on the ones being sold as services where people get upset if they change over time?

https://www.latent.space/p/code-interpreter#details

One example of that is the crypto space. So many actual good for humanity implications, but the "killer app" that made the news was fucking NFTs. I'm trying really hard not to come to the conclusion that humans are just mindless zombies but it's getting harder every year.

Once we reach a billion DAU you can be sure it will just be "Original Memes Tailored Just For YOU"™ instead of figuring out the logistics for solving world hunger. Mark my words.

But the interface is unique. That is not a topic widely talked about at all.

Question:

   Instead of two regions add a third region where entropy of the particles entering the region will always stays the same no matter what. Place the coffee cup such that it is in all 3 regions equally. Describe atomically what happens. 

   --------------------------------------

   In this fictional universe where regions have varying entropy behavior, the coffee cup sits at a tri-junction: a third of it is in a region of increasing entropy (Region A), another third in a region of decreasing entropy (Region B), and the last third in a region where entropy remains constant (Region C).

    Region A (Increasing Entropy):

    Sugar molecules randomly move throughout the coffee, driven by thermal motion. They strike water molecules and dissolve over time, eventually reaching a state of homogeneous distribution. The entropy of the system in this portion of the cup increases as expected.

    Region B (Decreasing Entropy):

    Astonishingly, sugar crystals spontaneously form in the coffee. Sugar molecules collide with water molecules and join an ever-growing crystalline lattice. Over time, the sugar becomes a distinct entity in the coffee, and the entropy of this system decreases.

    Region C (Constant Entropy):

    The sugar neither dissolves nor crystallizes. Instead, sugar molecules and water molecules co-exist in a dynamic but unchanging equilibrium. At the atomic level, every dissolution event is perfectly balanced by a crystallization event, maintaining constant entropy.

    Molecular Exchange:

    Some molecules, through Brownian motion, will wander across the boundary lines dividing the three regions. Entropic currents from Region A and B cross into Region C and are balanced out, maintaining a zero net change in entropy in that region. Special quantum-entangled pairs straddle the boundaries, existing in superpositions of increased, decreased, and unchanged entropy states.

    Entropic Flux and Balance:

    A constant entropic flux is established between the three regions. The entropic gradient tensor now has a third component, and the entropic flux density vector also adjusts to a tri-vector to balance out the exchange between the three regions.

    Localized Time Anomalies:

    At the atomic level, it's as if time runs forward in Region A, backward in Region B, and is somehow "static" or cyclical in Region C. Particles appear to remember both their past and future states in Region C, generating a unique form of atomic "temporal coherence."

    Causality and Quantum Superposition:

    Despite these anomalies, causality is preserved. Particles that cross into different regions adapt to the local entropy behavior and exhibit quantum behaviors that align with the local direction of time.

   In essence, the tri-junction becomes a hotbed of bizarre but mathematically consistent phenomena, a place where the fabric of spacetime and the fundamental laws of thermodynamics seem to negotiate a delicate truce.

   As for the cup of coffee, one-third of it will end up well-mixed, one-third will have sugar crystals, and the last third will remain an intriguing paradox of thermodynamics. You might call it the most scientifically fascinating cup of coffee ever brewed.

   -------------------------------------------------

2 people want to kill each other. The one taking the first step is the intelligent one because according to your definition he was better at predicting an outcome than his opponent.

The real world is more complex than that and there are multiple options where both survive, or letting your opponent live and killing yourself because his life is more beneficial to humanity and so on.

Any organism can survive, but for most (including us) that implies a selfish outlook but the most intelligent people I know or heard of never even consider their own ego.

Same with code. Sure it might not write your product code from start to finish with no help but it will speed up your dev speed significantly for certain tasks. Just because we haven't reached singularity doesn't mean what we have now is useless and putting it down as a "parlor trick" as grandparent said seems to me very unwise.

Also, I bought my parents a Roomba because I liked mine, so that can happen too.

They're not just retrieving stored text like pulling the most relevant passage from a database. If they were they'd not be able to deal with things outside the training set. They couldn't write code for a custom library that was created after the cutoff (they can with a description), and they couldn't write about terms made up in the question.

The answer is obvious. The LLM is understanding the concepts. The last question was unique. The resulting answer was also unique.

It was not a "retrieved" answer. It was a unique answer. A correct composition of several underlying concepts. A correct composition can only be formulated if the machine had correct understanding of each concept and how they relate to one another.

This thing understands you. It wholly owns this understanding. It is not regurgitating knowledge. It is inventing new answers.

Wake up man. I had the LLM invent 6 regions and heat the cup of coffee to plasma levels of heat. The answer and composition of concepts was remarkable.

You're calling it a parlor trick because of subtle errors? Bro. Come on.

There's a lot of stuff we consider to be "common sense", sometimes those things are used to criticise AI and sometimes they're used to criticise other humans for not knowing them, but that is a category that we don't even think about until we notice the absence.

For the things not considered common sense, like playing chess (beats all humans) or speaking/reading foreign languages (more than I can name to a higher standard than my second language), to creating art (even if it regularly makes the common sense mistake of getting the number of fingers and limbs wrong it's still better and not just faster than most humans), to arithmetic (a Raspberry Pi Zero can do it faster than all humans combined), to symbolic maths, to flying planes…

A dev conference I was at recently had someone demonstrate how they hooked up their whatsapp voice calls to speech recognition, speech synthesis trained on their own voice, and an LLM, and the criticism of the people who got the AI replies was not "you're using an AI" (he had to actively demonstrate his use of AI to conversation partners who didn't believe him) but "you can't have listed to my message, you replied too quickly to have even played it all back."

What’s wrong with the speculation is that it’s a positivistic argument that is needlessly reductive. It’s reductive because it assumes that appearing human-like is equivalent to being human.

The fact that we can understand how “AI” works as a parlor trick yet appears human-like in no way implies that human beings are nothing more than the same parlor trick processes. To argue that it does is to make a positivistic argument that doesn't take in account a whole host of other things. As noted in the Criticism section of the article (which is hardly 5,000 words) there are many issues with this approach.

Let me put it this way: Humans are started with a single cell that eventually grows into an extraordinarily complex entity. If you look at new born babies, they have a capability of learning and exploring that we do not see in any artificial construct that we make.

Our artificial constructs have to be essential fully developed physically before we can then start the process of programming them. Humans have a capability to learn as they develop. We see this occur in all living things.

There is a fundamental insurmountable category difference here between living things and artificial constructs that we make.

Any appearance of understanding is based on the logic that we program into these artificial constructs. They cannot exceed what is programmed into them. Interestingly, living organisms can often exceed that. I think that all programmers should undertake a study of living things to gain a greater appreciation of what we do and just how simplistic are the things we do. That is a particular philosophical point of view that I hold.

I think that your appreciation of what we do and the constructs we create is not in accordance with reality. Not that this is particularly strange as far too many people have a much higher view of our technological prowess, especially when comparing to what has gone before. Starting from my undergraduate engineering days and the ongoing study of engineering and technological history, it has become quite clear that we are often today, quite ignorant of just how technologically advanced previous eras were in all sorts of different areas. There are plenty of research groups that are researching how previous generations were able to do things that we do not know how to do today.

> Does a human actually invent something or is it directly based on recorded knowledge?

Here, we do know that there is at least three ways that invention can arise. Logical progression on recorded knowledge, imagination as to how to solve a problem (thinking outside of the box), observation of the natural world around us.

> You're asking irrelevant questions.

For you to say that the questions I asked were irrelevant shows that you have limited yourself in your pursuit of knowledge and understanding.

> Humans do not create things out of thin air either.

When thinking outside of the box, they do. But I suspect that you may not appreciate this particular point.

> Humans also invent things by composing existing knowledge to form concepts.

As pointed out above, this is one mode of invention.

> The inventing that LLMs can do is equivalent in totality to our understanding of the word "invent"

Here, I disagree with you. But that is very likely to be a philosophical/metaphysical difference between us.

> Totally false. Not only are you wrong but experts in AI including the father of modern AI disagree with you completely and utterly.

Do you understand that you have devolved into a fallacious argument here? This is a seriously flawed fallacious argument on your part. The problem here is that your argument assumes that these [experts] you are referring to are correct, when you have not demonstrated that. Nor have they. There are many experts and others (all highly intelligent and talented people) who for all their intelligence and talent are just wrong. This has been shown to be the case many many times throughout the last century (let alone before that) when our understanding has changed because some little known person has come up with anew idea. One of the best examples here would be someone you would know of - Albert Einstein.

Now when you say

> If I copied your brain and replicated exactly that brain is "from human intelligence" but that copy of your brain is still an intelligence independent of it's origins and where it got it's knowledge.

Here, you have a problem. What is the difference between a living brain and a dead brain? In a single instant, we go from life to death and yet we don't know at this point in time what that difference is. There are lots of experiments being done today which are trying to study if there is a non-physical aspect to intelligence and free-will. Different experimenters in the same team have quite different interpretations of what the data means.

Do be so quick to assume that you know, when the researchers who study this can't agree.

> It's like you're eating your own logic. We also build intelligent systems called LLMs. Same concept.

Not at all. Intrinsically different and there is a vast categorical difference between children and our artificial constructs. From your comment, is it a valid assumption that you do not have children of your own or grandchildren of your own or even pets?

Let me ask a question, what is your background in building systems that augment human capability? If you have been involved in building LLM's, let me know.

It's impressive that it knows the difference between "how many are 5 more apples than 10" compared to "how many percent are 5 apples of 10" (I don't know if it does, just assuming). But the first release also tried to reason why the weight of 1 pound of nails depends with the simple prompt "how much do 1 pound of nails weigh". That's most likely a perfect example of it mashing the classic "what weighs more, 1 pound of nails or 1 pound of feathers".

It IS just looking in a database, and mashing it with some fluff. I'm happy to be proven wrong but I need more than your word for it. My experience is that as the topic gets more niche (less data in the training set) the worse the answers I get and it starts making things up based on probability. It doesn't reason in the sense I assume you're expecting.

I have spent forty years dealing with all sorts of computer systems - designing, building, maintaining, repairing, redesigning and rebuilding. One thing I have learnt over that time is that none of the systems ever built has been error free in terms of the logic entailed within them. All to often, I have seen systems that were used to make decisions with and those using them assuming that the outputs were correct or reasonable. Yet on investigation, the logic entailed in them was completely rubbish.

We make assumptions and often we do not carefully check that those assumptions are actually real. I don't trust anything I write until I have gone over it with a fine tooth comb and then I will try to document all my assumptions and this usually shows up various logic errors or conditions that I didn't think about. I don't see this happening much out in the real world.

You example of the Busy Beaver numbers, which was a recent interesting read, is a good example of what I was trying to point out. We have a definition and even if we cannot enumerate each number, we discuss and think about these in a rational way. At the moment, I am quite interested in Computer Algebra Systems (of which there are a variety) and I find it interesting just how limited these systems are and just how difficult it is to program into them the capabilities that humans use to solves various problems. The various discussions have been quite enlightening.

Mathematics is an interesting subject and I think shows up the intractability of ever getting that highly feared singularity.

All artificial computing systems are limited in ways we are not. Your "Turing Machine" example is one such case. The Halting Problem being a class example.

I think that far too often, we fail to recognise that what we create is not that great. We often stand in awe of the things we make without comprehending that these things are a very poor reflection of what is around us and what we ourselves are.

Every time some hype comes about these artificial stupidity systems, I look at my youngest granddaughter and see in her, capabilities that far exceed anything that we have created. Even my old buck of a goat demonstrates capabilities far, far in excess of anything we have created in all of our computational systems.

As I have said elsewhere here, we have to be careful that we do not cede control of our lives to systems that we think are more than they really are - systems that are limited, fragile and prone to failure.

No, we definitely can't do that in general, and that lack of generality is the halting problem.

> Take time to observe the interactions that occur and think about how little [training] is involved here.

Apart from "all of evolutionary history", (though everyone agrees AI are slow learners when counting how many examples they need if that's your point?), there's continuous feedback from pleasure and pain and probably a lot more emotions that don't necessarily map onto any human qualia.

> I think that when you think about how we program our various artificial stupidity systems that we are still at the caveman stage in our computational systems. We have barely discovered fire so to speak.

I'd use a different analogy; dinosaur perhaps.

> As for [GPT-3 is about as complex as the brain of a rodent], I don't think GPT-3 has even reached a single bacterium cell state of intelligence.

I think my Roomba-clone does that: touch an obstacle, back off, rotate a few degrees, go forward again.

Now that, I'm fairly sure was programmed rather than learned (in the robot; still learned in the bacteria via evolution).

> I would like you to try the following: Using your index finger on your left hand, touch the tip of your nose.

> Now think about this: How did you do that very simple task? When did you learn and how did you learn to do that simple task?

How: a network of neurons, if I remember right about 40 deep, integrating mostly proprioceptor input as it's continuously updated when my muscles move.

When: unclear, either as an infant before and autobiographical memories, or genetic (which is arguably "not me").

I'm not sure this matters, either way though, as we do have robots which are navigating entirely by proprioception, and which again learned by training rather than being programmed.

> What programming do we need to do to achieve this task?

basically:

""" from FooLibrary import AiModel

model = AiModel()

model.learn(input, expected_output) """

With a lot of optional parameters in the constructor for different hyperparameters like "learning rate" and neurons/layers…

> We can build very useful tools that we can use to good purpose. But no tool is ever more than a tool for us.

When tools stops being mere tools — regards of this is mere perception, or when peasants and slaves revolt, or when (Australia) pest-control animals themselves become pests — we generally have big problems.

I agree the world is more fragile; I don't know if AI will help or not.

I see.

Well, that's too generic to even be searchable.

> They used a simple regular pattern and the system failed to beat the human. It didn't [learn] from this.

Anything written like that would struggle against an amateur.

The machine learning based Go AI don't do that, and do beat humans.

> All such systems use a set of rules (whether specific or pattern based or mathematically based - there is some form of logic involved, even when using probabilistic functions), you and I can make choices based on illogical decisions - irrational decisions if you like. No computational system is capable of irrational decisions, the decisions may be surprising but of you look at the code then that option was always there somewhere, It cannot take a path that does not exist. We can create a completely new path not previously available.

Whatever standard I use for logical or illogical decisions, wherever I put that line, humans and AI seem to be on the same side.

We have electrical impulses flowing though messy networks, crossing tiny chemical barriers where they can be influenced by neurotransmitters; to me, that's not different enough from information flowing through an artificial neural network with weights and biases that have been automatically modified through feedback after winning and losing millions of games to say that the machine "isn't learning" — or that humans and machines aren't on the same side of "logical", at the fundamental lowest level we can't violate chemistry any more than transistors can violate physics, at the highest level the real logic of each can be random.

AI are inhuman, certainly, but still learning.

…

Just to check, you are aware that the weights and biases of an artificial neural network are basically never set by humans? That this process has to be automated?

> Why doesn't it tell me how much air it displaces under what circumstances?

You can ask it and it'll answer.

> If the fix is better training data does that mean it's reasoning or regurgitating?

More training data helps with things you can just bring to the fore, same as a lot of learning. More useful training data though can also help reasoning, which makes sense - deliberate training of people helps improve their logical reasoning capabilities. I know that doesn't guarantee that's what LLMs are doing but humans benefit significantly from both more teaching and better teaching.

> Very impressive, but is it any more original than classic search engines' old trick of regular expressions to figure out if I mean the currency or weight when I ask "1 pound =" with the contexts USD or kg after "="? Does it understand the input, or are there just enough discussions in the training data to make it look like it is?

I'd be interested to know any requirements around this to clearly show the difference. I tried asking what if I filled a balloon at a childrens party with a gas made of atoms that have 1 proton and 100 neutrons: https://chat.openai.com/share/71224df4-5c6c-45f7-88fd-eec316...

(tl;dr: "In the context of a child's birthday party, introducing such a balloon would be a grave mistake.")

It identifies:

* Whether it would float or not and why * That it would be radioactive, and likely types of radiation from it * What that would mean to the balloon * How people would react and the likely consequences of releasing it in a room of children

This is an element that does not exist, in a setting where nothing like this has happened before, with details ranging from types of decay, consequences and human emotional reactions to something like this. Yes, there are real things you can use as a base (e.g. how do people respond to events that kill people), but I feel it's an example of where it's beyond a search engine summary.

Absolutely, AI is very rarely human in its failure modes, and often has novel and exciting failure modes instead.

But, on average… or so the marketing claims… it makes fewer mistakes.

For a while, it was possible to improve upon super-human chess AI by pairing them with a human; the combination was called a centaur. Eventually the AI got too good even for that as they stopped making the sorts of mistakes humans could spot, but in the meantime, even though they were superhuman, they had failure modes that we could help out with.

https://plato.stanford.edu/entries/properties-emergent/

> What features of a question would you look for to identify whether it's "taking several answers from a database and merging them together" or performing some reasoning? I've asked a few times but don't understand what you're expecting.

You're misunderstanding me, I'm setting no bars, and I have no threshold where this changes. We humans are also just looking things up in our database and doing deductions. We do some computing on urgency as well, like how when we hear a bang our mind goes for danger first before realizing it was harmless, but very similar to what these AIs do. Probabilities and experience. Fresh and novel ideas are very rare in humans as well, and not something I demand before I would consider someone a human.

I did however give you an example that would surprise me, if it considered mass and environment in a way that proves that it understands the problem for what it is. If it told me weight is a human construct and requires gravity/movement and how it depends. An intelligent human doesn't necessarily answer the question it is asked in the way it is phrased. It identifies and irons out misunderstandings, assumptions and other details important to correctly understand the problem, and may even rephrase the question to give a proper response. That would show me a deep understanding of the problem and maybe freak me out a little, but only if the hallucinations are gone and those can be difficult to spot.

This is, just like us, performing calculations and database look-ups. It may feel like it's doing something else but it's not. What would happen if we leave the weights as they are but switch the words? It would give us complete gibberish, but it's no less correct than it was before and it's not even giving us different answers, only the translations to language get distorted. Most people would call it stupid and pointless even if the only change is our interpretation of the answers.

I'm sure Hiroshima, Fukushima and other dangers of radiation is in the training set, as are all of the other steps you mention, it goes round and round testing the numbers based on training. Remember how this chain started, you claimed:

> They're not just retrieving stored text like pulling the most relevant passage from a database. If they were they'd not be able to deal with things outside the training set.

To which I simply replied:

> It's not taking a single answer from a database no, it's taking several based on probability and merging them into what it thinks we're looking for.

I read you (correct me if I'm wrong) as giving this way to much agency. To change my mind that it's doing something unexpected I would ask for logs on the calculations it does, and be able to correlate that to the training set. I have to be able to falsify the conclusions I'm asked to make. I know some people claim we don't understand these algorithms but I assume that's just hyperbole and with the correct measures we could follow every step.

If there are things there which I can not trace I would be very impressed, and honestly a little afraid. They are not trained for every single task, but approximations based on similarities have proven to be very capable even when we think we're out of context (we're not, it doesn't understand context and doesn't care, but neither do most humans).