Will scaling work?(dwarkeshpatel.com) |
Will scaling work?(dwarkeshpatel.com) |
I'd like to be wrong but I can't help but feel that people predicting a revolution are making the same, understandable mistake as my hypothetical 1950s person.
Productivity at information communication tasks just isn’t the entire economy.
I think we are massively more productive. Some of the biggest new companies are ad companies (Google, Facebook), or spend a ton of their time designing devices that can’t be modified by their users (Apple, Microsoft). Even old fashioned companies like tractor and train companies have time to waste on preventing users from performing maintenance. And then the economy has leftover effort to jailbreak all this stuff.
We’re very productive, we’ve just found room for unlimited zero or negative sum behavior.
For sure - I grew up in the mid-late 70s having to walk to the library to research stuff for homework, parents having to use the yellow-pages to find things, etc.
Maybe smartphones are more of a game changer than desk-bound internet though - a global communication device in your pocket that'll give you driving directions, etc, etc.
BUT ... does the world really FEEL that different now, than pre-internet? Only sort-of - more convenient, more connected, but not massively different in the ways that I imagine other inventions such as industrialization, electricity, cars may have done. The invention of the telephone and radio maybe would have felt a bit like the internet - a convenience that made you feel more connected, and maybe more startling being the first such capability?
The administration of the Tax Service uses 4% of the total tax revenue it generates. This percentage has stayed relatively fixed over time.
If IT really improved productivity, wouldn't you expect that that number would decrease, since Tax Administration is presumably an area that we should expect to see great gains from computerisation?
We should be able to do the same amount of work more efficiently with IT, thus decreasing the percentage. If instead the efficiency frees up time allowing more work to be done (because there are people dodging taxes and we need to discover that), then you should expect the amount of tax to increase relatively which should also cause the percentage to decrease.
Therefore IT has not increased productivity.
Either it doesn't do so directly, or it does do so directly, but all the efficiency gains are immediately consumed by more useless beurocracy.
https://archive.ph/baneA https://archive.ph/TrHYN
“Our central theme is that computers and the Internet do not measure up to the Great Inventions of the late nineteenth and early twentieth century, and in this do not merit the label of Industrial Revolution,”
— Robert Gordon, actual economist
On their current trajectory LLMs are just expert systems that will let certain types of simple job be automated. A potential productivity amplifier similar to having a personal assistant that you can assign tasks too. Handy (more so for people doing desk-bound jobs than others), but not a game changer.
An AGI far beyond human capability could certainly accelerate scientific advance and let us understand the world (e.g. how to combat climate change, how to address international conflicts, how to handle pandemics) so be very beneficial, but what that would feel like to us is hard to guess. We get used to slowly introduced (or even not so slowly) changes very quickly and just accept them, even though today's tech would look like science fiction 100 years ago.
What would certainly be a game changer, and presumably will eventually come (maybe only in hundreds of years?) would be if humans eventually relinquish control of government, industry, etc to AGIs. Maybe our egos will cause us to keep pretending we're in control - we're the ones asking the oracle, we could pull the plug anytime (we'll tell ourselves) etc, but it'll be a different world if all the decisions are nonetheless coming from something WAY more intelligent than ourselves.
Human civilization has accumulated many layers of systems since then and the internet changed all of them to the point that many are barely recognizable. Just ask someone who's been in prison since before the internet was a thing - there are plenty of them! They have extreme difficulty adapting to the outside world after they've been gone for forty or fifty years.
I see "AI safety" brought up as a laughable attempt at stopping the progress of LLMs, when in reality the people talking about "AI safety" are the people trying to say that the majority will not benefit from this technology.
When agents start being more reliable I think we will start seeing applications we couldn’t possibly anticipate today
I'd call the current generation "Social media natives", because that is the biggest difference from the previous generation. 90s kids grew up with games and communication, but they were free from facebook, youtube and instagram.
seems odd. What 'patterns' and 'limitations' do you still see? Because I see so much has changed.
My mom, who is 70 years old, regularly tells me how profoundly transformative the internet has been for society.
Food distribution has improved as have most logistics in general. These efficiencies have somewhat been shared with the general public but in a lot of cases, those gains were captured by private enterprise.
Healthcare has improved a little bit, iterative progress can be made more quickly, shared, and moved into translational medicine as practice. Drug discovery has improved quite a bit, as have logistics around getting said drugs in the hands of people who need them and doing so affordably. This improved lives and longevity.
Socially we can communicate far easier. It remains to be seen to me if thise is always an improvement. Humans seem to be designed for much smaller social circles and don't seem to be capable of taking much advantage in their daily lives of increases frequency, scale, and reach of socialization.
The list goes on. It's not exactly linearly correlated with technology growth because ultimately it boils down to actionable information. Just because we have more information or more processing capability around information doesn't mean we get direct returns from that or that we don't reach limits where we simply don't have use for the additional gains. Information has to be actionable in some way, otherwise it's just intermediate data products that may or may not benefit us. I know can ready daily news from some small town in Southern Japan if I wanted to. That doesn't improve my life mostly, but it's there.
We have piles and piles of scientific literature we could share and iterate on towards new discoveries for humanity. That doesn't mean in my daily need for survival and balance with recreation I have time to contribute to things I find interesting or necessary, after all I am to some degree a slave of my needs within the economic system I'm entrenched in. I have bills, I have to earn money, and I have to work.
Even if that wasn't the case maybe or maybe not would I be able to contribute more back to society than I do now at my paid profession. Currently I'd say I do pretty well in this department in terms of reach. Without that I might struggle.
by what criteria do you see the world as the same today vs 70 years ago?
Things "behind the scenes" have perhaps changed a lot -- e.g. financialization, more competitive markets, explosion of communication options, which are the driving force behind those visible changes.
- people eat plants and animals
- people pay money for goods and services
- there are countries, sometimes they fight, sometimes they work together
- men and women come together to create children, and often raise those children together
etc, etc, etc
The “bones” of what make up a capital-S Society are pretty much the same. None of these things had to stay the same, but they have so far.
With AI, the way I see it, it is just virtual other people. Of course, a bit stranger but more simillar than you think.
I've done many language exchanges with people using Google Translate and the lack of improvement/memory of past conversations is a real motivation killer; I'm concerned this will move on to general discourse on the internet with the proliferation of LLMs.
I'm sure many people have already gone around in circles with rules-based customer support. AI can make this worse.
We had designed many tools that beat us in various aspects. This is an invalid analogy.
If it’s closer to a midpoint between GPT-4 and true human intelligence, then sure, I agree with you, it’s a significant change to society but not an overhaul. But if it’s actually a human level (or better) general intelligence, it’ll be the biggest change to human society maybe ever.
I bet their reaction would be a facepalm.
Nothing I’ve seen or learned about LLMs leads me to believe that LLMs are in fact a pathway to AGI.
LLMs trained on more data with more efficient algorithms will make for more interesting tools built with LLMs, but I don’t see this technology as a foundation for AGI.
LLMs don’t “reason” in any sense of the word that I understand and I think the ability to reason is table stakes for AGI.
Never thought about it in this sense. Is he wrong?
Obviously scaling works in general, just ask anyone in HPC, haha.
I think Wittgenstein's ideas are pertinent to the discussion of the relation of language to intelligence (or reasoning in general). I don't meant this in a technical sense (I recall Chomsky mentioning that almost no ideas from Wittgenstein actually have a place in modern linguistics) but from a metaphysical sense (Chomsky also noted that Wittgenstein was one of his formative influences).
The video I linked is a worthy introduction and not too long so I recommend it to anyone interested in how language might be the key to intelligence.
My personal take, when I see skeptics of LLMs approaching AGI, is that they implicitly reject a Wittgenstein view of metaphysics without actually engaging with it. There is an implicit Cartesian aspect to their world view, where there is either some mental aspect not yet captured by machines (a primitive soul) or some physical process missing (some kind of non-language system).
Whenever I read skeptical arguments against LLMs they are not credibly evidence based, nor are they credibly theoretical. They almost always come down to the assumption that language alone isn't sufficient. Wittgenstein was arguing long before LLMs were even a possibility that language wasn't just sufficient, it was inextricably linked to reason.
What excites me about scaling LLMs, is we may actually build evidence that supports (or refutes) his metaphysical ideas.
1. https://www.youtube.com/watch?v=v_hQpvQYhOI&ab_channel=Philo...
I also don't understand the claims that it doesn't generalize. I currently use it to solve problems that I can absolutely guarantee were not in its training set, and it generalizes well enough. I also think that one of the easiest ways to get it to generalize better would simply be through giving it synthetic data which demonstrates the process of generalizing.
It also seems foolish to extrapolate on what we have under the assumption that there won't be key insights/changes in architecture as we get to the limitations of synthetic data wins/multi-modal wins.
Maintaining the illusion is important to keep the money flowing in.
LLMs can work as data extraction already, so one can build some prolog DB and update it as it consumes data. Then translate any logic problems into prolog queries. I want to see this in practice.
Similar with usage of logic engines and computation/programs.
I also think that RL can come up with better training function for LLMs. In the programming domain for example one could ask LLM to think about all possible test for given code and evaluate them automatically.
I was also thinking about using diffusER pattern where programming rules are kinda hardcoded (similar to add/replace/delete but instead algebra on functions/variables). Thats probably not AGI path but could be good for producing programs.
1 - Babies learn much more with much less 2 - Video training data can be made in theory at incredible rates
The questions becomes: why is the author focusing on approaches in AI investigated in like 2012? Does the author think SOTA is text only? Are OpenAI or other market leaders only focusing on text? Probably not.
If babies learn much more from much less, isn't that evidence that the LLM approach isn't as efficient as whatever approach humans implement biologically, so it's likely LLM processes won't "scale to ago"?
For video data, that's not how LLMs work(or any NNs for that matter). You have to train them on what you want them to look at, so if you want them to predict the next token of text given an input array, you need to train it on the input arrays and output tokens.
You can extract the data in the form you need from the video content, but presumably that's already been done for the most part, since video transcripts are likely included in the training data for gpt.
a lot of people have tried to replicate this, I have tried. It's very hard to get GPT-4 to draw a unicorn, also asking it to draw an upside down unicorn is even harder.
I think Google, Microsoft and facebook could easily have 5 OOM data than the entire public web combined if we just count text. Majority of people don't have any content on public web except for personal photos. A minority has few public social media posts and it is rare for people to write blog or research paper etc. And almost everyone has some content written in mail or docs or messaging.
Quality of data (which I believe is at least part of why synthetic data is being used) can perhaps make more of a difference and perhaps at least partly compensate in a crude way for these models lack of outlier rejection and any generalization prediction-feedback loop. Just feed them consistent correct data in the first place.
I’m worried that when people hear ‘5 OOMs off’, how they register it is, “Oh we have 5x less data than we need - we just need a couple of 2x improvements in data efficiency, and we’re golden”. After all, what’s a couple OOMs between friends?
No, 5 OOMs off means we have 100,000x less data than we need.
But every article/post of this kind immediately begs the question; What is AGI? I have yet to hear even a decent standard.
It always seems like I'm reading Greek philosophers struggling with things they have almost no understanding of and just throwing out the wildest theories.
Honestly, it raised my opinion of them seeing how hard it is to reason about things which we have no grasp of.
Assuming they have effectively a log of the internet, rather than counting the current state of the internet as usable data we should be thinking about the list of diffs that make up the internet.
Maybe this ends up like Millenium Management where a key differentiator is having access to deleted datasets.
"Faith and Fate: Limits of Transformers on Compositionality"
https://arxiv.org/abs/2305.18654
"Comparing Humans, GPT-4, and GPT-4V On Abstraction and Reasoning Tasks":
https://arxiv.org/abs/2311.09247
"Embers of Autoregression: Understanding Large Language Models Through the Problem They are Trained to Solve"
https://arxiv.org/abs/2309.13638
"Pretraining Data Mixtures Enable Narrow Model Selection Capabilities in Transformer Models"
You can feed your dog 100,000 times a day, but that won't make it a 1,000kg dog. The whole idea that AGI can be achieved by predicting the next word is just pure marketing nonsense at best.
Is there any existing cases of LLMs coming up with novel, useful and namely better architectures? Either related to AI/ML itself or any other field.
The entire subject of the article is concerned with what it will take and how likely it is than an AI will ever will able to generate improvements like this.
In my view, domain modeling, managing state, knowing when to transition between states, techniques for final decision making, consideration for the time domain, and prompt engineering are the real challenges.
It's a fact of neural networks that to train them supervised you need the training data in the expected input for(vector of n thousand preceding tokens for LLMs) with the expected output(the next token for LLMs). "Training them on video" would mean converting the video to a format we can train the llm with, then training the LLM with that info.
This would probably be a 1 OOM increase at maximum, if the video transcripts aren't already a part of the training data for gpt.
My estimation is about 200 years in future to have a "human-brain AI" that works.
All idea should be treated equally, not based on revenue metrics. If everyone could make a Youtube clone, the revenue should be divided equally to all of creator, that's the way the world should move forward, instead of monopoly.
Everything will be suck, forever.
There must be a reason we can do so much while consuming so little, and then again struggling with other tasks.
What is the success if we build a machine that consumes just heaps of energy and then is as bad in maths as us?
- to say that because we're "more optimised" we must be the most optimised. Our brains are optimised well for certain things, sure, but computers are far more efficient at e.g. crunching numbers than we are
- to say that there's no success in a machine that can't currently beat us at math - this year has already proven that false
Hint for everyone else here:
It's about scaling LLMs.
I think a clearer answer is that scaling alone will certainly NOT get us to AGI. There are some things that are just architecturally missing from current LLMs, and no amount of scaling or data cleaning or emergence will make them magically appear.
Some obvious architectural features from top of my list would include:
1) Some sort of planning ahead (cf tree of thought rollouts) which could be implemented in a variety of ways. A simple single-pass feed forward architecture, even a sophisticated one like a transformer, isn't enough. In humans this might be accomplished by some combination of short term memory and the thalamo-cortical feedback loop - iterating on one's perception/reaction to something before "drawing conclusions" (i.e. making predictions) based on it.
2) Online/continual learning so that the model/AGI can learn from it's prediction mistakes via feedback from their consequences, even if that is initially limited to conversational feedback in a ChatGPT setting. To get closer to human-level AGI the model would really need some type of embodiment (either robotic or in a physical simulation virtual word) so that it's actions and feedback go beyond a world of words and let it learn via experimentation how the real world works and responds. You really don't understand the world unless you can touch/poke/feel it, see it, hear it, smell it etc. Reading about it in a book/training set isn't the same.
I think any AGI would also benefit from a real short term memory that can be updated and referred to continuously, although "recalculating" it on each token in a long context window does kind of work. In an LLM-based AGI this could just be an internal context, separate from the input context, but otherwise updated and addressed in the same way via attention.
It depends too on what one means by AGI - is this implicitly human-like (not just human-level) AGI ? If so then it seems there are a host of other missing features too. Can we really call something AGI if it's missing animal capabilities such as emotion and empathy (roughly = predicting other's emotions, based on having learnt how we would feel in similar circumstances)? You can have some type of intelligence without emotion, but that intelligence won't extend to fully understanding humans and animals, and therefore being able to interact with them in a way we'd consider intelligent and natural.
Really we're still a long way from this type of human-like intelligence. What we've got via pre-trained LLMs is more like IBM Watson on steroids - an expert system that would do well on Jeopardy and increasingly well on IQ or SAT tests, and can fool people into thinking it's smarter and more human-like than it really is, just as much simpler systems like Eliza could. The Turing test of "can it fool a human" (in a limited Q&A setting) really doesn't indicate any deeper capability than exactly that ability. It's no indication of intelligence.
I'm not sure you realize this, but that is literally what this article was written to explore!
I feel like you just autocompleted what you believe about large language models in this thread, rather than engaging with the article. Engagement might look like "I hold the skeptic position because of X, Y, and Z, but I see that the other position has some really good, hard-to-answer points."
Instead, we just got the first thing that came to your mind talking about AI.
In fact, am I talking to a person?
Yeah but it's "exploration" answers all the reasonable objections by just extrapolating vague "smartness" (EDITED [1]). "LLMs seem smart, more data will make 'em smarter..."
If apparent intelligence were the only measure of where things are going, we could be certain GPT-5 or whatever would reach AGI. But I don't many people think that's the case.
The various critics of LLMs like Gary Marcus make the point that while LLMs increase in ability each iteration, they continue to be weak in particular areas.
My favorite measure is "query intelligence" versus "task accomplishment intelligence". Current "AI" (deep learning/transformers/etc) systems are great at query intelligence but don't seem to scale in their "task accomplishment intelligence" at the same rate. (Notice "baby AGI", ChatGPT+self-talk, fail to produce actual task intelligence).
[1] Edited, original "seemed remarkably unenlightening. Lots of generalities, on-the-one-hand-on-the-other descriptions". Actually, reading more closely the article does raise good objections - but still doesn't answer them well imo.
The point I was trying to make is that I think better LLMs won’t lead to AGI. The article focused on the mechanics and technology, but I feel that’s missing the point.
The point being, AGI is not going to be a direct outcome of LLM development, regardless of the efficiency or volume of data.
When we get to the point where LLMs are able to invoke these tools for a user, even if that user has no knowledge of them, and are able to translate the results of that reasoning back into the user's context... That'll start to smell like AGI.
The other piece, I think, is going to be improved cataloging of human reasoning. If you can ask a question and get the answer that a specialist who died fifty years ago would've given you because that specialist was a heavy AI user and so their specialty was available for query... That'll also start to smell like AGI.
The foundations have been there for 30 years, LLMs are the paint job, the door handles, and the windows.
I think OP meant other definition of reason, because by your definition calculator can also reason. These are tools created by humans, that help them to reason about stuff by offloading calculations for some of the tasks. They do not reason on their own and they can't extrapolate. They are expert systems.
> But of course we don’t actually care directly about performance on next-token prediction. The models already have humans beat on this loss function. We want to find out whether these scaling curves on next-token prediction actually correspond to true progress towards generality.
And:
> Why is it impressive that a model trained on internet text full of random facts happens to have a lot of random facts memorized? And why does that in any way indicate intelligence or creativity?
And:
> So it’s not even worth asking yet whether scaling will continue to work - we don’t even seem to have evidence that scaling has worked so far.
I don’t feel like we’re anywhere close given that we can’t even yet meaningfully define reasoning or consciousness… or as another commenter put it, what is it that differentiates us so significantly from other animals.
> I still think there are missing things with the current systems. […] I regard it a bit like the Industrial Revolution where there was all these amazing new ideas about energy and power and so on, but it was fueled by the fact that there were dead dinosaurs, and coal and oil just lying in the ground. Imagine how much harder the Industrial Revolution would have been without that. We would have had to jump to nuclear or solar somehow in one go. [In AI research,] the equivalent of that oil is just the Internet, this massive human-curated artefact. […] And of course, we can draw on that. And there's just a lot more information there, I think, it turns out than any of us can comprehend, really. […] [T]here's still things I think that are missing. I think we're not good at planning. We need to fix factuality. I also think there's room for memory and episodic memory.
[0]: https://cbmm.mit.edu/video/cbmm10-panel-research-intelligenc...
Societies pre-IR had multiple periods where energy usage increased significantly, some of them based specifically around coal. No IR.
Early IR was largely based around the usage of water power, not coal. IR was pure innovation, people being able to imagine and create the impossible, it was going straight to nuclear already.
Ironically, someone who is an innovator believes the very anti-innovation narrative of the IR (very roughly, this is the anti-Eurocentric stuff that began appearing in the 2000s...the world has moved on since then as these theories are obviously wrong). Nothing tells you more about how busted modern universities are than this fact.
We're using Transformer architecture right now. There's no reason there won't be further discoveries in AI that are as impactful as "Attention is All You Need".
We may be due for another "AI Winter" where we don't see dramatic improvement across the board. We may not. Regardless, LLMs using the Transformer architecture may not have human level intelligence, but they _are_ useful, and they'll continue to be useful. In the 90s, even during the AI winter, we were able to use Bayesian classification for such common tasks as email filtering. There's no reason we can't continue to use Transformer architecture LLMs for common purposes too. Content production alone makes it worth while.
We don't _need_ AGI, it just seems like the direction we are heading as a species. If we don't get there, it's fine. No need to throw the baby out with the bath water.
LLM may be a necessary step to get to AGI, but it (probably) won't be the one that achieves that goal.
Tardigrades might :)
We haven’t had ML models this large before. There’s innovation in architecture but we often come back to the bitter lesson: more data.
We’re likely going to see experimentation with language models to learn from few examples. Fine tuning pretrained LLMs shows they have quite a remarkable ability to learn from few examples.
Liquid AI has a new learning architecture for dynamic learning and much smaller models.
Some people seem mad about the bitter lesson, they want their model based on human features to work better when so far usually more data wins.
I think the next evolution here is in increasing the quality of the training data and giving it more structure. I suspect the right setup can seed emergent capabilities.
That will get us to what was previously known as AGI. The definition of AGI will change, but we will have systems that put perform humans in most ways.
"Are Emergent Abilities of Large Language Models a Mirage?"
https://arxiv.org/abs/2304.15004 https://blog.neurips.cc/2023/12/11/announcing-the-neurips-20...
What are you basing this claim on? There is no intelligence in an LLM, only humans fooled by randomness.
So if that continues then he is wrong unless he is defining LLMs in a strict way that does not include new improvement in the future
Humans are able to begin to generalize with a single persons experiences over less than a year, so the fact that LLMs cannot with billions of person-years of information could be an indicator of their inability to generalize no matter how much training data you throw at it.
Humans can learn using every ML learning paradigm in ever modality: unsupervised, self-supervised, semi-supervised, supervised, active, reinforcement based, and anything else I might be missing. Current LLMs are stuck with "self-supervised" with the occasional reinforced (RLHF) or supervised (DPO) cherry on top at the end. non multi-modal LLMs operate with one modality. We are hardly scratching the surface on what's possible with multi-modal LLMs today. We are hardly scratching the surface for training data for these models.
The overwhelming majority of todays LLMs are vastly undertrained and exhibit behavior of undertrained systems.
The claim from the OP about scale not giving us further emergent properties flies in the face of all of what we know about this field. Expect further significant gains despite nay-sayers claiming it's impossible.
The challenge is that we both do not understand which set of data is most beneficial for training, or how it could be efficiently ordered without triggering computationally infeasible problems. However we do know how to massively scale up training.
Edit: to expand, if the goal is AGI then yes we need all the help we can get. But even so, AGI is in a totally different league compared to human intelligence, they might as well be a different species.
LLMs provide some really nice text generation, summarization, and outstanding semantic search. It’s drop dead easy to make a natural language interface to anything now.
That’s a big deal. That’s what’s going to give this tech it’s longevity, imo.
Without the ability to interact with the physical world, LLMs will never be able to reach AGI. It can kind of simulate it to some extent, but it'll never get there
LLMs can't even form their own memories, the context has to be explicitly fed back to them.
That is one of the things that stood out to me in Searle's summary of his later work because I consider how the transformer architecture works and the way in which the surrounding context plays into the meaning of the words.
> Part of learning those language games is experimenting in the real world
It is interesting that the article we are responding to talks about how we have only just begun to experiment with RL on top of transformers. In the same way that Alpha Go engaged in adversarial play we can envision LLMs being augmented to play language games amongst themselves. That may result in their own language, distinct from human language. But it also may result in the formation of intelligence surpassing human intelligence.
> The fidelity of text is simply too low to communicate the amount of information humans use to build up general intelligence.
This does not at all follow from anything I've encountered in Wittgenstein. It is an empirical claim that we (as in humanity) are going to test and not something that I would argue either one of us can know simply reasoning from first principles.
What does follow for me is closer to what Steven Pinker has been proposing in his own critiques of LLMs and AGI, which is that there is no necessary correlation between goal seeking (or morality) and intelligence. I also feel this is concordant with Wittgenstein's own work.
> Without the ability to interact with the physical world, LLMs will never be able to reach AGI
Again, a confident claim that is based on nothing other than your own belief. As I stated in my last comment, I am excited to see if that is empirically true or false. We are definitely going to scale up LLMs in the coming decade and so we are likely to find out.
My suspicion is that people don't want this scaling up to work because it would force them to let go of metaphysical commitments they have on both the nature of intelligence as well as the nature of reality. And for this reason they are adamantly disbelieving in even the possibility before the evidence has been gathered.
I'm happy to stay agnostic until the evidence is in. Thankfully, it shouldn't take too long so I may be lucky enough to find out in my own lifetime.
lmao. I'd like to see you elaborate on what you think this means! The fact that you quote Searle, though, tells me all I need to know.
In his early work he appeared to believe that the way to do this categorization was to determine if the logical propositions corresponded in their structure to the real world (his picture theory of language). In his later work he appears to have believed the way to do this was to reference some "language game" among a set of communicating agents and then understand the words based on their context within that construct (use theory of language).
Both of these concepts do not reference any kind of subjective experience, aesthetics, morality, etc. They are simply a description of how to judge whether or not a proposition is "meaningful". In later Wittgenstein, meaning in language is entirely divorced from any necessity that the statements correspond to an independent objective reality. It is simply necessary that they are consistent amongst the participants in a particular language game.
I don't believe that means the entirety of our "consciousness" is related solely to our ability to categorize logical propositions. However, it may suggest that intelligence specifically is related to this ability.
In as much as we can say that an LLM is capable of participating in some particular human language game and can successfully categorize logical propositions within that language game - I would say that LLM is demonstrating "intelligence" within that language game. And if we can create LLMs that can participate in arbitrary (or general) language games across a wide variety of domains, we might call that LLM generally intelligent. I believe that current LLMs have achieved the first (demonstrating some intelligence in particular domains) but we have yet to achieve the second (demonstrating consistent intelligence in a wide range of general domains).
As for metaphysics, I would argue that Wittgenstein saw this general ability (to categorize logical propositions) as a subset of all possible experience. I believe he saw this categorization activity as the primary aim of philosophy. However, the kinds of experience that were outside of this categorization activity could not be spoken about at all.
You might want to reconsider your stance on emergent abilities in LLMs considering the NeurIPS 2023 best paper winner is titled:
"Are Emergent Abilities of Large Language Models a Mirage?"
https://arxiv.org/abs/2304.15004 https://blog.neurips.cc/2023/12/11/announcing-the-neurips-20...
Also, consider that some work gets a lot of positivity not for the work itself, but for the people who wrote it. Timnit Gebaru's work was effectively ignored until she got famous for her spat with jeff dean at google. Her citations have exploded as a result, and I don't think that most in the field think that the "stochastic parrot" paper was especially good, and certainly not her other papers which include significant amounts of work dedicated to claiming that LLM training is really bad for the environment (despite a single jet taking AI researchers to conferences being worse for the environment than LLM training circa that paper being written was taking). Doesn't matter that the paper was wrong, it's now highly cited because you get brownie points for citing her work in grievance studies influenced subfields of AI.
To me the most interesting aspect of LLMs is the way that they reveal cognitive 0-days in humans.
The human race needs patches to cognitive firmware to deal with predictive text... Which is a fascinating revelation to me. Sure it's backed up by psych analysis for decades but it's interesting to watch it play out on such a large scale.
I don't think what LLMs are currently doing is really generalizing, but rather:
1) Multiple occurrences of something in the dataset are mutually statistically reinforcing. This isn't generalization (abstraction) but rather reinforcement through repetition.
2) Multiple different statistical patterns are being recalled/combined in novel ways such that it seems able to "correctly" respond to things out of dataset, but really this only due to these novel combinations, not due to it having abstracted it's knowledge and applying a more general (or analogical) rule than present in it's individual training points.
But if it were so settled and obvious there would be a clear line of reasoning to make that plain. And there is not. Instead, there is a very vibrant debate on the topic with tons of nuance and good faith (and bad) on each side, if we want to talk about sides.
And, of course, one of the implications of this very real and significant inquiry that needs to be made and that requires real contributions from informed individuals, is that whenever anyone is dismissive or reductive regarding the unresolved difficulties, you can be sure they have absolutely no clue what they are talking about.
In part, whether conscious or not, people see the bright future of LLMs as a kind of redemption for the world so far wrought from a Silicon Valley ideology; its almost too on-the-nose the way chatgpt "fixes" internet search.
But on a deeper level, consider how many hn posts we saw before chatgpt that were some variation of "I have reached a pinnacle of career accomplishment in the tech world, but I can't find meaning or value in my life." We don't seem to see those posts quite as much with all this AI stuff in the air. People seem to find some kind of existential value in the LLMs, one with an urgency that does not permit skepticism or critique.
And, of course, in this thread alone, there is the constant refrain: "well, perhaps we are large language models ourselves after all..." This reflex to crude Skinnerism says a lot too: there are some that, I think, seek to be able to conquer even themselves; to reduce their inner life to python code and data, because it is something they can know and understand and thus have some kind of (sense) of control or insight about it.
I don't want to be harsh saying this, people need something to believe in. I just think we can't discount how personal all this appears to be for a lot of regular, non-AI-CEO people. It is just extremely interesting, this culture and ideology being built around this. To me it rivals the LLMs themselves as a kind fascinating subject of inquiry.
What is experience? We are in state S, and take action A, and observe feedback R. The environment is the teacher, giving us reward signals. We can only increase our knowledge incrementally, by trying our many bad ideas, and sometimes paying with our lives. But we still leave morsels of newly acquired experience for future generations.
We are experience machines, both individually and socially. And intelligence is the distilled experience of the past, encoded in concepts, methods and knowledge. Intelligence is a collective process. None of us could reach our current level without language and society.
Human language is in a way smarter than humans.
What I say is not just regurgitation of my past experiences; there is a logic to it.
The author also cited a few human assisted efforts as ML only.
The fact that the author also is surprised that GPT is better at falsifying user input while it struggles at new ideas demonstrates the fact that those who are hyping LLMs as getting us closer to strong AI don't know or ate ignoring the know limitations of problems like automated theorem solving.
I think generative AI is powerful and useful. But the AGI is near camp is starting to make it a hard sell because the general public is discovering the limits and people are trying to force it into inappropriate domains.
Over parameterization and double decent is great at expanding what it can do, but I haven't seen anything that justifies the AGI hype yet.
But - it's only a pre-print and I don't think that they have taken it forward to a publication so handle with care. Anyway, the relevant evidence and thinking that I would highlight and somewhat agrees with my findings is seen in figure 7 and section 5.6. I struggle to get the quality of results that they saw, but its believed by some people that ongoing development of GPT-4 and throttling of the reasoning for cost reasons may have limited some of its capabilities by the end of this year so that may be some of my problem.
I've did this and got a result. Asked for a cat wearing a hat. It drew a circle with dots as eyes and sorta-whiskers with lines and a triangle for the hat. All using just SVG vector code.
The claim is result shows that there is understanding of not just words about cats and hats and connections to shapes but also a bit of spatial awareness in the x,y coords needed on the SVG canvas to place the shapes.
I was able to do with with several different little characters / cartoons in SVG and while it completely fails every now and then it was better than I would have thought.
I think it has been exposed to SVG (via the web) way more than LaTeX vector drawings.
100,000x is only seventeen 2x improvements. This[1] says world volume of data doubles every two years, citing a 2016 McKinsey study as the source. That puts it 34 years away. 2056. This[2] says the estimated compound growth rate of data creation is around 61%, that's a doubling time of 1.32 years and puts it 22 years away. 2045.
It's not tomorrow, but it may not be all that far away.
[1] https://rivery.io/blog/big-data-statistics-how-much-data-is-...
[2] https://theconversation.com/the-worlds-data-explained-how-mu...
[1]: Or came from wealthy families elsewhere.
I think most people realize there are some gains to be had here, the trick is to do so in a way that doesn't, yet again, massively redistribute wealth and power to a select few and instead share some of those gains. Right now there's not enough legitimate competition to keep things in check and that should be concerning. I'm all for rewarding the early successors who invested and took risk but let's not pretend those investments weren't captured through all sorts of other unequitable approaches to begin with.
Human brains aren't trained on video transcripts, which leave out a lot of information from the video that human brains have a shared understanding of due to training. You would train on video embeddings and predict next embeddings, thus learning physics and other properties of the world and the objects within it and how they interact. This is many more than 5 orders of magnitude more data than the text that today's LLMs are trained on.
What I’m saying is, I suspect information technology has made classic production companies vastly more efficient and productive. To the point where we can afford to have massive companies like Facebook that are almost entirely based on value capture.
That’s my speculation at least. Your example puts me in a tough spot, in the sense that Louis Vuitton is pretty old and pretty big. I’d have to know more about the company to quibble, and I don’t feel like researching it. I wonder if the proportion of their value that comes from pointless fashion branding was originally smaller. Or if the whole pointless fashion branding segment was originally just smaller itself. But I’m just spitballing.
In the past we also had mercenary companies and the like to capture value without producing much, so I could just be wrong.
What definition of 'valuable' are you using here?
I think it is a fair point TBH, my original comment could have been more clear about this aspect.
I just got back from the shop with my dad, he was born in 1935 .. neither of us struggled to survive the 1950s.
It's dropped from a standard 40 hour work week to a 38 hour week for indexing a living wage, but things are more or less still as they were in 1907 (inflation adjusted) albeit with greater choice of consumerables.
https://www.fwc.gov.au/about-us/history/waltzing-matilda-and...
Either way, it does highlight a serious economic concern as computing continues to advance. The majority of the wealth is increasingly being concentrated by our tech overlords.
As you said, we may very well be a couple iterations away from a chatbot that is truly indistinguishable from a human, but I still strongly assert that even a perfectly coherent chatbot is nothing more than an automaton and we humans are not automatons.
The fact that a couple replies in this thread made me feel defensive and a bit discouraged with their condescending tone is to me an internal reaction that an LLM or similar system will never have. Maybe an appropriate emotional reaction can be calculated and simulated, but I think the nature of the experience itself is truly beyond our current comprehension.
Maybe I’m grasping at the metaphysical to rationalize my fear that we’re on the cusp of understanding consciousness… and it turns out to be pretty boring and will be included with Microsoft O365 in a couple years.
Personally, I don't think we can construct an equivalent intelligence to a human out of silicon. That's not say AGI is unachievable or that it can't surpass human intelligence and be superficially undistinguishable from a human, but it will always be different and alien in some way. I believe our intelligence is fundamentally closer to other earth animals descended from common genetic ancestors than it can be to an artificial intelligence. As the creators of AI, we can and will paper over these differences enough to Get The Job Done™, but the uncanny valley will always be there if you know where to look.
The article does address this regarding abysmal performance on the GitHub PR benchmark. It’s one of the big “ifs” for sure.
Not so much a them waking up as an us falling asleep.
It's unclear whether a rocket ship is a multimodal neural net. Or some sort of swarm of LLM's in an adversarial relationship, or something completely novel. Regardless, we might be as far between LLM's to ASI's, as airplanes are to rocket ships. Or not.
Electrical parts ran at aviation-standard 400hz. Aviation gyroscopes and aviation instruments. Structural parts made of aviation aluminum alloys. Astronauts that are all airplane test pilots. I can imagine doing Apollo from complete scratch (using car manufacturers that have to invent aluminum-handling tech starting from nothing) but it would have taken a lot more than the decade Apollo took.
I don't think it's likely to be a helpful one in this case.
For a system to be able to solve the same classes of problems human can solve it would need to be able to invent their own rules just like humans can.
Though, I will admit, a system that acts in a way that’s indistinguishable from a human will be awful hard to classify as anything but AGI.
Maybe I’m conflating AGI and consciousness, though given that we don’t understand consciousness and there’s no clear definition of AGI, maybe they ought to be inclusive of each other until we can figure out how to differentiate them.
Still, one interesting outcome, I think, should consciousness be included in the definition of AGI, is that LLMs are deterministic, which, if conscious, would (maybe) eliminate the notion of free will.
I feel like this whole exercise may end up representing a tiny, microscopic scratch on the surface of what it will actually take to build AGI. It feels like we’re extrapolating the capabilities of LLMs far too easily from capable chat bots to full on artificial beings.
We humans are great at imagining the future, but not so good at estimating how long it will take to get there.
Expert systems, for instance, use a set of if-then rules derived from human expertise to make decisions in specific domains. This is a form of deductive reasoning, albeit limited and highly structured. They don't 'understand' in a human sense but operate within a framework of logic provided by humans.
LLMs, on the other hand, use statistical methods to generate responses based on patterns learned from vast amounts of data. This isn't reasoning in the traditional philosophical sense, but it's a kind of probabilistic reasoning. They can infer, locally generalize, and even 'extrapolate' to some extent within the bounds of their training data. However, this is not the same as human extrapolation, which often involves creativity and a deep understanding of context.
I actually asked ChatGPT 4 to do so, and it produced the sort of reasonable but unremarkable stuff I've come to expect from it.
However whatever we're doing seems to be different from what LLMs do, at least because of the huge difference in how we train.
It's possible that it will end up like airplanes and birds. Airplanes can bring us to the other side of the world in a day by burning a lot of fuel. Birds can get there too in a much longer time and more cheaply. They can also land on a branch of a tree. Airplanes can't and it's too risky for drones.
From my perspective theres intelligence in a how to manual.
It seems like maybe you mean consciousness? Or creativity?
Is there another kind?
Not sure why I would need to. I'm general, arguments can simply be understood. If they are good, good. If not, then fix them or move along.
Good and bad faith are about people, not arguments.
It sounds like you have a different experience(copilot is useful for business logic). Do you have maybe any examples of what you mean by "any coding problems"?
Is it similar to what I've heard previously(copilot can replace boilerplate) or have LLMs actually solved business problems for you in code?
But if you do want to appeal to Lecun as an authority, then maybe you'll accept that these (re)tweets that clearly indicate he finds the insights from the paper to be valid:
https://nitter.1d4.us/ylecun/status/1736479356917063847 https://nitter.1d4.us/rao2z/status/1736464000836309259 (retweeted)
As for Timnit, I think you have your timeline confused. Model cards are what put her on the map for most general NLP researchers, which predates her difficulties at Google.
2018: Model cards paper was put on arXiv https://arxiv.org/abs/1810.03993
2019: Major ML organizations start using model cards https://github.com/openai/gpt-2/blob/master/model_card.md
2020: Model cards become fairly standard https://blog.research.google/2020/07/introducing-model-card-...
Dec 2020: Timnit is let go from the ethics team at Google https://www.bbc.com/news/technology-55187611
EDIT:formatting
Perhaps also provide other researchers with vetted access. There are a lot of groups trying to evaluate these things systematically - for example "Faith and Fate", "Jumbled thoughts","Emergent abilities are a Mirage" were all very good papers published this year which really highlighted hype in LLM evaluation.
Everyone can see that modern LLM's have some great capabilities, the flexibility you can get in an interface by doing intent detection and categorizations using an LLM is great and it is so much easier and quicker than using previous techniques. It's more expensive, but that's improving rapidly. I firmly believe that a new era of great new systems with better interfaces and more functionality will be built on LLM's and other models from this wave of Big Data / Big Model AI, but these are not the precursors of AGI.
The problem with the looky looky AGI bunkum show is that it's pulling money into crappy projects that are going to fail hard and this will then stop a lot of money going into projects that could be successful fast. I am seeing the shape of the dotcom boom/bust in what's happening. Microsoft and Intel used dotcom to build and maintain their monopoly position, I think AWS, MS and Google will do the same this time. I think we will see a wave of new companies like Amazon that will "fail" some of them will really fail and disappear, some will half fail like Sun did, but some will go on to build monopolies anew. In the meantime the technology will evolve not for the greater good but instead to serve purposes like advertising distribution that are trivial compared to the benefit we could have seen. Over all we will not capitalise on the potential of what we have for several decades, ironically because of the failures of capitalism. Children will die, wars will be fought but some of us will have nice sweat pants and fun playing paddleball in the sunshine while it all happens.
When historians write this up in 100 years they won't really see any of this - they will just see a huge surge of innovation. The dead have no voices...
> I think better LLMs won’t lead to AGI.
Does this mean you believe that the Transformer architecture won't be an eventual part of AGI? (possibly true, though I wouldn't bet on it)
Does this mean that you see no path for GPT-4 to become an AGI if we just leave it alone sitting on its server? I could certainly agree with that.
Does this mean that something like large language models will not be used for their ability to model the world, or plan, or even just complete patterns as does our own System one in an eventual AGI architecture? I would have a lot more trouble agreeing with that.
In general, it seems like these sequence modelers that actually work right is a big primitive we didn't have in 2016 and they certainly seem to me as an important step. Something that will carry us far past human-level, whatever that means for textual tasks.
To bring it back to the article, probably pure scale isn't quite the secret sauce, but it's a good 80-90% and the rest will come from the increased interest, the shear number of human-level intelligences now working on this problem.
Too bad we haven't scaled safety nearly as fast though!
Having said that, I do agree that LLMs will be transformative technology. As important perhaps as the transistor or the wheel.
I think LLMs will accelerate our ability as a species to solve problems even more than the calculator, computer or internet has.
I think the boost in human capability provided by LLMs will help us more rapidly discover the true nature of reasoning, intelligence and consciousness.
But, like the wheel, transistor, calculator, computer and internet; I feel strongly that LLMs will prove to be just another tool and not a foundational technology for AGI.
This is a bit far from the economic aspect, but the world currently seemed to be utterly suffused with a looming sense of dread, I think because we have, or know other people have, news notifications in their pockets telling us all about how bad things are.
I don’t remember that feeling from the 90’s, but then, I was a kid. And of course before that there was the constant fear of nuclear annihilation, which we’ve only recently brought back really. Maybe growing up in the end of history warped my perspective, haha.
That’s not to say all eras are the same. Clearly there’s better and worse times to be alive, but it’s hard to be objective about our childhoods.
But I do think that perhaps the 70's was a somewhat more decent time than today. Lines have been crossed and levels of violence normalized that it seems really didn't exist back then, or certainly were not as widespread. e.g. I grew up with the IRA constantly in the news - often bombings in the UK as well as violence in Northern Ireland. But, by today's standard the IRA's terrorism was almost quaint and gentlemanly ... they'd plant a bomb, but then call it into the police and/or media so that people could be evacuated - they still created terror/disruption which realistically probably did help them achieve their goals, but without the level of ultra violence and complete disregard for human life that we see today, such as ISIS beheadings posted on FaceBook or Twitter that some people happily watch and forward to their friends, or the 9/11 attack which was really inconceivable beforehand.
But at least the dread was about things that seemed vaguely tractable and somewhat local, rather than the dizzyingly complex, global and existential threats the news delivers these days.
And of course not everyone read newspapers as intentionally-alarming as the Mail. Whereas now many more people’s information supply is mediated by channels with that brief.
Feels to me like a double-whammy of the alarm-maximising sections of the internet developing at the same time as the climate crisis becomes more imminent, maybe?
I grew up in the UK, so news was mainly from the BBC which was pretty decent although bad news (e.g. IRA bombings) was still front and center. US TV news doesn't even pretend/try to be unbiased and is all about shock value, reinforcing their viewers political beliefs and of course advertizing (which the BBC didn't have, being state funded).
Internet takes bad news and misinformation to a whole new and massively distorted level.
I gave up watching TV many years ago (nowadays primarily YouTube & Netflix for entertainment), and mostly just skim headlines (e.g. Google news) to get an idea of what's going on.
Today, I got to do some miniature painting while hanging out on video with someone in England. I get to buy books digitally the same day they are published, and I don't have to travel a suitcase full of them, plus a cd collection for a 1 month vacation. My son can talk to his grandma, on video, whenever he likes: Too cheap to meter. Food? I can find an importer that already has what I want most of the time, and if not, i can get anything shipped, from anywhere. A boardgame from germany, along with some cookies? Trivial. Spanish TV, including soccer games, which before were impossible. My hometown's newspaper, along with one from Madrid, and a few international ones.
An immigrant in the 90s basically left their culture behind with no recourse. Today I can be American, and a Spaniard, at the same time with minimal loss of context by being away. All while working on a product used by hundreds of millions of people, every day, with a team that spans 16 timezones, yet manages to have standups.
A lot of people's lives haven't changed that much, because their day to day is still very local. If you work at the oil field, and then go to the local high school to watch your kid's game on friday night, and all your family is local, a big part of your life wouldn't have been so different in the 90s, or even in the 60s. But I look at the things my family did week that I couldn't have possibly done in 98, and it's most of my life. My dad's brain would have melted if he could hear a description of the things I get to do today that were just sci-fi when he died. It's just that the future involved fewer people wielding katanas in the metaverse than our teenage selves might have liked.
Yes. You said it yourself: you used to have to WALK somewhere to look things up. Added convenience isn't the only side affect; that walk wasn't instantaneous. During the intervening time, you were stimulated in other ways on your trek. You saw, smelled, and heard things and people you wouldn't have otherwise. You may have tried different routes and learned more about your surroundings.
I imagine you, like I, grew up outside, sometimes with friends from a street or two over, that small distance itself requiring some exploration and learning. Running in fresh air, falling down and getting hurt, brushing it off because there was still more woods/quarry/whatever to see, sneaking, imagining what might lie behind the next hill/building; all of that mattered. The minutae people are immersed in today is vastly different in societies where constant internet access is available than it was before, and the people themselves are very different for it. My experience with current teens and very young adults indicates they're plenty bright and capable (30-somethings seem mostly like us older folks, IMO), but many lack the ability or desire to focus long enough to obtain real understanding of context and the details supporting it to really EXPERIENCE things meaningfully.
Admittedly anecdotal example: Explaining to someone why the blue-ish dot that forms in the center of the screen in the final scene of Breaking Bad is meaningful, after watching the series together, is very disheartening. Extrapolation and understanding through collation of subtle details seems to be losing ground to black and white binaries easily digested in minutes without further inquiry as to historical context for those options.
I abhor broad generalizations, and parenting plays a large part in this, but I see a concerning detachment among whatever we're calling post-millenials, and that's a major, real world difference coming after consecutive generations of increasing engagement and activism confronting the real problems we face.
Another thing that's changed is engineering. The US has moved up the stack. Engineering is now mostly software development and within that it's mostly web development. Engineering and manufacturing has largely moved overseas to Asia and that's where most of the expertise lies. The only thing off the top of my head that the US still dominates in engineering is software/aerospace/defense. In general though everything else is dominated by Asia, if you want the top hardware technology the US is no longer the place to get it. In Silicon Valley there used to be a good mix of different types of engineers, now everyone is SWE, and most likely doing web stuff. But here's the thing, you most likely wouldn't have noticed this unless you thought hard about it because either you're too young or because the change happened so slowly.
The same will be for AGI if it comes into fruition. A lot of jobs will be replaced, slowly. Then when AGI replacement reaches saturation most people will be used to the status quo whether it's better or worse. It will seem like nothing has changed.
And historically, TV news isn't that old, especially the 24-hour variety. The Apollo landings and Vietnam War are often cited as landmarks in TV news, where for the first time large numbers of people watched things as they occurred. But it's only about 25 years from those events to Netscape Navigator, where the web became widely available (at least in the developed world). That's a long time in most people's lives, but I wouldn't be surprised if future historians will see TV as something like an early, one-way Internet.
The tax administration is far more efficient than that. The IRS has 79K workers out of a total workforce of 158M, or 1/2000 workers. Federal taxes are about 19% GDP (28% of GDP including state and local taxes.) The IRS costs $14.3B to run and collects 19% of $25.46T = $4,800B or 0.3% of collected taxes.
> If IT really improved productivity, wouldn't you expect that that number would decrease, since Tax Administration is presumably an area that we should expect to see great gains from computerisation?
Those gains are so great that tax administration has been computerised for more than half a century now.
We could certainly save an awful lot more in tax preparation for the economy as a whole if we sent out pre-filled tax forms (the IRS already has all the information required for most people) like other countries do but the tax preparation companies have made a lot of contributions to politicians to prevent this.
Don’t get me wrong, I’m not a “taxes are theft” dummy or anything like that. Taxes are an important knob in shaping the economy. But a better functioning tax collection agency should more effectively implement the rules of whose money is collected for deletion, not just collect more money generally.
Not exactly, it's more about redistributing or reappropriating wealth because we know there exists flaws in the economic system we have (progressive tax systems). In more general terms, ignoring progressive structures, it's about investing in necessary shared services for everyone and to maintain the government that does such and more.
Looking at taxes as if they destroy wealth is a bit bleak. Governments may not be the most efficient institutions in all possible metrics but they're not out to destroy wealth exactly.
That's how government digitalization has functioned in my country. It hasn't improved things, it just moved all the paper hassle to a digital hassle now where I need to go to Reddit to find out how to use it right and then do a back and forth to get it right. Same with the new digitalization of medical activities, a lot of doctors I know say it actually slows them down instead of making them more productive as they say they're now drowning in even more bureaucracy.
So depending on how you design and use your IT systems, they can improve things for you if done well, but they cal also slow you down if done poorly. And they're more often done poorly than great because the people in charge of ordering and buying them (governments, managers, execs, bean counters, etc) are not the same people who have to use them every day (doctors, taxpayers, clerks, employees in the trenches, etc).
I kind of feel the same way about the Slack "revolution". It hasn't made me more productive compared to the days when I was using IBM Lotus Sametime. Come to think of it, Slack and Teams, and all these IM apps designed around constant group chatting instead of 1-1, is actually making me less productive since it's full of SO .... MUCH ... NOISE, that I need to go out of my way to turn off or tune out in order to get any work done.
The famous F1 aerodinamic engineer, Arain Newey, doesn't even use computers, he has his secretary print out his emails every day which he reads at home and replies through his secretary the next day, and draws everything by hand on the drafting board and has the people below him draw them in CAD and send him the printed simulation results through his secretary, and guess what, his cars have been world class winning designs. So more IT and more sync communication, doesn't necessarily mean more results.
> The specificity matters here because each innovation in the chain required not merely the discovery of the principle, but also the design and an economically viable use-case to all line up in order to have impact.
https://acoup.blog/2022/08/26/collections-why-no-roman-indus...
That's a straight up misstatement of the parent argument - the parent argued that coal was necessary, not that coal sufficient. True or not, the argument isn't refuted by the IR starting with water power either.
And pairing this with "anti-woke" jabs is discourse-diminishing stuff. The theory that petroleum was a key ingredient of the IR is much older than that (I don't even agree with it but it's better than "pure innovation" fluff).
Perhaps we could be optimistic about people and assume the amount of real, legitimate tax fraud and evasion is pretty low. If we took the latter scenario you present—increasing efficiency means the same amount of people will do more work—and assumed this effort is instead applied to decreasing the number of random errors (which might result in someone overpaying or underpaying), we wouldn’t necessarily expect to see a change in the expected value of the taxes. But, it could be “better” in the sense that it is more fair.
A third option: technology investments improved the efficiency of the previous tax base, which allowed the expansion of the tax base - through additional enforcement activity, increasing the tax base in absolute terms but also returning the overhead to its historical norms.
Without tracking the size of the tax base in inflation-adjusted terms, hard to account for.
(The cynically, you’re probably right re: useless bureaucratic expansion)
A large change on an otherwise stagnant activity... I would expect it to increase quickly.
If that number is stable for that long, it means it's defined by some factor that doesn't depend on its performance.
What is anti-woke? You realise that stuff existed before zoomers starting saying everything was woke/anti-woke. Eurocentrism is a school of thought within economic history, it is nothing to do with wokeism...I have no idea how these two things are related apart from you trying to relate it to something you understand, i.e. pop culture.
"Pure innovation" fluff is the dominant theory today, McCloskey's books are the most important ones in this school. To call this "fluff" suggests ignorance rather than the superiority that you seem to be trying to portray.
Petroleum wasn't a key ingredient of IR...at this point, I am assuming you know nothing about basic aspects of economic history because petroleum wasn't widely used as a fuel until the 1930/40s (again, you seem intent on talking about things that you know rather than the actual subject).
For reference, there are 175,000 authors in medium compared to billions using whatsapp or gmail or difference of around 50,000.
Internet and the last 30 years tech did change things dramatically. I bet that most people would feel handicapped if they were teleported just 50 years back. We got into this type of life progressively, so people didn't notice the change, even though it was dramatic. The same phenomena with gradient changes happen on physiological level too, this is not different.
At least if we're using the technological singularity was what constitutes fundamental societal change in unpredictable ways. The singularity people think AGI is going to fundamentally change everything, even more than what the past 500 years has done. Certainly many magnitudes of order more than the last 50 years. And they think it will happen much faster.
As well as those, the change from food and goods being scarce to abundant roughly corresponding with the industrial revolution (abundant textiles and clothes) and the early to mid 1900s (factories), labour receding from sunrise to sundown changing to a working week with days off (various, but early 1900s official 5 day week[1] and 8 hour day), changing to the more recent thing where both parents have to work to get enough income while the child is away all day, massively increased free time (particularly household chore automation - electricity, light, central heating, food mixers, washing machines, mostly early to mid 1900s).
Compared to those things, the internet gets you something else to read or watch (instead of TV, newspaper, book, radio) and some other way to talk (instead of letter, telegram, postcard, telephone). Yes the organisation of things happens quicker and information comes from farther away, and can be more up to date, but you spend your time sitting in a chair watching or reading (office, home, school) like you did before, you buy things and have them delivered or go collect them (like you did before), you consult maps and directories and consumer advice and government documents (like you did before), you take and share holiday photos (like before). It's different, but it's not all that different.
[1] https://www.bbc.co.uk/bitesize/articles/zf22kmn (1932 in America)
[2] https://researchbriefings.files.parliament.uk/documents/SN03... - the UK had 1.7M people working in farming in 1851, down to 182k today while the population has roughly 4x'd in the same time.
Not sure how much stock I put in that, though.
It is a very different kind of life form though so many things that applies to other complex being doesn't apply to them. Being a large single cell means that they learn by changing its proteins and other internals, very hard for us humans to reason about and understand since it is so alien compared to just having nerve cells with physical connections.
The illusion of emergence is fact not fiction. The cognitive biases exposed by stochastic parrots are fact not fiction.
It is a philosophical argument to say that a machine isn't truly intelligent because it isn't using the same type of neural network as a human
There are some really good bulshitters who have led smart people into deep trouble. These bulshitters behaved really like ducks but they weren't ducks. The duck test just isn't good enough.
The -1 day is where people say that because LLMs behave like humans then humans must be based on the same tech. I just wonder if these people have ever debugged a complex system only to discover that their initial model of how it worked was way off.
I'm excited to see how LLMs scale but it won't reach AGI without a much richer architecture that is capable of experimentation, capable of playing "language games" with other humans and remembering what it learned.
(I'm fairly certain of my views given my experience in neuroscience but it's fun to talk Wittgenstein in the context of LLMs, something that's been conspicuously missing. Sadly I don't believe discussions of AGI are fruitful, just what LLMs can teach us about the nature of language)
> That is one of the things that stood out to me in Searle's summary of his later work because I consider how the transformer architecture works and the way in which the surrounding context plays into the meaning of the words.
That's what makes transformer LLMs so interesting! Clearly they have captured a lot of what it means to be intelligent vis a vis language use, but is that enough to capture the kind of innate knowledge that defines intelligence at a human level? Based on my experiments with LLMs, it hasn't (yet). One of the clearest signs IMO is that there is no pedagological foundation to the LLM's answers. It can mimick explanations it learned on the internet but it cannot predict how to best explain a concept by implicitly picking up context from wrong answers or confusing questions. There is no "self reflection" because the algorithm as designed is incapable except for RLHF and finetuning.
> It is interesting that the article we are responding to talks about how we have only just begun to experiment with RL on top of transformers. In the same way that Alpha Go engaged in adversarial play we can envision LLMs being augmented to play language games amongst themselves. That may result in their own language, distinct from human language. But it also may result in the formation of intelligence surpassing human intelligence.
I think they already have their own language - embeddings! That really shines through with the multi-modal LLMs like GPT4V and LLaVa. What's curious is that we stumbled onto the same concept long before our algorithms showed any "intelligence" and it even helped Google move past the PageRank days. That's probably one of the fist steps towards intelligence but far from sufficient.
That brings up the fun question of what is enough to surpass human intelligence? I'm trying to apply LLMs to help make sense of the insane size of the American legal code and I can scale that process up to thousands of GPUs in a matter of seconds (as long as I can afford it). Even if it's at the level of a relatively dumb intern, that's a huge upside when talking about documents that would otherwise take years to read. Is that enough to claim intelligence, even if its not superior to a trained paralegal/lawyer?
>> The fidelity of text is simply too low to communicate the amount of information humans use to build up general intelligence.
> This does not at all follow from anything I've encountered in Wittgenstein. It is an empirical claim that we (as in humanity) are going to test and not something that I would argue either one of us can know simply reasoning from first principles.
Wittgenstein alone is not enough to come to this conclusion because it requires a peek at cognitive neuroscience and information theory which Mr W would have been woefully behind on given his time period. In short, just like the LLM "compresses" its training data to weights, all of human perception has to be compressed into language to communicate, which I think is impossible. We're talking about (age in years) * (365 days/year) * (X hours awake per day) * (500 megapixels per eye) * (2 eyes) + (all the other senses) versus however many bits it takes to represent language. I don't want to do the math on the latter cause I'm several beers in but it's not even close. 10 orders of magnitude wouldn't surprise me. 10 gigabytes of visual and other sensory input per 1 byte of language isn't out of the question.
I'm totally speculating and pulling numbers out of my ass here but the information theoretic part is undeniable: each human has access to more training data than it is possible for ChatGPT to experience. The quality of that training data ranges from "PEEKABOO!" to graduate textbooks, but its volume is incalculable and volume matters a lot to unsupervised algorithms like humans and LLMs.
> What does follow for me is closer to what Steven Pinker has been proposing in his own critiques of LLMs and AGI, which is that there is no necessary correlation between goal seeking (or morality) and intelligence. I also feel this is concordant with Wittgenstein's own work.
I haven't read Steven Pinkers critiques (could you link them please) so I can't say much about that. What does he mean by goal seeking?
IMO the only goal that matters is the will to survive, but lets assume for the sake of this discussion that it's not necessary to intelligence (otherwise we'll have to force our AGI bots to fight in a thunderdome and that's how we probably get Battlestar Gallactica all over again)
> My suspicion is that people don't want this scaling up to work because it would force them to let go of metaphysical commitments they have on both the nature of intelligence as well as the nature of reality. And for this reason they are adamantly disbelieving in even the possibility before the evidence has been gathered.
Let me be clear: I have zero metaphysical commitments and I can't wait until we come up with a richer vocabulary to describe intelligence. LLMs are clearly "intelligent", just not in any human sense quite yet. They don't have the will to survive, or any sense of agency, or even any permanence beyond the hard drive they exist on, but damn if they're not intelligent in some way. We just need better words to describe the levels of intelligence than "human, dog/cat/pig/pet, and everyone else"
However, I have some very strong physical commitments that must be met before I can even consider any algorithm as intelligent:
Neuroplasticity: human brains are incredibly capable of adapting all throughout life. That ranges from the simplest of drug tolerance to neurotransmitter attenuation/potentiation to the growth of new ion channels on cell membranes to very complex rewiring of axons and dendrites. That change is constant. It never stops, and LLMs don't have anything remotely like it. The brain rearchitects itself constantly and it's controlled as much by higher order processes as the neuron itself.
Scale: last time I did the math the minimum number of parameters required to represent the human connectome was over 500 quadrillion. 10+ quintillion is probably more accurate. That's 6-8 orders of magnitude more than we have in SOTA LLMs running on the best of the best hardware and Moore's law isn't going to take us that far. A 2.5D CPU/GPU might not even be theoretically capable of enough elements to simulate a fraction of that.
Quantization: I'm not sure neurons can be fully simulated with the limited precision of FP64, let alone FP32/16 or Q8/7/6/5/4. I've got far less evidence for this point than the others but it's a deeply held suspicion.
He has spoken about it multiple times on a few different podcasts. Here is a recent discussion he had with physicist David Deutsch [1] where he references this idea (See chapter timestamp for "Does AGI need agency to be 'creative'?").
1. https://www.youtube.com/watch?v=3Ho-vJZsMgk&t=2363s&ab_chann...
It just isn’t an efficiency, in the <achieved quantity>/<total quantity> sense. The measurement of <tax revenue>/<money spend on tax collection> is meaningless in the same way that a comparison between the energy required to flip a light switch and the energy that is sent to that socket as a result is basically meaningless.
I mean, sure, if tending a light switch required an appreciable percentage of the amount of the energy the socket could provide, that would be bad, but it is a ridiculous scenario.
It is fungible anyway, so I think it is really just a matter of semantics or philosophy if the government is collecting and redistributing dollars, or if it is destroying and creating them. My (outsider) understanding is that modern monetary theory leans toward the latter, because it more accurately reflects the latitude the government has, working with a fiat currency and all that.
So your statement just seems to be your bias thinking that a slime mold couldn't possible do any reasoning. Cells are much smarter than most thinks.
Edit: Anyway, apparently slime molds can communicate what they learn by sharing those proteins. So they do have a language, it is like a primitive version of how human bodies cells communicate. So your point still stands, reasoning seems to go hand in hand with communication. If you can reason then it is worth it to share those conclusions with your friends and family.
They also taught slime molds to cross a bridge for food, and it learned to do it. Then they got the slime mold to tell other slime molds and now those also knew how to cross the bridge. It is pretty cool that slime molds can be that smart.
https://asknature.org/strategy/brainless-slime-molds-both-le...
Your brain is several orders of magnitude more complex than even the largest LLM.
GPT4 has 1 trillion parameters? Big deal. Your brain has 1 quadrillion synapses, constantly shifting. Beyond that the synapses are analog messages, not binary. Each synapse is approximately like 1000 transistors based on the granularity of messaging it can send and receive.
It is temporally complex as well as structurally complex, well beyond anything we've ever made.
I'm strongly in favor of AGI, for what it's worth, but LLMs aren't even scratching the surface. They're nowhere close to a human. They're a mediocre pastiche and it's equally possible that they're a dead end as it is that they'll ever be AGI.
But then you get into the nuance of spiking vs non spiking neurons etc, which to my knowledge isn't emulated. The brain is extremely complex. I don't believe LLMs do anything similar to inhibitor neuronal function for example.
The amount of hubris we have in our field is deeply embarrassing. Imagine a neuroscientists reading that. The thought makes me blush.
It's our field that are the alchemist mystics, rambling about AGI/Philosopher's Stone in ever increasingly unhinged ways, while stirring our ML-pots that we have never even tried to prove have a chance to be anymore successful than the alchemists.
Consciousness?
No telephone to telephone is a HUGE change. Telephone call to digital PBX over fibre to packet switched VoIP to WhatsApp feels like no change at all.
I don’t really have hard data that proves the impact of the internet at a macro level, just anecdotal stuff. I think it’s easy to overplay the impact of past innovations bc we see the late waves - eg the telephone wasn’t really a thing that impacted most of the world for decades, and there’s entirely countries that went from no telephone at all to whatsapp directly (brazil for instance)
When it comes to jobs, I think I know maybe a dozen people whose jobs/industries only exist because of the Internet - and that’s counting outside the tech bubble, mind you. I’m sure this is verifiable with labor data somehow. Not to mention the entire middle class slipping into gigs in a model that only exists because of the internet (maybe a bit too recent for most to grasp the consequences).
On a personal level - quite literally everything in my life would’ve been completely different. I can’t even imagine the kind of local job I’d have at a small town, watching about stuff on the TV and hearing about tech only from a friend over radio
> It's _our field_ that are the alchemist mystics, rambling about AGI/Philosopher's Stone in ever increasingly unhinged ways,
These "ramblings" are what scientists call hypotheses. The people making these hypotheses have even proposed how to test them.
You can accurately describe any AGI or reasoning problem as an open domain sequence modeling problem. It is not an unreasonable hypothesis that brains evolved to solve a similar sequence modeling problem.
The real world is random, requires making decisions on incomplete information in situations that have never happened before. The real world is not a sequence of tokens.
Consciousness requires instincts in order to prioritize the endless streams of information. One thing people dont want to accept about any AI is that humans always have to tell it WHAT to think about. Our base reptilian brains are the core driver behind all behavior. AI cannot learn that
What if "instinct" is also just (pretrained) model weight?
The human brain is very complex and far from understood and definitely does NOT work like a LLM. But it likely shares some core concepts. Neuronal networks were inspired by brain synapses after all.
We are clearly a product of our past experience (in LLMs this is called our datasets). If you go back to the beginning of our experiences, there is little identity, consciousness, or ability to reason. These things are learned indirectly, (in LLMs this is called an emergent property). We don't learn indiscriminately, evolved instinct, social pressure and culture guide and bias our data consumption (in LLMs this is called our weights).
I can't think of any other way our minds could work, on some level they must function like a LLM, Language perhaps supplemented with general Data, but the principle being the same. Every new idea has been an abstraction or supposition of someones current dataset, which is why technological and general societal advancement has not been linear but closer to exponential.
> If you go back to the beginning of our experiences, there is little identity, consciousness, or ability to reason.
That is because babies brains aren't properly developed. There is nothing preventing a fully conscious being from being born, you see that among animals etc. A newborn foal is a fully functional animal for example. Genes encode the ability to move around, identify objects, follow other beings, collision avoidance etc.
I'm not ignoring that, I'm just saying that in LLMs we call these things weights. And i don't want to downplay the importance of weights, its probably a significant difference between us and other hominids.
But even if you considered some behaviors to be more akin to the server or interface or preprocess in LLMs it still wouldn't detract from the fact that the vast majority of the things that make us autonomous logical sentient beings come about through a process that is very similar to the core workings of LLMs. I'm also not saying that all animal brains function like LLMs, though that's an interesting thought to consider.
I admit that humans don’t progress much behaviorally, outside of intellect, past our teen years; we’re very instinct driven.
But still, I think even very young children have a spark that’s something far beyond rote token generation.
I think it’s typical human hubris (and clever marketing) to believe that we can invent AGI in less than 100 years when it took nature millions of years to develop.
Until we understand consciousness, we won’t be able to replicate it and we’re a very long way from that leap.
What LLMs learn is exactly the diff between primitive humans and us. It's such a huge jump a human alone can't make it. If we were smarter we should have figured out the germ theory of disease sooner, as we were dying from infections.
So don't praise the learning abilities of little children, without language and social support they would not develop very much. We develop not just by our DNA and direct experiences but also by assimilating past experiences through language. It's a huge cache of crystallized intelligence from the past, without which we would not rule this planet.
That's also why I agree LLMs are stalling because we can't quickly scale a few more orders of magnitude the organic text inputs. So there must the a different way to learn, and that is by putting AI in contact with environments and letting it do its own actions and learn from its mistakes just like us.
I believe humans are "just" contextual language and action models. We apply language to understand, reason and direct our actions. We are GPTs with better feedback from outside, and optimized for surviving in this environment. That explains why we need so few samples to learn, the hard work has been done by many previous generations, brains are fit for their own culture.
So the path forward will imply creating synthetic data, and then somehow evaluating the good from the bad. This will be task specific. For coding, we can execute tests. For math, we can use theorem provers to validate. But for chemistry we need simulations or labs. For physics, we need the particle accelerator to get feedback. But for games - we can just use the score - that's super easy, and already led to super-human level players like AlphaZero.
Each topic has its own slowness and cost. It will be a slow grind ahead. And it can't be any other way, AI and AGI are not magic. They must use the scientific method to make progress just like us.
Sure - then it will take the same amount of energy to train as our reptilian and higher brains took. That means trillions of real life experiences over millions of years.
"Neuroscience is the study of the nervous system, the collection of nerve cells that interpret all sorts of information which allows the body to coordinate activity in response to the environment."
https://openbooks.lib.msu.edu/introneuroscience1/chapter/wha...
This is even simpler than the RL agent description of the brain that visagra provided earlier, ignoring that the brain must have some internal state instead of being a pure function from each input to each response. Would you say that neuroscientists have even more hubris?
Have you not understood by now that my critic of our field's AI-hubris is a general one - and thus not hinged on exact wordings? I could've written similar hubris related replies on multiple other comments in this thread. It's my reaction. It's my exasperation.
I think it does, and I've explained exactly how but you just pivots to snarky comments on specific words instead - first scientists, then magic, then oversimplified. Furthermore you seemingly can't accept no matter how many times that it's not specific to this particular comment but meant as a response to the general "AI-hype. So judging by this you're either wilfully obtuse or completely unable to read between the lines no matter how many times I explain it to you. Did you miss that all sibling comments to mine seemed to - shockingly - be able to react the same way I did - albeit less exasperated? But somehow you're completely unable to accept that reaction.
Seriously though, to make my general non-comment specific point for the last time; just do a Occam's razor on the "AI"-hype timeline between if it's a bog standard self-interested tech-hype, using the same methods but with more hardware, or an extraordinary scientific breakthrough justifiably triggering all the extraordinary claims about AGI being close and even an existential threat (!). I think it's the former. If you think that the latter is likelier, fine, but I do think that that requires an embarrassing combination of hubris & confirmation bias because it strokes our egos.
I assume that you're invested in this hype and has taken offense by me calling it embarrassing hubris. I hope you'll be okay when this bubble inevitably bursts. I'll leave it to it now, since I think I've made my general point even though it's clear that it will never be acknowledged.
Did you miss all the upvotes that my comment and responses to sibling comments that show that your reaction is completely out of place?
> to make my general non-comment specific point
OK, so you have no justification for your reaction to Visarga's comment, just as I and the voters suspected.
Added emphasis, and others reacted the same way. You can disagree with the interpretation but to like disallow it like you seem to be attempting is really weird.
you mean by picking the intro one-liner from a textbook? haha, pathetic. whatever floats your boat mate.