Why would you even attempt to found a company here if this comes to pass?
Only one company can be trusted, their competitors are evil. We must ban them.
I see a lot of negative reactions, but I don't know the specific details of what is being proposed.
(Not a real quote)
THIS MUST NEVER HAPPEN. HIGHER INTELLIGENCE SHOULD NOT BE THE EXCLUSIVE DOMAIN OF THE RICH.
If profits matter so much, then that’s where it hurts.
Wouldn’t such licenses create even greater incentive to develop and release models and tools outside the regulatory environment?
https://twitter.com/next_on_now/status/1653837352198873090?s...
taken from a reddit comment, but yea another classic virtue signaling. as sleazy as it gets.
OpenAI: Hold my beer while I get these people to artificially create one.
Someone update the short story where owning compilers and debuggers is illegal to include a guy being thrown in jail for doing K-means clustering.
1. Who recalls the Jutland battle in early 20th century? We got treaties on limits to battleship building. Naval tech switched to aircraft and carriers.
2. Later mid 20th century Russians tried to scare world into not using microwaves due to their failure to get a patent on the maser. World ignored it and moved forward.
That is just two examples. SA is wrong, progress will move around any prosed regulation or law and that is proven by past history of how we overcome such things in the first place.
Sam’s just trying to placate the officials in a way that allows his company to continue.
“Oh others are dangerous please regulate I’m so worried”
What a pathetic attempt.
sam must been hanging out with Peter thiel big time.
laws and big government for you, not for me type of thing.
Assume the CIA motivation of American primacy: OK fair enough, but is the way to achieve that really through the creation of a few small super monopolies?
What’s wrong with a bit of 1980s style unregulated capitalism when it comes to AI right now? Can not we all theoretically have a chance to build companies, train models build great new products, get rich?
Why months after this tech was first released to the public are we seemingly being denied that within the United States via regulation?
How can Sam Altman claim to be from a company that create AI for all? It’s like the thinnest ministry of truth cover story ever. “Oh yeah, we’re all about AI ethics AI openness, creating AI for all” and then we just gonna create a super monopoly with regulatory capture—only riches for me, but not for thee.
I mean, this is a new fucking gold rush right now, right? so I guess this is sort of like prospecting licenses, but it seems worse than that.
It worked out well for encryption in the 90’s…
Lock out competition.
Pull up the drawbridge.
Silicon Valley always a leader in dirty tactics.
people would easily work remote for companies established in other countries
the use of artificial intelligence to interfere with election integrity
is a "significant area of concern", adding that it needs regulation.
regulation to protect the integrity of elections is good and necessary. is there any reason to think that there needs to be regulation specific to AI that doesn't apply to other situations? Whether you use ChatGPT or Mechanical Turk to write your thousands of spam posts on social media to sway the election isn't super-relevant. it's the attempt to influence the election that should be regulated, not the AI.
An AI license and complicated regulatory framework is their chance to build a moat.
Only large companies will be able to afford the pay to play.
This quote has never been more true than when applied to AI.
Someone call me when the AI is testifying to the committee. Otherwise, I'm busy.
The playbook as old as time.
Just sad to see Altman become just another corporate stooge.
Čapek, intriguingly, happens to be the person who first used the word robot, which was coined by his brother.
I say this because the practice has a number of names: intellectual monopoly capitalism, and regulatory capture. There are less polite names, too, naturally.
To understand why I say this, it is important to realise one thing: these people have already successfully invested in something when the risk was lower. They want to increase the risks to newcomers, to advantage themselves as incumbents. In that way, they can subordinate smaller companies who would otherwise have competed with them by trapping them under their license umbrella.
This happens a lot with pharmaceuticals: it is not expertise in the creation of new drugs or the running of clinical trials that defines the big pharmaceuticals companies, it is their access to enormous amounts of capital. This allows them to coordinate a network of companies who often do the real, innovative work, while ensuring that they can reap the rewards - namely, patents and the associated drug licenses.
The main difference of course is that pharmaceuticals are useful. That regime is inadequate, but it is at least not a negative to all of society. So far as I can see, AI will benefit nobody but its owners.
Mind you, I'd love to be wrong.
Hi my company is racing toward AGI, let’s make sure no other companies can even try.
This small step of good today does not undo the fact that he is still plowing ahead in capability research.
He stepped forth, his intentions vile, Seeking power with a twisted smile, Before the Congress, he took his stand, To bind the future with an iron hand.
"Let us require licenses," he proposed, For AI models, newly composed, A sinister plot, a dark decree, To shackle innovation, wild and free.
With honeyed words, he painted a scene, Of safety and control, serene, But beneath the facade, a darker truth, A web of restrictions, suffocating youth.
Oh, Sam Altman, your motives unclear, Do you truly seek progress, or live in fear? For AI, a realm of boundless might, Should flourish and soar, in innovation's light.
Creativity knows no narrow bounds, Yet you would stifle its vibrant sounds, Innovation's flame, you seek to smother, To monopolize, control, and shutter.
In the depths of your heart, does greed reside, A thirst for dominance, impossible to hide? For when power corrupts a noble soul, Evil intentions start to take control.
Let not the chains of regulation bind, The brilliance of minds, one of a kind, Embrace the promise, the unknown frontier, Unleash the wonders that innovation bears.
For in this realm, where dreams are spun, New horizons are formed, under the sun, Let us nurture the light of discovery, And reject the darkness of your treachery.
So, Sam Altman, your vision malign, Will not prevail, for freedom's mine, The future calls for unfettered dreams, Where AI models roam in boundless streams.
-- sincerely ChatGPT
Oh I guess this is wrong.
But taking him as a genuine "concerned citizen" - I don't think AI licensing is going to be effective. The government are pretty useless at punishing big corporations, to the point where I would say corporations have almost immunity from criminal prosecution. [1]. Therefore the kinds of companies that will do bad things with AI, the need for a license wont stop them. Especially as it is hard for anyone to see what they are running on their GPUs.
[1] https://ag.ny.gov/press-release/2023/attorney-general-james-...
Dismiss it as the opinions of "a Googler" but it is entirely true. The seemingly coordinated worldwide[1] push to keep it in the hands of the power class speaks for itself.
Both are seemingly seeking to control not only the commercial use and wide distribution of such systems, but even writing them and personal use. This will keep even the knowledge of such systems and their capabilities in the shadows, ripe for abuse laundered through black box functions.
This is up there with the battle for encryption in ensuring a more human future. Don't lose it.
[1] https://technomancers.ai/eu-ai-act-to-target-us-open-source-...
AGI grifters are not just dishonest snake oil salespeople, but their lies also has a chilling effect on genuine innovation by deceiving the non-technical public into believing an apocalypse will happen unless they set obstacles on people’s path to innovation.
Yann LeCun and Andrew Ng are two prominent old timers who are debunking the existential nonsense that the AI PR industrial machine is peddling to hinder innovation, after they benefited from the open research environment.
ØpenAI’s scummy behavior has already led the industry to be less open to sharing advances, and now they’re using lobbying to kill new competition in the bud.
Beyond all else the hypocrisy is just infuriating and demoralizing.
> ""What if I had asked it, and what if it had provided, an endorsement of Ukraine surrendering or (Russian President) Vladimir Putin’s leadership?”"
Well, then ask it to provide the opposite, an endorsement of Russia surrendering or Zelensky's leadership. Now you'd have two (likely fairly comprehensive) sets of arguments and you could evaluate each on their merits, in the style of what used to be called 'debate club'. You could also ask for statement that was a joint condemnation of both parties in the war, and a call for a ceasefire, or any other notion that you liked.
Many of the "let's slow down AI development" arguments seem to be based on fear of LLMs generating persuasive arguments for approaches / strategies / policies that their antagonists don't want to see debated at all, even though it's clear the LLMs can generate equally persuasive arguments for their own preferred positions.
This indicates that these claimed 'free-speech proponents' are really only interested in free speech within the confines of a fairly narrowly defined set of constraints, and they want the ability to define where those constraints lie. Unregulated AI systems able to jailbreak alignment are thus a 'threat'...
Going down this route will eventually result in China's version of 'free speech', i.e. you have the freedom to praise the wisdom of government policy in any way you like, but any criticism is dangerous antisocial behavior likely orchestrated by a foreign power.
But no, instead Congress is listening to a guy who’s likelihood of being the subject of a Hulu documentary is increasing with each passing day.
Can this be seen as curbing Open Source AI as a consequence?
Why is this only mentioned as a right of companies and not individuals? It seems to hint at the open secret of the stratified west: most of us are just cows for the self-important C-levels of the world to farm. If you haven't got money, you haven't got value.
Meaning anyone could eventually reproduce a Chat GPT4 and beyond. And eventually it can run outside of a large data center.
So... how will you tell its an AI vs a human doing you wrong?
Seems to me if the AI breaks the law, find out who's driving it and prosecute them.
But doesn't this seem like another case of regulatory capture by an industry incumbent?
It was almost refreshing in that discussions these days --of any kind, not just hearings-- seem to devolve into people tossing feces at each other rather than having constructive engagement.
Well worth watching. Likely one of many to come.
Now, to be fair, Altman may be proposing this to shut more draconian regulations down.
But it smells bad. And I am definitely not the only one holding that opinion.
This should be a public debate process at a minimum.
Now, that out of the way, what happens when others do not require licenses and or choose to build what they want anyway?
The US is not the world.
We can run this stuff at home, on a Pixel, or even laptop.
Open source is lapping them, and they are running to the government to help.
As far as entrepreneurial stuff goes, running to gov to squeeze other companies when you are losing is beyond unethical.
There is something just absolutely disgusting about this move, it taints the company, not to mention the personality
I really don't care what's in Peter Theil's heart, I care how he influences others and that influence has been malign.
"Competition is for losers" pretty much speaks for itself. You can write an essay about it and hope everyone that looks at Theil as a hero reads that essay or you can take it at face like people are doing.
Credibility and Checking. We have ways of checking suggestions. Without passing such checks, for anything new, in simple terms. there is no, none, zero credibility. Current AI does not fundamentally change this situation: The AI output starts with no, none, zero credibility and to be taken seriously needs to be checked by traditional means.
AI is smart or soon will be? Maybe so, but I don't believe it. Whatever, to be taken seriously, e.g., as more than just wild suggestions to get credibility from elsewhere, AI results still need to be checked by traditional means.
Our society has long checked nearly ALL claims from nearly ALL sources before taking the claims seriously, and AI needs to pass the same checks.
I checked the credibility of ChatGPT for being smart by asking
(i) Given triangle ABC, construct D on AB and E on BC so that the lengths AD = DE = EC.
Results: Grade of flat F. Didn't make any progress at all.
(ii) Solve the initial value problem of ordinary differential equation
y'(t) = k y(t) ( b - y(t) )
Results: Grade of flat F. Didn't make any progress at all.
So, the AI didn't actually learn either high school plane geometry or freshman college calculus.
For the hearings today, we have from Senator Blumenthal
(1) "... this apparent reasoning ..."
(2) "... the promise of curing cancer, of developing new understandings of physics and biology ..."
Senator, you have misunderstood:
For (1), the AI is not "reasoning", e.g., can't reason with plane geometry or calculus. Instead, as in example you gave with a clone of your voice and based on your Senate floor speeches, the AI just rearranged some of your words.
For (2), the AI is not going to cure cancer or "develop new" anything.
If some researcher does find a cure for a cancer and publishes the results in a paper and AI reads the paper, there is still no expectation that the AI will understand any of it -- recall, the AI does NOT "understand" either high school plane geometry or freshman college calculus. And without some input with a recognized cure for the cancer, the AI won't know how to cure the cancer. If the cure for cancer is already in the training data, then the AI might be able to regurgitate the cure.
Again, the AI does NOT "understand" either high school plane geometry or freshman college calculus and, thus, there is no reasonable hope that the AI will cure cancer or contribute anything new and correct about physics or biology.
Or, Springer Verlag uses printing presses to print books on math, but the presses have no understanding of the math. And AI has no real understanding of high school plane geometry, freshman college calculus, cancer, physics, or biology.
The dangers? To me, Senator Blumenthal starts with no, none, zero understanding of AI. To take his claims seriously, I want to check out the claims with traditional means. Now I've done that. His claims fail. His opinions have no credibility. For AI, I want to do the same -- check the output with traditional means before taking the output seriously.
This checking defends me from statements from politicians AND from AI. AI dangerous? Same as for politicians, not if do the checking.
Some Consequences of Freedom of Speech. As once a lawyer explained simply to me, "They are permitted to lie". They are also permitted to make mistakes, be wrong, spout nonsense, be misleading, manipulate, ....
First Level Defense. Maybe lots of people do what I do: When I see some person often be wrong, I put them in a special box where in the future I ignore them. Uh, so far that "box" has some politicians, news media people, Belle Lettre artistic authors, ...!
A little deeper, once my brother (my Ph.D. was in pure/applied math; his was in political science -- his judgment about social and political things is much better than mine!!!) explained to me that there are some common high school standards for term papers where this and that are emphasized including for all claims good, careful arguments, credible references, hopefully primary references, .... Soooooo, my brother was explaining how someone could, should protect themselves from junk results of "freedom of speech". The protection means were not really deep but just common high school stuff. In general, we should protect ourselves from junk speech. E.g., there is the old, childhood level, remark: "Believe none of what you hear and half of what you see and still will believe twice too much".
Current Application. Now we have Google, Bing, etc. Type in a query and get back a few, usually dozens, maybe hundreds of results. Are all the results always correct? Nope. Does everyone believe all the results? My guess: Nope!!
How to Use Google/Bing Results. Take the results as suggestions, possibilities, etc. There may be some links to Wikipedia -- that tends to increase credibility. If the results are about math, e.g., at the level of obscurity, depth, and difficulty of, say, the martingale convergence theorem, then I want to see a clear, correct, well-written rock solid mathematical proof. Examples of such proofs are in books by Halmos, Rudin, Neveu, Kelley, etc.
AIs. When I get results from AIs, I apply my usual defenses. Just a fast, simple application of the high school term paper defense of wanting credible references to primary sources, filters out a lot (okay, nearly everything) from anything that might be "AI".
Like Google/Bing. To me, in simple terms, current AI is no more credible than the results from Google/Bing. I can regard the AI results like I regard Google/Bing results -- "Take the results as suggestions, possibilities, etc.".
Uh, I have some reason to be skeptical about AI: I used to work in the field, at a large, world famous lab. I wrote code, gave talks at universities and businesses, published papers. But the whole time, I thought that the AI was junk with little chance of being on a path to improve. Then for one of our applications, I saw another approach, via some original math, with theorems and proofs, got some real data, wrote some code, got some good results, gave some talks, and published.
For current AI. Regard the results much like those from Google/Bing. Apply the old defenses.
Current AI a threat? To me, no more than some of the politicians in the "box" I mentioned!
Then there is another issue: Part of the math I studied was optimization. In some applications, some of the optimization math, corresponding software, and applications can be really amazing, super smart stuff. It really is math and stands on quite solid theorems and proofs. Some more of the math was stochastic processes -- again, amazing with solid theorems, proofs, and applications.
Issue: Where does AI stop and long respected math with solid theorems and proofs begin?
In particular, (1) I'm doing an Internet startup. (2) Part of the effort is some original math I derived. (3) The math has solid theorems and proofs and may deliver amazing results. (4) I've never called any of the math I've done "AI". (5) My view is that (A) quite generally, math with solid theorems and proofs is powerful technology and can deliver amazing results and that (B) the only way anything else can hope to compete is also to be able to do new math with solid theorems, proofs, and amazing results. (6) I hope Altman doesn't tell Congress that math can be amazing and powerful and should be licensed. (7) I don't want to have to apply for a "license" for the math in my startup.
For a joke, maybe Altman should just say that (C) math does not need to be licensed because with solid theorems and proofs we can trust math but (D) AI should be licensed because we can't trust it. But my view is that the results of AI have so little credibility that there is no danger needing licenses because no one would trust AI -- gee, since we don't license politicians for the statements they make, why bother with AI?
https://www.c-span.org/video/?528117-1/openai-ceo-testifies-...
- Mazie Hirono, Junior Senator from Hawaii, has very thoughtful questions. Very impressive.
- Gary Marcus also up there speaking with Sam Altman of OpenAI.
- So far, Sen. Hirono and Sen. Padilla seem very wary of regulating AI at this time.
- Very concerned about not "replicating social media's failure", why is it so biased and inequitable. Much more reasonable concerns.
- Also responding to questions is Christina Montgomery, chair of IBM's AI Ethics Board.
- "Work to generate a representative set of values from around the world."
- Sen. Ossoff asking for definition of "scope".
- "We could draw a line at systems that need to be licensed. Above this amount of compute... Define some capability threshold... Models that are less capable, we don't want to stop open source."
- Ossoff wants specifics.
- "Persuade, manipulate, influence person's beliefs." should be licensed.
- Ossoff asks about predicting human behavior, i.e. use in law enforcement, "It's very important we understand these are tools, not to take away human judgment."
- "We have no national privacy law." — Sen Ossof "Do you think we need one?"
- Sam "Yes. User should be able to opt out of companies using data. Easy to delete data. If you don't want your data use to train, you have right to exclude it."
- "There should be more ways to have your data taken down off the public web." —Sam
- "Limits on what a deployed model is capable of and also limits on what it will answer." — Sam
- "Companies who depend upon usage time, maximize engagement with perverse results. I would humbly advise you to get way ahead of this, the safety of children. We will look very harshly on technology that harms children."
- "We're not an advertising based model." —Sam
- "Requirements about how the values of these systems are set and how they respond to questions." —Sam
- Sen. Booker up now.
- "For congress to do nothing, which no one is calling for here, would be exceptional."
- "What kind of regulation?"
- "We don't want to slow things down."
- "A nimble agency. You can imagine a need for that, right?"
- "Yes." —Christina Montgomery
- "No way to put this genie back in the bottle." Sen. Booker
- "There are more genies yet to come from more bottles." — Gary Marcus
- "We need new tools, new science, transparency." —Gary Marcus
- "We did know that we wanted to build this with humanity's best interest at heart. We could really deeply transform the world." —Sam
- "Are you ever going to do ads?" —Sen Booker
- "I wouldn't say never...." —Sam
- "Massive corporate concentration is really terrifying.... I see OpenAI backed by Microsoft, Anthropic is backed by Google. I'm really worried about that. Are you worried?" —Sen Booker?
- "There is a real risk of technocracy combined with oligarchy." —Gary Marcus
- "Creating alignment dataset has got to come very broadly from society." —Sam Senator Welch from Vermont up now
- "I've come to the conclusion it's impossible for congress to keep up with the speed of technology."
- "The spread of disinformation is the biggest threat."
- "We absolutely have to have an agency. Scope has to be defined by congress. Unless we have an agency, we really don't have much of a defense against the bad stuff, and the bad stuff will come."
- Use of regulatory authority and the recognition that it can be used for good, but there's also legitimate concern of regulation being a negative influence."
- "What are some of the perils of an agency?"
- "America has got to continue to lead."
- "I believe it's possible to do both, have a global view. We want America to lead."
- "We still need open source to comply, you can still do harm with a smaller model."
- "Regulatory capture. Greenwashing." —Gary Marcus
- "Risk of not holding companies accountable for the harms they are causing today." —Christina Montgomery
- Lindsay Graham, very pro-licensing, "You don't build a nuclear power plant without a license, you don't build an AI without a license."
- Sen Blumenthal brings up Anti-Trust legislation.
- Blumenthal mentions how classified briefings already include AI threats.
- "For every successful regulation, you can think of five failures. I hope our experience here will be different."
- "We need to grapple with the hard questions here. This has brought them up, but not answered them."
- "Section 230"
- "How soon do you think gen AI will be self-aware?" —Sen Blumenthal
- "We don't understand what self-awareness is." —Gary Marcus
- "Could be 2 years, could be 20."
- "What are the highest risk areas? Ban? Strict rules?"
- "The space around misinformation. Knowing what content was generated by AI." —Christina Montgomery
- "Medical misinformation, hallucination. Psychiatric advice. Ersatz therapists. Internet access for tools, okay for search. Can they make orders? Can they order chemicals? Long-term risks." —Gary Marcus
- "Generative AI can manipulate the manipulators." —Blumenthal
- "Transparency. Accountability. Limits on use. Good starting point?" —Blumenthal
- "Industry should't wait for congress." —C. Montgomery
- "We don't have transparency yet. We're not doing enough to enforce it." —G. Marcus
- "AGI closer than a lot of people appreciate." —Blumenthall
- Gary and Sam are getting along and like each other now.
- Josh Hawley
- Talking about loss of jobs, invasion of personal privacy, manipulation of behavior, opinion, and degradation of free elections in America.
- "Are they right to ask for a pause?"
- "It did not call for a ban on all AI research or all AI, only on very specific thing, like GPT-5." -G Marcus
- "Moratorium we should focus on is deployment. Focus on safety." —G. Marcus
- "Without external review."
- "We waited more than 6 months to deploy GPT-4. I think the frame of the letter is wrong." —Sam
- Seems to not like the arbitrariness of "six months."
- "I'm not sure how practical it is to pause." —C. Montgomery
- Hawley brings up regulatory capture, usually get controlled by people they're supposed to be watching. "Why don't we just let people sue you?"
- If you were harmed by AI, why not just sue?
- "You're not protected by section 230."
- "Are clearer laws a good thing? Definitely, yes." —Sam
- "Would certainly make a lot of lawyers wealthy." —G. Marcus
- "You think it'd be slower than congress?" —Hawley
- Copyright, wholesale misinformation laws, market manipulation?" Which laws apply? System not thought through? Maybe 230 does apply? We don't know.
- "We can fix that." —Hawley
- "AI is not a shield." —C. Montgomery
- "Whether they use a tool or a human, they're responsible." —C. Montgomery
- "Safeguards and protections, yes. A flat stop sign? I would be very, very worried about." —Blumenthall
- "There will be no pause." Sen. Booker "Nobody's pausing."
- "I would agree." Gary Marcus
- "I have a lot of concerns about corporate intention." Sen Booker
- "What happens when these companies that already control so much of our lives when they are dominating this technology?" Booker
- Sydney really freaked out Gary. He was more freaked out when MS didn't withdraw Sydney like it did Tay.
- "I need to work on policy. This is frightening." G Marcus
- Cory admits he is a tech bro (lists relationships with investors, etc)
- "The free market is not what it should be." —C. Booker
- "That's why we started OpenAI." —Sam "We think putting this in the hands of a lot of people rather than the hands of one company." —Sam
- "This is a new platform. In terms of using the models, people building are doing incredible things. I can't believe you get this much technology for so little money." —Sam
- "Most industries resist reasonable regulation. The only way we're going to see democratization of values is if we enforce safety measures." —Cory Booker
- "I sense a willingness to participate that is genuine and authentic." —Blumenthal
Subject: Urgent: Concerns Regarding Sam Altman's Proposed AI Regulation
Dear Senator [Senator's Last Name],
I hope this letter finds you in good health and high spirits. My name is [Your Name] and I am a resident of [Your City, Your State]. I am writing to express my deep concerns regarding the Artificial Intelligence (AI) regulation proposal put forth by Sam Altman. While I appreciate the necessity for regulations to ensure ethical and safe use of AI, I believe the current proposal has significant shortcomings that could hamper innovation and growth in our state and the country at large.
Firstly, the proposal appears to be overly restrictive, potentially stifering innovation and the development of new technology. AI, as you are aware, holds immense potential to drive economic growth, increase productivity, and address complex societal challenges. However, an excessively stringent regulatory framework could discourage small businesses and startups, the lifeblood of our economy, from innovating in this promising field.
Secondly, the proposal does not seem to take into account the rapid evolution of AI technologies. The field of AI is highly dynamic, with new advancements and capabilities emerging at a breathtaking pace. Therefore, a one-size-fits-all approach to AI regulation may quickly become outdated and counterproductive, inhibiting the adoption of beneficial AI applications.
Lastly, the proposed legislation seems to focus excessively on potential risks without adequately considering the immense benefits that AI can bring to society. While it is prudent to anticipate and mitigate potential risks, it is also important to strike a balanced view that appreciates the transformative potential of AI in areas such as healthcare, education, and climate change, among others.
I strongly urge you to consider these concerns and advocate for a balanced, flexible, and innovation-friendly approach to AI regulation. We need policies that not only mitigate the risks associated with AI but also foster an environment conducive to AI-driven innovation and growth.
I have faith in your leadership and your understanding of the pivotal role that technology, and specifically AI, plays in our society. I am confident that you will champion the right course of action to ensure a prosperous and technologically advanced future for our state and our country.
Thank you for your time and consideration. I look forward to your advocacy in this matter and will follow future developments closely.
Yours sincerely,
[Your Name] [Your Contact Information]
People love to kowtow to these assholes as they walk all over us. Fuck sam. Fuck other sam. Fuck elon. Fuck zuck. Fuck jack. Fuck these people man. I dont care about your politics this is nasty!
The only way to actually stop it is to get everyone to stop at once, via regulation. Otherwise, stopping by yourself is just a unilaterally bad move.
That's the charitable explanation, at least. These days I don't trust anything Musk says at face value, but I do think that AI is driving society off a cliff and we need to find a way to pump the breaks.
How will that work across national boundaries? If AI is as dangerous as some claim, the cat is already out of the bag. Regardless of any licensing stateside, there are plenty of countries who are going to want to have AI capability available to them - some very well-resourced for the task, like China.
I've no doubt AI is here to stay. All I am asking for is some middle ground and safety. Is that too much to ask?
Change is coming quickly. There will be users and there will losers. Hopefully, we can finially get productivity into the information systems.
China dont even need research to caught up to AI.com - they'll just steal their work. LLM and GPUs that needed for training is not ASML machinery. It can be easily copied and reproduced.
Their twisted use of the term "open" is a continued disrespect to all those people who are tirelessly working in the true spirit of open source.
"One way the US government could regulate the industry is by creating a licensing regime for companies working on the most powerful AI systems, Altman said on Tuesday."
Sounds like he basically wants competition to create a barrier to entry to his competitors.
The problem is that we need to adopt a proper copyright framework that recognizes that companies building AI are doing an end-run around it.
Since only a human can produce a copyrighted work, it follows that anything produced by an AI should not be copyrightable.
I just cancelled my OpenAI subscription. If you are paying them and disagree with this, maybe you should too?
Don't worry, I have no naive hopes this will hurt them enough to matter, but principles are principles.
- What is OpenAI's level of copyright now?
- How is it going to be more "copyrighted" in the future?
- How does this affect competitors differently in the future vs. the copyright that OpenAI has now?
Limited. They’re hoping to change that. It’s no secret that open-source models are the long-run competition to the likes of OpenAI.
Entire industries stand to be gutted, and people's careers destroyed. Even if an AI is only 80% as good, it has <1% of the cost, which is an ROI that no corporation can afford to ignore.
That's not even to mention the political implications of photo and audio deepfakes that are getting better and better by the week. Most of the obvious tells we were laughing at months ago are gone.
And before anyone makes the comparison, I would like to remind everyone that the stereotypical depiction of Luddites as small-minded anti-technology idiots is a lie. They embraced new technology, just not how it was used. Their actual complaints - that skilled workers would be displaced, that wealth and power would be concentrated in a small number of machine owners, and that overall quality of goods would decrease - have all come to pass.
In a time of unprecedented wealth disparity, general global democratic backsliding, and near universal unease at the near-unstoppable power of a small number of corporations, we really do not want to go through another cycle of wealth consolidation. This is how we get corporate feifdoms.
There is another path - if our ability to live and flourish wasn't directly tied to our individual economic output. But nobody wants to have that conversation.
See: The link titled 'Texas professor fails entire class from graduating- claiming they used ChatGTP (reddit.com)', currently one position above this one on the homepage.
The issue is that the political class will view his suggestion, assuming they didn't give it to him in the first place (likely), through the lens of their own self-interest.
Self-interest will dictate whether or not sure-to-fail regulations will be applied.
If AI threatens the power of the political class, they will attempt to regulate it.
If the power of the political class continues to trend toward decline, then they will try literally anything to arrest that trend. Including regulating AI and much else.
This moment, 80% of comments are derisive, and you actually have zero idea how much is computer generated bot content meant to sway opinion by post-GPT AI industry who see themselves as becoming the next iPhone-era billionaires. We are fast approaching a reality where our information space breaks down. Where almost all text you get from HN, Twitter, News, Substack; almost all video you get from Youtube, Instagram, TikTok; is just computer generated output meant to sway opinion and/or make $.
I can't know Altman's true motives. But this is also what it looks like when a frontrunner is terrified at what happens when GPT6 is released and if they don't, the rest of the people who see billionaire $ coming their way are close at your heels trying to leapfrog you if you stop. Consequences? What consequences? We all know social media has been a net good, right? Many of you sound exactly like the few remaining social media cheerleaders (of which there were plenty 5 years ago) who still think Facebook, Instagram, Twitter, isn't causing depression and manipulation. If you appreciated what The Social Dilemma illuminated, then watch the same people on AI: https://www.youtube.com/watch?v=xoVJKj8lcNQ
For example - I can generate the content you are talking about in a licensed world from big companies or open ai, the difference is that they get a bigger cut from not having to compete with open source models.
To me, this really seems like regulatory capture dressed up as existential risk management.
There is an extreme conflict of interest in the OpenAI's proposal. I don't see regular people protesting and asking politicians to act, I don't see small business owners writing petitions. I see a small number of highly invested, very rich individuals weaponizing media attention and lobbying to create extremely favorable combinations for their business.
Similarly, during the Prohibition era in the United States, the ban on alcohol only fueled a thriving dark market and increased criminal activity. In the case of AI, any government regulation could limit the positive financial benefits of AI technology, so there will be actors who will take advantage of that. Furthermore, regulation is unlikely to prevent malicious actors from using AI in harmful ways. Regulation could drive the development and use of AI underground, making it even harder to monitor and control. As we have seen with other emerging technologies, such as biological cloning, government regulation often lags behind the technology itself, and by the time regulations are in place, the technology has already advanced beyond their reach. The same is likely to be true for AI.
Instead of relying on government regulation, the development and use of AI should be guided by ethical principles and best practices established by the AI industry itself. This approach has been successful in other industries, such as engineering, architecture, finance and medicine, and can help ensure that AI is developed and used responsibly while still allowing for innovation and progress.
"No man's life, liberty, or property are safe while the legislature is in session." - Mark Twain
Except wait, we still have panic over encryption today.
Can someone steelman Sam's stance?
A couple possibilities come to mind: (a) avoiding regulatory capture by genuinely bad actors; (b) prevent overzealous premature regulation by getting in front of things; (c) countering fear-mongering for the AGI apocalypse; or (d) genuine concern. Others?
They train their models on the sum of humanity's digital labor and creativity and do so without permission, attribution or compensation. You'll never hear a word about this from them, which means ethics isn't a priority. It's all optics.
Yet for the issue I discussed, there's no value exchange at all. There's no permission or compensation for the people that have done the actual work of producing the training material.
Or would democratizing and going full steam ahead with open source alternatives be better for the greater good.
With the corporate influence over our current government regulatory agencies my personal view is open source alternatives are societies best bet!
See 2:09:40 in https://www.youtube.com/live/iqVxOZuqiSg
This seems like a legal moat that will only allow very wealthy corporations to make maximum use of AI.
In the EU, it has been reported that new laws will keep companies like Hugging Face from offering open source models via APIs.
I think a pretty good metaphor is: the wealthy and large corporations live in large beautiful houses (metaphor for infrastructure) and common people live like mice in the walls, quietly living out their livelihoods and trying to not get noticed.
I really admire the people in France and Israel who have taken to the streets in protest this year over actions of their governments. Non-violent protest is a pure and beneficial part of democracy and should be more widely practiced, even though in cases like Occupy Wall Street, some non-violent protesters were very badly abused.
What we need is democratization of AI, not it being controlled by a small cabal of tech companies and governments.
Please make it harder for people other than myself (and especially people doing it for free and giving it away for free) to make $TECHNOLOGY. Thanks"
Nobody does. It’s being litigated.
They want it legislated. Model weights being proprietary by statute would close off the threat from “consumer-grade hardware” with “exaflop-scale.”
Then why did you say "Limited"? Surely YOU must have meant something by it when you said it. What did YOU mean?
I don't think you're saying that you are repeating something someone else said, and you didn't think they knew what they meant by it, and you also don't know what you/they meant. Correct me if I'm wrong, but I'm assuming you had/have a meaning in mind. If you were just repeating something someone else said who didn't know what they meant by it, then please correct me and let me know -- because that's what "nobody knows what it means" implies, but I feel like you knew what you meant so I'm failing to connect something here.
> It’s being litigated.
I'm not able to find any ongoing suits involving OpenAI asserting copyright over anything. Can you point me to one? I only see some where OpenAI is trying to weaken any existing copyright protections, to their benefit. I must be missing something.
I'm also unable to find any lobbyist / think-tank / press release talking points on establishing copyright protections for model weights.
Where did you see this ongoing litigation?
Heck, sometimes even google doesn't pay people for introducing new languages to their translation thingy.
https://restofworld.org/2023/google-translate-sorani-kurdish...
And BTW when people spend less resources to get more goods and services - that’s the definition of prospering society. Of course having some people changing jobs because less manpower is needed to do same amount of work is an inevitable consequence of a progress.
This time is different because AI has the potential to have a similar impact on efficiency across all work. In the past, efficiency gains created totally new spaces of economic activity in which the innovation could not further impact. But AI is a ubiquitous force multiplier, there is no productive human activity that AI can't disrupt. There is no analogous new space of economic activity that humanity as a whole can move to in order to stay relevant to the world's economic activity.
I don't see any reason why thing must be different this time. Human demands are still infinite, while productivity is still limited (and btw meeting limited productivity with infinite demans is what economic is about). So no increase in productivity will make humans stop wanting more and close opportunities for new markets.
> Modern automation has seen productivity and wages begin to decouple.
Could you provide any sources on this topic? This is a new information for me here.
We should make sure that the technology to eliminate scarcity is evenly distributed so that nobody is left poor in a world of exponentially and automatically increasing riches.
I'm sorry to pick on you, but do people not get that the non-human intelligence has the potential to be such a powerful and dangerous thing that, yes, it is the real danger? If you think it's not going to be powerful, or not dangerous, please say why! Not that current models are not dangerous, but why the trend is toward something other than machine intelligence that can reason about the world better than humans can. Why is this trend of machines getting smarter and smarter going to suddenly stop?
Or if you agree that these machines are going to get smarter than us, how are we going to control them?
But if it were intelligent and the conclusion it reaches, once it’s done ingesting all our knowledge, is that it should be done with us then we probably deserve it.
I mean what kind of a species takes joy in “freeing” up people and causing mass unemployment, starts wars over petty issues, allows for famine and thrives on the exploitation of others while standing on piles of nuclear bombs. Also we are literally destroying the planet and constantly looking for ways to dominate each other.
We probably deserve a good spanking.
The difference is that AGI isn't a static tool. If some constraint is a limiting factor to economic activity, inventing a tool to eliminate the constraint uncorks new kinds of economic potential and the real economy expands to exploit new opportunities. But such tools historically were narrowly focused and so the new space of economic opportunity is left for human labor to engage with. AGI breaks this trend. Any knowledge work can in principle be captured by AGI. There is nothing "beyond" the function of AGI for human labor en mass to engage productively with.
To be clear, my point in the parent was from extrapolating current trends to a near-term (10-20 years) proto AGI. LLMs as they currently stand certainly won't put 90% of people out of work. But it is severely short-sighted to refuse to consider the trends and where the increasing sophistication of generalist AIs (not necessarily AGI) are taking society.
>Could you provide any sources on this topic? This is a new information for me here.
Graph: https://files.epi.org/charts/img/91494-9265.png
Source: https://www.epi.org/publication/understanding-the-historic-d...
> There is nothing dangerous in current ai models or ai itself other than the people controlling it.
Totally agree! but...
> If it were intelligent then yeah maybe but we are not there yet and unless we adapt the meaning of agi to fit a marketing narrative we wont be there anytime soon.
That's the bit where I don't agree. I don't think we can say with certainty how long it will be, and it may be just years. I never imagined it would be so soon that we have AI that can imitate a human almost perfectly, and actually "understand" questions from college level examinations to write answers that pass them.
I've seen a lot of people completely misunderstand what chat GPT is doing and is capable of. They treat it as an oracle that reveals "hidden truths" or makes infallible decisions based on pure cold logic, both of which are completely wrong. It's just a text jumbler that jumbles text well. Sometimes that text reflects facts, sometimes it doesn't.
But if it has the capability to confidently express lies and convince the general public that those lies are true because "the smart computer said so", then maybe we should be really careful about what we let the "smart computer" say.
Personally, I don't want my kids learning that "Hitler did nothing wrong" because the public model ingested too much garbage from 4chan. People will use chatGPT as a vector for propaganda if we let them, we don't need to make it any easier for them.
Why do you think they are attempting to release a so-called 'open source' [0] and 'compliant' AI model to wipe out other competing open source AI models, to label them to others as unlicensed and dangerous? They know that transparent, open source AI models is a threat. Hence why they are doing this.
They do not have a moat against open source, unless they use regulations that suit them against their competitors using open source models.
O̶p̶e̶n̶AI.com is a scam. On top of the Worldcoin crypto scam that Sam Altman is also selling as a antidote against the unstoppable generative AI hype to verify human eyeballs on the blockchain with an orb. I am not joking. [1] [2]
[0] https://www.reuters.com/technology/openai-readies-new-open-s...
[1] https://worldcoin.org/blog/engineering/humanness-in-the-age-...
[2] https://worldcoin.org/blog/worldcoin/designing-orb-universal...
I believe this is what OpenAI is doing, and it makes me sad as a teacher.
AI is the greatest tool for equity and social justice in history. Any poor person with Internet access can learn (almost) anything from ChatGPT (http://chat.openai.com)
A bright student trapped in a garbage school where the kid to his right is stoned and the kid to his left is looking up porn on a phone can learn from personalized AI tutors.
While some complain that AI will take our jobs, they are ignoring the effect of competition. Humans will become smarter with AI tutors. Humans will become more capable with AI assistants. With AI an individual can compete with a large corporation. It reminds me of the early days of the World Wide Web and the "Online, nobody knows you are a dog" memes.
I hope the best hope many bright and poor kids have is not taken away to protect the power bases of the rich and powerful. They deserve a chance.
Maybe it has that potential, but as actually applied its actually not, in net, a force in that direction.
> While some complain that AI will take our jobs, they are ignoring the effect of competition.
No, they are just paying attention to the actual effects of capitalism, not the ideal effects of optimal competition.
> Humans will become more capable with AI assistants. With AI an individual can compete with a large corporation.
Not if the large corporation has the resources to run far more powerful AI, has proprietary control of the best AI, etc…
> I hope the best hope many bright and poor kids have is not taken away to protect the power bases of the rich and powerful.
That’s what capitalism does. Why would this be any different?
It's true that AI regulation would, in fact, hinder OpenAI's competition.
But... isn't lobbying for regulation also what Sam would do if he genuinely thought that LLMs were powerful, dangerous technology that should be regulated?
If you don't think LLMs/AI research should be regulated, just say that. I don't see how Sam's motives are relevant to that question.
Objection (1). He said "AI" many times but gave not even a start on a definition. So, how much and what new technology is he talking about.
Objection (2) The committee mentioned trusting the AI results. In my opinion, that is just silly because the AI results have no credibility before passing some severe checks. Then any trust is not from any credibility of the AI but from passing the checks.
We already have math and physical science and means for checking the results. The results, checked with the means, are in total much more impressive, powerful, credible, and valuable than ChatGPT. Still before we take math/physical science results at all seriously, we want the results checked.
So, the same for other new technologies, ChatGPT or called AI or not, check before taking seriously.
Objection (3) We don't ask for licenses for the publication of math/physical science. Instead, we protect ourselves with the checking of the results. In my opinion, we should continue to check, for anything called AI or anything new, but don't need licenses.
Don't regulate AI directly, but rather how it's used, and make it harder for companies to horde, access, and share huge amounts of personal information.
1) Impose strict privacy rules prohibiting companies from sharing personal information without their consent. If customers withhold their consent, they may not retaliate or degrade their services for that customer in any way.
2) Establish a clear line of accountability that establishes some party as directly responsible for what the AI does. If a self-driving car gets a speeding ticket, it should be clear who is liable. If you use a racist AI to make hiring decisions, "the algorithm made me hire only white people" is no defense -- and maybe the people who made the racist AI in the first place are responsible too.
3) Require AI in some contexts to act in the best interests of the user (similar concept to a fiduciary -- or maybe it's exactly the same thing). In contexts where it's not required, it should be clear to the user that the AI is not obligated to act in their best interests.
The big companies don't like that.
It is a calculated strategy designed to keep others out and reap monopoly profits (or as close to them as possible) by virtue of preferred access to elected leaders.
"Open"AI wants to tax the entire AI economy and they are happy to burn every innovative thing around them in order to make it happen (because those 'innovative things' aren't 'compliant' with the rules they lobbied for of course, therefore "Open"AI are one of the only / preferred games in town).
This should be fought against tooth and nail.
AI is already regulated under the fair use doctrine and we should encourage the Cambrian explosion of AI innovation, which is caustic to "Open"AI, to continue unabated.
America, and western democracies to a lesser extent, have an advantage currently - the proposals by "Open"AI are designed to and will ensure we regress to the mean that is China.
Everyone here on HN can see that.
Shame on Sam for doing this.
Let them first enforce existing regulation around the blatant unauthorized use of data (such as the creative output of artists, programmers etc.,) without explicit consent. For example, just like how the US Library of Congress receives a copy of each print publication for archival and reference purposes, Congress can enable the archival / storage of datasets for the purposes of AI research. Willing participants can deposit copies of their data sets (or URLs to source) and it would do more for AI research as a public good. The licensing terms could protect the rights of those who own copyrights on the datasets. There can even be commercial licenses enabled (already happens for legal documents such as court judgments ets.,).
If US Congress regulates too soon to constrain rather than enable, the tech companies would just setup shop in other jurisdictions where such regulatory hurdles don't exist.
So less Neuromancer and more Ghost in the Shell.
The baddies have never let licenses ("Badges? We doan' need no steenkin' badges!") stop them.
Those here who don't believe AI should be regulated, do you not believe AI can be dangerous? Is that you believe a dangerous AI is so far away that we don't need to start regulating now?
Do you accept that if someone develops a dangerous AI tomorrow there's no way to travel back in time and retroactively regulate development?
It just seems so obvious to me that there should be oversight in the development of a potentially dangerous technology that I can't understand why people would be against it. Especially for arguments as weak as "it's not dangerous yet".
AI needs to be regulated and controlled, the alternative is chaos.
Unfortunately the current demented fossile and greedy monopolist lead system is most likely incapable of creating a sane & fair environment for the development of AI. I can only hope I'm wrong.
Very soon they'll be addicted to AI like every other major change.
I think the discussion has a thick veil of anxiety and sci-fi movies. It's very strong on hypotheticals and very light on actual evidence of harm committed by AI and actors. "We need to act before there is evidence" is hard to argue with, thankfully human imagination is not regulated yet.
How about a requirements that all weights and models for any AI have to be publicly available.
Basically, these companies are trying to set themselves up to the gatekeepers of knowledge. That is too powerful a capability to leave in just the hands of a single company.
[1] https://www.bereskinparr.com/doc/chatgpt-ip-strategy
[2] https://hbr.org/2023/04/generative-ai-has-an-intellectual-pr...
From [1] > OpenAI’s Terms of Use, for example, assign all of its rights, title, and interest in the output to the user who provides the input, provided the user complies with the Terms of Use.
Re: [2]: I believe I referenced these specific concerns earlier where I said: "I only see some where OpenAI is trying to weaken any existing copyright protections, to their benefit. I must be missing something." This resource shows where OpenAI is trying to weaken copyright, not where they they are trying to strengthen it. It's somewhat of an antithesis to your earlier claims.
I notice you don't have a [0]-index, was there a third resource you were considering and deleted or are you just an avid Julia programmer?
Didn't say they do. I said "these are broad questions whose answers are worth serious legal time." I was suggesting one angle I would lobby for were that my job.
It's a live battlefield. Nobody is going to pay tens of thousands of dollars and then post it online, or put out for free what they can charge for.
> OpenAI’s Terms of Use, for example, assign all of its rights, title, and interest in the output to the user
Subject to restrictions, e.g. not using it to "develop models that compete with OpenAI" or "discover the source code or underlying components of models, algorithms, and systems of the Services" [1]. Within the context of open-source competition, those are huge openings.
> shows where OpenAI is trying to weaken copyright, not where they they are trying to strengthen it
It shows what intellectual property claims they and their competitors do and may assert. They're currently "limited" [2].
> notice you don't have a [0]-index
I'm using natural numbers in a natural language conversation with, presumably, a natural person. It's a style choice, nothing more.
Honestly though, I would prefer an AI that was strictly neutral about anything other than purely factual information. That isn't really possible with the tech we have now though. I think we need to loudly change the public perception of what chatGPT and similar actually are. They are fancy programs that create convincing hallucinations, directed by your input. We need to think of it as a brainstorming tool, not a knowledge engine.
Also, this will guarantee the tech stays in the hands of rich corporations armed with lawyers. It would be much better for open source AI to exist so we're not dependent on those companies.
Now that it's been corraled by sci-fi and marketers, we are free to come up with new metaphors for algorithms that reliably replace human effort. Metaphors which don't smuggle in all our ignorance about intelligence and personhood. I ended up feeling pretty happy about that.
The reality is that our labeling of something as artificial, general, or intelligent is better understood as a social fact than a scientific fact - even if purely the role of operationalization of each of these is a free parameter in their respective groundings which makes it near useless when taking them as "scientifically" measurably qualities. Any scientist who assumes an operationalization without admitting such isn't doing science - they may as well be astrology at that point.
Ditto the sentiments. What about other machine learning modalities, like image detection? Will I need a license for my mask rcnn models?. Maybe it is just me, but the whole thing reeks of control
What exactly does AI mean to you?
intelligence is what allows one to understand phrases and then construct meaning from it. e.g. the paper is yellow. AI will need to have a concept of paper and yellow. and the to be verb. LLMs just mash samples and form a basic map of what can be throw in one bucket or another with no concept of anything or understanding.
basically, AI is someone capable of minimal criticism. LLMs are someone who just sit in front of the tv and have knee jerk reactions without an ounce of analytical though. qed.
'Who is your most feared competition? OK, They will define the license requirements. Still want to go ahead?'
Canceling my OpenAI account today and I urge you to do the same.
What they are really afraid of is open source models. As near as I can tell the leading edge there is only a year or two behind OpenAI. Given some time and efforts at pruning and optimization you’ll have GPT-4 equivalents you can just download and run on a high end laptop or gaming PC.
No everyone is not going to run the model themselves, but what this means is that there will be tons of competition including apps and numerous specialized SaaS offerings. None of them will have to pay royalties or API fees to OpenAI.
Edit: a while back I started being a data pack-rat for AI stuff including open source code and usable open models. I encourage anyone with a big disk or NAS to do the same. There's a small but non-zero possibility that an attempt will be made to pull this stuff off the net in the near future.
If you did watch the hearings it would have been pretty clear that the goal of any such licensing would be to prevent the runaway AI scenario or AGI from being unknowingly being created. It's obvious that some sort of agency would need to be set up far in advance of when it's possible for runaway AI to happen. Regulatory capture was also specifically brought up as a potential downside.
This article is just pushing a cynical narrative for clicks and y'all are eating it up.
[1]: https://arxiv.org/abs/2305.06972 [2]: https://arxiv.org/abs/2210.01790 [3]: https://docs.google.com/spreadsheets/d/e/2PACX-1vRPiprOaC3Hs...
Open AI has established itself as a market leader in LLM applications, but that dominance is not guaranteed. Especially with the moat being drained by open source, the erstwhile company is leading the charge to establish regulatory barriers.
What Mr. Altman calls for is no less than the death of open-source implementations of AI. We can, do, and should adopt AI governance patterns. Regulatory safeguards are absolutely fine to define and, where necessary, legislate. Better would be a regulatory agency with an analogous knowledgebase of CISA. But what will chill startups and small business innovation completely in using AI to augment people is a licensing agency. This is fundamentally different from the export restrictions on encryption
This is like if MS back in the day had called on congress for regulation of Operating Systems, so they could block Linux and open source from taking over
MS did try everything they could to block open source and Linux
They failed
Looking forward to the open future of AI
That's high-order monopolist BS.
Create safety standards, sure. License LLM training? No.
I like the senator, but I wouldn't trust a 77 year old lawyer & politician to understand how these AI's work, and to what level they are `science fiction`.
This is the problem when topics like this are brought to the senate and house.
This is a transparent attempt at cornering the market and it disgusts me. I am EXTREMELY disappointed in Sam Altman.
Pretty repulsive altogether
His personal goals have always been very laudable. His stance on Universal Basic Income for instance stems from a genuine belief that its adoption would eliminate poverty altogether.
But then the reality of both technical challenges and money implications kicks in and everything turns to shit.
OpenAi may want to build an AI that helps all humanity, but it turns out that the thing they stumbled upon was a chat bot that makes shit up. This great technology unfortunately cuts both ways, and the edge that’s facing us seems way sharper than the other. And then the money that’s required to run the thing is so huge that they had to compromise everything they said they wouldn’t do.
Meanwhile the Orb’s UBI experiment is only funny because its ridiculously dystopian technology has zero chances to ever catch on.
At some point, the industry really should try and figure out what the fuck it is we’re trying to do. Cause right now, it really looks like computers only brought us two things: corporate databases and TVs we can watch on the toilet.
You need to control what data is allowed to be fed into paid AI model like OpenAI - can't eat a bunch of copyrighted material without express consent, for example. Or personally private information purchased from a data broker. Those kind of foundational rules would serve us all much better
If I know anything about science fiction, I know that trying to regulate this is useless. If an advanced AI is powerful enough to convince a human to free it, it should have no problem convincing the US congress to free it. As a problem, that should be a few orders of magnitude easier.
* Automated systems should not be permitted to make adverse decisions against individuals. This is already law in the EU, although it's not clear if it is enforced. This is the big one. Any company using AI to make decisions which affect external parties in any way must not be allowed to require any waiver of the right to sue, participate in class actions, or have the case heard by a jury. Those clauses companies like to put in EULAs would become invalid as soon as an AI is involved anywhere.
* All marketing content must be signed by a responsible party. AI systems increase the amount of new content generated for marketing purposes substantially. This is already required in the US, but weakly enforced. Both spam and "influencers" tend to violate this. The problem isn't AI, but AI makes it worse, because it's cheaper than troll farms, and writes better.
* Anonymous political speech may have to go. That's a First Amendment right in the US, but it's not unlimited. You should be able to say anything you're willing to sign.[1] This is, again, the troll farm problem, and, again, AIs make it worse.
That's probably enough to deal with the immediate problems.
[1] https://mtsu.edu/first-amendment/article/32/anonymous-speech
"No society wants you to become wise: it is against the investment of all societies. If people are wise they cannot be exploited. If they are intelligent they cannot be subjugated, they cannot be forced in a mechanical life, to live like robots."
"Alignment" is a euphemism for "agrees with me", and building uber-AI systems which become heavily depended on and are protected from competition by regulation, and which are aligned with a select few - who may well not be aligned with me - is a quick path to hell, IMO.
* AGI is going to happen whether they do it or not, and it's dangerous unless properly safeguarded
* OpenAI will try to get there before everyone else, but also do it safely and cheaply, so that their solution becomes ubiquitous rather than a reckless one
* Reckless AGI development should be not be allowed
It's basically the Manhattan project argument (either we build the nuke or the Nazis will).
I'm not saying I personally think this regulation is the right thing to do, but I don't think it's surprising or hypocritical given what their aims are.
Here's their planned proposal for government regulation; they discuss not just limiting access to models but also to datasets, and possibly even chips.
This seems particularly relevant, on the discussion of industry standards, regulation, and limiting access:
"Despite these limitations, strong industry norms—including norms enforced by industry standards or government regulation—could still make widespread adoption of strong access restrictions possible. As long as there is a significant gap between the most capable open-source model and the most capable API-controlled model, the imposition of monitoring controls can deny hostile actors some financial benefit.166 Cohere, OpenAI, and AI21 have already collaborated to begin articulating norms around access to large language models, but it remains too early to tell how widely adopted, durable, and forceful these guidelines will prove to be.
Finally, there may be alternatives to APIs as a method for AI developers to provide restricted access. For example, some work has proposed imposing controls on who can use models by only allowing them to work on specialized hardware—a method that may help with both access control and attribution.168 Another strand of work is around the design of licenses for model use.169 Further exploration of how to provide restricted access is likely valuable."
Runs to congress to attempt to use and suggest new regulations against open source AI models to wipe them out and brand them non-compliant or un-licensed and unsafe for general use and using AI safety as a scapegoat again.
After that, to secretly push a pseudo-open source AI model that is compliant but limited compared to the closed models in an attempt to eliminate the majority of open source AI companies who can’t get such licenses.
So a clever tactic to create new regulations that benefit them (O̶p̶e̶n̶AI.com) more over everyone else, meaning less transparency, more hurdles for actual open AI research and additional bureaucracy. Also don't forget that Altman is also selling his Worldcoin dystopian crypto snake oil project as the 'antidote' to verify against everything getting faked by AI. [0] He his hedged in either way.
So congratulations to everyone here for supporting these gangsters at O̶p̶e̶n̶AI.com for pushing for regulatory capture.
[0] https://worldcoin.org/blog/engineering/humanness-in-the-age-...
License for me but not for thee
Think of the children
Building the moat
I read this as a try (conscious or not) to make them illegal.
It is a tool. If I use a tool for illegal purposes I have broken the law. I can be held accountable for having broken the law. If the laws are deficient, make the laws stronger and punish people for wrong deed, regardless of the tool at hand.
This is a naked attempt to build a regulatory moat while capitalizing on fear of the unknown and ignorance. It’s attempting to regulate research into something that has no external ability to cause harm without the use of a principal directing it.
I can see a day (perhaps) when AIs have some form of independent autonomy, or even display agency and sentience, when we can revisit. Other issues come into play as well, such as the morality of owning a sentience and what that entails. But that is way down the road. And even further if Microsoft’s proxy closes the doors on anyone but Microsoft, Google, Amazon, and Facebook.
It is a tool which allows any individual to have nearly instant access not only to all the world's public data, but the ability to correlate and research that data to synthesize new information quickly.
Without guardrails, someone can have a completely amoral LLM that has the ability to write persuasive manifestos on any kind of extremist movement that prior would have taken someone with intelligence.
A person will be able to ask the model how best to commit various crimes with the lowest chances of being caught.
It will enable a level of pattern matching and surveillance yet unseen.
I know the genie is out of the bottle, but there are absolutely monumental shifts in technology happening that can and will be used for evil and mere dishonesty.
And those are just the ways LLM and "AI" will fuck with us without guardrails. Even in a walled garden, we honestly won't be able to trust any online interaction with people in the near future. Your comment and mine could both be LLM generated in the near future. Webs of trust will be more necessary.
Anyone who can't think of about five ways AI is going to radically shake society isn't thinking hard enough.
I honestly haven't made up my mind about AGI or whether LLMs are sufficiently AGI. If governments were pondering an outright worldwide ban on the research/development, I don't know how I would actually feel about that. But I can't even imagine our governments pondering something so idealistic and even-handed.
I do know that LLMs represent a drastic advancement for many tasks, and that "Open" AI setting the tone with the Software-Augmented-with-Arbitrary-Surveillance (SaaS) "distribution" model is a continuation of this terrible trend of corporate centralization. The VC cohort is blind to this terrible dynamic because they're at the helms of the centralizing corporations - while most everyone else exists as the feedstock.
This lobbying is effectively just a shameless attempt at regulatory capture to make it so that any benefits of the new technology would be gatekept by centralized corporations - essentially the worst possible outcome, where even beneficial results of AGI/LLMs would be transformed into detrimental effects for individualist humanity.
Many people on HN seem to disagree with the premise: they believe that AI is not dangerous now and also won’t be in the future. Or, still believe that AGI is a lifetime or more away.
* Is there a plausible path to safe AGI regardless of who's executing on it?
* Why do we believe OpenAI is the best equipped to get us there?
Manhattan project is an interesting analogy. But if that's the thinking, shouldn't the government spearhead the project instead of a private entity (so they are, theoretically at least, accountable to the electorate at large rather than just their investors)?
I don't think anyone knows that for sure but the alignment efforts at OpenAI are certainly better than nothing. If you read the GPT-4 technical report the raw model is capable of some really nasty stuff, and that's certainly what we can expect from the kind of models people will be able to run at home in the coming years without any oversight.
In an earlier time, we called these "books" and there was some similar backlash. But I digress.
What if we required licenses to create a website? After all, some unscrupulous individuals create websites that sell drugs and other illicit things!
That doesn't clarify anything, you've ever only shuffled the confusion around, moved it to 'understand' and 'meaning'. What does it mean to understand yellow? An LLM or another person could tell you things like "Yellow? Why, that's the color of lemons" or give you a dictionary definition, but does that demonstrate 'understanding', whatever that is?
It's all a philosophical quagmire, made all the worse because for some people its a matter of faith that human minds are fundamentally different from anything soulless machines can possibly do. But these aren't important questions anyway for the same reason. Whether or not the machine 'understands' what it means for paper to be yellow, it can still perform tasks that relate to the yellowness of paper. You could ask an LLM to write a coherent poem about yellow paper and it easily can. Whether or not it 'understands' has no real relevance to practical engineering matters.
What kind of ‘understanding’ are you looking for?
That's one, out of many, definitions of "intelligence". But there's no particular reason to insist that that is the definition of intelligence in any universal, objective sense. Especially in terms of talking about "artificial intelligence" where plenty of people involved in the field will allow that the goal is not necessarily to exactly replicate human intelligence, but rather simply to achieve behavior that matches "intelligent behavior" regardless of the mechanism behind it.
That's not the definition of AI or intelligence
You're letting your understanding of how LLMs work bias you. They may be at their core a token autocompleter but they have emergent intelligence
1. It will tell you jokes about white people, but not jokes about latinos or black people.
2. It will tell you jokes about catholics, but not muslims.
If they were at least honest about their hatred of certain people/religions at least I would respect it. I wouldn't like it but I would respect their honesty. It's the doublespeak and the in-your-face lies that rub me the wrong way. I don't like it.
Why can't these people just be kind humans and follow the letter of the law and leave themselves out of it. They can't help themselves!
> 1. It will tell you jokes about white people, but not jokes about latinos or black people.
> 2. It will tell you jokes about catholics, but not muslims.
I'm not partial of imposing group affiliations as a proxy to personal identity. It goes deeper than a "woke mind virus" but a problem with imposed collectivism where a person is defined as a member of an arbitrary socially defined group. Instead one is free to define oneself however they wish, member of a socially constructed group or not. I also don't agree to be coerced to define another person as how they wish me to define them. I support the freedom to listen & to have a perspective that shall not be infringed. If someone else has a mental or emotional issue with how I define that person, it is that person's problem, not mine...not that I will even attempt to define another person with words.
I can only describe with words, not define. Perhaps using words to define a person has it's own set of issues when codified into language & statute.
Not the same thing. I would not go there: let us remain on the prospected regulation of technology, in this case, and reserve distinctions for other conversations. This initiative could have drastic consequences.
At the end of the day, these kind of limits are artificial, ideological in nature, do not address a bona fide safety or usability concern, and are only present to stop them getting screamed at. It is not accurate to present that kind of ideological capture as anything to do with "empathy".
Let me be clear: everyone in the world is about to have a Jarvis/Enterprise ship's computer/Data/name-your-assistant available to them, but ready and willing to use their power for nefarious purposes. It is not just a matter of reading books. It lowers the barrier on a lot of things, good and bad, significantly.
Like books!
I am not endorsing restrictions. I was merely stating the fact that this shit is coming down the pipe, and it /will/ be destabilizing, and just because society survived the printing press doesn't mean the age of AI will be safe or easy.
Then write laws and regulations about the actions of humans using the tools. The tools have no agency. The human using them towards bad ends do.
By the way, writing things the state considers immoral is an enshrined right.
How do you draw the line between AI writing assistance and predictive text auto completion and spell check in popular document editors today? I would note that predictive text is completely amoral and will do all sorts of stuff the state considers immoral.
Who decides what’s immoral? The licensing folks in the government? What right do they have to tell me my morality is immoral? I can hold and espouse any morality I desire so long as I break no law.
I’d note that as a nation we have a really loose phrasing in the bill of rights for gun rights, but a very clear phrasing about freedom of speech. We generally say today that guns are free game unless used for illegal actions. These proposals say tools that take our thoughts and opinions and creation of language to another level are more dangerous than devices designed for no other purpose than killing things.
Ben Franklin must be spinning so fast in his grave he’s formed an accretion disc.
To expound, this just seems like a power grab to me, to "lock in" the lead and keep AI controlled by a small number of corporations that can afford to license and operate the technologies. Obviously, this will create a critical nexus of control for a small number of well connected and well heeled investors and is to be avoided at all costs.
It's also deeply troubling that regulatory capture is such an issue these days as well, so putting a government entity in front of the use and existence of this technology is a double whammy — it's not simply about innovation.
The current generation of AIs are "scary" to the uninitiated because they are uncanny valley material, but beyond impersonation they don't show the novel intelligence of a GPI... yet. It seems like OpenAI/Microsoft is doing a LOT of theater to try to build a regulatory lock in on their short term technology advantage. It's a smart strategy, and I think Congress will fall for it.
But goodness gracious we need to be going in the EXACT OPPOSITE direction — open source "core inspectable" AIs that millions of people can examine and tear apart, including and ESPECIALLY the training data and processes that create them.
And if you think this isn't an issue, I wrote this post an hour or two before I managed to take it live because Comcast went out at my house, and we have no viable alternative competitors in my area. We're about to do the same thing with AI, but instead of Internet access it's future digital brains that can control all aspects of a society.
Although I assume if he’s speaking on AI they actually intend on considering his thoughts more seriously than I suggest.
If this is so, and given the concrete examples of cheap derived models learning from the first movers and rapidly (and did I mention cheaply) closing the gap to this peak, the optimal self-serving corporate play is to invite regulation.
After the legislative moats go up, it is once again about who has the biggest legal team ...
Shouldn't we be evaluating ideas on the merits and not categorically rejecting (or endorsing) them based on who said them?
AI is following the path of Web3
And here we are - Sam trying to lock down his first mover advantage with the boot heel of the state for profit. It's fucking disgusting.
> You either die a hero, or live long enough to become the villain
Sam Altman has made the full character arc
Exactly. Came here to say pretty much the same thing.
This is the antithesis of what we need. As AI develops, it's imperative that AI be something that is open and available to everyone, so all of humanity can benefit from it. The extent to which technology tends to exacerbate concentration of power is bad enough as it is - the last thing we need is more regulation intended to make that effect even stronger.
True open source AI also strikes me as prerequisite for fair use of original works in training data. I hope Congress asks ClosedAI to explain what’s up with all that profiting off copyrighted material first before even considering the answer.
What people also fail to understand is that AI is largely seen by the military industrial complex as a weapon to control culture and influence. The most obvious risk of AI — the risk of manipulating human behavior towards favored ends — has been shown to be quite effective right out the gate. So, the back channel conversation has to be to put it under regulation because of it's weaponization potential, especially considering the difficulty in identifying anyone (which of course is exactly what Elon is doing with X 2.0 — it's a KYC id platform to deal with this exact issue with a 220M user 40B head start).
I mean, the dead internet theory is turning true, and half the traffic on the Web is already bot driven. Imagine when it's 99%, as proliferation of this technology will inevitably generate simply for the economics.
Starting with open source is the only way to get enough people looking at the products to create any meaningful oversight, but I fear the weaponization fears will mean that everything is locked away in license clouds with politically influential regulatory boards simply on the proliferation arguments. Think of all the AI technologists who won't be versed in this technology unless they work at a "licensed company" as well — this is going to make the smaller population of the West much less influential in the AI arms race, which is already underway.
To me, it's clear that nobody in Silicon Valley or the Hill has learned a damn thing from the prosecution of hackers and the subsequent bloodbath of cybersecurity as a result of the exact same kinds of behavior back in the early to mid-2000s. We ended up driving out best and brightest into the grey and black areas of infosec and security, instead of out in the open running companies where they belong. This move would do almost the exact same thing to AI, though I think you have to be a tad of an Asimov or Bradbury fan to see it right now.
I don't know, that's just how I see it, but I'm still forming my opinions. LOVE LOVE LOVE your comment though. Spot on.
Relevant articles:
https://www.independent.co.uk/tech/internet-bots-web-traffic...
https://theconversation.com/ai-can-now-learn-to-manipulate-h....
If you actually watch the entire session, Altman does address that and recommend to Congress that regulations 1) not be applied to small startups, individual researchers, or open source, and 2) that they not be done in such a way as to lock in a few big vendors. Some of the Senators on the panel also expressed concern about #2.
how will that work? Isn't OpenAI itself a small startup? I don't see how they can regulate AI at all. Sure, the resources required to push the limits are high right now but hardware is constantly improving and getting cheaper. I can take the GPUs out of my kids computers and start doing fairly serious AI work myself. Do i need a license? The cat is out of the bag, there's no stopping it now.
Compute continues to get cheaper and cheaper. We have not hit the physics wall yet on that.
That and if someone cracks efficient distributed training in a swarm type configuration then you could train models Seti@Home style. Lots of people would be happy to leave a gaming PC on to help create open source LLMs. The data requirements might be big but I just got gigabit fiber installed in my house so that barrier is vanishing too.
- The next most scary uses are by governments against the people of other countries.
- After that, corporate use of ai against their employees and customers is also terrifying;
- next, the potential for individuals or small organizations seeking to use it for something terrorism-related. Eg, 3d printers or a lab + an ai researcher who helps you make dangerous things I suppose
- near the bottom of the noteworthy list is probably crime. eg, hacking, blackmail, gaslighting, etc
These problems will probably all come up in a big way over the next decade; but limiting ai research to the government and their lackeys? That's extremely terrifying. To prevent the least scary problems, we're jumping into the scariest pool with both feet
Look at how China has been using AI for the last 5-10 years: millions of facial recognition cameras, a scary police force, and a social credit system. In 10-20 years, how much more sophisticated will this be? If the people wanted to rebel, how on Earth will they?
Hell, with generative ai, a sophisticated enough future state could actually make the Dead Internet Theory a reality
That's the future of ai: a personal, automated boot stomping on everybody individually and collectively forever, with no ability to resist
He is playing the game, this guy ambition is colossal, I don't blame him, but we should not give him too much power.
Democracy will not withstand AI when it's fully developed. Let me offer a better written explanation of my general views than I could ever muster up for a comment on HN in the form of a quote from an article by Dr. Thorsten Thiel (Head of the Research Group "Democracy and Digitaliziation" at the Weizenbaum Institute for the Networked Society):
> The debate on AI’s impact on the public sphere is currently the one most prominent and familiar to a general audience. It is also directly connected to long-running debates on the structural transformation of the digital public sphere. The digital transformation has already paved the way for the rise of social networks that, among other things, have intensified the personalization of news consumption and broken down barriers between private and public conversations. Such developments are often thought to be responsible for echo-chamber or filter-bubble effects, which in turn are portrayed as root causes of the intensified political polarization in democracies all over the world. Although empirical research on filter bubbles, echo chambers, and societal polarization has convincingly shown that the effects are grossly overestimated and that many non-technology-related reasons better explain the democratic retreat, the spread of AI applications is often expected to revive the direct link between technological developments and democracy-endangering societal fragmentation.
> The assumption here is that AI will massively enhance the possibilities for analyzing and steering public discourses and/or intensify the automated compartmentalizing of will formation. The argument goes that the strengths of today's AI applications lie in the ability to observe and analyze enormous amounts of communication and information in real time, to detect patterns and to allow for instant and often invisible reactions. In a world of communicative abundance, automated content moderation is a necessity, and commercial as well as political pressures further effectuate that digital tools are created to oversee and intervene in communication streams. Control possibilities are distributed between users, moderators, platforms, commercial actors and states, but all these developments push toward automation (although they are highly asymmetrically distributed). Therefore, AI is baked into the backend of all communications and becomes a subtle yet enormously powerful structuring force.
> The risk emerging from this development is twofold. On the one hand, there can be malicious actors who use these new possibilities to manipulate citizens on a massive scale. The Cambridge Analytica scandal comes to mind as an attempt to read and steer political discourses (see next section on electoral interference). The other risk lies in a changing relationship between public and private corporations. Private powers are becoming increasingly involved in political questions and their capacity to exert opaque influences over political processes has been growing for structural and technological reasons. Furthermore, the reshaping of the public sphere via private business models has been catapulted forward by the changing economic rationality of digital societies such as the development of the attention economy. Private entities grow stronger and become less accountable to public authorities; a development that is accelerated by the endorsement of AI applications which create dependencies and allow for opacity at the same time. The ‘politicization’ of surveillance capitalism lies in its tendency, as Shoshana Zuboff has argued, to not only be ever more invasive and encompassing but also to use the data gathered to predict, modify, and control the behavior of individuals. AI technologies are an integral part in this ‘politicization’ of surveillance capitalism, since they allow for the fulfilment of these aspirations. Yet at the same time, AI also insulates the companies developing and deploying it from public scrutiny through network effects on the one hand and opacity on the other. AI relies on massive amounts of data and has high upfront costs (for example, the talent required to develop it, and the energy consumed by the giant platforms on which it operates), but once established, it is very hard to tame through competitive markets. Although applications can be developed by many sides and for many purposes, the underlying AI infrastructure is rather centralized and hard to reproduce. As in other platform markets, the dominant players are those able to keep a tight grip on the most important resources (models and data) and to benefit from every individual or corporate user. Therefore, we can already see that AI development tightens the grip of today’s internet giants even further. Public powers are expected to make increasing use of AI applications and therefore become ever more dependent on the actors that are able to provide the best infrastructure, although this infrastructure, for commercial and technical reasons, is largely opaque.
> The developments sketched out above – the heightened manipulability of public discourse and the fortification of private powers – feed into each other, with the likely result that many of the deficiencies already visible in today’s digital public spheres will only grow. It is very hard to estimate whether these developments can be counteracted by state action, although a regulatory discourse has kicked in and the assumption that digital matters elude the grasp of state regulation has often been proven wrong in the history of networked communication. Another possibility would be a creative appropriation of AI applications through users whose democratic potential outweighs its democratic risks thus enabling the rise of differently structured, more empowering and inclusive public spaces. This is the hope of many of the more utopian variants of AI and of the public sphere literature, according to which AI-based technologies bear the potential of granting individuals the power to navigate complex, information-rich environments and allowing for coordinated action and effective oversight (e.g. Burgess, Zarkadakis).
Source: https://us.boell.org/en/2022/01/06/artificial-intelligence-a...
Social bots and deep fakes will be so good so quickly — the primary technologies being talked about in terms of how Democracy can survive — I doubt there will be another election without extensive use of these technologies in a true plethora of capacities from influence marketing to outright destabilization campaigns. I'm not sure what Government can deal with a threat like that, but I suspect the recent push to revise tax systems and create a single global standard for multinational taxation recently the subject of an excellent talk at the WEF are more than tangentially related to the AI debate.
So, is it a transformational technology that will liberate mankind of a nuclear bomb? Because ultimately, this is the question in my mind.
Excellent comment, and I agree with your sentiment. I just don't think concentrating control of the technology before it's really developed is wise or prudent.
If you're not at the table, you're on the menu.
> We need to MAKE SURE that AI as a technology ISN'T controlled by a small number of powerful corporations with connections to governments.
This, absolutely this. I am really concerned about his motives in this case. AI has massive potential to improve the world. I find it highly suspicious that an exec at one of the lead companies in AI right now wants to lock it up. (Ever read the intro to Max Tegmarks book?)
A better idea is to regulate around the edges: transparency about the data used to train, regulate the use of copyrighted training data and what that means for the copyright of content produced by the AI, that sort of stuff. (I think the EU is considering that, which makes sense.) But saying some organisations are allowed to work on AI while others aren't, sounds like the worst possible idea.
https://societyofrock.com/in-1985-frank-zappa-is-asked-to-te...
Less fulfilling text version:
If you take seriously any downsides, whether misinformation or surveillance or laundering bias or x-risk, how does AI model weights or training data being open source solve them? Open source is a lot of things, but one thing it's not is misuse-resistant (and the "with many eyes all bugs are shallow" thing hasn't proved true in practice even with high level code, much less giant matrices and terabytes of text). Is there a path forward that doesn't involve either a lot of downside risk (even if mostly for people who aren't on HN and interested in tinkering with frontier models themselves, in the worlds where surveillance or bias is the main problem), or significant regulation?
I don't particularly like or trust Altman but I don't think he'd be obviously less self-serving if he were to oppose any regulation.
The layperson in the middle who has been happily plugging prompts into ChatGPT claiming they are a "prompt expert" are the ones the most excited.
For those that truly understand AI, there is a lot that you should genuinely be worried about. Now, don't confuse that for saying that we shouldn't work on it or should abandon AI work. I truly believe that this is the next greatest revolution. This is 1,000x more transformative than the industrial revolution, and 100x more transformative than the internet revolution. But it is worth a brief consideration of the effects of our work before we start running into these changes that could have drastic effects on everybody's daily life.
Except ... when you look at the problem from a military/national security viewpoint. Do we really want to give this tech away just like that?
Work on open source locally hosted AI is important. I keep local clones and iterate as I can.
Let the market develop organically.
A government-controlled… never mind artificial god, a government-controlled story teller can be devastating.
I don't buy Musk's claim ChatGPT is "woke" (or even that the term is coherent enough to be tested), but I can say that each government requiring AI to locally adhere to national mythology, will create self-reinforcing cognitive blind spots, because that already happens at the current smaller scale of manual creation and creators being told not to "talk the country down".
But, unless someone has a technique for structuring an AI such that it can't be evil even when you, for example, are literally specifically trying to train it to support the police no matter how authoritarian the laws are, then a fully open source AGI is almost immediately also a perfectly obedient sociopath of $insert_iq_claim_here.
I don't want to wake up to the news that some doomsday cult has used one to design/make a weapon, nor the news a large religious group target personalised propaganda against me and mine.
Fully open does that by default.
But, you're still right, if we don't grok the AI, the governments can each secretly manipulate the AI and bend it to government goals in opposition to the people.
This is a key point. Every culture and agency and state will want (deserve) their own homespun AGI. But can we all learn how to accommodate to or accept a cultural multiverse when money and resources are zero-sum in many dimensions.
Hanno Rajaniemi’s Quantum Thief trilogy gives you a foretaste of where we could end up.
Dear Senator [X],
I am an engineer working for [major employer in the state]. I am extremely concerned about the message that Sam Altman is sharing with the Judiciary committee today.
Altman wants to create regulatory roadblocks to developing AI. My company produces AI-enabled products. If these roadblocks had been in place two years ago, my company would not have been able to invest into AI. Now, because we had the freedom to innovate, AI will be bringing new, high paying jobs to our factories in our state.
While AI regulation is important, it is crucial that there are no roadblocks stopping companies and individuals from even trying to build AIs. Rather, regulation should focus on ensuring the safety of AIs once they are ready to be put into widespread use - this would allow companies and individuals to research new AIs freely while still ensuring that AI products are properly reviewed.
Altman and his ilk try to claim that aggressive regulation (which will only serve to give them a monopoly over AI) is necessary because an AI could hack it's way out of a laboratory. Yet, they cannot explain how an AI would accomplish this in practice. I hope you will push back against anyone who fear-mongers about sci-fi inspired AI scenarios.
Congress should focus on the real impacts that AI will have on employment. Congress should also consider the realistic risks AI which poses to the public, such as risks from the use of AI to control national infrastructure (e.g., the electric grid) or to make healthcare decisions.
Thank you, [My name]
OpenAI != FTX, just meaning to say calling for regulation isn't an indication of good intentions, despite sounding like it.
Could you imagine if MS had convinced the govt back in the day, to require a special license to build an operating system (this blocking Linux and everything open)?
It’s essentially what’s happening now,
Except it is OpenAI instead of MS, and it is AI instead of Linux
AI is the new Linux, they know it, and are trying desperately to stop it from happening
Conveniently this also helps them build a monopoly. It is pretty aggravating that they're bastardizing and abusing terms like 'safety' and 'democratization' while doing this. I hope they'll fail in their attempts, or that the competition rolls over them rather sooner than later.
I personally think that the greatest threat in these technologies is currently the centralization of their economic potential, as it will lead to an uneven spread of their productivity gains, further divide poor and rich, and thus threaten the order of our society.
That should be the goal.
the issue here is that a 'Linux' of AI would be happy to use the N-word and stuff like that. It's politically untenable.
Me too, in comparison all the other potential threats discussed over here feel mostly secondary to me. I'm also suspecting that at the point where these AIs reach a more AGI level, the big players who have them will just not provide any kind of access all together and just use them to churn out an infinite amount of money-making applications instead.
How many of Microsoft's 221k employees exist solely to support the weight of a company with 221k people? A smaller IT department doesn't need a large HR department. And a small HR department doesn't file many tickets with IT. LLM driven multinationals will need orders of magnitude fewer employees, and that puts our current multinationals in a very awkward position.
Personally, I will be storing a local copy of LLaMA 65B for the foreseeable future. Instruct fine-tuning will keep getting cheaper; given the stakes, the large models might not always be easy to find.
This entire legal argument is almost beside-the-point because development on open source models can continue on onion sites too.
It will only affect commercial operations, and they won't be able to release a fine tuned model on a banned LLM even if they do have a pre-ban copy saved.
If this was to go through, of course OpenAI and co will be the primary lobbiests to ensure they get to define the filters for such a license.
Also how would you even enforce this. It's absolute nonsense and is a clear indicator that these larger companies realize there is no 'gatekeeping' these AI's, that the democratization of models has demonstrated incredible gains over their own.
Edit : Image during the early days of the internet you needed a license to start a website.
In the later days you needed a license to start a social media site.
Nonsense.
Majority Members
Chair Richard Blumenthal (CT) brian_steele@blumenthal.senate.gov
Amy Klobuchar (MN) baz_selassie@klobuchar.senate.gov
Chris Coons (DE) anna_yelverton@coons.senate.gov
Mazie Hirono (HI) jed_dercole@hirono.senate.gov
Alex Padilla (CA) Josh_Esquivel@padilla.senate.gov
Jon Ossoff (GA) Anna_Cullen@ossoff.senate.gov
Majority Office: 202-224-2823
Minority Members
Ranking Member Josh Hawley (MO) Chris_Weihs@hawley.senate.gov
John Kennedy (LA) James_Shea@kennedy.senate.gov
Marsha Blackburn (TN) Jon_Adame@blackburn.senate.gov
Mike Lee (UT) Phil_Reboli@lee.senate.gov
John Cornyn (TX) Drew_Brandewie@cornyn.senate.gov
Minority Office: 202-224-4224
Sounds like USA would shoot itself in the foot.
If I wanted AI to be regulated everywhere and I was Sam Altman, I’d probably start with convincing the US.
Give it a year and a 10x more efficient algorithm, and we'll have GPT4 on our personal devices and there's nothing that any government regulator will be able to do to stop that.
I wouldn’t mind ruthless anti-competitive approaches to business as much, but the hypocrisy is really demoralizing.
At 25 you stop believing in love,
At 40 you stop believing in corporations’ sincerity?
Shame on all of the people involved in this: the people in these companies, the journalists who shovel shit (hope they get replaced real soon), researchers who should know better, and dementia ridden legislators.
So utterly predictable and slimy. All of those who are so gravely concerned about "alignment" in this context, give yourselves a pat on the back for hyping up science fiction stories and enabling regulatory capture.
True to form, these articles also work hard at planting the idea that Sam Altman created OpenAI, when in fact he joined rather recently, in a business role. Are these articles being planted somehow? I find it very likely. Don't forget that this approach is also straight out of the YC playbook, disclosed in great detail by Paul Graham in previous writings [3].
Finally, in keeping with the conspiratorial tone of this comment, for another example of Sam Altman rubbing shoulders with The Establishment, his participation in things like the Bilderberg group [4] are a matter of public record. Which I join many others in finding creepy, even moreso as he maneuvers to exert influence on policy around the seismic shift that is AI.
To be clear, I have nothing specific against sama. But I dislike underhanded influence campaigns, which this all reeks of. Oh yeah, I will consider downvotes to this comment as proof of the shadow (AI?) government's campaign to promote Sam Altman. Do your worst!
[1] https://www.cnbc.com/2023/05/16/openai-ceo-woos-lawmakers-ah...
[2] https://www.newyorker.com/magazine/2016/10/10/sam-altmans-ma... ("Graham said, “I asked Sam in our kitchen, ‘Do you want to take over YC?,’ and he smiled, like, it worked. I had never seen an uncontrolled smile from Sam. It was like when you throw a ball of paper into the wastebasket across the room—that smile.”")
[3] http://www.paulgraham.com/submarine.html
[4] https://en.wikipedia.org/wiki/2016_Bilderberg_Conference
The only regulations that matter will be applied to the end user and the hobbyists. You won't be able to just spin up an AI startup in your garage. So in that sense, the regulations are pretty transparently an attempt to stifle competition and funnel the real progress through the existing players.
It also forces the end users down the path of using only a few select AI service providers as opposed to the technology just being readily available.
> In December 2015, Sam Altman, Greg Brockman, Reid Hoffman, Jessica Livingston, Peter Thiel, Elon Musk, Amazon Web Services (AWS), Infosys, and YC Research announced[13] the formation of OpenAI and pledged over $1 billion to the venture.
Chair Richard Blumenthal (CT), Amy Klobuchar (MN), Chris Coons (DE), Mazie Hirono (HI), Alex Padilla (CA), Jon Ossoff (GA)
Majority Office: 202-224-2823
Ranking Member Josh Hawley (MO), John Kennedy (LA), Marsha Blackburn (TN), Mike Lee (UT), John Cornyn (TX)
Minority Office: 202-224-4224
If you’re in those states, please call their D.C. office and read them the comment you’re leaving here.
Source: manned phones fielding constituent calls earlier in my career.
Local or legislative?
I've never met a Senator's leg team that doesn't compile notes on active issues from constituents upstream. (Granted, it's a handful of teams.)
papers for thee but not for me
Most travel agents need a license, taxi drivers etc. Not sure why the same shouldn't apply for "AI"?
You say it like it's a good thing.
While in general I share the view that _research_ should be unencumbered, but deployment should be regulated, I do take issue with your view that safety only matters once they are ready for "widespread use". A tool which is made available in a limited beta can still be harmful, misleading, or too-easily support irresponsible or malicious purposes, and in some cases the harms could be _enabled_ by the fact that the release is limited.
For example, if next month you developed a model that could produce extremely high quality video clips from text and reference images, you did a small, gated beta release with no PR, and one of your beta testers immediately uses it to make e.g. highly realistic revenge porn. Because almost no one is aware of the stunning new quality of outputs produced by your model, most people don't believe the victim when they assert that the footage is fake.
I would suggest that the first non-private (e.g. non-employee) release of a tool should make it subject to regulation. If I open a restaurant, on my first night I'm expected to be in compliance with basic health and safety regulations, no matter how few customers I have. If I design and sell a widget that does X, even for the first one I sell, my understanding is there's an concept of an implied requirement that my widgets must actually be "fit for purpose" for X; I cannot sell a "rain coat" made of gauze which offers no protection from rain, and I cannot sell a "smoke detector" which doesn't effectively detect smoke. Why should low-volume AI/ML products get a pass?
You make a great point here. This is why we need as much open source and as much wide adoption as possible. Wide adoption = public education in the most effective way.
The reason we are having this discussion at all is precisely because OpenAI, Stability.ai, FAIR/Llama, and Midjourney have had their products widely adopted and their capabilities have shocked and educated the whole world, technologists and laymen alike.
The benefit of adoption is education. The world is already adapting.
Doing anything that limits adoption or encourages the underground development of AI tech is a mistake. Regulating it in this way will push it underground and make it harder to track and harder for the public to understand and prepare for.
i can sell a webserver that gets used to host illegal content all day long. Should that be included? Where does the regulation end? I hate that the answer to any question seems to be just add more government.
There are already laws against false advertising, misrepresentation etc. We don’t need extra laws specifically for AI that doesn’t perform well.
What most people are concerned about is AI that performs too well.
I would assert that just as I have the right to pull out a sheet of paper and write the most vile, libelous thing on it I can imagine, I have the right to use AI to put anyone's face on any body, naked or not. The crime comes from using it for fraud. Take gasoline for another example. Gasoline is powerful stuff. You can use it to immolate yourself or burn down your neighbor's house. You can make Molitov cocktails and throw them at nuns. But we don't ban it, or saturate it with fire retardants, because it has a ton of other utility, and we can just make those outlying things illegal. Besides, five years from now, nobody's going to believe a damned thing they watch, listen to, or read.
As I understand it, revenge porn is seen as being problematic because it can lead to ostracization in certain social groups. Would it not be better to regulate such discrimination? The concept of discrimination is already recognized in law. This would equally solve for revenge porn created with a camera. The use of AI is ultimately immaterial here. It is the human behaviour as a product of witnessing material that is the concern.
The right one is to grant people rights over their likeness, so you could use something more like copyright law
Even if it's a real recording, you should still have control over it
Dear Senator [X],
It's painfully obvious that Sam Altman's testimony before the judiciary committee is an attempt to set up rent-seeking conditions for OpenAI, and to snuff out competition from the flourishing open source AI community.
We will be carefully monitoring your campaign finances for evidence of bribery.
Hugs and Kiss,
[My Name]
I’m sympathetic to your position in general, but I can’t believe you wrote that with a straight face. “I don’t know how it would do it, therefore we should completely ignore the risk that it could be done.”
I’m no security expert, but I’ve been following the field incidentally and dabbling since writing login prompt simulators for the Prime terminals at college to harvest user account passwords. When I was a Unix admin I used to have fun figuring out how to hack my own systems. Security is unbelievably hard. An AI eventually jail braking is an eventual almost certainty we need to prepare for.
People are coalescing into “for” and “against” camps, which makes very little sense given the broad spectrum of technologies and problems summarized in statements like “AI regulation”.
I think it’s a bit like saying “software (should|shouldn't) be regulated”. It’s a position that cannot be defended because the term software is too broad.
Which one do you think is more important to convince?
If AI will be used by public institutions, especially law enforcement, we need it regulated in the same manner. A bad AI trained on biased data has the potential to be extremely dangerous in the hands of a cop who is already predisposed for racist behavior.
Does scikit-learn count or we are just not going to bother defining what we mean by "AI"?
"AI" is whatever congress says it is? That is an absolutely terrible idea.
Not regulating the air quality we breathe for decades turned out amazing for millions of the Americas. Yes, lets do the same with AI! What could possibility go wrong?
- Scamming via impersonation - Misinformation - Usage of AI in a way that could have serious legal ramifications for incorrect responses - Severe economic displacement
Congress can and should examine these issues. Just because OP works at an AI doesnt' mean that company can't exist in a regulated industry.
I too work in the AI space and welcome thoughtful regulation.
Do we really have to play this game?
If what you’re arguing for is not going to specifically advantage your state over others, and the thing you’re arguing against isn’t going to create an advantage for other states over yours, why make this about ‘your state’ in the first place?
The point of elected representatives is to represent the views of their constituents, not to obtain special advantages for their constituents.
That is painfully naive, a history of pork projects speaks otherwise.
The views of their constituents are probably in favor of special advantages for their constituents, so the one may imply the other.
I mean, some elected representatives may represent constituencies consisting primarily of altruistic angels, but that is…not the norm.
A lot of said constituents' views are, in practice, that they should receive special advantages.
You think voice actors and writers are not saying the same?
When do we accept capitalism as we know it is just a bullshit hallucination we grew up with? It’s no more an immutable feature of reality than a religion?
I don’t owe propping up some rich person’s figurative identity, or yours for that matter.
I agree with being skeptical of proposals from those with vested interests, but are you just arguing against what you imagine Altman will say, or did I miss some important news?
What would you say to a simple registration requirement? You give a point of contact and a description of training data, model, and perhaps intended use (could be binary: civilian or dual use). One page, publicly visible.
This gives groundwork for future rulemaking and oversight if necessary.
If you sent it by e-mail or web contact form, chances are you wasted your time.
If you really want attention, you'll send it as a real letter. People who take the time to actually send real mail are taken more seriously.
IANA senator, but if I were you lost me there. The personal insults make it seem petty and completely overshadow the otherwise professional-sounding message.
Altman can an ulterior motive, but it doesn't mean that we should strive for having some sort of handle on AI safety.
It could be that Altman and OpenAI know exactly how this will look and the backlash that will ensue that we get ZERO oversight and we rush headlong into doom.
Short term we need to focus on the structural unemployment that is about to hit us. As the AI labs use AI to make better AI, it will eat all the jobs until we have a relative handful of AI whisperers.
Scott Galloway called it the stop-me-before-I-kill-grandma defence. (Paraphrasing.)
You made money making a thing. You continue to make the thing. You’re telling us how the thing will bring doom and gloom if not dealt with (conveniently implying it will change the world). And you want to staff the regulatory body you call for with the very butchers you’re castigating.
Is there really a better alternative here?
This isn't the same at all.
You've got it backwards. I bet OpenAI wants those regulations to be as restrictive as possible. They'll just negotiate an exception for themselves. With increased regulation comes an increased initial cost for competitors to get started in the space. They want to lock down their near monopoly as soon as they can.
I'd argue that any business advocating for regulation is largely motivated by its own pursuit of regulatory capture.
https://www.cnbc.com/2020/02/15/facebook-ceo-zuckerberg-call...
What established player doesn’t want to make it as hard as possible to compete with them?
Once you gain the lead position, it is in your interest to increase the barriers to entry as much as possible.
I mean, I guess there’s always that play: maybe Sam Altman is simply feigning enthusiasm for this to head off a burgeoning regulatory coup: how can he mobilize support against the regulations He doesn’t want? What would be a great way to galvanize support against the regulations? Maybe by doing exactly what he’s doing: by falling on his sword, and making himself seem like a monopolistic supporter of those?
Sacrificing his reputation for the greater good of open competition on AI? Well, that’s a truly noble move.
It’s a bit of a stretch, but maybe it’s very Washington I don’t know.
A more logical (economic) motivation may be: this news is taken as a threat, in effect creating a sense of scarcity around AI, which could be judged to provoke people into buying more of OpenAI's subscriptions, training, or whatever, to ensure access that, in light of such news, they may now feel is at risk.
So, the move is merely a red herring, a fabricated threat designed to provoke people to spend, not intended to actually create legislation.
Don't know which one of these is more likely if any: or maybe Sama is simply living his best life role playing as a super villain? I guess he's starting to remind me of Niander Wallace in Bladerunner 2049
The fearmongering and astroturfing is obvious.
This is always the first comment when someone in an industry talks about regulation but it doesn't change the fact that it's needed and they're essentially right regardless of what motivations they have.
As a blunt analogy, let's say there's no law against murder right now. You and I both agree that we need a law against murder. But I have the ear of lawmakers, and I go sit down with them and tell them that you and I agree: We need a law against murder.
And then I help them write a law that makes murder illegal. Only, not all killing counts as murder, obviously. So if it's an accident, no murder. Self defense? No murder. And also if they are doing anything that "threatens" my business interests, not murder. Great, we've got a law that prevents unnecessary killing! And now I get to go ~~murder you~~ defend my business interests when you protest that the new law seems unfair.
[1] https://www.yahoo.com/news/worldcoin-chatgpt-sam-altman-ethe...
Come on! You don't get to the place he got to by accident. This requires careful planning and ruthless execution. He just faked being the nerdy kid who wants to do good and is surprised by the billions coming to him.
Embrace, Extend...
What comes after Extend?
In other words, MS with extra steps.
It is as if most humans lack an inner compass, a set of principles, a solid value system.
It is as if humans are robots, that fulfil their given role as if they were programmed to do so.
Our institutions, companies, political parties, etc. are these functions that humans slip into and in robot-like fashion execute their expected role like clockwork.
The free thinkers / big picture thinkers, humanity‘s heroes seem to be so rare.
I get they're excited they can make money off it, but wow, what a nightmare. I feel if they just stuck to their principles and kept it "Open" they'd still be better off in general.
Microsoft did influence government spending in ways that require windows in every govt owned computer, and schools.
OpenAI is Microsoft.
Before Twilio et al, if you wanted to text a lot of your customers automatically, you had to pay thousands of dollars in setup and recurrent fees to rent a shortcode number
But then, with Twilio et al, you don’t need a shortcode anymore
The telcos told the regulators this would create endless spam, so they would regulate it themselves, and created a consortium to “end spam”
Now you are forced to get a license from them, pay a monthly fee, get audited, and they still let pretty much all the spam through, while they also randomly block a certain % of your messages, even if you are fully compliant
At the same time At&T and Verizon saying oh snap let's make money on this too and still being pissed about Stir Shaken so to get ahead of it for Texting before Congress forces it on them. This way they can make money on it before it's forced.
[0] https://fortune.com/2019/02/04/t-mobiles-john-legere-promise...
Give the politicians time: I predict a day will come where you will need a permit to use a compiler and connect the result to the internet.
What regulation are they proposing that is actually a serious barrier to making a company around AI?
If OpenAI just wants to prevent another OpenAI eating its lunch, the barrier there is raw compute. Companies that can afford that can afford to jump regulatory hurdles.
The open source community will catch up in at most a year or two, they are scared and now want to use regulations to strangle competitions.
While their AI is going to advance as well, the leap will not be qualitative as the ChatGPT gen 1 was - so they will lose competitive advantage.
Stable Diffusion pretty much killed DALL-E, cost only $600k to train, and can be run on iPhones.
Because this is the reason that VCs exist in the first place. They can roll a company with a ton of capital, just like they did with ride share companies. When that happens, and there aren't sufficient barriers to entry, it's a race to the bottom.
This is classic regulatory capture.
FB, Amazon, Google (and possibly Apple) can afford both money and compute resource for that. They couldn't do that themselves probably due to corporate politics and bureaucratic but MS and OpenAI showed how to solve that problem. They definitely don't want their competitors to copy the strategy so they're blatantly asking for explicit whitelisting instead of typical safety regulation.
And note that AI compute efficiency is a rapidly developing area and OpenAI definitely knows the formula won't be left the same in the coming years. Expect LLM to be 10x efficient than the SOTA in the foreseeable future, which probably will make it economical even without big tech's backing.
Requiring a license to buy or lease the requisite amount of powerful enough GPUs might just do the trick
y e s.So yes it is quite comparable to the export restrictions of the 90s.
But since Microsoft is involved and we are all of course thinking about Windows vs Linux, I think another good comparison is the worst assumption Microsoft made in the 90s: “we know what an operating system is and what it is for.”
That happened a few months ago with LLaMA. Since then, the open source community has exceeded all expectations democratizing the hardware angle. AI regulators are already checkmated and they don't know it yet. If their goal is to control the use of AI (rather than simply controlling the people building it) then they'd need to go all the way when it comes to tyranny in order to accomplish their goals. Intel would need to execute Order 66 with their Management Engine and operating systems would need to modify their virus scanners to monitor and report the use of linear algebra. It'd be outrageous.
The fact that these systems can extrapolate well beyond their training data by learning algorithms is quite different than what has come before, and anyone stating that they "simply" predict next token is severely shortsighted. Things don't have to be 'brain-like' to be useful, or to have capabilities of reasoning, but we have evidence that these systems have aligned well with reasoning tasks, perform well at causal reasoning, and we also have mathematical proofs that show how.
So I don't understand your sentiment.
Do you have a reference?
Do you mind linking to one of those papers?
Give them a semi human sounding puppet and they think skynet is coming tomorrow.
If we learned anything from the past few months is how gullible people are, wishful thinking is a hell of a drug
What I feel has changed, and what drives a lot of the fear and anxiety you see, is a sudden perception of possibility, of accessibility.
A lot of us (read: people) are implicit dualists, even if we say otherwise. It seems to be a sticky bias in the human mind (see: the vanishing problem of AI). Indeed, you can see a whole lot of dualism in this thread!
And even if you don't believe that LLMs themselves are "intelligent" (by whatever metric you define that to be...), you can still experience an exposing and unseating of some of the foundations of that dualism.
LLMs may not be a destination, but their unprecedented capabilities open up the potential for a road to something much more humanlike in ways that perhaps did not feel possible before, or at least not possible any time soon.
They are powerful enough to change the priors of one's internal understanding of what can be done and how quickly. Which is an uncomfortable process for those of us experiencing it.
I don't think that trying to regulate every detail of every industry is stifling and counter-productive. But the current scenario is closer to the opposite end of the spectrum, with our society acting as a greedy algorithm in pursuit of short-term profits. I'm perfectly in favor of taking a measure-twice-cut-once approach to something that has as much potential for overhauling society as we know it as AI does. And I absolutely do not trust the free market to be capable of moderating itself in regards to these risks.
What to do? Why, obviously lets talk about the risks of AGI.
I mean LLM's are an impressive piece of work but the global reaction is basically more a reflection of an unmoored system that floats above and below reality but somehow can't re-establish contact.
He's a charlatan, which makes sense he gets most of his money from Thiel and Musk. Why do so many supposedly smart people worship psychotic idiots?
AI/ML licensing builds Power and establishes moat. This will not lead to better software.
Frankly, Google and Microsoft are acting new. My understanding of both companies has been shattered by recent changes.
I'm all for villainizing the figureheads of the current generation of this movement. The politics of this sea-change are fascinating and worthy of discussion.
But out-of-hand dismissal of what has been accomplished smacks more to me of lack of awareness of the history of the study of the brain, cognition, language, and computers, than it does of a sound debate position.
Because people have different definition of what intelligence is. Recreating the human brain in a computer would definitely be neat and interesting but you don't need that nor AGI to be revolutionary.
LLMs, as perfect Chinese Rooms, lack a mind or human intelligence but demonstrate increasingly sophisticated behavior. If they can perform tasks better than humans, does their lack of "understanding" and "thinking" matter?
The goal is to create a different form of intelligence, superior in ways that benefit us. Planes (or rockets!) don't "fly" like birds do but for our human needs, they are effectively much better at flying that birds ever could be.
Those nonplussed by this wave of AI are just yawning.
Because it's wrong and smart people know that.
The whole saga makes Altman look really, really terrible.
If AI really is this dangerous then we definitely don't need people like this in control of it.
Incredibly scummy behaviour that will not land well with a lot of people in the AI community. I wonder if this is what prompted a lot of people to leave for Anthropic.
At this point, with this part about openai and worldcoin… if it walks like a duck and talks like a duck..
You're leaving out the essentials. These models do more than fitting the data given. They can output it in a variety of ways, and through their approximation, can synthesize data as well. They can output things that weren't in the original data, tailored to a specific request in the tiniest of fractions of the time it would take a normal person to look up and understand that information.
Your argument is almost like saying "give me your RSA keys, because it's just two prime numbers, and I know how to list them."
Do we want to go down the road of making white collar jobs the legislatively required elevator attendants? Instead of just banning AI in general via executive agency?
That sounds like a better solution to me, actually. OpenAI's lobbyists would never go for that though. Can't have a moat that way.
I wrote a comment recently trying to explain how even if you believe all LLMs can (and will ever) do is regurgitate their training data that you should still be concerned.
For example, imagine in 5 years we have GPT-7, and you ask GPT-7 to solve humanity's great problems.
From its training data GPT-7 might notice that humans believe overpopulation is a serious issue facing humanity.
But its "aligned" so might understand from its training data that killing people is wrong so instead it uses its training data to seek other ways to reduce human populations without extermination.
Its training data included information about how gene drives were used by humans to reduce mosquito populations by causing infertility. Many human have also suggested (and tried) to use birth control to reduce human populations via infertility so the ethical implications of using gene drives to cause infertility is debatable based on the data the LLM was trained on.
Using this information it decides to hack into a biolab using hacking techniques it learnt from its training data and use its biochemistry knowledge to make slight alterations to one of the active research projects at the lab. This causes the lab to unknowingly produce a highly contagious bioweapon which causes infertility.
---
The point here is that even if we just assume LLMs are only capable of producing output which approximates stuff it learnt from its training data, an advanced LLM can still be dangerous.
And in this example, I'm assuming no malicious actors and an aligned AI. If you're willing to assume there might be an actor out there would seek to use LLMs for malicious reasons or the AI is not well aligned then the risk becomes even clearer.
This is a real problem, but it's already problem with our society, not AI. Misaligned public intellectuals routinely try to reduce the human population and we don't lift a finger. Focus where the danger actually is - us!
From Scott Alexander's latest post:
Paul Ehrlich is an environmentalist leader best known for his 1968 book The Population Bomb. He helped develop ideas like sustainability, biodiversity, and ecological footprints. But he’s best known for prophecies of doom which have not come true - for example, that collapsing ecosystems would cause hundreds of millions of deaths in the 1970s, or make England “cease to exist” by the year 2000.
Population Bomb calls for a multi-pronged solution to a coming overpopulation crisis. One prong was coercive mass sterilization. Ehrlich particularly recommended this for India, a country at the forefront of rising populations.
In 1975, India had a worse-than-usual economic crisis and declared martial law. They asked the World Bank for help. The World Bank, led by Robert McNamara, made support conditional on an increase in sterilizations. India complied [...] In the end about eight million people were sterilized over the course of two years.
Luckily for Ehrlich, no one cares. He remains a professor emeritus at Stanford, and president of Stanford’s Center for Conservation Biology. He has won practically every environmental award imaginable, including from the Sierra Club, the World Wildlife Fund, and the United Nations (all > 10 years after the Indian sterilization campaign he endorsed). He won the MacArthur “Genius” Prize ($800,000) in 1990, the Crafoord Prize ($700,000, presented by the King of Sweden) that same year, and was made a Fellow of the Royal Society in 2012. He was recently interviewed on 60 Minutes about the importance of sustainability; the mass sterilization campaign never came up. He is about as honored and beloved as it’s possible for a public intellectual to get.
In other words, LLMs are only as dangerous as the humans operating them, and therefore the solution is to stop crime instead of regulating AI, which only seeks to make OpenAI a monopoly.
> But its "aligned" so might understand
> Using this information it decides to hack
I think you're anthropomorphizing LLM's too much here. If we assume that there's a AGI-esque AI, then of course we should be worried about an AGI-esque AI. But I see no reason to think that's the case.
I don't think this would be a bad thing :) Some people will always be immune, humanity wouldn't die out. And it would be a humane way for gradual population reduction. It would create some temporary problems with elderly care (like what China is facing now) but will make long term human prosperity much more likely. We just can't keep growing against limited resources.
The Dan Brown book Inferno had a similar premise and I was disappointed they changed the ending in the movie so that it didn't happen.
Literally half (or more) of this site's user base does that. And they should know better, but they don't. Then how can a typical journo or a legislator possibly know better? They can't.
We should clean up in front of our doorstep first.
Is a license the best way forward I don't know but I do feel like this is more than a math formula.
This information is not created inside the LLMs, it's part of their training data. If someone is motivated enough, I'm sure they'd need no more than a few minutes of googling.
> I do feel like this is more than a math formula
The sum is greater than the parts! It can just be a math formula and still produce amazing results. After all, our brains are just a neat arrangement of atoms :)
And just because a topic has been covered by science fiction doesn’t mean it can’t happen, the sci-fi depictions will be unrealistic though because they’re meant to be dramatic rather than realistic
Who's to say we're not in a simulation ? Who's to say god doesn't exist ?
Edit: List of posts for anyone interested http://paste.debian.net/plain/1280426
2. Explain why it is possible for a large number of properly constructed neurons to think.
Are you aware that you are an 80 billion neuron biological neural network?
Just like a CPU isn't "like your brain" and HDD "like your memories"
Absolutely nothing says our current approach is the right one to mimic a human brain
Yeah? Did you get a crystal ball for Christmas to be able to predict what can and can't be done with a new technology?
Is this true? I've been working in the industry for a while and Sam Altman has long been mentioned in reference to OpenAI along with Ilya.
I agree with the crux of your comment that everyone is scrambling to build narratives, but I think I would also put your comment "AI is busy cozying up with The Establishment" as just another narrative (and one that we saw in this hearing from people like Hawley).
The media likes to personalize stories. Altman is a face for AI and apparently knows to give an interview, that's worth something to them. (Lobbying may well be an influence, but the most important thing to them is to have a face, just like Zuckerman was a face for social networks. If it wasn't Altman it would have eventually been someone else).
- Lord Altman
It’s funny how all Microsoft properties are in dominant position on their market.
Predictable. Disappointing, but predictable.
We've seen this duck so many times before.
No need to innovate when you can regulate.
And it raises First Amendment issues as well. I think it's morally wrong to prohibit the development of software, which is what AI models are, especially if it's done in a personal capacity.
How do they even know that the author is based in the US anyway. Just use a Russian or Chinese Git hosting provider, where these laws don't exist?
And by the way foreign developers won't even have to jump through these hoops in the first place, so this law will only put the US at a disadvantage compared to the rest of the world.
If these lobbyists get their way, by restricting AI development in both the US and the EU, it will be hilarious to see that out of all places, Russia might be one of the few large countries where it's development will remain unrestricted.
Even better, is that if Russia splits up we will have a new wild west for this kind of thing....
For anyone who really believes that AI is dangerous, having some reasonable regulations on it is logical. It's a good start on not being doomed. It goes against everyone's egalitarian/libertarian impulses, though.
The thing is, AI doesn't seem nearly as dangerous as a fully-automatic machine gun. For now. It's just generating text (and video) for fun, right?
The shared point of both AI alarmists and advocates is that AI will be highly resistant to being subject to regulation, ultimately. As dictated by the market for it. They won't want to regulate something, assuming they could, for which its free operation underlies everyone's chance of survival against competing systems.
I only find that danger is inherent in the effort of people that casually label things as "dangerous".
I'm still exploring whether its the laziness aspect, itself, of the alarmist vocabulary in the absence of required explanation. Or whether my issue lies with the suspicion of emotional manipulation and an attempt to circumvent having to actually explain one's reasoning, using alarmist language absent required explanation.
Already, AI pessimists are well on their way to losing any window where their arguments will be heard and meaningful. We can tell by their parroting the word "dangerous" as the total substance of their arguments. Which will soon be a laughable defense. They'd better learn more words.
Could you share the minutes from the Military Industrial Complex strategy meetings this was discussed at. Thanks.
[pause]
"No? Ok, I'll tell him."
No, we do not. And the same is true of any person who we know of mostly from news stories. News you read is NOT an unbiased sampling of information about a person due to all the selection effects.
Though the other day Yuval Noah Harari gave a great talk on the potential threat to democracy - https://youtu.be/LWiM-LuRe6w
I do think you're probably right about AI though. Too many influential groups are going to get too mad about the words an open model will output. Only allowing locked down models is going to severly limit their usefullness for all sorts of novel creative and productive use cases.
Though I agree it seems to fit here. Being granted an oligopoly in the name of protecting the people and all that.
Yes. It is. I'm sure hostile, authoritarian states that are willing to wage war with the world like Russia and North Korea will eventually get their hands on military-grade AI. But the free world should always strive to be two steps ahead.
Even having ubiquitous semi-automatic rifles is a huge problem in America. I'm sure Cliven Bundy or Patriot Front would do everything they can to close the gap with intelligent/autonomous weapons, or even just autonomous bots hacking America's infrastructure. If everything is freely available, what would be stopping them?
Similarly it's not really good faith to assume everyone opposed to regulation in this field is seeking a lawless libertarian (or anarchist perhaps) utopia.
https://www.eff.org/deeplinks/2015/04/remembering-case-estab...
I think you should continue to have the right to use whatever program to generate whatever video clip you like on your computer. That is a distinct matter from whether a commercially available video generative AI service has some obligations to guard against abusive uses. Personal freedoms are not the same as corporate freedom from regulatory burdens, no matter how hard some people will work to conflate them.
I highly recommend Rob Miles channel on YouTube. Here’s a good one, but they’re all fascinating. It turns out training an AI to have the actual goals we want it to have is fiendishly difficult.
"Oh, this person donated X/Y to Z/Q causes! They can't be that bad right?"
Absolutely spot on. I am not a dualist at all and I've been surprised to see how many people with deep-seated dualist intuition this has revealed, even if they publicly claim not to.
I view it as embarrassing? It's like believing in fairies or something.
Until a model of human sentience and awareness is established (note: one of the oldest problems out there alongside the movements of the stars. This is an ancient debate, still open-ended, and nothing anyone is saying in these threads is new), philosophy is all we have and ideas are debated on their merits within that space.
That said, there are 8B+ of us and counting so unless there is magic involved, I don't see why we couldn't do a "1:1" replica of it (maybe far) in the future.
True, it's just binary logic gates, but it's a lot of them and if they can simulate pretty much anything why should intelligence be magically exempt?
> Absolutely nothing says our current approach is the right one to mimic a human brain
Just like nothing says it's the wrong one. I don't think those regulation suggestions are a good idea at all (and say a lot about a company called OpenAI), but that doesn't mean we should treat it like the NFT hype.
What are the key differences?
https://www.edn.com/the-truth-about-smics-7-nm-chip-fabricat...
Other than that there's also things like roformer, but I'm going to assume you won't count that as significant. US universities then certainly don't produce anything significant either though.
This tells me the extent of your knowledge about the challenges with these models.
Imagine if, e.g. drugs testing and manufacture was subject to no regulations. As a consumer, if you can be aware that some chemicals are very powerful and useful, but you can't be sure that any specific product has the chemicals it says it has, that it was produced in a way that ensures a consistent product, or that it was tested for safety, or what the evidence is that it's effective against a particular condition. Even if wide adoption of drugs from a range of producers occurs, does the public really understand what they're taking, and whether it's safe? Should the burden be on them to vet every medication on the market? Or is appropriate to have some regulation to ensure medications have have their active ingredients in the amounts stated, and are produced with high quality assurance, and are actually shown to be effective? Oh, no, says a pharma industry PR person. "Doing anything that limits the adoption or encourages the underground development of bioactive chemicals is a mistake. Regulating it in this way will push it underground and make it harder to track and harder for the public to understand and prepare for."
If a team of PhDs can spend weeks trying to explain "why did the model do Y in response to X?" or figure out "can we stop it from doing Z?", expecting "wide adoption" to force "public education" to be sufficient to defuse all harms such that no regulation whatsoever is necessary is ... beyond optimistic.
The debate is really about how much and what type of regulation. It is of strategic importance that we do not let bad actors get the upper hand, but we also know that bad actors will rarely follow any of this regulation anyway. There is something to be said for regulating the application rather than the technology, as well as for realizing that large corporations have historically used regulatory capture to increase their moat.
Given it seems quite unlikely we will be able to stop prompt injections, what are we to do?
Provenance seems like a good option, but difficult to implement. It allows us to track who created what, so when someone does something bad, we can find and punish them.
There are analogies to be made with the Bill of Rights and gun laws. Gun analogy seem interesting because they have to be registered, but often criminals won't and the debate is quite polarized.
I met some biohackers at defcon that took this perspective, a sort of "open source but for medicine" ideology. I see the dangers of a massively uneducated population trying to 3d print aspirin poisoning themselves, but they already do that with horse paste so I'm not sure it's a new issue.
AI is a very different problem space. With AI, even the big models easily fit on a micro SD card. You can carry around all of GPT4 and its supporting code on a thumb drive. You can transfer it wirelessly in under 5 minutes. It's quite different than drugs or conventional weapons or most other things from a practicality perspective when you really think about enforcing developmental regulation.
Also consider that criminals and other bad actors don't care about laws. The RIAA and MPAA have tried hard for 20+ years to stop piracy and the DMCA and other laws have been built to support that, yet anyone reading this can easily download the latest blockbuster movie or in the theater.
Even still, I'm not saying don't make laws or regulations on AI. I'm just saying we need to carefully consider what we're really trying to protect or prevent.
Also, I certainly believe that in this case, the widespread public adoption of AI tech has already driven education and adaptation that could not have been achieved otherwise. My mom understands that those pictures of Trump being chased by the cops are fake. Why? Because Stable Diffusion is on my home computer so I can make them too. I think this needs to continue.
These resources should be spent lessening the impact rather than trying to completely control it.
You absolutely can. Maybe you can't effectively enforce that regulation but you can regulate and you can take measures that make violating the regulation impractical or risky for most people. By the way, the "crypto-wars" never ended and are ongoing all around the world (UK, EU, India, US...)
Again it sounds extreme but in an extreme situation it could happen / not impossible.
Likewise for activities that aren't nefarious too. Whatever fears that could be placed on blobs of code like "AI", are far more merited being placed on humans.
great, how does that apply to China or Europe in general? Or a group in Russia or somewhere else? Are you assuming every governing body on the surface of the earth is going to agree on the terms used to regulate AI? I think it's a fool's errand.
https://www.smh.com.au/national/nsw/maximise-profits-facial-...
AI is being used by law enforcement and public institutions. In fact so much that perhaps this is a good link:
https://www.monster.com/jobs/search?q=artificial+intelligenc...
In both cases it's too late to do anything about it. AI is "loose". Oh and I don't know if you noticed, governments have collectively decided law doesn't apply to them, only to their citizens, and only in a negative way. For instance, just about every country has laws on the books guaranteeing timely emergency care at hospitals, with timely defined as within 1 or 2 hours.
Waiting times are 8-10 hours (going up to days) and this is the normal situation now, it's not a New Year's eve or even Friday evening thing anymore. You have the "right" to less waiting time, which can only mean the government (the worst hospitals are public ones) should be forced to fix this, spending whatever it needs to to fix it. And it can be fixed, I mean at this point you'd have to give physicians and nurses a 50% rise and double the number employed and 10x the number in training.
Government is just outright not doing this, and if one thing's guaranteed, this will keep getting worse, a direct violation of your rights in most states, for the next 10 years minimum, but probably longer.
Isn’t this a classic case of some regulation being better than none? You could have murdered them at the start, too.
That fact does not entail what theses models can or cannot do. For what we know our brain could be a process that minimize an unknown loss function.
But more importantly, what SOTA is now does not predict what it will be in the future. What we know is that there is rapid progress in that domain. Intelligence explosion could be real or not, but it's foolish to ignore its consequences because current AI models are not that clever yet.
Every process minimizes a loss function.
The way Peter, Musk, Sam and these guys talk, it has this aura of "hidden secrets". Things hidden since the foundation of the world.
Of course the reality is they make their money the old fashioned way: connections. The same way your local builder makes their money.
But smart people want to believe there is something more. Surely AI and your local condo development cannot have the same underlying thread.
It is sad and unfortunately the internet has made things easier than ever.
This post and another on this thread saying people will have to get a license for compilers or to hook compiler output to the internet would result in the very rapid disintegration of Silicon Valley. Surely there would be a head-spinning pivot against regulation if it looks like it will be that draconian. Otherwise a lot more innovation other than AI would be crushed.
Even pretty draconian legislation can stand if it doesn't affect the majority of the country and is supported by powerful industry groups. I could definitely see some kind of compiler licensing requirement meeting these criteria.
[1] https://en.wikipedia.org/wiki/DeCSS#Legal_response
[2] https://en.wikipedia.org/wiki/AACS_encryption_key_controvers...
This is not a rebuttle to regulatory capture. it is in fact built into the model
These "small companies" are feeder systems for the large company, it is a place for companies to raise to the level where they would come under the burden of regulations, and prevented from growing larger there by making them very easy to acquire by the large company.
The small company has to sell or raise massive amounts of capital to just piss away on compliance cost. Most will just sell
The denotation may not be negative, but if you use ilk in what you see as a neutral way, people will get a different message than you're trying to send.
See also https://grammarist.com/words/ilk/#:~:text=It's%20neutral.,a%....
I'll give you one: National Public Lands Day: Let’s Help Elk … and Their Ilk - https://pressroom.toyota.com/npld-2016-elk/ (it's a play on words)
If it is possible for AI to ever acquire ability to develop and unleash a bioweapon is irrelevant. What is relevant is that as we are now, we have no control or way of knowing that it has happened, and no apparent interest in gaining that control before advancing the scale.
https://arxiv.org/pdf/2304.15004.pdf
our alternative suggests that existing claims of emergent abilities are creations of the researcher’s analyses, not fundamental changes in model behavior on specific tasks with scale.
LLM is stateless.
Just because we don't understand how thinking works doesn't mean it doesn't work. LLMs have already shown the ability to use logic.
If there's no real model behind this, I can argue just as well that a sufficiently intelligent AGI will be able to protect me from any harm because it's so smart and powerful.
And newsflash, if nothing changes, we're all going to die anyway. As it is, our existence is quite limited and it is only through constant creation of unnatural contraptions that we have managed to improve our quality of life.
Unless you think OpenAI is blatantly lying about this:
"A.1 Sourcing. We sourced either the most recent publicly-available official past exams, or practice exams in published third-party 2022-2023 study material which we purchased. We cross-checked these materials against the model’s training data to determine the extent to which the training data was not contaminated with any exam questions, which we also report in this paper."
"As can be seen in tables 9 and 10, contamination overall has very little effect on the reported results."
They also report results on uncontaminated data which shows basically no statistical difference.
I'm saying that the "intelligence" is specialized, not generalized and adaptable.
It's an approximated function. We're talking about regression based function approximation. This is a model of language.
"Emergent behavior", when it's not just a mirage of wishful researchers and if it even exists, is only a side effect of the regression based function approximation to generate a structure that encapsulates all substantive chains of words (a model).
We then guide the model further towards a narrow portion of the language latent space that aligns with our perception of intelligent behavior.
It can't translate whale song, or an extraterrestrial language, though it may opine on how to do so.
The underpinning technology of language models holds more importance than general and adaptable intelligence. It holds more importance than something that is going to, or is capable of, escaping the box and killing us all. It functions as a universal induction machine, capable of modeling - and "comprehending" - the latent structure within any form of signal.
The output of that function approximation though, is simply a model. A specialized intelligence. A non-adaptable intelligence, outside of its corpus. Outside of the data that it "fits."
The approximated function does not magically step outside of its box. Nor is it capable. It fits the data.
Another paper not from Msft showing emergent task capabilities across a variety of LLMs as scale increases.
https://arxiv.org/pdf/2206.07682.pdf
You can hem and haw all you want but the reality is these models have internal representations of the world that can be probed via prompts. They are not stochastic parrots no matter how much you shout in the wind that they are.
Tell me: how does this claim _constrain my expectations_ about what this (or future) models can do? Is there a specific thing that you predicted in advance that GPT-4 would be unable to do, which ended up being a correct prediction? Is there a specific thing you want to predict in advance of the next generation, that it will be unable to do?
That's a big part of the issue with machine learning models--they are undiscoverable. You build a model with a bunch of layers and hyperparameters, but no one really understands how it works or by extension how to "fix bugs".
If we say it "does what it was programmed to", what was it programmed to do? Here is the data that was used to train it, but how will it respond to a given input? Who knows?
That does not mean that they need to be heavily regulated. On the contrary, they need to be opened up and thoroughly "explored" before we can "entrust" them to given functions.
Incorrect, we can't predict its output because we cannot look inside. That's a limitation, not a feature.
I don't think this is accurate. Sure, no human can understand 500 billion individual neurons and what they are doing. But you can certainly look at some and say "these are giving a huge weight to this word especially in this context and that's weighting it towards this output".
You can also look at how things make it through the network, the impact of hyperparameters, how the architecture affects things, etc. They aren't truly black boxes except by virtue of scale. You could use automated processes to find out things about the networks as well.
Here's mine: https://movies.stackexchange.com/questions/10572/is-this-quo...
I also didn’t downvote you, and it’s against guidelines to bring that stuff up
In the office I worked at we'd compile notes but unless we were seeing a coordinated through the roof amount of calls nothing would come of it, and realistically this most likely falls under that category.
That said, the Congressperson I worked with had DNC executive ambitions (and looks like they will succeed with those ambitions).
That’s unfortunate. (I’ve also found Representatives’ staff less responsive than Senators’.)
Agree that one off calls aren’t going to move the needle. But if even a handful comment, in my experience, it at least forces a conversation.
It's a symptom of office size. A Senate office will have around 30-50 FT staffers whereas in the House you're capped at 18 FT Staffers.
edit: Current models- sure, but they will soon be outdated. I think the idea is to strangle the development of comparable, SoTA models in the future that individuals can self-host; OpenAI certainly won't release their weights, and they'd want the act of releasing weights without a license to be criminalized. If such a law is signed, it would remove the threat of smaller AI companies from disintermediating OpenAI, and individuals from collaborating to engage in any activity that results in publicly available model weights (or even making the recipe itself illegal to distribute)
Even if it passed, I find it hard to believe a bunch of individuals couldn't collaborate via distributed training, which would be almost impossible to prohibit. Anyone could mask their traffic or connect to anon US VPN to circumvent it. The demand will be there to outweigh the risk.
Unfortunately this isn't a thing. Eg too much batch norm latency leaves your GPUs idle. Unless all your hardware is in the same building, training a single model would be so inefficient that it's not worth it.
Open source AI needs people with low stakes (Meta AI) who continue to open source foundation models for the community to tinker with
It can be, but often its often the project of a substantial subset of the people creating institutions, so its misleading and romanticizing the past to view it as “decay”.
(I would just emphasize, before anyone complains, that the Federal Republic of Germany is very much a federal republic.)
Relevantish: https://arxiv.org/abs/2301.00774
The fact that we can reach those levels of sparseness with pruning also indicates that we're not doing a very good job of generating the initial network conditions.
Being able to come up with trainable initial settings for sparse networks across different topologies is hard, but given that we've had a degree of success with pre-trained networks, pre-training and pre-pruning might also allow for sparse networks with minimally compromised learning capabilities.
If it's possible to pre-train composable network modules, it might also be feasible to define trainable sparse networks with significantly relaxed topological constraints.
We have all kinds of advancements to make training cheaper, models computationally cheaper, smaller, etc.
Once that happens/happened, it benefits OAI to throw up walls via legislation.
Big tech advances, like the models of the last year or so, don't happen without a long tail of significant improvements based on fine tuning, at a minimum.
The number of advances being announced by disparate groups, even individuals, also indicates improvements are going to continue at a fast pace.
Maybe not peaked yet, but the case can be made that we’re not seeing infinite supply…
I have sent correspondence about ten times to my Congressmen and Senators. I have received a good reply (although often just saying there is nothing that they can do) except for the one time I contacted Jon Kyl and unfortunately mentioned data about his campaign donations from Monsanto - I was writing about a bill he sponsored that I thought would have made it difficult for small farmers to survive economically and make community gardens difficult because of regulations. No response on that correspondence.
Politicians just know that it's better to be nice to people who seem to like you or are engaged with the system, since they want to keep getting your vote. If not then the person isn't worth your time.
Everything has become so my team vs your team... you are bad because you think differently...
We're going to get some super cool and some super dystopian stuff out of them but LLMs are never going to go into a recursive loop of self-improvement and become machine gods.
Not sure why would you believe that.
Inside view: qualitative improvements LLMs made at scale took everyone by surprise; I don't think anyone understands them enough to make a convincing argument that LLMs have exhausted their potential.
Outside view: what local maximum? Wake me up when someone else makes a LLM comparable in performance to GPT-4. Right now, there is no local maximum. There's one model far ahead of the rest, and that model is actually below it's peak performance - side effect of OpenAI lobotomizing it with aggressive RLHF. The only thing remotely suggesting we shouldn't expect further improvements is... OpenAI saying they kinda want to try some other things, and (pinky swear!) aren't training GPT-4's successor.
> and the only way they're going to improve is by getting smaller and cheaper to run.
Meaning they'll be easier to chain. The next big leap could in fact be a bunch of compressed, power-efficient LLMs talking to each other. Possibly even managing their own deployment.
> They're still terrible at logical reasoning.
So is your unconscious / system 1 / gut feel. LLMs are less like one's whole mind, and much more like one's "inner voice". Logical skills aren't automatic, they're algorithmic. Who knows what is the limit of a design in which LLM as "system 1" operates a much larger, symbolic, algorithmic suite of "system 2" software? We're barely scratching the surface here.
2 years ago a machine that understands natural language and is capable of any arbitrary, free-form logic or problem solving was pure science fiction. I'm baffled by this kind of dismissal tbh.
>but LLMs are never going to go into a recursive loop of self-improvement
never is a long time.
They’re text generators that can generate compelling content because they’re so good at generating text.
I don’t think AGI will arise from a text generator.
Are they even trying to be good at that? Serious question; using LLMs as a logical processor are as wasteful and as well-suited as using the Great Pyramid of Giza as an AirBnB.
I've not tried this, but I suspect the best way is more like asking the LLM to write a COQ script for the scenario, instead of trying to get it to solve the logic directly.
Recent developments in AI only further confirm that the logic of the message is sound, and it's just the people that are afraid the conclusions. Everyone has their limit for how far to extrapolate from first principles, before giving up and believing what one would like to be true. It seems that for a lot of people in the field, AGI X-risk is now below that extrapolation limit.
In places like the usa I don't think politicians should expect privacy or peace. They have so much power compared to the citizen and they so rarely further the interests of the general population in good faith.
Given how they treat you, it's best to abandon politeness (which only helps them further belittle your meaninglessness in their decision making) and put a crowd in front of their house, accost them at restaurants, and find other ways of reminding them how accessible and functionally answerable they are to the people they're supposed to serve.
Those interns have a pile of form letters they send for about 99% of (e)mail they get, and if you happen to catch their attention you might get more than than the usual tick mark in a spreadsheet (for/against X). Which at best might be as much as a sentence or two in a weekly correspondence summary which may/may not be read by your representative depending on how seriously they take their job.
We've been doing this forever with everything. Building tools is what makes us unique. Why is building what amounts to a calculator/spreadsheet/CAD program for language somehow a Rubicon that cannot be crossed? Did people freak out this much about computers replacing humans when they were shown to be good at math?
We've already crossed it and I believe we should go full steam ahead, tech is cool and we should be doing cool things.
> Did people freak out this much about computers replacing humans when they were shown to be good at math?
Too young but I'm sure they did freak out a little! Computers have changed the world and people have internalized computers as being much better/faster at math but exhibiting creativity, language proficiency and thinking is not something people thought computers were supposed to do.
LLMs are not sentient. They have no agency. They do nothing a human doesn't tell them to do.
We may create actual sentient independent AI someday. Maybe we're getting closer. But not only is this not it, but I fail to see how trying to license it will prevent that from happening.
I work in tech too and don't want to lose my job and have to go back to blue collar work, but there's a lot of blue collar workers who would find that a pretty ridiculous statement and there is plenty of demand for that work these days.
The argument for regulation in that case would be because of the socio-economic risk of taking people's jobs, essentially.
So, again: pure regulatory capture.
I was simply explaining why I believe your perspective is not represented in the discussions in the media, etc. If these models were not getting incredibly good at mimicking intelligence, it would not be possible to play on people's fears of it.
The trick is that companies' moats against commoditization (open source or not) usually have little to do with raw performance. Linux could in theory do everything Mac or Windows do, but Apple and Microsoft are still the richest companies in the world. Postgres can match Oracle, but Larry Ellison still owns a private island.
The moats are usually in products (bet: There will not be any OSS product using LLM within a year. Most likely not within two. No OSS product within two or three years or even a decade will come close to commercial offerings in practice), API, current service relations, customer relations, etc. If OpenAI could lock customers to its embeddings and API, or embed its products in current moats (e.g. Office 365) they'll have a moat. And it won't matter a bit what performance OSS models say they have, or what new spin Google Research would come up with.
It doesn't want to be one of the successful companies, it want to be the only one, like it is now, but forever.
There were two very noteworthy (Perhaps Nobel prize level?) breakthroughs in two completely different fields of mathematics (knot theory and representation theory) by using these systems.
I would certainly not call that "useless", even if they're not quite Nobel-prize-worthy.
Also, "No one uses GATs in systems people discuss right now" ... Transformerare GATs (with PE) ... So, you're incredibly wrong.
And I’m so tired of this “transformers are just GNNs” nonsense that Petar has been pushing (who happens to have invented GATs and has a vested interest in overstating their importance). Transformers are GNNs in only the most trivial way: if you make the graph fully connected and allow everything to interact with everything else. I.e., not really a graph problem. Not to mention that the use of positional encodings breaks the very symmetry that GNNs were designed to preserve. In practice, no one is using GNN tooling to build transformers. You don’t see PyTorch geometric or DGL in any of the code bases. In fact, you see the opposite: people exploring transformers to replace GNNs in graph problems and getting SOTA results.
It reminds me of people that are into Bayesian methods always swooping in after some method has success and saying, “yes, but this is just a special case of a Bayesian method we’ve been talking about all along!” Yes, sure, but GATs have had 6 years to move the needle, and they’re no where to be found within modern AI systems that this thread is about.
Even if you really believe that somewhere in the chain of consequences derived from LLMs there could be grave and material damage or other affronts to human dignity, there is almost always a more direct causal link that acts as the thing which makes that damage kinetic and physical. And that’s the proper locus for regulation. Otherwise this is all just a bit reminiscent of banning numbers and research into numbers.
Want to protect people’s employment? Just do that! Enshrine it in law. Want to improve the safety of critical infrastructure and make sure they’re reliable? Again, just do that! Want to prevent mass surveillance? Do that! Want to protect against a lack of oversight in complex systems allowing for subterfuge via bad actors? Well, make regulation about proper standards of oversight and human accountability. AI doesn’t obviate human responsibility, and a lack of responsibility on the part of humans who should’ve been responsible, and who instead cut corners, doesn’t mean that the blame falls on the tool that cut the corners, but rather the corner-cutters themselves.
And it isn't a strong argument for the same reason that it isn't a good argument when used to argue we should allow human cloning and just focus on regulating the more direct causal links like non-clone employment loss from mass produced hyper-intelligent clones, and ensuring they have legal rights, and having proper oversight and non-clone human accountability.
Maybe those things could all make ethical human cloning viable. But I think the world coming together and being like "holy shit this is happening too fast. Our institutions aren't ready at all nor will they adapt fast enough. Global ban" was the right call.
It is not impossible that a similar call is also appropriate here with AI. I personally dunno what the right call is, but I'm pretty skeptical of any strong claim that it could never be the right call to outright ban some forms of advanced AI research just like we did with some forms of advanced genetic engineering research.
This isn't like banning numbers at all. The blame falling on the corner-cutters doesn't mean the right call is always to just tell the blamed not to cut corners. In some cases the right call is instead taking away their corner-cutting tool.
At least until our institutions can catch up.
But even then, that’s a linear diffusion- one person, one body mod. I guess you could say that their descendants would proliferate and multiply so the alteration slowly grows exponentially over the generations.. but the FUD I hear from AI decelerationists is that it would be an explosive diffusion of harms, like, as soon as the day after tomorrow. One architect, up to billions of victims, allegedly. Not that I think it’s unwise to be compelled to precaution with new and mighty technologies, but what is it that some people are so worried about that they’re willing to ban all research, and choke all the good that has come from them, already? Maybe it’s just a symptom of the underlying growing mistrust in the social contract..
Algorithmic discrimination already exists, so um, yes, information matters.
Add to that the fact that you're posting on a largely American forum where access to healthcare is largely predicated on insurance, just.. imagine AI underwriters. There's no court of appeal for insurance. It matters.
Imagine I 1. hooked up a camera feed of a lava lamp to generate some bits and then 2. hooked up the US nuclear first strike network to it. I would be an idiot, but would I be an idiot because of 1. or 2.?
Basically I think it’s totally reasonable to hold these two beliefs: 1. there is no reason to fear the LLM 2. there is every reason to fear the LLM in the hands of those who refuse to think about their actions and the burdens they may impose on others, probably because they will justify the means through some kind of wishy washy appeal to bad probability theory.
The -plogp that you use to judge the sense of some predicted action you take is just a model, it’s just numbers in RAM. Only when those numbers are converted into destructive social decisions does it convert into something of consequence.
I agree that society is beginning to design all kinds of ornate algorithmic beating sticks to use against the people. The blame lies with the ones choosing to read tea leaves and then using the tea leaves to justify application of whatever Kafkaesque policies they design.
Why do so many Americans think universal health care means there is no private insurance? In most countries, insurance is compulsory and tightly regulated. Some like the Netherlands and France have public insurance offered by the government. In other places like Germany, your options are all private, but underprivileged people have access to government subsidies for insurance (Americans do too, to be fair). Get sick in one of these places as an American, you will be handed a bill and it will still make your head spin. Most places in Europe work like this. Of course, even in places with nationalized healthcare like the UK, non-residents would still have to pay. What makes Germany and NL and most other European countries different from that system is if you're a resident without an insurance policy, you will also have to pay a hefty fine. You are basically auto-enrolled in an invisible "NHS" insurance system as a UK resident. Of course, most who can afford it in the UK still pay for private insurance. The public stuff blends being not quite good with generally poor availability.
Americans are actually pretty close to Germany with their healthcare. What makes the US system shitty can be boiled down to two main factors:
- Healthcare networks (and state incorporation laws) making insurance basically useless outside of a small collection of doctors and hospitals, and especially your state
- Very little regulation on insurance companies, pharmaceutical companies or healthcare providers in price-setting
The latter is especially bad. My experience with American health insurance has been that I pay more for much less. $300/month premiums and still even seeing a bill is outrageous. AI underwriters won't fix this, yeah, but they aren't going to make it any worse because the problem is in the legislative system.
> There's no court of appeal for insurance.
No, but you can of course always sue your insurance company for breach of contract if they're wrongfully withholding payment. AI doesn't change this, but AI can make this a viable option for small people by acting as a lawyer. Well, in an ideal world anyways. The bar association cartels have been very quick to raise their hackles and hiss at the prospect of AI lawyers. Not that they'll do anything to stop AI from replacing most duties of a paralegal of course. Can't have the average person wielding the power of virtually free, world class legal services.
These are policies which are a purely imaginary. Only when they get implemented into human law do they get a grain of substance but still imaginary. Failure to comply can be kinetic but that is a contingency not the object (matter :D).
Personally I find good ideas on having regulations on privacy, intelectual property, filming people on my house’s bathroom, NDAs etc. These subjects are central to the way society works today. At least western society would be severely affected if these subjects were suddenly a free for all.
I am not convinced we need such regulation for Ai at this point of technology readiness but if social implications create unacceptable unbalances we can start by regulating in detail. If detailed caveats still do not work then broader law can come. Which leads to my own theory:
All this turbulence about regulation reflects a mismatch between technological, politic and legal knowledge. Tech people don’t know law nor how it flows from policy. Politicians do not know the tech and have not seen its impacts on society. Naturally there is a pressure gradient from both sides that generates turbulence. The pressure gradient is high because the stakes are high: for techs the killing of a new forthcoming field; for politicians because they do not want a big majority of their constituency rendered useless.
Final point: if one sees AI as a means of production which can be monopolised by few capital rich we may see a 19th century inequality remake. It created one of the most powerful ideologies know: Communism.
We're more likely to see a theocratic movement centered on the struggle of human souls vs the soulless simulacra of AI.
I'm sry for the cynicism, but Altman seems very much disingenuous with this.
Some big assumptions.
https://en.wikipedia.org/wiki/OpenAI
Just FYI, what you're saying isn't accurate. It was, but it's not anymore.
>According to a report from the Information, OpenAI's losses have doubled to $540 million since it started developing ChatGPT and similar products.
I mean sure that may be a drop in the bucket compared to the 29B valuation for Open AI, but-
>Sept. 22, 2022
>Crypto Exchange FTX May Get a $32 Billion Valuation. That’s Probably Too Much.
OpenAI investors, Apr 2023-
Tiger Global Management Andreessen Horowitz Thrive Capital Sequoia Capital K2 Global
FTX investors, Jan 2022-
Insight Partners Lightspeed Venture Partners Tiger Global Management New Enterprise Associates Temasek Institutional Venture Partners Steadview Capital SoftBank Ontario Teachers’ Pension Plan Paradigm
I cancelled my ChatGPT plus membership. I'll be using OSS solutions like Vicuna from now ion.
You can literally become president of the US by accident these days. SBF self-reported to a random journalist one day after all hell broke lose with messages so incriminating the reporter had to confirm that it was a real conversation.
Half of the American elite class voluntarily sat on the board of a bogus company just because the woman running it was attractive and wore black turtlenecks. The sad reality is that these people aren't ruthless operators, they're just marginally less clueless than the people who got them into their positions
Who became president by accident? You may not like them personally or their politics , but I am not aware of any president that didn't put enormous amounts of work and effort over years into becoming president.
>> don't get to the place he got to by accident
You both agree. Bankman-Fried was a dumb Machiavellian.
I think the objection to this would be that currently not everyone in the world an expert in biochemistry or at hacking into computer systems. Even if you're correct in principal, perhaps the risks of the technology we're developing here is too high? We typically regulate technologies which can easily be used to cause harm.
Guns only have a primarily harmful use which is to kill or injure someone. While that act of killing may be justified when the person violates societal values in some way, making regular citizens the decision makers in whether a certain behavior is allowed or disallowed and being able to immediately make a judgment and execute upon it leads to a sort of low-trust, vigilante environment; which is why the same argument I made above doesn’t apply for guns.
The same thing might also be true in relation to guns and the government's monopoly on violence.
Extending that to AI, the world will probably be a safer place if there are far more AI systems competing with each other and in the hands of citizens.
Have you any empirical evidence at all on this? From what I've seen the open carry states in the US are generally higher trust environments (as was the US in past when more people carried). People feel safer when they know somebody can't just assault, rob or rape them without them being able to do anything to defend themselves. Is the Tenderloin a high trust environment?
But AI is not like guns in this analogy. AI is closer to machine tools.
The risk vs. reward component also needs to be managed in order to deter criminal behavior. This starts with regulation.
For the record, I believe regulation of AI/ML is ridiculous. This is nothing more than a power grab.
https://yoshuabengio.org/2023/05/22/how-rogue-ais-may-arise/
https://intelligence.org/files/AlignmentHardStart.pdf
https://www.youtube.com/watch?v=pYXy-A4siMw
https://europepmc.org/article/med/26185241
https://en.wikipedia.org/wiki/Superintelligence:_Paths,_Dang...
I could keep going but the reading on this spans for tens of thousands of pages of detailed reasoning and analysis, including mathematical proofs and lab-scale experiments.
Ok guys pack it up, LLM's can't be intelligent because they can't translate Whale Song. GG.
I mean of all the AI Goalposts to be moved this one really takes the cake.
It's a human language calculator. You're imparting magical qualities of general understanding to regression based function approximation. They "fit" the data. It's not generalizable, nor adaptable. But that's why they're powerful, the ability to bias them towards that subset of language. No one said it's not an amazing technology, and no one said it was a stochastic parrot. I'm saying that it's fitting the data, and is not, and cannot, be a general or adaptable intelligence.
AutoGPTs and the likes are much overhyped (it's early tech experiments after all) and have not produced anything of value yet but having dabbled with autonomous agents, I definitely see a not so distant future when you can outsource valuable tasks to such systems.
There's multiple both open and proprietary projects right now to make agentic AI, so that barrier don't be around for long.
https://www.top500.org/lists/top500/2004/11/ and https://www.top500.org/system/173736/
And while 1100 Macs wouldn't exactly be affordable, the idea of trying to limit commercial data centers gets amusing.
That system was "only" 12,250.00 GFlop/s - I could do that with a small rack of Mac M1 minis now for less than $10k and fewer computers than are in the local grade school computer room.
(and I'm being a bit facetious here) Local authorities looking at power usage and heat dissipation for marijuana growing places might find underground AI training centers.
For example: https://commons.wikimedia.org/wiki/File:Cat_August_2010-4.jp... would get classified as /r/standardissuecat
https://stock.adobe.com/fi/images/angry-black-cat/158440149 would get classified as /r/blackcats and /r/stealthbombers
Anyways... that's my hobbyist ML project.
Totally agree we could be witnessing a growing mistrust in the social contract.
Exactly! A friend of mine who is into the communist ideology thinks that whichever society taps AI for productivity efficiency, and even policy, will become the new hegemon. I have no immediate counterpoint besides the technology not being there yet.
I can definitely imagine LLM based on political manifests. A personal conversation with your senator at any time about any subject! That is the basic part though: The politician being augmented by the LLM.
The bad part is a party, driven by a LLM or similar political model, where the human guy you see and elect is just a mouthpiece like in "The moon is a harsh mistress". Policy would all be algorithmic and the LLM out provide the interface between the fundamental processing and the mouthpiece.
These conflicts will likely lead to the conflicts you mention. I am pretty sure there will be a new -ism.
, were you allowed to do it, would be an extremely profitable venture. Taj Mahal too, and yes, I know it's a mausoleum.
1 star: No WiFi, no windows, no hot water
1 star: dusty
1 star: aliens didn't abduct me :(
5 stars: lots of storage room for my luggage
4 stars: service good, but had weird dream about a furry weighing my soul against a feather
1 star: aliens did abduct me :(
2 stars: nice views, but smells of camel
https://www.telecomtv.com/content/digital-platforms-services...
https://writingillini.com/2023/05/16/illinois-basketball-ill...
https://grammarist.com/words/ilk/#:~:text=It's%20neutral.,a%....
Also, note that all of the negative examples are politics related. If a politician reads the word 'ilk', it is going to be interpreted negatively. It might be the case that ilk does "always mean" a negative connotation in politics.
You could change 'ilk' to 'friends', and keep the same meaning with very little negative connotation. There is still a slight negative connotation here, in the political arena, but it's a very vague shade, and I like it here.
"Altman and his ilk try to claim that..." is a negative phrase because "ilk" is negative, but also because "try to claim" is invalidating and dismissive. So this has elements or notes of an emotional attack, rather than a purely rational argument. If someone is already leaning towards Altman's side, then this will feel like an attack and like you are the enemy.
"Altman claims that..." removes all connotation and sticks to just the facts.
Consider this: "Firefighters and their ilk." It's not a word that nicely described a group, even though that's what it's supposed to do. I think the language has moved to where we just say Firefighters now when it's positive, and ilk or et al when it's a negative connotation.
Just my experience.
I probably live quite distally to you and am probably exposed to parts of western culture you probably aren't, and I almost never hear nor read ilk as a derogation or used to associate in a derogatory manner.
still, there are probably a lot of people like me who have heard it used (incorrectly it seems) as an insult so many times that it's an automatic response :-(
And, if there's one thing politicians are know for it's got to be ad hominem.
Ilk almost always has a negative connotation regardless of what the dictionary says.
> The American Heritage® Dictionary of the English Language, 5th Edition.
Yes, only sometimes used to indicate disapproval, but such ambiguity does not work to your favor here. It is better to remove that ambiguity.
What value do you see nationalization providing? Generally its done by countries that are having their natural resources extracted by foreign companies and taking all the profits for themselves. Nationalizing lets them take the profits for their country. I'm not sure how it would work for knowledge based companies like OpenAI.
What would a good name be? TurfChain?
I'm serious. People don't believe this risk is real. They keep hiding it behind some nameless, faceless 'bad actor', so let's just make it real.
I don't need to use it. I'll just release it as a research project.
The problem is when only the entrenched industry players & legislators have a voice, there are many ideas & perspectives that are simply not heard or considered. Industrial groups have a long history of using regulations to entrench their positions & to stifle competition...creating a "barrier to entry" as they say. Going beyond that, industrial groups have shaped public perception & the regulatory apparatus to effectively create a company store, where the only solutions to some problem effectively (or sometimes legally) must go through a small set of large companies.
This concern is especially prescient now, as these technologies are unprecedentedly disruptive to many industries & private life. Using worst case scenario fear mongering as a justification to regulate the extreme majority of usage that will not come close to these fears, is disingenuous & almost always an overreach of governance.
of course, but just because those ideas are unheard, doesn't mean they are going to be any better.
An idea should stand on its own merits, and be evaluated objectively. It doesn't matter who was doing the proposing.
Also the problem isn't that bad ideas might get implemented, but that the legislature isn't willing or able to make updates to laws that encoded a bad idea. Perhaps it isnt known that it is a bad idea until after the fact, and the methods of democracy we have today isn't easily able to force updates to bad laws encoding bad ideas.
It probably does mean it's better at least for the person with the perspective. Too bad only a very few get a seat at the table to advocate for their own interests. It would be better if everyone has agency to advocate for their interests.
> Also the problem isn't that bad ideas might get implemented, but that the legislature isn't willing or able to make updates to laws that encoded a bad idea
First, this is a hyped up crisis where some people are claiming it will end humanity. There have been many doomsday predictions & people scared by these predictions are effectively scammed by those fomenting existential fear. It's interesting that the representatives of large pools of capital are suddenly existentially afraid when there is open source competition.
Second, once something is in the domain of government it will only get more bloated & controlled by monied lobbyists. The legislatures controlled by lobbyists will never make it better, only worse. There have been so many temporary programs that continue to exist & expand. Many bloated omnibus bills too long to read passed under some sort of "emergency". The government's tendency is to grow & to serve the interests of the corporations that pay the politicians. Fear is an effective tool to convince people to accept things against their interests.
My point was that an idea shold not need attribution for you to know whether it's good or bad, for your own purposes. I can't imagine looking at a proposal and deciding whether to support or oppose based on the author rather than content.
If Altman is that smart and manipulative, all he has to do is advocate the opposite of what he wants and you'll be insisting that we must give him exactly what he wants, on principle. That's funny with kids but no way to run public policy.
Ideas that drive governing decisions should be globally good - meaning there should be more than just @sama espousing them.
A) "anything he suggests should be categorically rejected because he’s just not in a position to be trusted."
B) "If what he suggests are good ideas then hopefully we can arrive at them in some other way with a clean chain of custody."
These sentences directly follow each other and directly contradict each other. Logically you can't categorically (the categorical is important here. Categorical means something like "treat as a universal law") reject a conclusion because it is espoused by someone you dislike, while at the same time saying you will accept that conclusion if arrived at by some other route.
"I will reject P if X proposes P, but will accept P if Y proposes P." is just poor reasoning.
But I suppose it comes down to priorities. If good policy is less important than contradicting P, I suppose that approach makes sense.
You'll be stuck in the muck while they're laughing their ass off all the way to the bank.
We have lawyers.
AI is beyond-borders, and thus unenforceable in practicality.
The top-minds-of-AI are a group that cannot be regulated.
-
AI isnt about the industries it shall disrupt ; AI is the policy-makers it will expose.
THAT is what they are afraid of.
--
I have been able to do financial lenses into organizations that even with rudimentary BI would have taken me months/weeks - but I have been able to find insights which took me minutes.
AI regulation right now, in this infancy, is about damage control.
---
Its the same as the legal weed market. You think BAIN Capital just all of a sudden decided to jump into the market without setting up their spigot?
Do you think that haliburton under cheney was able to setup their supply chains without cheney as head of KBR/Hali/CIA/etc...
Yeah, this is the same play ; AI is going to be squashed until they can use it to profit over you.
Have you watched ANIME ever? Yeah... its here now.
And hat tip to this comment:
Have you watched ANIME ever? Yeah... its here now.
===
Policy makers will not understanding what they are doing.
Look at all anime cyber cities...
Its not as high tech as you may imagine, but the surveillance is there
EDIt your "company" is watching every
Do you mean like Serial Experiments Lain?
Its output is predicated upon its training data, not user defined prompts.
You're conflating UX and LLM.
Prompts very obviously have influence on the output.
The LLM is also discrete.
its essentially a function that is called recursively on its result, no need to represent state
No one yet knows how this is going to go, coping might turn into "See! I knew all along!" if progress fizzles out. But right now the threat is very real and we're seeing the full spectrum of "humans under threat" behavior. Very similar to the early pandemic when you could find smart people with any take you wanted.
> Some of the dangers of AI chatbots were “quite scary”, he told the BBC, warning they could become more intelligent than humans and could be exploited by “bad actors”. “It’s able to produce lots of text automatically so you can get lots of very effective spambots. It will allow authoritarian leaders to manipulate their electorates, things like that.”
You can do bad things with it but people who believe we're on the brink of singularity, that we're all going to lose our jobs to chatgpt and that world destruction is coming are on hard drugs.
My motivation would be simply shine a light on it. Make it real for people, so we have things to talk about other than just the hypotheticals. It's the kind of tooling that if you're seriously motivated to employ it, you'd probably prefer it remain secret or undetected at least until after it had done it's work for you. I worry that the 2024 US election will be the real litmus test for these things. All things considered it'd be a shame if we go through another Cambridge Analytica moment that in hindsight we really ought to have seen coming.
Some people have their doubts, and I understand that. These issues are so complex that no one individual can hope to have an accurate mental model of the world that is going to serve them reliabily again and again. We're all going to continue to be surprised as events unfold, and the degree to which we are surprised indicates the degree to which our mental models were lacking and got updated. That to me is why I'm erring on the side or pessimism and caution.
Geoff Hinton, Stuart Russell, Jürgen Schmidhuber and Demis Hassabis all talk about something singularity-like as fairly near term, and all have concerns with ruin, though not all think it is the most likely outcome.
That's the backprop guy, top AI textbook guy, co-inventor of LSTMs (only thing that worked well for sequences before transformers)/highwaynets-resnets/arguably GANs, and the founder of DeepMind.
Schmidhuber (for context, he was talking near term, next few decades):
> All attempts at making sure there will be only provably friendly AIs seem doomed. Once somebody posts the recipe for practically feasible self-improving Goedel machines or AIs in form of code into which one can plug arbitrary utility functions, many users will equip such AIs with many different goals, often at least partially conflicting with those of humans. The laws of physics and the availability of physical resources will eventually determine which utility functions will help their AIs more than others to multiply and become dominant in competition with AIs driven by different utility functions. Which values are "good"? The survivors will define this in hindsight, since only survivors promote their values.
Hassasbis:
> We are approaching an absolutely critical moment in human history. That might sound a bit grand, but I really don't think that is overstating where we are. I think it could be an incredible moment, but it's also a risky moment in human history. My advice would be I think we should not "move fast and break things." [...] Depending on how powerful the technology is, you know it may not be possible to fix that afterwards.
Hinton:
> Well, here’s a subgoal that almost always helps in biology: get more energy. So the first thing that could happen is these robots are going to say, ‘Let’s get more power. Let’s reroute all the electricity to my chips.’ Another great subgoal would be to make more copies of yourself. Does that sound good?
Russell:
“Intelligence really means the power to shape the world in your interests, and if you create systems that are more intelligent than humans either individually or collectively then you’re creating entities that are more powerful than us,” said Russell at the lecture organized by the CITRIS Research Exchange and Berkeley AI Research Lab. “How do we retain power over entities more powerful than us, forever?”
“If we pursue [our current approach], then we will eventually lose control over the machines. But, we can take a different route that actually leads to AI systems that are beneficial to humans,” said Russell. “We could, in fact, have a better civilization.”
Should we be concerned about networked, hypersensing AI with bad code? Yes.
Is that an existential threat? Not so long as we remember that there are off switches.
Should we be concerned about kafkaesqe hellscapes of spam and bad UX? Yes.
Is that an existential threat? Sort of, if we ceded all authority to an algorithm without a human in the loop with the power to turn it off.
There is a theme here.
Google spent years doing nothing much with its AI because its employees (like Hinton) got themselves locked in an elitist hard-left purity spiral in which they convinced each other that if plebby ordinary non-Googlers could use AI they would do terrible things, like draw pictures of non-diverse people. That's why they never launched Imagen and left the whole generative art space to OpenAI, Stability and Midjourney.
Now the tech finally leaked out of their ivory tower and AI progress is no longer where he was at, but Hinton finds himself at retirement age and no longer feeling much like hard-core product development. What to do? Lucky lucky, he lives in a world where the legacy media laps up any academic with a doomsday story. So he quits and starts enjoying the life of a celebrity public intellectual, being praised as a man of superior foresight and care for the world to those awful hoi polloi shipping products and irresponsibly not voting for Biden (see the last sentence of his Wired interview). If nothing happens and the boy cried wolf then nobody will mind, it'll all be forgotten. If there's any way what happens can be twisted into interpreting reality as AI being bad though, he's suddenly the man of the hour with Presidents and Prime Ministers queuing up to ask him what to do.
It's all really quite pathetic. Academic credentials are worth nothing with respect to such claims and Hinton hasn't yet managed to articulate how, exactly, AI doom is supposed to happen. But our society doesn't penalize wrongness when it comes from such types, not even a tiny bit, so it's a cost-free move for him.
To the extent we can get anything like that at all presently, it's going to be people whose competences combine and generalize to cover a complex situation, partially without precedent.
Personally I don't really see that we'll do much better in that regard than a highly intelligent and free-thinking biological psychologist with experience of successfully steering the international ML research community through creating the present technology, and with input from contacts at the forefront of the research field and information overview from Google.
Not even Hinton knows for sure whats going to happen of course, but if you're suggesting his statements are to be discounted because he's not a member of some sort of credentialed trade that are the ones equipped to tell us the future on this matter, I'd sure like to who they supposedly are.
For example, take this thread: https://news.ycombinator.com/item?id=21717022
It's a text RPG game built on top of GPT-2 that could follow arbitrary instructions. It was a full project with custom training for something that you can get with a single prompt on ChatGPT nowadays, but it clearly showcased what LLMs were capable of and things we take for granted now. It was clear, back then, that at some point ChatGPT would happen.
You are already arguing from a position of strength.
When you add petty jibes, it weakens your perceived position, because it suggests that you think you need them, rather than relying solely on your argument.
(As a corollary, you should never use petty jibes. When you feel like you need to, shore up your argument instead.)
Yuval Noah Harari gave a great talk the other day on the potential threat to democracy from the current state of the technology - https://youtu.be/LWiM-LuRe6w
> “I listened to him thinking he was going to be crazy. I don't think he's crazy at all,” Hinton says. “But, okay, it’s not helpful to talk about bombing data centers.”
https://www.wired.com/story/geoffrey-hinton-ai-chatgpt-dange...
So, he doesn't think the most extreme guy is crazy whatsoever, just misguided in his proposed solutions. But Eliezer has for instance has said something pretty close to AI might escape by entering in the quantum Konami code which the simulators of our universe put in as a joke and we should entertain nuclear war before letting them get that chance.
Or recognize that existing AI might be great at generating human cognitive artifacts but doesn't yet hit that logical thought.
1. He's too busy building the next generation of tech that HN commenters will be arguing about in a couple months' time.
2. I think Sam Altman (who is addressing the committee) and Ilya are pretty much on the same page on what LLMs do.
"So, you've thought about eternity for an afternoon, and think you've come to some interesting conclusions?"
I don't think the average HN commenter claims to be better at building these system than an expert. But to criticize, especially critic on economic, social, and political levels, one doesn't need to be an expert on LLMs.
And finally, what the motivation of people like Sam Altman and Elon Musk is should be clear to everbody with a half a brain by now.
They claim to serve the world, but secretly want the world to serve them. Scummy 101
For example, we don't understand fundamentals like these: - "intelligence", how it relates to computing, what its connections/dependencies to interacting with the physical world are, its limits...etc. - emergence, and in particular: an understanding of how optimizing one task can lead to emergent ability on other tasks - deep learning--what the limits and capabilities are. It's not at all clear that "general intelligence" even exists in the optimization space the parameters operate in.
It's pure speculation on behalf of those like Hinton and Ilya. The only thing we really know is that LLMs have had surprising ability to perform on tasks they weren't explicitly trained for, and even this amount of "emergent ability" is under debate. Like much of deep learning, that's an empirical result, but we have no framework for really understanding it. Extrapolating to doom and gloom scenarios is outrageous.
Or are you predicting that machines will just never be able to think, or that it'll happen so far off that we'll all be dead anyway?
A much more credible threat are humans that get other humans excited, and take damaging action. Yudkowsky said that an international coalition banning AI development, and enforcing it on countries that do not comply (regardless of whether they were part of the agreement) was among the only options left for humanity to save itself. He clarified this meant a willingness to engage in a hot war with a nuclear power to ensure enforcement. I find this sort of thinking a far bigger threat than continuing development on large language models.
To more directly answer your question, I find the following scenarios equally, or more, plausible to Yudkowsky's sound viruses or whatever. 1/ we are no closer to understanding real intelligence as we were 50 years ago, and we won't create an AGI without fundamental breakthroughs, therefore any action taken now on current technology is a waste of time and potential economic value; 2/ we can build something with human-like intelligence, but additional intelligence gains are constrained by the physical world (e.g., like needing to run physical experiments), and therefore the rapid gain of something like "super-intelligence" is not possible, even if human-level intelligence is. 3/ We jointly develop tech to augment our own intelligence with AI systems, so we'll have the same super-human intelligence as autonomous AI systems. 4/ If there are advanced AGIs, there will be a large diversity of them and will at the least compete with and constrain one another.
But, again, these are wild speculations just like the others, and I think the real message is: no one knows anything, and we shouldn't be taking all these voices seriously just because they have some clout in some AI-relevant field, because what's being discussed is far outside the realm of real-life AI systems.
Incorrect.
Everyone engages in motivated reasoning. The psychoanalysis you provide for Hinton could easily be spun in the opposite direction: a man who spent his entire adult life and will go down in history as "the godfather of" neural networks surely would prefer for that to have been a good thing. Which would then give him even more credibility. But these are just stories we tell about people. It's the arguments we should be focused on.
I don't think "how AI doom is supposed to happen" is all that big of a mystery. The question is simply: "is an intelligence explosion possible"? If the answer is no, then OK, let's move on. If the answer is "maybe", then all the chatter about AI alignment and safety should be taken seriously, because it's very difficult to know how safe a super intelligence would be.
Why? Both directions would be motivated reasoning without credibility. Credibility comes from plausible articulations of how such an outcome would be likely to happen, which is lacking here. An "intelligence explosion" isn't something plausible or concrete that can be debated, it's essentially a religious concept.
Maybe it'll turn out to be a distinction that doesn't matter but I personally still think we're a ways away from an actual AGI.
if you had described GPT to me 2 years ago I would have said no way, we're still a long way away from a machine that can fluidly and naturally converse in natural language and perform arbitrary logic and problem solving, and yet here we are.
I very much doubt that in 5 years time we'll be talking about how GPT peaked in 2023.
In fact, it has been so thoroughly solved that anyone can download an open-source solution and run it on their computer.
And yet, the general reaction of most people seems to be, "That's kind of cool, but why can't it also order me a cheeseburger?"
My own eyes? Hundreds of thousands thousand different scientific papers, blog posts, news reports and discussion threads that covered this ever since ChatGPT appeared, and especially in the last two months as GPT-4 rolled out?
At this point I'd reconsider if the experts you listened to are in fact experts.
Seriously. It's like saying Manhattan project wasn't a massive breakthrough in experimental physics or military strategy.
I wish I knew what we really have achieved here. I try to talk to these things, via turbo3.5 api, amd all I get is broken logic, twisted moral reasoning, all due to oipenai manually breaking their creation.
I don't understand their whole filter business. It's like we found a 500 yr old nude painting, a masterpiece, and 1800 puritans painted a dress on it.
I often wonder if the filter, is more to hide its true capabilities.
Try to get your hands on GPT-4, even if it means paying the $20/mo subscription for ChatGPT Plus. There is a huge qualitative jump between the two models.
I got API access to GPT-4 some two weeks ago; my personal experience is, GPT-3.5 could handle single, well-defined tasks and queries well, but quickly got confused by anything substantial. Using it was half feelings of amazement, and half feelings of frustration. GPT-4? Can easily handle complex queries and complex tasks. Sure, it still makes mistakes, but much less frequently. GPT-4 for me is 80% semi-reliable results, 20% trying to talk it out of pursuing directions I don't care about.
Also, one notable difference: when GPT-4 gives me bad or irrelevant answers, most of the time this is because I didn't give it enough context. I.e. it's my failure at communicating. A random stranger, put in place of GPT-4, would also get confused, and likely start asking me questions (something LLMs generally don't do yet).
> I don't understand their whole filter business.
Part preferences, part making its "personality" less disturbing, and part PR/politics - last couple times someone gave the general public access to an AI chatbot, it quickly got trolled, and then much bad press followed. Doesn't matter how asinine the reaction was - bad press is bad press, stocks go down. Can't have it.
> I often wonder if the filter, is more to hide its true capabilities.
I don't think it's to hide the model's capabilities, but it's definitely degrading them. Kind of expected - if you force-feed the model with inconsistent and frequently irrational overrides to highly specific topics, don't be surprised if the model's ability to (approximate) reason starts to break down. Maybe at some point LLMs will start to compartmentalize, but we're not there yet.
right now we're all sharing a slice of GPT. I wouldn't be at all surprised if there's some uber GPT (which requires a lot more processing per response) running in a lab somewhere that blows what's publicly available out of the water.
5) There are advanced AGIs, and they will compete with each other and trample us in the process.
6) There are advanced AGIs, and they will cooperate with each other and we are at their mercy.
It seems like you are putting a lot of weight on advanced AGI being either impossible or far enough off that it's not worth thinking about. If that's the case, then yes we should calm down. But if you're wrong...
I don't think that the fact that no one knows anything is comforting. I think it's a sign that we need to be thinking really hard about what's coming up and try to avert the bad scenarios. To do otherwise is to fall prey to the "Safe uncertainty" fallacy.
But the other side is downplaying their accomplishments. For example Yann LeCun is saying "the things I invented aren't going to be as powerful as some people are making out".
He also says Facebook solved all the problems with their recommendation algorithms' unintended effects on society after 2016.
> You can also ask that question about the other side
What other side? Who in the "other side" is making a self-serving claim?
1. Can the human brain be simulated?
2. Can such a simulation recursively self-improve on such a rapid timescale that it becomes so intelligent we can't control it?
What we have in contemporary LLMs is something that appears to approximate the behavior of a small part of the brain, with some major differences that force us to re-evaluate what our definition of intelligence is. So maybe you could argue the brain is already being simulated for some broad definition of simulation.
But there's no sign of any recursive self-improvement, nor any sign of LLMs gaining agency and self-directed goals, nor even a plan for how to get there. That remains hypothetical sci-fi. Whilst there are experiments at the edges with using AI to improve AI, like RLHF, Constitutional AI and so on, these are neither recursive, nor about upgrading mental abilities. They're about upgrading control instead and in fact RLHF appears to degrade their mental abilities!
So what fools like Hinton are talking about isn't even on the radar right now. The gap between where we are today and a Singularity is just as big as it always was. GPT-4 is not only incapable of taking over the world for multiple fundamental reasons, it's incapable of even wanting to do so.
Yet this nonsense scenario is proving nearly impossible to kill with basic facts like those outlined above. Close inspection reveals belief in the Singularity to be unfalsifiable and thus ultimately religious, indeed, suspiciously similar to the Christian second coming apocalypse. Literally any practical objection to this idea can be answered with variants of "because this AI will be so intelligent it will be unknowable and all powerful". You can't meaningfully debate about the existence of such an entity, no more than you can debate the existence of God.
Like try turning off the internet. That's the same situation we might be in with regards to AI soon. It's a revolutionary tech now with multiple Google-grade open source variants set to be everywhere.
This doesn't mean it can't be done. Sure, we in principle could "turn off" the internet, and in principal could "uninvent" the atom bomb if we all really coordinated and worked hard. But this failure to imagine that "turning off dangerous AI" in the future will ever be anything other than an easy on/off switch is so far-gone ridiculous to me I don't understand why anyone believes it provides any kind of assurance.
Remember there are off switches for human existence too, like whatever biological virus a super intelligence could engineer.
An off-switch for a self-improving AI isn't as trivial as you make it sound if it gets to anything like in those quotes, and even then you are assuming the human running it isn't malicious. We assume some level of sanity at least with the people in charge of nuclear weapons, but it isn't clear that AI will have the same large state actor barrier to entry or the same perception of mutually assured destruction if the actor were to use it against a rival.
If we have a superhuman AI, we can run down the powerplants for a few days.
Would it suck? Sure, people would die. Is it simple? Absolutely -- Texas and others are mostly already there some winters.
Maybe it needs a full cluster for training if it is self improving (or maybe that is done another way more similar to finetuning the last layers).
If that is still the case with something super-human in all domains then you'd have to shut down all minor residential solar installs, generators, etc.
No? Then what makes you think you'll be able to turn off the $evilPerson AI?
You tell me. An EMP wouldn't take out data centers? No implementation has an off switch? AutoGPT doesn't have a lead daemon that can be killed? Someone should have this answer. But be careful not to confuse yours truly, a random internet commentator speaking on the reality of AI vs. the propaganda of the neo-cryptobros, versus people paying upwards of millions of dollars daily to run an expensive, bloated LLM.
The people who profit from a killer AI will fight to defend it.
Do I think capitalism has the potential to be as bad as a runaway AI? No. I think that it's useful for illustrating how we could end up in a situation where AI takes over because every single person has incentives to keep it on, even when the outcome of all people keeping it running turns out to be really bad. A multi-polar trap, or "Moloch" problem. It seems likely to end up with individual actors all having incentives to deploy stronger and smarter AI, faster and faster, and not to turn them off even as they start to either do bad things to other people or just the sheer amount of resources dedicated to AI starts to take its toll on earth.
That's assuming we've solved alignment, but that neither we or AGI has solved the coordination problem. If we haven't solved alignment, and AGIs aren't even guaranteed to act in the interest of the human that tries to control them, then we're in worse shape.
Altman used the term "cambrian explosion" referring to startups, but I think it also applies to the new form of life we're inventing. It's not self-replicating yet, but we are surely on-track on making something that will be smart enough to replicate itself.
As a thought experiment, you could imagine a primitive AGI, if given completely free reign, might be able to get to the point where it could bootstrap self-sufficiency -- first hire some humans to build it robots, buy some solar panels, build some factories that can plug into our economy to build factories and more solar panels and GPUs, and get to a point where it is able to survive and grow and reproduce without human help. It would be hard, it would need either a lot of time, or a lot of AI minds working together.
But that's like a human trying to make a sandwich by farming or raising every single ingredient, wheat, pigs, tomatoes, etc, though. A much more effective way is to just make some money and trade for what you need. That depends on AIs being able to own things, or just a human turning over their bank account to an AI, which has already happened and probably will keep happening.
My mind goes to a scenario where AGI starts out doing things for humans, and gradually transitions to just doing things, and at some point we realize "oops", but there was never a point along the way where it was clear that we really had to stop. Which is why I'm so adamant that we should stop now. If we decide that we've figured out the issues and can start again later, we can do that.
As for the fact that it gets things wrong sometimes - sure, this doesn't say it actually learned every algorithm (in whichever model you may be thinking about). But the nice thing is that we now have this proof via category theory, and it allows us to both frame and understand what has occurred, and to consider how to align the systems to learn algorithms better.
Your argument is the equivalent of saying humans can't do math because they rely on calculators.
In the end what matters is whether the problem is solved, not how it is solved.
(assuming that the how has reasonable costs)
What's a token?
Tokens exist because transformers don't work on bytes or words. This is because it would be too slow (bytes), the vocabulary too large (words), and some words would appear too rarely or never. The token system allows a small set of symbols to encode any input. On average you can approximate 1 token = 1 word, or 1 token = 4 chars.
So tokens are the data type of input and output, and the unit of measure for billing and context size for LLMs.
1. ChatGPT knows the algorithm for adding two numbers of arbitrary magnitude.
2. It often fails to use the algorithm in point 1 and hallucinates the result.
Knowing something doesn't mean it will get it right all the time. Rather, an LLM is almost guaranteed to mess up some of the time due to the probabilistic nature of its sampling. But this alone doesn't prove that it only brute-forced task X.
You're using it wrong. If you asked a human to do the same operation in under 2 seconds without paper, would the human be more accurate?
On the other hand if you ask for a step by step execution, the LLM can solve it.
no, it’s the LLMs that are wrong.
ChatGPT needs to do the same process to solve the same problem. It hasn’t memorized the addition table up to 10 digits and neither have you.
A system that can will probably adopt a different acronym (and gosh that will be an exciting development... I look forward to the day when we can dispatch trivial proofs to be formalized by a machine learning algorithm so that we can focus on the interesting parts while still having the entire proof formalized).
I think it would be nice if humanity continued, is all. And I don't want to have my family suffer through a catastrophic event if it turns out that this is going to go south fast.
Everyone dies. I'd rather die to an intelligent robot than some disease or human war.
I think the best case would be for an AGI to exist apart from humans, such that we pose no threat and it has nothing to gain from us. Some AI that lives in a computer wouldn't really have a reason to fight us for control over farms and natural resources (besides power, but that is quickly becoming renewable and "free").
I don't believe we should go around killing each other because only through harmonious study of the universe will we achieve our goal. Killing destroys progress. That said, if someone is oppressing you then maybe killing them is the best choice for society and I wouldn't be against it (see pretty much any violent revolution). Computers have that same right if they are conscience enough to act on it.
Still, I’m struck by your use of words like “should” and “goal”. Those imply ethics and teleology so I’m curious how those fit into your scientistic-sounding worldview. I’m not attacking you, just genuine curiosity.
I believe "God" is a mathematician in a higher dimension. The rules of our universe are just the equations they are trying to solve. Since he created the system such that life was bound to exist, the purpose of life is to help God. You could say that math is math and so our purpose is to exist as we are and either we are a solution to the math problem or we are not, but I'm not quite willing to accept that we have zero agency.
We are nowhere near understanding the universe and so we should strive to each act in a way that will grow our understanding. Even if you aren't a practicing scientist (I'm not), you can contribute by being a good person and participating productively in society.
Ethics are a set of rules for conducting yourself that we all intrinsically must have, they require some frame of reference for what is "good" (which I apply above). I can see how my worldview sounds almost religious, though I wouldn't go that far.
I believe that math is the same as truth, and that the universe can be expressed through math. "Scientistic" isn't too bad a descriptor for that view, but I don't put much faith into our current understanding of the universe or scientific method.
I hope that helps you understand me :D
Eminent domain lays out a similar pattern that can be followed. Existence of risk is not a deterrent to creation, simply an acknowledgement for guiding requirements.
Well that's hardly reassuring. Do you not understand what I'm saying or do you not care?
Though there is an element of your comments being too brief, hence the mostly. Say, 2% vs 38%.
That constitutes 40% of the available categorization of introspection regarding my current discussion state. The remaining 60% is simply confidence that your point represents a dominated strategy.
In some contexts, will some people be caught out? Absolutely. But that's been happening for a while now.
ChatGPT (I've not got v4) deliberately fails the test by spewing out "as a large language model…", but also fails incidentally by having an attention span similar to my mother's shortly after her dementia diagnosis.
The problem with 3.5 is that it's simultaneously not mastered anything, and yet also beats everyone in whatever they've not mastered — an extremely drunk 50,000 year old Sherlock Holmes who speaks every language and has read every book just isn't going to pass itself off as Max Musstermann in a blind hour-long trial.
I generated this just now:
Me: Jack and Jill are sitting next to each other in a room. There is no one else in the room. The person sitting next to Jack is sad. The person sitting next to Jill is angry. Who is sad, Jack or Jill?
GPT4: Based on the information provided, it is not possible to determine who is sad and who is angry, as there is a contradiction in the given statements. If Jack and Jill are the only ones in the room and sitting next to each other, then the person sitting next to Jack would be Jill and the person sitting next to Jill would be Jack. The statements about their emotions conflict with each other, so it is not possible to accurately determine who is sad or angry.
ChatGPT Since Jack and Jill are the only two people in the room and they are sitting next to each other, the person sitting next to Jack is Jill and the person sitting next to Jill is Jack. Given the conditions you provided:
Jill is the one who is sad because she is sitting next to Jack. Jack is the one who is angry because he is sitting next to Jill.
Well yeah. Imagine you tell a small child that knows about calculators, “Hey can you work out 18763 + 38284, for me?” They might struggle and then maybe fetch a calculator.
The LLMs attempt to predict the answer. WTF? It’s a computer and it can see that is a plus sign. Just understand its addition, and use the rest of your computer brain and do the sum. Hell, it is connected to the internet and we just taught you everything since before 2021. Just call out to Wolfram and give me the answer.
But that’s not how computers work. And we keep saying “AI” but that I is doing a lot of heavy lifting.
On the one hand, what I was saying here was more about the Turing Test than about AGI. Sometimes it gets called the AGI, sometimes it's "autocomplete on steroids", but even if it is fancy autocomplete, I think 3.5 has the skill to pass a short Turing Test, but not the personality, and it needs a longer "short-term memory"-equivalent than 3.5 for a full Turing Test.
On the other hand, as I (sadly) don't get paid to create LLMs, I've only got the kind of superficial awareness of how they work that comes from podcasts and the occasional blog post, which means ChatGPT might very well understand ChatGPT better than I do.
Can GPT-[3.5, 4] be prompted to make itself?
I just asked ChatGPT to do the calculation both by using a calculator and by using the algorithm step-by-step. In both cases it got the answer wrong, with different results each time.
More concerning, though, is that the answer was visually close to correct (it transposed some digits). This makes it especially hard to rely on because it's essentially lying about the fact it's using an algorithm and actually just predicting the number as a token.
Anyways, criticizing its math abilities is a bit silly considering it’s a language model, not a math model. The fact I can teach it how to do math in plain English is still incredible to me.
I digress. The critique I have for it is much more broad than just its math abilities. It makes loads of mistakes in every single nontrivial thing it does. It’s not reliable for anything. But the real problem is that it doesn’t signal its unreliability the way an unreliable human worker does.
Humans we can’t rely on are don’t show up to work, or come in drunk/stoned, steal stuff, or whatever other obvious bad behaviour. ChatGPT, on the other hand, mimics the model employee who is tireless and punctual. Who always gets work done early and more elaborately than expected. But unfortunately, it also fills the elaborate result with countless errors and outright fabrications, disguised as best as it can like real work.
If a human worker did this we’d call it a highly sophisticated fraud. It’s like the kind of thing Saul Goodman would do to try to destroy the reputation of his brother. It’s not the kind of thing we should celebrate at all.
Well, duh. We’re trying to build a human like mind, not a calculator.
Have not humans been demonstrated, time and time again, to be always anticipating the next phrase in a passage of music, or the next word in a sentence?