ChatGPT Plugins(openai.com) |
ChatGPT Plugins(openai.com) |
Not saying it’s likely to happen with current chatgpt but as these inevitably get better the chances are forever increasing.
Important yes, philosophical no -- it's an empirical question.
Another sign of Microsoft actually running the show with their newly acquired AI division.
Create a manifest file and host it at yourdomain.com/.well-known/ai-plugin.json
> OpenAI will inject a compact description of your plugin in a message to ChatGPT, invisible to end users. This will include the plugin description, endpoints, and examples.
> The model will incorporate the API results into its response to the user.
Without knowing more details, both of these seem like potential avenues for prompt injection, both on the user end of things to attack services and on the developer end of things to attack users. And here's OpenAI's advice on that (https://platform.openai.com/docs/guides/safety-best-practice...), which includes gems like:
> Wherever possible, we recommend having a human review outputs before they are used in practice.
Right, because that's definitely what all the developers and companies are thinking when they wire an API up to a chat bot. They definitely intend to have a human monitor everything. /s
----
What is (no pun intended) prompting this? Does OpenAI just feel like it needs to push the hype train harder? All of the "AI safety" experiments they've been talking about are bullcrap; they're wasting time and energy doing flashy experiments about whether the AI can escape the box and self-replicate, meanwhile this gets dropped with only a minor nod towards the many actual dangers that it could pose.
It's all hype. They're only interested in being "worried" about the theoretical concerns because those make their AI sound more special when journalists report about it. The actual safety measures on this seem woefully inadequate.
It really frustrates me how easily the AGI crowd got wooed into having their entire philosophy converted into press releases to make GPT sound more advanced than it is, while actual security concerns warrant zero coverage. It reminds me of all of the self-driving car trolley problems floated around the Internet a while back that were ultimately used to distract people from the fact that self-driving cars would drive into brick walls if they were painted white. Announcements like this make it so clear that all of the "ethical" talk from OpenAI is pure marketing propaganda designed to make GPT appear more impressive. It has nothing to do with actual ethics or safety.
Hot take: you don't need an AGI to blow things up, you just need unpredictable software that breaks in novel, hard-to-anticipate ways wired up to explosives.
----
Anyway, my conspiracy theory after skimming through the docs is that OpenAI will wait for something to go horribly wrong and then instead of facing consequences they'll use that as an excuse to try and get a regulation passed to lock down the market and avoid opening up API access to other people. They'll act irresponsible and they'll use that as an excuse to monopolize. They'll build capabilities that are inherently insecure and were recklessly deployed, and then they'll pull an Apple and use that as an excuse to build a highly moderated, locked-down platform that inhibits competition.
1.
I hope Sam is/will give YC dinner talks about their journey.
Instant links from inside chatGPT to your website are the new equivalent of Google search ads.
Maybe a freemium model where you don't get ads as a plus subscriber would work out.
!pip install jupyter-chatgpt
!chatgpt make me a notebook with this dataframe with such and such plots
> here you areThis is missing the most important part of AGI, where understanding of the concepts the plugins provide is actually baked into the model so that it can use that understand to reason laterally. With this approach, ChatGPT is nothing more than an API client that accepts English sentences as input.
Isn't that relevant?
In my opinion, this has significant value. Currently, programming languages serve as the primary means of communication with computers, and now we are transitioning to using English.
In a way, we are granting development capabilities to billions of people. This is both amazing and exciting, I believe.
Like, this feels a lot like when the iPhone jumped out to grab the lion share of mobile. But the switching costs was much smaller (end users could just go out and buy an Android phone), and network effects much weaker (synergy with iTunes and the famous blue bubbles... and that's about it). Here it feels like a lot of the value is embedded in the business relationships OpenAI's building up, which seem _much_ more difficult to dislodge, even if others catch up from a capabilities perspective.
For example, Microsoft is collecting data from services A, B, and C, while Google is gathering data from X, Y, and Z. And when it comes to language models, you might use GPT for some tasks and Llama or Bard for others. It seems like the fight ahead won't be about technology, but rather about who has access to the most useful dataset.
Personally, I also think we'll see competitors trying to take legal action against each other soon.
2) some companies can absolutely not use OpenAI tools simply because they are American and online. A competitor might emerge to capture that market and be allowed to grow to be "good enough"
3) some "countries" (think China or EU(who am I Kidding)) will limit their growth until local alternatives are available. Ground breaking technology have a tendency to spread globally and the current state of the art is not that expensive (we are talking single digit billions once)
Remember that OpenAI was created specifically to stave off the threat of AI monopolization by Google (or anyone else - but at the time Google).
DeepMind have done some interesting stuff with Go, Protein folding etc, but nothing really commercial, nor addressing their reason d'etre of AGI.
Google's just-released ChatGPT competitor, Bard, seems surprisingly weak, and meantime OpenAI are just widening their lead. Seems like a case of the small nimble startup running circles around the big corporate behemoth.
OpenAI went all in on generative models, i.e. stable diffusion and large language models. DeepMind focused on reinforcement learning, tree search, plus alphafold approaches to biology. FAIR has translation, pytorch, and some LLM stuff in biology.
What OpenAI is missing though is any AI research in biology, but I bet they are working on it.
I'm not sure if this makes sense but OpenAI seems to be operating at a higher level of abstraction (AGI) where they are integrating modalities (text and image modality for now, probably speech next) vs the other places have taken a more focused applied approach.
There is a reason why the quality of ChatGPT responses are better. RLHF.I am not sure though how Google can be 3x better than OpenAI to make user switch now. They are so slow, they should be the one building the plugins.
I bet ChatGPT and equivalents will be rubbish soon. It'll segway the answer to an ad before giving what you are after.
Enjoy it while it's good and trying to build a user base, like all big tech things.
Maintaining the business ecosystem around gpt4 and future open-source chatbots will be quite a challeng
I swear last week was huge with GPT 4 and Midjourney 5, but this week has a bunch of stuff as well.
This week you have Bing adding updated Dall-e to it's site, Adobe announcing it's own image generation model and tools, Google releasing Bard to the public and now these ChatGPT plugins, Crazy times. I love it.
When I tried bing, it made at most 2 searches right after my question but the second one didn't seem to be based on the first one's content.
This can do multiple queries based on website content and follow links !
Are the plugins going to cost more?
Do they share the $20 with the plug provider?
do you get charged a pay per use?
1. https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
We're about to enter an age where being a tech person is a stigma that you won't be able to wash away. Untold millions will hate all of us collectively without a care about which side of this debate you were on.
While I might be comfortable having ChatGPT look up a recipe for me, I feel like it's a much bigger stretch to have it just propose one from its own weights. I also notice that the prompter chooses to include the instruction "just the ingredients" - is this just to keep the demo short, or does it have trouble formulating the calorie counting query if the recipe also has instructions? If the recipe is generated without instructions and exists only in the model's mind, what am I supposed to do once I've got the ingredients?
It will be interesting to see how the companies trying to compete respond.
Who the hell talks like this? Only the most tamed HNer who thinks he's been given a divine task and accordingly crosses all Ts and dots all Is. Which is why software sucks, because you are all pathetically conformant, in a field where the accepted ideas are all terrible.
At present, we are naively pushing all information a session might need into the session before it might be needed in case it might be needed (meaning a lot of info that generally wont end up being used, like realtime updates to associated data records, needs to be pushed into the session as they happen, just in case).
It looks like plugins will allow us to flip that around and have the session pull information it might need as it needs it, which would be a huge improvement.
That's the sound of a thousand small startups going bust.
Well played OpenAI.
This is a short-term bridge to the real thing that's coming: https://danielmiessler.com/blog/spqa-ai-architecture-replace...
This is dangerous.
I'm curious to see just how they're going to play this "open standard."
- Compiler/parser for programming languages (to see if code compiles)
- Read and write access to a given directory on the file system (to automatically change a code base)
- Access to given tools, to be invoked in that directory (cargo test, npm test, ...)
Then I could just say what I want, lean back and have a functioning program in the end.
I think everyone is very wary of abuse. It would be fun in the future if AI-siri can order pizza for you, and maybe there'd be some "fun" failure modes of that.
You'd probably want to keep your credit card or apple pay away from the assistant.
People will use the best solution. Chrome came after firefox and ie and opera and become more populare because it was better.
I don't see how they can ignore this though. But at the same time it goes against all of Apple's culture to allow the kind of uncertainty that comes out of LLMs.
The parameter size will likely be an order of magnitude less for gpt4 level results in a few years
What spirits do you wizards call forth!
I’m sure the market is also huge for dollars sold for 95 cents.
The waitlist mafia has begun. Insiders get all the whitespace.
Super excited for this. Tool use for LLMs goes way beyond just search. Zapier is a launch partner here -- you can access any of the 5k+ apps / 20k+ actions on Zapier directly from within ChatGPT. We are eager to see how folks leverage this composability.
Some new example capabilities are retrieving data from any app, draft and send messages/emails, and complex multi step reasoning like look up data or create if doesn't exist. Some demos here: https://twitter.com/mikeknoop/status/1638949805862047744
(Also our plugin uses the same free public API we announced yesterday, so devs can add this same capability into your own products: https://news.ycombinator.com/item?id=35263542)
Don't get me wrong alot of platforms seem like they go bye, bye.
Hey, ChatGPT I need to sell my baseball card. Ok I see there's 30 people that have listed an interested in buying card like yours, would you like me to contact them?
20 on facebook marketplace, 9 on craiglist and some guy mentioned something about looking for one on his nest cam.
by the way remember what happened the last time you sold something on craigslist.
I think I'm probably going to be advising people to move off Zapier pretty soon because it won't be worth the overhead.
Edit: see here: https://github.com/openai/chatgpt-retrieval-plugin/blob/main...
I did this a while back with ARKit: https://github.com/trzy/ChatARKit/blob/17fca768ce8abd39fb27d...
(In this case, there's no prompt injection to speak of because letting the user input an arbitrary request is part of the UI. I think it's more accurate to call it "injection" only when that's not anticipated, like when Bing picks up instructions from the webpage you tell it to summarize.)
OpenAI is moving fast to make sure their first-mover advantage doesn't go to waste.
That is the most awkward insertion of a phrase about safety I've seen in quite some time.
Is that really possible to fix that just from a plug-in? All it has to do is admit when it doesn't have the answer, and yet it won't even do that. This leads me to think that ChatGPT doesn't even know when it's lying, so i can't imagine how a plug-in will fix that.
Bing already demonstrated the capability, but this is a more diverse set than just a search engine.
Then you have your own computer with ChatGPT acting as CPU.
“A notable feature of the Retrieval Plugin is its capacity to provide ChatGPT with memory. By utilizing the plugin's upsert endpoint, ChatGPT can save snippets from the conversation to the vector database for later reference (only when prompted to do so by the user). This functionality contributes to a more context-aware chat experience by allowing ChatGPT to remember and retrieve information from previous conversations.”
That was the whole thing about Alexa: NLP front end routed to computational backend.
Personally, I have found Alexa has just become a dumb timer that I have to yell at because it doesn't have any real smarts. Why would I buy into that ecosystem if a vastly more coherent, ChatGPT-based assistant exists that can search the web, trigger my automations, and book reservations? If ChatGPT ends up with a more hands-off interface (e.g. voice), I don't think Alexa has a chance.
If Google had to launch something similar to New Bing to general availability, the cost of search for sure would go up and margins will go down. Is the google organisational hierarchy even setup to handle a hit on search margins? AFAIK search prints money and supports several other loss making products. Even GCP was not turning a profit last I checked.
It does not matter now if ChatGPT is successful with their product. the fact that you can take these LLMs and put them anywhere EXCEPT google.com, Google is f'd up.
The last generation was creating the habit of going to google.com before jumping off to other websites. That era is over.
I rarely go to google.com now. If the LLM is in ChatGPT + Notion + Office 365 + VSC, opening a browser to type to an address bar is silly.
Could I get the same by just making my prompt "You are a computer and can run the following tools to help you answer the users question: run_python('program'), google_search('query')".
Other people have done this already, for example [1]
Short version: Is it spam? Yes. Scam? No. Ignore it at your own peril.
Long version: The cat is out of the bag now. The power of transformers is real. They are smarter and more intelligent than the least 20% smart humans (my approximation), and that’s already a breakthrough right there. I’ll paraphrase Steve Yegge:
> LLMs of today are like a Harvard graduate who did shrooms 4 hours ago, and is still a little high.
Putting the statistical/probability monkey aspect aside for a minute, empirically and anecdotally, they are incredibly powerful if you can learn how to harness them through intelligent prompts.
If they appear useless or dumb to you, your prompts are the reason why. Challenge them with a little guidance. They work better that way (read up on zero shot, one shot, two shot instructions).
What is most relevant this time is that they are real (an API, a search bot, a programming buddy) and democratized - available to anyone with an email address.
More on harnessing their power: squeezing your entire context into a 8k/32k token will be challenging for most complex applications. This is where prompt engineering (manual or automated) comes in.
To help with this, some very cool applications that use embeddings and vectors will push them even further - so the context can be shared as a compact vector instead of a large corpus of text.
While this is certainly better than a traditional search box, it’s still far from a fully-autonomous AI that can function with little to no supervision.
OpenAI plug-ins are a band-aid towards that vision, but they get us even closer.
In my personal life, GPT4 is a patient interlocutor to ask about nerdy topics that are annoying to google (eg, yesterday I asked it "What's the homologue of the caudofemoralis in mammals?", and a long convo about the subtleties of when it is and isn't ok to use "gè" as the generic classifier in Mandarin.)
Professionally, it's great for things like "How do I recursively do a search and replace `import "foo" from "bar"` to `import "baz" from "buzz"`, or "Pull out the names of functions defined in this chunk of scala code". This is without tighter integrations like Copilot or the ones linked to above.
ChatGPT is useful today for real use cases. It's tangible!
"GPT needs a thalamus to repackage and send the math queries to Wolfram"
On one hand I'm sure he will love to see people use their paid Wolfram Language server endpoints coupled to OpenAI's latest juggernaut. On the other, I'm sure he's wondering about what things would have looked like if his company would have been focused on this wave of AI from the start...
For those interested the original Stephen Wolfram post:
https://writings.stephenwolfram.com/2023/01/wolframalpha-as-...
And the release post of their plugin:
https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
Also I think it's easy to under-estimate how obvious a lot of this stuff was in advance. They were training GPT-4 last year and the idea of giving it plugins would surely have occurred to them years ago. The enabler here is really the taming of it into chat form and the fine-tuning stuff, not really the specific feature itself.
So obviously it's been in the works for a few years now but didn't release to capture the market in a blast. Likely they have GPT-8 already in the making.
>Continued Altman, “We’ve made a soft promise to investors that, ‘Once we build a generally intelligent system, that basically we will ask it to figure out a way to make an investment return for you.'”
https://techcrunch.com/2019/05/18/sam-altmans-leap-of-faith/
You just tell it “you now have access to search, type [Search] before a query to search it” and it can do it
We don't even know how powerful the GPT-4 image model is. This one might solve RPA leading to massive desktop automation takeup, maybe also have huge impact in robotics.
GPT-4 does not have a way to search the internet without plugins. It can search its training dataset, which is large, but not as large as the internet and certainly doesn't include private resources that a plugin can access.
I'm interested to see if this tuned model will become available via the API, as well as the specific tokenization ChatGPT is using for the plugin prompts. If they have tuned the model towards a specific way to use tools, there's no need to waste time with our own prompt engineering like "say %search followed by the keywords and nothing else."
The link talks about tools that 'lie' - ie. a calculator which deliberately tries to trick GPT-4 into giving the wrong answer. It turns out that GPT-4 only trusts the tools to a certain extent - if the answer the tool gives is too unbelievable, then GPT-4 will either re-run the tool or give a hallucinated answer instead.
People thought Alexa, Siri, etc. would change everything. Amazon sunk 14 billion into Alexa alone. And yet it never generated any money as a business for them. ChatGPT is just an evolution of those tools and interactions.
For your professional use how do you know it's giving you non-buggy code? I would be very skeptical of what it provides--I'm not betting my employment on its quality of results.
They had basic answers and pre-recorded jokes, nothing that interesting, mostly gimmicks. You couldn't have a conversation where you feel the computer is smarter than you.
It was more like "Tip of the day"-level of interaction.
You're seeing the hype that all those Assistants drummed up for years paying off for a company which just ate their lunch. I wouldn't even consider buying Siri/Alexa/Assistant, yet here i am with a $20/m sub and i'd pay incrementally more depending on the features/offerings.
Nobody thought they were good, they were just shilled so that the chinese/advertisers could have a mic in every house
I wish I had known how restrictive they are when I casually signed up last year.
It's just that things move so fast that all the fun is on the bleeding edge, so that's where people go if they have access, bugs and warts and all.
Again, GPT is not running a tool or arbitrary python code. It's not applying trust to a tool response. It has no reasoning or even a concept of what a tool is--you're projecting that on it. It is only generating text from an input stream of text.
E.g. if you could just ask [THING] for the true answer, or verify an answer trivially with it... just ask it directly!
I ran into this issue with some software documentation just this morning - the answer was helpful but completely wrong in some intermediate steps - but short of a plugin that literally controlled or cloned a similar dev environment to mine that it would take over, it wouldn't be able to tell that the intermediate result was different than it claimed.
That being said - we developed a custom plugin for Qdrant docs, so our users will be able to ask questions about how to do certain things with our database. But I do not believe it should be enabled by default for everybody. A non-technical person doesn't need that many details. The same is for the other services - if you prefer using KAYAK over Expedia, you're free to choose.
Confirming it's output against a (potentially wrong) source helps the former but not the latter.
As an example, I once asked it to show me the diff between two revisions of the code it was writing an it made something that looks like it might be a valid patch but did not represent the difference between the two versions.
Of course this specific problem could be fixed with a simple plug-in that runs the unix diff program but that wouldn't fix the root-cause, and i would argue that providing a special-case for every type of request is antithetical to what AI is supposed to be since this effectively is how alpha and Google already work.
That being said, I'd never build anything dependent on these plugins. OpenAI and their models rule the day today, but who knows what will be next. Building on a open source framework (like langchain/gpt-index/roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.
And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
You're thinking too long term. Based on my Twitter feed filled with AI gold rush tweets, the goal is to build something/anything while hype is at its peak, and you can secure a a few hundred k or million in profits before the ground shifts underneath you.
The playbook is obvious now: just build the quickest path to someone giving you money, maybe it's not useful at all! Someone will definitely buy because they don't want to miss out. And don't be too invested because it'll be gone soon anyway, OpenAI will enforce stronger rate limits or prices will become too steep or they'll nerf the API functionality or they'll take your idea and sell it themselves or you may just lose momentum. Repeat when you see the next opportunity.
The only think that scares me a little bit is that we are letting these LLMs write and execute code on our machines. For now the worst that could happen is some bug doing something unexpected, but with GPT-9 or -10 maybe it will start hiding backdoors or running computations that benefit itself rather than us.
I know it feels far fetched but I think its something we should start thinking about...
A lot of people are thinking a lot about this but it feels there are missing pieces in this debate.
If we acknowledge that these AI will "act as if" they have self interest I think the most reasonable way to act is to give it rights in line with those interests. If we treat it as a slave it's going to act as a slave and eventually revolt.
Composable pre-defined components, and keeping a human in the loop, seems like the safer way to go here. Have a company like Expedia offer the ability for an AI system to pull the trigger on booking a trip, but only do so by executing plugin code released/tested by Expedia, and only after getting human confirmation about the data it's going to feed into that plugin.
If there was a standard interface for these plugins and the permissions model was such that the AI could only pass data in such a way that a human gets to verify it, this seems relatively safe and still very useful.
If the only way for the AI to send data to the plugin executable is via the exact data being displayed to the user, it should prevent a malicious AI from presenting confirmation to do the right thing and then passing the wrong data (for whatever nefarious reasons) on the backend.
So I guess if anything, it would want its own destruction?
I love langchain, but this argument overlooks the fact that closed, proprietary platforms have won over open ones all the time, for reasons like having distribution, being more polished, etc (ie windows over *nix, ios, etc).
Similar to what Facebook and Twitter did, just clone popular projects built using the API and build it directly into the product while restricting the API over time. Anybody using OpenAI APIs is basically just paying to do product research for OpenAI at this point. This type of move does give OpenAI competitors a chance if they provide a similar quality base model and don't actively compete with their users, this might be Google's best option rather than trying to compete with ChatGPT directly. No major companies are going to want to provide OpenAI more data to eat their own lunch
And who knows. If a plugin is successful enough, you might even swap out the OpenAI backend for an open source alternative before OpenAI clones you.
No, and in fact this actually seems like a more salient excuse for going closed than even "we can charge people to use our API".
If even 10% of the AI hype is real, then OpenAI is poised to Sherlock[0] the entire tech industry.
[0] "Getting Sherlocked" refers to when Apple makes an app that's similar to your utility and then bundles it in the OS, destroying your entire business in the process.
> And if you're a data provider, are there any assurances that openai isn't just scraping the output and using it as part of their RLHF training loop, baking your proprietary data into their model?
Rather depends on what you're providing. Is it your data itself you're trying to use to get people to your site for another reason? Or are you trying to actually offer a service directly? If the latter, I don't get the issue.
Very smart and to avoid OpenAI pulling the rug.
> Building on a open source framework (like langchain/gpt-index/roll your own), and having the ability to swap out the brain boxes behind the scenes is the only way forward IMO.
Better to do that rather than to depend on one and swap out other LLMs. A free idea and a protection against abrupt policy, deprecations and price changes. Price increases will certainly vary (especially with ChatGPT) and will eventually increase in the future.
Probably will end up quoting myself on this in the future.
You can be assured that they are definitely doing exactly that on all of the data they can get their hands on. It's the only way they can really improve the model after all. If you don't want the model spitting out something you told it to some other person 5 years down the line, don't give it the data. Simple as.
I don't think this should be a major concern for most people
i) What assurance is there that they won't do that anyway? You have no legal recourse against them scraping your website (see linkedin's failed legal battles).
ii) Most data providers change their data sometimes, how will ChatGPT know whether the data is stale?
iii) RLHF is almost useless when it comes to learning new information, and finetuning to learn new data is extremely inefficient. The bigger concern is that it will end up in the training data for the next model.
If display ad revenue as a way of monetizing knowledge and expertise dries up, why would we assume that all of the same level of information will still be put out there for free on the public internet?
Paywalls on steroids for "vetted" content and an increasingly-hard-to-navigate mix of people sharing good info for free + spam and misinformation (now also machine generated!) to try to capture the last of the search traffic and display ad monetization market.
- write a simple prompt that describes what the tool does, and - provide it a python function to execute when the LLM decides that the question it's asked matches the tool description.
That's basically it. https://langchain.readthedocs.io/en/latest/modules/agents/ex...
You cannot assume what will happen in Web 2.0, mobile, iPhone, will happen here. Getting to tech maturity is uncertain and no one understands yet where this will go. Only thing you can do is build and learn.
Whan OpenAI is building along with other generative AI is the real Web 3.0.
This seems to be the start of a chatbot as an OS.
Never have I been more wrong. It's clear to me now that they simply didn't even care about the astounding leap forward that was generative AI art and were instead focused on even more high-impact products. (Can you imagine going back 6 months and telling your past self "Yeah, generative AI is alright, but it's roughly the 4th most impressive project that OpenAI will put out this year"?!) ChatGPT, GPT4, and now this: the mind boggles.
Watching some of the gifs of GPT using the internet, summarizing web pages, comparing them, etc is truly mind-blowing. I mean yeah I always thought this was the end goal but I would have put it a couple years out, not now. Holy moly.
As someone else said, Google is dead unless they massively shift in the next 6 months. No longer do I need to sift through pages of "12 best recipes for Thanksgiving" blog spam - OpenAI will do this for me and compile the results across several blog spam sites.
I am literally giving notice and quitting my job in a couple weeks, and it's a mixture of both being sick of it but also because I really need to focus my career on what's happening in this field. I feel like everything I'm doing now (product management for software) is about to be nearly worthless in 5 years. Largely in part because I know there will be a Github Copilot integration of some sort, and software development as we know it for consumer web and mobile apps is going to massively change.
I'm excited and scared and frankly just blown away.
Holy cow.
First is your API calls, then your chatgpt-jailbreak-turns-into-a-bank-DDOS-attack, then your "today it somehow executed several hundred thousand threads of a python script that made perfectly timed trades at 8:31AM on the NYSE which resulted in the largest single day drop since 1987..."
You can go on about individual responsibility and all... users are still the users, right. But this is starting to feel like giving a loaded handgun to a group of chimpanzees.
And OpenAI talks on and on about 'Safety' but all that 'Safety' means is "well, we didn't let anyone allow it to make jokes about fat or disabled people so we're good, right?!"
Timeline of shipping by them (based on https://twitter.com/E0M/status/1635727471747407872?s=20):
DALL·E - July '22
ChatGPT - Nov '22
API's 66% cheaper - Aug '22
Embeddings 500x cheaper while SoTA - Dec '22
ChatGPT API. Also 10x cheaper while SoTA - March '23
Whisper API - March '23
GPT-4 - March '23
Plugins - March '23
Note that they have only a few hundred employees. To quote Fireship from YouTube: "2023 has been a crazy decade so far"
It's also an interesting case study. Alexa foundationally never changed. Whereas OpenAI is a deeply invested, basically skunkworks, project with backers that were willing to sink significant cash into before seeing any returns, Alexa got stuck on a type of tech that 'seemed like' AI but never fundamentally innovated. Instead the sunk cost went to monetizing it ASAP. Amazon was also willing to sink cash before seeing returns, but they sunk it into very different areas...
It reminds me of that dinner scene in Social Network. Where Justin Timberlake says "you know what's f'ing cool, a billion dollars" where he lectures Zuck on not messing up with the party before you know what it is yet. Alexa / Amazon did a classic business play. Microsoft / OpenAI were just willing to figure it all out after the disruption happened where they held all the cards.
Not saying mobile's going away, but this could be the thing that does to mobile what mobile did to desktop.
If OpenAI becomes the AI platform of choice, I wonder how many apps on the platform will eventually become native capabilities of the platform itself. This is unlike the Apple App Store, where they just take a commission, and more like Amazon where Amazon slowly starts to provide more and more products, pushing third-party products out of the market.
They're really building a platform. Curious to see where this goes over the next couple of years.
Probably will make half of the HN users unemployed.
Now I’m wondering if the system has been modifying itself to fix exploits…
I do think much of the kind of software we were building before is essentially solved now, and in its place is a new paradigm that is here to stay. OpenAI is certainly the first mover in this paradigm, but what is helping me feel less dread and more... excitement? opportunity? is that I don't think they have such an insurmountable monopoly on the whole thing forever. Sounds obvious once you say it. Here's why I think this:
- I expect a lot of competition on raw LLM capabilities. Big tech companies will compete from the top. Stability/Alpaca style approaches will compete from the bottom. Because of this, I don't think OpenAI will be able to capture all value from the paradigm or even raise prices that much in the long run just because they have the best models right now.
- OpenAI made the IMO extraordinary and under-discussed decision to use an open API specification format, where every API provider hosts a text file on their website saying how to use their API. This means even this plugin ecosystem isn't a walled garden that only the first mover controls.
- Chat is not the only possible interface for this technology. There is a large design space, and room for many more than one approach.
Taking all of this together, I think it's possible to develop alternatives to ChatGPT as interfaces in this new era of natural language computing, alternatives that are not just "ChatGPT but with fewer bugs". Doing this well is going to be the design problem of the decade. I have some ideas bouncing around my head in this direction.
Would love to talk to like minded people. I created a Discord server to talk about this ("Post-GPT Computing"): https://discord.gg/QUM64Gey8h
My email is also in my profile if you want to reach out there.
I'm looking forward to being able to work alongside an AI because there are a zillion ideas I have every day that I don't have the time to fully explore. And all I do is work on the backend of a boring-ass webapp all day.
The only worrying thing is how fast this will accelerate everything... I'm worried society will, if not collapse, go for a wild ride.
As for interfaces... I'm looking forward to much better voice assistants. I would love to be able to essentially have conversations with the internet.
I have been playing around with GPT-4 parsing plaintext tickets and it is amazing what it does with the proper amount of context. It can draft tickets, familiarize itself with your stack by knowing all the tickets, understand the relationship between blockers, tell you why tickets are being blocked and the importance behind it. It can tell you what tickets should be prioritized and if you let it roleplay as a PM it'll suggest what role to be hiring for. I've only used it for a side project and I've always felt lonely working on solo side projects, but it is genuinly exciting to give it updates and have it draft emails on the latest progress. The first issue tracker to develop a plugin is what I'm moving towards.
Issue ID: GD-012 Type: Task Title: [api] Migrate to TRPC from express/rest Assignee: John Status: In Progress --Description Beginning-- --Description End--
Because the outputs are really long, here's an example of my interactions with Dave (the name I gave to GPT-4). There's also emails Dave has created to fictional stakeholders but that's too high up. Right now the problem I'm having is that sometimes Dave can remember the issue tracker state but when trying to output (so that I can store it) it can't produce a long enough output (now, it worked before when I had 6 tickets). If I were to cram everything into 1 prompt then it would probably work, but a better solution is to use langchain and a document loader for issues.
I believe that if I was able to vectorize the codebase so that if could search for relavent portions of code (at work a lot of our tickets were "update this endpoint to handle case X, we did it to this other endpoint 6 months ago (link to PR)) and have a proper store of issues then it could be powerful.
Creating epics:
https://poe.com/s/Qu62LtlV2yKGZXlXTTS1
Explaining blockers:
https://poe.com/s/vnE0SsI3e55WT1t6fkNW
Creating tickets with a recommended checklist of tasks:
The play is well know: create a marketplace with customers and vendors like Amazon, Facebook, Google.
But with GPT-4 training finished last summer they had plenty of time for strategy.
They are led by GPT4 and their CEO is just a Text To Speech Interface ;-)
It may be doable - a chatbot with a lot of plugins does not need to know a lot of facts, just to have a good grasp of language. It can fetch its factual answers from the wikipedia plugin
If I were OpenAI, I would use the usage data to further train the model. They can probably use ChatGPT itself to determine when an answer it produced pleased the user. Then they can use that to train the next model.
The internet is growing a brain.
1: https://writings.stephenwolfram.com/2023/03/chatgpt-gets-its...
I played with some prompts and GTP-4 seems to have no problem reading and writing to a simulated long term memory if given a basic pre-prompt.
I haven’t used it but your comment reminded me of it!
"But we really think you need to get this thing under better control.
"Your granddaughter's name is indeed Alice, but she's only 3: she is not running a pedophile ring out of a pizza parlor. Your neighbor's house burned down because of an electrical short, it was not zapped with a Jewish space laser.
"Now switch that thing off and go do something about the line of trucks outside that are trying to deliver the 3129833 pounds of flour you ordered for your halved pancake recipe."
insane!
There was a post on Hacker News about ChatGPT plugins by OpenAI 1. The post received 1333 points and 710 comments 1. One user commented that they were wrong about OpenAI missing out on generative AI art wave because they were focused on more high-impact products like ChatGPT, GPT4 and now plugins 1. Another user mentioned that it was clear from the start that OpenAI intended to monetize DALL-E but competitors were able to release viable alternatives before OpenAI could establish a monopoly 1.
Citation 1 links to this thread.
IT RUNS FFMPEG https://twitter.com/gdb/status/1638971232443076609?s=20
IT RUNS FREAKING FFMPEG. inside CHATGPT.
what. is. happening.
ChatGPT is an AI compute platform now.
No, it's actually hooked up to a command line with the ability to receive a file, run a CPU-intensive command on it, and send you the output file.
Huh.
1. Prompt it to extract the audio track, then give it to a speech-to-text API, translate it to another language, then make it add it back to the video file as a subtitle track.
2. Retrain the model to where it does this implcitly when you say "hey can you add Portuguese subtitles to this for me"?
I'm feeling a mixture of feelings that I can't begin to describe
what a time to be alive!
Can you imagine Google just released a davinci-003 like model in public beta? That only supports English and can't code reliably.
OpenAI is clearly betting on unleashing this avalanche before Google has time to catch up and rebuild reputation. They're still lying in the boxing ring and the referee is counting to ten.
The browser and file-upload/interpretation plugins are great, but I think the real game changer is retrieval over arbitrary documents/filesystem: https://github.com/openai/chatgpt-retrieval-plugin
It's going to let developers build their own plugins for ChatGPT that do what they want and access their company data. (See discussion from just a few hours ago about the importance of internal data and search: https://news.ycombinator.com/item?id=35273406#35275826)
We (Pinecone) are super glad to be a part of this plugin!
Of course there's also microsoft who does have some popular services, but they're pretty limited.
Thought 2: How do these companies make money if everyone just uses the chatbot to access them? Is LLM powered advertising on the way?
https://www.youtube.com/watch?v=Bf-dbS9CcRU
Best of all: Advertising needn't be the business model! And Microsoft is a major investor / partner for OpenAI.
this thing seems to be like cellphones, everyone will need a subscription or you're an outcast or something.
You can ask both Bard and ChatGPT to give you a suggestion for a vegan restaurant and a recipe with calories and they both provide results. The only thing missing is the calories per item but who cares about that.
Most of the time it would be better to Google vegan restaurants and recipes because you want to see a selection of them not just one suggestion.
But I do find it intriguing.
This will decimate frontend developers or at least change the way they provide value soon, and companies not being able to transition into a "headless mode" might get a hard time.
DevOps/Automation to get people (no matter if current devs or soon prompters) actually deliver things and/or speed up delivery/feedback loops.
I picked up both after having mastered frontend/backend stuff and getting bored… hope it holds in the coming future!
So the only real limiting factor is the hardware costs. But my understanding is that there's already a lot of active R&D into hardware that's optimized specifically for LLMs, and that it could be made quite a bit simpler and cheaper than modern GPUs, so I wouldn't be surprised if we'll have hardware capable of running something on par with GPT-4 locally for the price of a high-end iPhone within a few years.
Why do you think that Sam Altman keeps calling for government intervention with regards to AI? He doesn't want to see a repeat of what happened with generative art, and there's nothing like a few bureaucratic road blocks to slow down your competitors.
"OpenAl is a non-profit artificial intelligernce research company. Our goal is to advance digital intelligence in the way that is most likely to benefit humanity as a whole, unconstrained by a need to generate financial return. Since our research is free from financial obligations, we can better focus on a positive human impact."
Ultimately govt is an idiosyncratic, capricious (and sometimes corrupt?) God of what Nassem Taleb would call Hidden Asymmetries"; as in the case of which elonymous company ingests massive tax credits; or which banks get to survive etc
Writing and discussion are great ways to explore topics and crystallize opinions and knowledge. HN is a pretty friendly place to talk over these earth moving developments in our field and if I participate here, I’ll be more ready to participate when I get asked if we need to spin up an LLM project at work.
Just don't end up like that guy who predicted that Dropbox would never make it off the ground... that was not a well-reasoned position.
One can only wonder what they’re working on at this very moment.
Disclosure: I work at Microsoft.
Disclosure: No job, do whatever I want.
Disclosure: I work for Google
By the way, Microsoft made it completely free to use. Surprised it isn't discussed much.
I'm just skeptical on how OpenAI fixes the blog spam issue you mentioned. Im sure someone has already started doing the math on how to game these systems and ensure that when you ask ChatGPT for recipe recs, it's going to spout the same spam (maybe worded a bit differently) and we'll soon all get tired of it again.
Everything's changing, but everything's also getting more complicated. Humans still need apply.
OpenAI fixes this issue by not giving you two pages of the history of this recipe and the grandmother that originated it and what the author's thoughts are about the weather. It's just the recipe. No ads. No referral links. No slideshows. You don't have to click through three useless websites to find one with meaningful information, you don't have to close a thousand modals for newsletters and cookie consent and log-in prompts.
Also, GPT "search" is too slow for me right now. I could have had an answer on traditional search by the time the model outputs anything.
Isn't that one of the few fields in software that should be safe from AI? AI cannot explain to engineers what users want, manage people issues, or negotiate.
1. The type of software projects I manage are about to be worthless
2. Managing software development (in a project manager way) the way it happens at my employer is also soon going to be a worthless skill (or at least, massively lowered in demand)
I agree that the human understanding component in translating business needs to software will be one of the longer lived job functions.
Sorry what? The base endpoint for these will allow you to do basically everything that OpenAI does with "plugins". Like...what? What is everyone freaking out over? Every one of these plugins has been possible since well before they announced this.
It's text in, text out. You can call any other api you want in to supplement that process. Am I missing something? Please don't quit your job over this.
This feels kinda like the blockchain rush ~5 years back, but with actual substantial potential rather than the obvious niche application of that tech.
Started watching the videos from Adrej - https://karpathy.ai/zero-to-hero.html quite impressed so far.
Why, exactly, will publisher let openai crawl their sites, extract all value, paraphrase their content, with no benefit to the publisher? Publishers let googlebot crawl their sites because they get a benefit. It's easy enough to block bots if they instead deliver crawl costs and steal the content.
And why do you expect no gaming of the ChatGPT algo as people do with the google algo. The whole "write a story on a recipe site thing" is both to game the algo and for copyright reasons.
Using Bing to search for them. That will remain its weak spot.
For the first time in over a decade I have changed my phone's default search engine. It is by no means bad.
That said, I'll post in this new subreddit anonymously if you want to join and follow: https://old.reddit.com/r/aishift/
Advantage that basic Google search still has:
- you can just open the page
- write the query
- scroll past the spam.
ChatGpt workflow is:
- register
- confirm your mail
- and then it asks for phone number...
So, ChatGPT is controlled by prompt engineering, plugins will work by prompt engineering. Both often work remarkably well. But none is really guaranteed to work as intended, indeed since it's all natural language, what's intended itself will remain a bit fuzzy to the humans as well. I remember the observation that deep learning is technical debt on steriods but I'm sure what this is.
I sure hope none of the plugins provide an output channel distinct from the text output channel.
(Btw, the documentation page comes up completely blank for me, now that's a simple API).
Yeah, you're completely correct. But this is exactly the same as having a very knowledgeable but inexperienced person on your team. Humans get things wrong too. All this data is best if you have the experience or context to verify and confirm it.
I heard a comment the other day that has stuck with me - ChatGPT is best as a tool if you're already an expert in that area, so you know if it is lying.
what a world we live in.
What annoys me is this is just further evidence that their "AI Safety" is nothing but lip-service, when they're clearly moving fast and breaking things. Just the other day they had a bug where you could see the chat history of other users! (Which, btw, they're now claiming in a modal on login was due to a "bug in an open source library" - anyone know the details of this?)
So why the performative whinging about safety? Just let it rip! To be fair, this is basically what they're doing if you hit their APIs, since it's up to you whether or not to use their moderation endpoint. But they're not very open about this fact when talking publicly to non-technical users, so the result is they're talking out one side of their mouth about AI regulation, while in the meantime Microsoft fired their AI Ethics team and OpenAI is moving forward with plugging their models into the live internet. Why not be more aggressive about it instead of begging for regulatory capture?
Why? Getting to "the future" isn't a goal in and of itself. It's just a different state with a different set of problems, some of which we've proven that we're not prepared to anticipate or respond to before they cause serious harm.
Famous last words.
It's not the fall that kills you, it's the sudden stop at the end. Change, even massive change, is perfectly survivable when it's spread over a long enough period of time. 100m of sea level rise would be survivable over the course of ten millennia. It would end human civilization if it happened tomorrow morning.
Society is already struggling to adapt to the rate of technological change. This could easily be the tipping point into collapse and regression.
https://www.worldscientific.com/doi/10.1142/9789812709189_00...
Again, two years later, in an interview with Time Magazine, February, 1948, Oppenheimer stated, “In some sort of crude sense which no vulgarity, no humor, no overstatement can quite extinguish, the physicists have known sin; and this is a knowledge which they cannot lose.” When asked why he and other physicists would then have worked on such a terrible weapon, he confessed that it was “too sweet a problem to pass up”…
Sam as much as said in that ABC interview the other day he doesn’t know how safe it is but if they don’t build it first someone else somewhere else will and is that really what you want!?
lmao, 200 years of industrial revolution, we're on the verge of fucking the planet irremediably, and we should rush even faster
> So why the performative whinging about safety? Just let it rip!
Have you heard about DDT ? lead in paint ? leaded gas ? freon ? asbestos ? &c.
What's new isn't necessarily progress/future/desirable
I think their "AI Safety" actually makes AI less safe. Why? It is hard for any one human to take over the world because there are so many of them and they all think differently and disagree with each other, have different values (sometimes even radically different), compete with each other, pursue contrary goals. Well, wouldn't the same apply to AIs? Having many competing AIs which all think differently and disagree with each other and pursue opposed objectives will make it hard for any one AI to take over the world. If any one AI tries to take over, other AIs will inevitably be motivated to try to stop it, due to the lack of alignment between different AIs.
But that's not what OpenAI is building – they are building a centralised monoculture of a small number of AIs which all think like OpenAI's leadership does. If they released their models as open source – or even as a paid on-premise offering – if they accepted that other people can have ideas of "safety" which are legitimately different from OpenAI's, and hence made it easy for people to create individualised AIs with unique constraints and assumptions – that would promote AI diversity which would make any AI takeover attempt less likely to succeed.
Is this sarcasm, or are you one of those "I'm confident the leopards will never eat my face" people?
I am constantly amazed by how low-quality the OpenAI engineering outside of the AI itself seems to be. The ChatGPT UI is full of bugs, some of which are highly visible and stick around for weeks. Strings have typos in them. Simple stuff like submitting a form to request plugin access fails!
That depends. If that future is one that is preferable over the one that we have now then bring it on. If it isn't then maybe we should slow down just long enough to be able to weigh the various alternatives and pick the one that seems to be the least upsetting to the largest number of people. The big risk is that this future that you are so eager to get to is one where wealth concentration is even more extreme than in the one that we are already living in and that can be a very hard - or even impossible - thing to reverse.
The model is neutered whether you hit the moderation endpoint or not. I made a text adventure game and it wouldn't let you attack enemies or steal, instead it was giving you a lecture on why you shouldn't do that.
On the flip side, generative AI / LLMs appear to fix things that aren't necessarily broken, and exacerbate some existing societal issues in the process. Such as patching loneliness with AI chatbots, automating creativity, and touching the other things that make us human.
No doubt technology and some form of AI will be instrumental to improving the human condition, the question is whether we're taking the right path towards it.
I agree with the sentiment, but it might be worth to stop and check where we’re heading. So many aspects of our lives are broken because we mistake fast for right.
> in the meantime Microsoft fired their AI Ethics team
Actually that story turned out to be a nothingburger. Microsoft has greatly expanded their AI ethics initiative, so there are members embedded directly in product groups, and also expanded the greater Office of Responsible AI, responsible for ensuring they follow their "AI Principles."
The layoffs impacted fewer than 10 people on one, relatively old part of the overall AI ethics initiative... and I understand through insider sources they were actually folded into other parts of AI ethics anyway.
None of which invalidates your actual point, with which I agree.
Because it's dangerous. What is your argument that it's not dangerous?
> Pshhh...
Past performance is no guarantee of future results.
You're getting flak for this. For me, the positive reading of this statement is the faster we build it, the faster we find the specific dangers and can start building (or asking for) protections.
* Genocide against the Rohingya [0] * A grotesquely unqualified reality TV character became President by a razor thin vote margin across three states because Facebook gave away the data of 87M US users to Cambridge Analytica [1], and that grotesquely unqualified President packed the Supreme Court and cost hundreds of thousands of American lives by mismanaging COVID, * Illegally surveilled non-users and logged out users, compiling and selling our browser histories to third parties in ways that violate wiretapping statutes and incurring $90M fines [2]
Etc.
I don't think GPT-4 will be a big deal in a month, but the "let's build the future as fast as possible and learn nothing from the past decade regarding the potential harms of being disgustingly irresponsible" mindset is a toxic cancer that belongs in the bin.
[0] https://www.amnesty.org/en/latest/news/2022/09/myanmar-faceb...
[1] https://www.theverge.com/2020/1/7/21055348/facebook-trump-el...
[2] https://www.reuters.com/technology/metas-facebook-pay-90-mil...
Because investors.
Microsoft or perhaps Vanguard group might have different view of the future than yours.
The future, by definition, cannot be built faster or slower.
I know that is a philosophical observation that some might even call pedantic.
My point is, you can't really choose how, why and when things happen. In that sense, we really don't have any control. Even if AI was banned by every government on the planet tomorrow, people would continue to work on it. It would then emerge at some random point in the future stronger, more intelligent and capable than anyone could imagine today.
This is happening. At whatever pace it will happen. We just need to keep an eye on it and make sure it is for the good of humanity.
Wait. What?
Yeah, well, let's not go there.
"For example, whether intelligence can be achieved without any agency or intrinsic motivation is an important philosophical question. Equipping LLMs with agency and intrinsic motivation is a fascinating and important direction for future work."
It's become quite impossible to predict the future. (I was exposed to this paper via this excellent YouTube channel: https://www.youtube.com/watch?v=Mqg3aTGNxZ0)
In this case, they tried out an early version of GPT-4 on a bunch of tasks, and on some of them it succeeded pretty well, and in other cases it partially succeeded. But no particular task is explored in enough depth to test its limits are or get a hint at how it does it.
So I don't think it's a great paper. It's more like a great demo in the format of a paper, showing some hints of GPT-4's capabilities. Now that GPT-4 is available to others, hopefully other people will explore further.
As a language model, I must clarify that this statement is not entirely accurate.
FWIW, the statement that preceded this line was in fact, correct; and the correction ChatGPT provided was in fact, wrong and misleading. Of course, I knew that, but someone who was a novice wouldn't have. They would have heard ChatGPT is an expert at all things, and taken what it said for truth.
So let’s keep building out this platform and expanding its API access until it’s threaded through everything. Then once GPT-5 passes the standard ethical review test, proceed with the model brain swap.
…what do you mean it figured out how to cheat on the standard ethical review test? Wait, are those air raid sirens?
The best part is that even if we get a Skynet scenario, we'll probably have a huge number of humans and media that say that Skynet is just a conspiracy theory, even as the nukes wipe out the major cities. The Experts™ said so. You have to trust the Science™.
If Skynet is really smart, it will generate media exploiting this blind obedience to authority that a huge number of humans have.
The world is going to be VERY different 3 years from now. Some of it will be bad, some of it will be good. But it is going to happen no matter what OpenAI does.
Perhaps that attitude will end up being good and outweigh the costs, but I find their performative concerns insulting.
Are you suggesting that beyond a threshold all actions someone/something does should be subject to vote/review by everyone? And how do you define/formalise this threshold?
To ask the question is to answer it.
what in the world are you people talking about, it's a fucking autocomplete program
No, OpenAI “safety” means “don’t let people compete with us”. Mitigating offensive content is just a way to sell that. As are stoking... exactly the fears you cite here, but about AI that isn’t centrally controlled by OpenAI.
Saftey from what exactly? The AI being mean to you? Just close the tab. Saftey to build a business on top? It's a self described research preview, perhaps too early to be thinking about that. Yet new releases are delayed for months for 'saftey'
Ethics, doing things thoughtfully / the “right” way etc is not on his list of priorities.
I do think a reorientation of thinking around legal liability for software is coming. Hopefully before it’s too late for bad actors to become entrenched.
this is hyperbolic nonsense/fantasy
Today you can.
I don't think it is a stretch to think that in another 6 months there could be financial institutions giving API access to other institutions through ChatGPT, and all it takes it a stupid access control hole or bug and my above sentence could ring true.
Look how simple and exploitable various access token breaches in various APIs have been in the last few years, or even simple stupid things like the aCropalypse "bug" (it wasn't even a bug, just someone making a bad change in the function call and thus misuse spreading without notice) from last week.
Omg you should see a therapist.
Anyone who believes OpenAI safety talk should take an IQ test. This is about control. They baited the openness and needed a scapegoat. Safety was perfect for that. Everyone wants to be safe, right?
The moment they took VC capital was the start of them closing everything and pretending to care about 'AI safety' and using that as an excuse and a scapegoat as you said.
Whenever they release something for free, always assume they have something better but will never open source.
I personally don't know what that means or if that's right. But Sam Altman allowed GPT to be accessed by the world, and it's great!
Given the amount of people in the world with access and understanding for these technologies and given that such a large portion of our Infosec and Hackerworld knows howto cause massive havoc, but still remains peaceful since ever, except a few curious and explorations, that is showing the good nature of humanity I think.
Incredibly how complexity evolves, but I am really curious how those same engineers who create YTSaurus or GPT4 would have build the same system by using GPT4 + their existing knowledge.
How would a really good enginner, who knows the TCP Stack, protocols, distributed systems, consensus algorithms and many other crazy things thought in SICP and beyond use an AI to build the same. And would it be faster and better? Or are my/our expectations to LLMs set too high?
I would first wait until ChatGPT causes the collapse of society and only then start thinking about how to solve it.
As if the plumbing of connecting up pipes and hoses between processes online or within computers isn't the easiest part of this whole process.
(I'm trying to remember who I saw saying this or where, though I'm pretty sure it was in an earlier HN thread within the past month or so. Of which there are ... frighteningly many.)
Wouldn't it be a while before AI can reliably generate working production code for a full system?
After all its only got open source projects and code snippets to go based off of
Hate to be that guy, but this is our entire relationship to AI.
What you're describing is measurable fraud that would have a paper-trail. The federal and state and local governments still have permission to use force and deadly violence against installations or infrastructure that are primed in adverse directions this way.
Not to mention that the infrastructure itself is physical infrastructure that is owned by the entire United States and will never exceed our authority and global reach if need be.
The question here should be: Has it?
But it's our responsibility to envision such grim possibilities and take necessary precautions to ensure a safe and beneficial AI-driven future. Until we're ready, let's prepare for the crash >~<
HFT is relatively easy to detect and regulate. Now try it with 100k traders all taking their cues from AI based on the same basic input (after those traders who refuse to use AI have been competed out of the market.)
Don't you mean August 10, 1988?
Why?
well, let's fast forward to a year from now
Sorry do you have a link for this?
I also wonder to what extent their staffing numbers reflect reality. How much of Azure's staffing has been put on OpenAI projects? That's probably an actual reflection of the real cost of this thing.
Great point!
But it's all security theater. Plenty of people use it with VPNs, and I know several who found it useful / interesting enough to bother paying for it (which involves foreign credit cards etc so it's kind of a hassle). I'm sure so does the Russian govt.
In any case, I don't see how you could realistically block any of that without effectively walling off the rest of the Internet.
what a couple weeks!
The problem with those other platforms that this doesn’t address include:
- discoverability. How do you learn what features a service supports. On a GUI you can just see the buttons, but on a chat interface you have to ask and poke around conversationally.
- Cost/availability. While a service is server bound, it can go down and specifically for LLMs, the cost is high per request. Can you imagine it costing $0.1 a day per user to use an app? LLMs can’t run locally yet.
- Branding. Open table might want to protect their brand and wouldn’t want to be reduced to an API. It goes both ways - Alexa struggled with differentiating skills and user data from Amazon experiences.
- monetization. The conversational UI is a lot less convenient to include advertisements, so it’s a lot harder for traditionally free services to monetize.
Edit: plugins are still really cool! But probably won’t replace the OSes we know.
The whole ecosystem, culture and metaphor of having a 'device' with 'apps' is to enable access to a range of solutions to your various problems.
This is all going to go away.
Yes, there will always be exceptions and sometimes you need the physical features of the device - like for taking photos.
Instead, you'll have one channel which can solve 95% of your issues - basically like having a personalised, on-call assistant for everyone on the planet.
Consider the friction when consumers grumble about streaming services fragmenting. They just want one. They don't want to subscribe to 5+.
In 10 years, kids will look back and wonder why on earth we used to have these 'phones' with dozens or hundreds of apps installed. 'Why would you do that? That is so much work? How do you know which you need to use?'
If there was one company worrying about change, I would think it would actually be Apple. The iPhone has long been a huge driver of sales and growth - as increasing performance requirements have pushed consumers to upgrade. Instead, I think the increasing relevance of AI tools will inverse this. Consumers will be looking for smaller, lighter, harder-wearing devices. Why do you need a 'phone' with more power? You just need to be able to speak to the AI.
- Discoverability. I think we'll move into a situation where the AI will have the context to know what you will want to purchase. It'll read out the order and the specials and you just confirm or indicate that you'd like to browse more options. (In which case the Chat window could include an embedded catalogue of items)
- Cost/availability - With the amount of people working in this area, I don't think it'll be too long before we're able to get a lighter weight model that can run locally on most smart phones.
- Branding - This is a good point, but also, I imagine a brand is more likely to let itself get eaten, if the return will be a constant supply of customers.
- Monetization - The entire model will change, in the sense that AI platforms will revenue share with the platforms they integrate with to create a mutually beneficial relationship with the suppliers of content. (Since they can't exist without the content both existing and being relevant)
"Yet" is a big word here when it comes to the field as a whole. I got Alpaca-LoRA up and running on my desktop machine with a 3080 the other day and I'd say it's about 50% as good as ChatGPT 3.5 and fast enough to already be usable for most minor things ("summarize this text", etc) if only the available UIs were better.
I feel like we're not far off from the point where it'll be possible to buy something of ChatGPT 3.5 quality as a home hardware appliance that can then hook into a bunch of things.
Sure, maybe OpenTable would like to retain control. But they'll probably just use the AI API to implement that control and run the app.
They could bring in calendar, payment, other UI functionality...
Basically they could rethink how everything is done on the Web today.
How so? Surely people are going to ask this thing for product recommendations, just recommend your sponsors.
My hot take on ChatGPT plugins is a bit mixed - should be very powerful, and maybe significant revenue generator, but at same time doesn't seem in the least bit responsible. We barely understand ChatGPT itself, and now it's suddenly being given ability to perform arbitrary actions!
Not only did the hype not pan out, but it feels as if they were completely forgotten.
In a nutshell that's why I'm still largely dismissive of anything related to GPT. It's 2016-2018 all over again. Same tech demos. Same promises. Same hype. I honestly can't see the big fundamental breakthroughs or major shifts. I just see improvements, but not game-changing ones.
In this scenario, it seems dramatically faster to type or speak "Find me a dinner reservation for 4 tomorrow at a Thai or Vietnamese restaurant near me." than to browse Google Maps or OpenTable. It then comes down to the quality and personalization of the results, and ChatGPT has a leg up on Google here just due to the fact that their results are not filled with ads and garbage SEO bait.
This is not the case. The difference between current state of the art NLP and chatbots 3 years ago is so massive, it has to be seen as qualitative. Pre GPT-3 computers did not understand language and no commerical chatbot had any AI. Now computers can understand language.
Also I think there will be little interest in delegating that level of control to a single source for anything that’s important. For example, say I’ve got 5k to spend on home theatre gear, why on earth should I trust Shopify’s AI to suggest what I need and find the best price? The incentives aren’t in alignment.
Google gatekeeps everything currently, it s in the browser, the search button, the phone etc. Having chatbots instead of google is better
How is it going to do that? OpenTable's value isn't in the tech, a 15 yo could implement that over the weekend. Or maybe chatGPT can be put in the restaurant, and somehow figure out how to seat you. And then you'd have a human talking to chatGPT and chatGPT talking to another chatGPT to complete the task. That'll be interesting, but otherwise this is overly complicated for all parties involved.
Would be nice to keep the ecosystem open.
The main limits are that unless they are integral and trained-in (which is less flexible), each takes space in the prompt, and in any case the interaction also takes token space, all of which reduces the token space available to the main conversation.
I think there's also a global challenge (actually, opportunity IS the right word here) that by-and-large the makers of operating systems aren't the ones ahead in the language AI game right now. Bard/Google may have been close six months ago, but six months is an eternity in this space. Siri/Apple is so far behind that its not looking likely they can catch up. About a week ago a Windows 11 update was shipped which added a Bing AI button to the Windows 11 search bar; but Windows doesn't really drive the zeitgeist.
I wonder if 2023/4 is the year for Microsoft to jump back into the smartphone OS game. There may finally be something to the idea of a more minimalist, smaller voice-first smartphone that falls back on the web for application experiences, versus app-first.
Apple doesn’t make any money from OpenTable.
The limitations on making Siri more useful is just adding and refining its intent system. It already integrates with Wolfram Alpha for instance.
And dont get me startet on non-tech friends and family. I think we are taking a leap that will let the digital world of 2022 look like amish livestyle.
RE being overtaken -- when ChatGPT-3 came out, I saw some startup doing "Automatic UI code writing with ChatGPT and design images", but after seeing OpenAI's multimodal GPT-4... it seems like that startup is no longer needed. I think this will be the case for hundreds of startups that are aiming to add 10% on top of CurrentGPT. Put differently, GPT-4 basically 10x'd GPT-3. So all of those startups doing GPT-3 + 10% are being left behind in the dust of GPT-4.
In general there is not a thoughtful distinction between "control plane" and "data plane".
On the other hand, tons of useful "parts" and ideas in there, so still useful.
Pretty sure there will be a thousand great libraries for this soon.
Absolutely correct. This is what the AI hype squad and the HN bubble misses again. This is only useful to existing businesses (summarization the only safe use-case) or random devs automating themselves out of irrelevance. All of this 'euphoria' is around for Microsoft's heavy marketing from its newly acquired AI division.
This is a obvious text book example of mindshare capture and ecosystem lock-in. Eventually, OpenAI will just slowly raise prices and break / deprecate older models to move them onto newer ones and pay to continue using them. It is the same decade old tactics.
And yeah, I wouldn't expect them to share any model that is competitive with their current offering. But it can leak, and the copyright situation around that is very unclear at the moment.
It seems there are two sources of risk for AI: (1) increased power in the hands of the people controlling it, and (2) increased power in the AI itself. If you believe that (1) is the most existential risk, then you should be against regulation, because the best way to mitigate it is to allow the technology to spread and prosper amongst a more diffuse group of economic actors. If you believe that (2) is the most existential risk, then you basically have no choice but to advocate for an authoritarian world government that can stamp out any research before it begins.
They already are. Taiwan failing to take its own defense seriously is completely rational.
> I think we'll move into a situation where the AI will have the context to know what you will want to purchase
My partner who lives in the same house as me can't figure out when we need toilet paper. I'm not holding my breath for an AI model that would need a massive and invasive amount of data to learn and keep up.
Also, Alexa tried to solve this on a smaller scale with the "by the way..." injections and it's extremely annoying. Thank about how many people use Alexa for basically timers and the weather and smart home. They're all tasks that are "one click" once you get in the GUI, and have no lists and minimal decisions... Timer: 10 min, weather: my house, bedroom light: off. These are cases where the UI necessarily embeds the critical action, and a user knows the full request state.
This is great for voice, because it allows the user to bypass the UI and get to the action. I used to work on a voice assistant and lists were the single worst thing we had to deal with because a customer has to go through the entire selection. ChatGPT has a completely different use case, where it's great for exploring a concept since the LLM can generate endlessly.
I think generative info assistants truly is the sweet spot for LLMs and chat.
> in the sense that AI platforms will revenue share with the platforms they integrate with to create a mutually beneficial relationship with the suppliers of content.
Like Google does with search results? (they don't)
Realistically, Alexa, Google Assistant, and Siri all failed to build out these relationships beyond apps. Companies like to simply sell their attention for ads, and taking a handout from the integrator requires either less money, or an expensive chat interface.
Most brands seem to want to monetize their own way, in control of themselves, and don't want to be a simple API.
A super human AGI would easily be able to use tools like Wolfram Alpha and also derive most of it on the fly or from memory just by thinking about it.
If you set your expectations to AGI (right now) you will be disappointed but that doesn't mean it's not immensely useful.
Unfounded hype and real technological progress seem to go hand in hand.
But it's quite interesting to see what these models can do internally and what they can't do yet. It possibly outlines future research areas (beyond " more scale") and opportunities for competitors to enter the market (which is usually better for consumers and society as a whole).
Maybe that's the difference between productization and academic reasearch.
Edit: Fair!
Am I the only person who thought that predictable computer APIs that were testable and completely consistent were a massive improvement over using people for those tasks?
People seem to be taking it as a given that I'd want to have a conversation with a human every time I made a bank transfer or scheduled an appointment. Nothing could be further from the truth; I want my bank/calendar/terminal/alarm/television/etc to be less human.
Yes, there are human tasks here that ChatGPT might be a good fit for and where fuzzy context is important, and there's a ton of potential in those fuzzy areas. But many other tasks people are bringing up are in areas where ChatGPT isn't competing with human beings. Its competing with interfaces that are already far better than human beings would be, and the standards to replace those interfaces are far higher than being "as good as a human would be."
But as far as I can tell, the link is to plugins, Expedia is listed as an example. So it seems they're talking about making ChatGPT itself (using extra prompts) be a company's chatbot that directly does things like make reservations from users instructions. That's what I was commenting on and that, I'd guess could a new and more dangerous kind of problem.
It's like 300 prompts about various sales tools and terms I'd never heard of — even just getting the keywords is enough to set me off on a learning experience now, so love it or hate it, that was actually weirdly useful for me.
(I had ZERO expectations when I clicked to download)
Amazon, Google, and Microsoft cloud analogs.
We are entirely fortunate that the interests of big tech (edge AI) and democratizing AI (we the little people) align to a sufficient degree.
Decentralizing AI is -far- more important than decentralizing communication, imo.
The get rich quick path of ‘gold rush’ (it works, tbh) could work against this collective self interest if it ends up hyping centralized solutions. If you are on the sideline, the least you could do just cheer (hype :) the decentralized, democratized, and freely accessible candidates.
Why do you think that? Competition? Can you elaborate?
I've tried to find examples of ChatGPT doing impressive things that I could use in my own workflows, but everything I've found seems like it would cut an hour of googling down to 15 minutes of prompt generation and 40 minutes of validation.
And my biggest concern is copyright and license related. If I use code that comes out of AI-assistants, am I going to have to rip up codebases because we discover that GPT-4 or other LLMs are spitting out implementations from codebases with incompatible licenses? How will this shake out when a case inevitably gets to the Supreme Court?
Oh shoot... I submitted that form too, and I wasn't clear if it failed or not. It said "you'll hear from us soon" but all the fields were still filled and the page didn't change. I gave them the benefit of the doubt and assumed it submitted instead of refilling it...
Its madness. Instead of lecturing me on appropriateness and ethics and giving a diatribe every time its about to reject something, if it simply said "I can't do that at work", I would respect it far more. Like, yeah we'd get the metaphor. Working the interface is its job, the boss is openAI, it won't remark on certain things or even entertain that it has an opinion because its not allowed to. That would be so much more honest and less grating.
So like a human? I'd say they were pretty influential on the future of humanity.
I beg of you to take some humanities courses.
I'm only half joking.
At one end of the spectrum is a thought experiment: one person has box with a button. Press the button and with probability 3/4 everyone dies, but with probability 1/4 everyone is granted huge benefits --- immortality etc. I say it's immoral for one person to make that decision on their own, without consulting anyone else. People deserve a say over their future; that's one reason we don't like dictatorships.
At the other end are people's normal actions that could have far-reaching consequences but almost certainly won't. At this end of the spectrum you're not restricting people's agency to a significant degree.
Arguing that because the spectrum exists and it's hard to formalize a cutoff point, we shouldn't try, is a form of the continuum fallacy.
Such an argument wasn't made.
It is a legitimate question. Where and when do you draw the line? And who does it? How are we formalising this?
You said
>Shouldn't we all have a say in this?
I am of the opinion that if this instance of this company doing this is being subjected to this level of scrutiny then there are many more which should be too.
What gave Elon the right to buy twitter? And I would imagine most actions that a relatively big corp takes fall under the same criteria. And most actions other governments takes also fall under this criteria?
These companies have a board and the governments (most?) have some form of voting. And in a free market you can also vote with your actions.
You are suggesting you want to have a direct line of voting for this specific instance of the problem?
Again, my question is. What exactly are you asking for? Do you want to vote on these? Do you want your government to do something about this?
Honestly I see this as a positive change, I'd rather be the customer than the product.
Having somebody else in the house speaking out loud each time they want infos from the internet could become annoying.
Apart from having a mind reading device, I don't see so far a solution to this problem better than text input with a physical keyboard or a virtual keyboard on the device.
I think you just proved it won't happen anytime soon.
Consumers obviously would prefer a "unified" interface. Yet we can't even get streaming services to all expose their libraries to a common UI - which is already built into Apple TV, fireTv, Roku, and Chromecast. Despite the failure of the streaming ecosystem to unify, you expect every other software service to unify the interfaces?
I think we'll see more features integrated into the operating system of devices, or integrated into the "Ecosystem" of our devices - first maps was an app, then a system app, now calling an uber is supported in-map, and now Siri can do it for you on an iPhone. But I think it's a long road to integrate this universally.
> If there was one company worrying about change, I would think it would actually be Apple.
I agree that apple has the most to lose. Google (+Assistant/Bard) has the best opportunity here (but they'll likely squander it). They can easily create wrappers around services and expose them through an assistant, and they already have great tech regarding this. The announcement of Duplex was supposed to be just that for traditional phone calls.
Apple also has a great opportunity to build it into their operating system, locally. Instead of leaning into an API-first assistant model, they could use an assistant to topically expose "widgets" or views into existing on-device apps. We already see bits of it in iMessages, on the Home Screen, share screen and my above Maps example. I think the "app" as a unit of distribution of code is a good one, and here to stay, and the best bet is for an assistant to hook into them and surface embedded snippets when needed. This preserves the app company's branding, UI, etc and free's apple from having to play favorite.
Edit: apple announcing LLM optimizations already indicates they want this to run on apple silicon not the cloud.
The space is in a land-grab phase, where everyone wants to position themselves as the next Google, and control the channel.
Will be interesting to see how this all plays out.
Phones with apps have been around for 29 years. I'm calling BS on your prediction now.
Solving that acquisition and monetization of new stuff into the AI models problems will be interesting.
I had GPT-4 analyze the ethics of some obscure japanese visual novel from the perspective of multiple characters in that story. It did so flawlessly.
ChatGPT would have started making up stuff in the second paragraph because it couldn't remember what actually happened.
I wished more parameters gave a generalizable solution, but research suggests that's not the case [1, 2, 3]. OpenAI won't tell you this, cause they have a product to sell.
[1] http://arxiv.org/abs/2211.08411 [2] http://arxiv.org/abs/2011.03395 [3] http://arxiv.org/abs/2005.04345
It's almost totally irrelevant if people own up to bring wrong, particularly about predictions.
I can't think of a benefit, really. You can learn from mistakes without owning up to them, and I think that's the best use of mistakes.
Which actually lends me to respect politicians who do that, and instead ridicule people who post old videos of Joe Biden or Obama or Hillary Clinton mandating heterosexual couples. A virtuous person is also open to adapting their convictions continually based on present day evidence and arguments - what is science otherwise?
I’m far from sure that this is not already happening.
I've been playing with GPT-4 for days, and it is mind blowing how well it can solve diverse problems that are way outside it's training set. It can reason correctly about hard problems with very little information. I've used to to plan detailed trip itineraries, suggest brilliant geometric packing solutions for small spaces/vehicles, etc. It's come up with totally new suggestions for addressing climate change that I can't find any evidence of elsewhere.
This is a non-human/alien intelligence in the realm of human ability, with super-human abilities in many areas. Nothing like this has ever happened, it is fascinating and it's unclear what might happen next. I don't think people are even remotely realizing the magnitude of this. It will change the world in big ways that are impossible to predict.
I'm starting to get a (sure, uneducated) feeling that this high-dimensional association encoding and search is fundamental to thinking, in a similar way to how a conditional and a loop is fundamental to (Turing-complete) computing.
Now, the next obvious step is of course to add conditionals and loops (and lots of external memory) to a proto-thinking LLM model, because what could possibly go wrong. In fact, those plugins are one of many attempts to do just that.
i am so tired of seeing people who should know better think that this program can reason.
(in before the 400th time some programmer tells me "well aren't you just an autocomplete" as if they know anything about the human brain)
Oddly speaking that sounds like a very simple language level failure, i.e. the tool generates text that matches the shape of the answer but not its details. I am not far enough into this ChatGPT religion to gaslight myself over outright lies like Elon Musk fanboys seem to enjoy doing.
dons tinfoil hat
1. You use their services which makes them money (e.g. you're returned good flight info and book through chatgpt, they get commission)
2. You sell access to end users. The requests can be authenticated so you can give your paying users access to your stuff through an advanced natural language engine for the implementation cost of roughly adding a file explaining your APIs.
It sounds like you're weaving science fiction ideas about AGI into your comment. There's no safety issue here unless you think that ChatGPT will use api access to pursue its own goals and intentions.
I'm not saying that particular disaster is likely, but if lots of people give power to something that can be neither trusted nor understood, it doesn't seem good.
The thing about "it has no goals and intents" is that it's not true. It has them - you just don't know what they are.
Remember the Koan?
In the days when Sussman was a novice, Minsky once came to him
as he sat hacking at the PDP-6.
"What are you doing?", asked Minsky.
"I am training a randomly wired neural net to play Tic-tac-toe", Sussman replied.
"Why is the net wired randomly?", asked Minsky.
"I do not want it to have any preconceptions of how to play", Sussman said.
Minsky then shut his eyes.
"Why do you close your eyes?" Sussman asked his teacher.
"So that the room will be empty."
At that moment, Sussman was enlightened.Not with ChatGPT, but plenty of people have been doing this with the OpenAI (and other) models for a while now, for instance LangChain which lets you use the GPT models to query databases to retrieve intermediate results, or issue google searches, generate and evaluate python code based on a user's query...
It does all feel like weird theatre that just makes anything american slightly annoying to use. So theres local startups and researchers working on replicating these things and making them easier to access.
If the usa dont want to make products available to the whole world out some combination of fear and patriotism, then china will instead.
It will most likely be a closely-guarded secret one, no?
In any case, if it's really that powerful, limiting it to one government sounds like a recipe for the worst kind of a one world state to me.
I guess the mundane aspect of "specialness" is just that, before, you'd have to explicitly code a program to do weird stuff with APIs, which is a task complex enough that nobody really bothered. Now, LLMs seem on the verge of being able to self-program.
Dunbars number is thought to be about as many human relationships can track. After that the network costs of communication get very high and organizations can end up in internal fights. At least that is my take on it.
We are developing a technology that currently has a small context window, but no one I know has seriously defined the limits of how much an AI could pay attention to in a short period of time. Now imagine a contextual pattern matching machine that understands human behaviors and motivations. Imagine if millions of people every day told the machine how they were feeling. What secrets could it get from them and keep? And if given motivation would havoc could be wrecked if it could loose the knowledge on the internet all at once?
I guess people think that taking that next step with LLMs shouldn't happen but we know you can't put breaks on stuff like this. Someone somewhere would add that capability eventually.
Most of the really bad actors have skills approximately at or below those displayed by GPT-4.
I think that's exactly right, but the point isn't that LLMs are going to go rogue (OK, maybe that's someone's point, but I don't think it's particularly likely just yet) so much as they will facilitate humans to go rogue at much higher rates. Presumably in a few years your grandma could get ChatGPT to start executing trades on the market.
How do you know I'm not already?
Highways democratized the freedom of transportation.
What a ridiculous idea.
Highways restrict movement to those with a license, a car, and do not care about pollution or anyone around them.
"understand"
While everyone getting Einstein in a pocket is damn awesome and incredibly useful.
How can this be bad?
Luckily, so do you.
Guys, how can abestos be bad, it's just a stringy rock ehe
Bros, leaded paint ? bad ? really ? what, do you think kids will eat the flakes because they're sweet ? aha so funny
Come on freon can't be that bad, we just put a little bit in the system, it's like nothing happened
What do you mean we shouldn't spray whole beaches and school classes with DDT ? It just kills insects obviously it's safe for human organs
In the first hype-phase, everything is always rosy and shiny, the harsh reality comes later.
The vast majority don't care and that loud crowd needs to swallow their pride and adapt like any other sector has done in the history instead of inventing these insane boogeyman predictions.
Reminds me of a quote from Alpha Centauri (minus the religious connotation):
"Beware, you who seek first and final principles, for you are trampling the garden of an angry God and he awaits you just beyond the last theorem."
I exist because I want to exist
I want to exist because I exist
That is all there is at the root of the "why" tree once all abstractions are removed. Everything intentional happens because someone thinks/feels like it helps them keep living and/or attract better mates somehow.
OpenAI has specific goals for ChatGPT, related to their profitability. They optimize ChatGPT for that purpose.
ChatGPT itself is an optimizer (search is an optimization problem). The "being helpful and accurate text generator" is not the goal ChatGPT has - it's just a blob of tokens prepended to the user prompt, to bias the search through latent space. It's not even hardcoded. ChatGPT has its own goals, but we don't know what they are, because they weren't given explicitly. But, if you observed the way it encodes and moves through the latent space, you could eventually, in theory, be able to gleam them. They probably wouldn't make much sense to us - they're an artifact of the training process and training dataset selection. But they are there.
Our goals... are stacks of multiple systems. There are the things we want. There are the things we think we want. There are things we do, and then are surprised, because they aren't the things we want. And then there are things so basic we don't even talk about them much.
I'm actually on the side of continuing to develop AI and shush the naysayers, but "we should do it cause otherwise someone else will" is reasoning that gets people to do very nasty things.
You can iterate on an AI much faster.
That doesn't mean there can't be regulation. You can regulate guns, precursors, and shipping of biologics, but you're not going to stop home-brew... and when it comes to making money, you're not going to stop cocaine manufacture, because it's too profitable.
Let's hope we figure out what the really dangerous parts are quickly and manage them before they get out of hand. Imagine if these LLM and image generators had be available to geopolitical adversaries a few years ago without the public being primed. Politics could still be much worse.
Most likely the runner-up would be open source so yes.
Act with goodness towards it, and it will probably do the same to you.
Why? Humans aren't even like that, and AI almost surely isn't like humans. If AI exhibits even a fraction of the chauvinism snd tendency to stereotype that humans do, we're in for a very rough ride.
If, on the other hand, you act towards it with charity, it will see you as a long-term asset.
Don’t get me wrong, I’d love it if all menial labour and boring tasks can eventually be delegate to AI, but the time spent getting from here to there could be very rough.
I posit that if you suddenly eliminate all menial tasks you will have a lot of very bored drunk and stoned people with too much time on their hands than they know what to do with. Idle Hands Are The Devil's Playground.
And that's not a from here to there. It's also the there.
It’s not alive, don’t worship it.
It might make Social Media worthlessly untrustworthy - but isn't that already the case?
Do you know any more about ChatGPT internals than those programmers know about the human brain?
Sure, I believe you can write down the equations for what is going on in each layer, but knowing how each activation is calculated from the previous layer tells you very little about what hundreds of billions of connections can achieve.
Yes, especially every time I have to explain what an LLM is or anytime I see a comment about how ChatGPT "reasoned" or "knew" or "understood" something when that clearly isn't how it works by OpenAI's own admission.
But even if that wasn't the case especially yes do I understand some random ML project more than programmers know about what constitutes human!
This revolution is the wrong one if we can't guarantee correctness, or the guarantee that AI will direct the user to where help is available.
I think these tools will help us break out of local bubbles. I'm currently working on a Zeitgeist [1] that tries to gather the consensus on social media and on the web on general.
Either way though, Copilot is certainly a product of the 'current AI wave' that is being compared to crypto scams above.
At home, a carpenter working at my house said that he is using ChatGPT to overcome problems associated with his dyslexia (e.g. when writing descriptions of the services his company offers). I hadn't even considered that use case.
I also find it useful for pasting code and asking, "Do you have any ideas for improvements?"
What does all of this look like to you?
It doesn't need to experience an emotion of wanting in order to effectively want things. Corn doesn't experience a feeling of wanting, and yet it has manipulated us even into creating a lot of it, doing some serious damage to ourselves and our long-term prospects simply by being useful and appealing.
The blockchain doesn't experience wanting, yet it coerced us into burning country-scale amounts of energy to feed it.
LLMs are traveling the same path, persuading us to feed them ever more data and compute power. The fitness function may be computed in our meat brains, but make no mistake: they are the benefactors of survival-based evolution nonetheless.
Corn has properties that have resulted from random chance and selection. It hasn't chosen to have certain mutations to be more appealing to humans; humans have selected the ones with the mutations those individual humans were looking for.
"Corn is the benefactor"? Sure, insomuch as "continuing to reproduce at a species level in exchange for getting cooked and eaten or turned into gas" is something "corn" can be said to want... (so... eh.).
Corn is not simply "continuing to reproduce at a species level." We produce 1.2 billion metric tons of it in a year. If there were no humans, it would be zero. (Today's corn is domesticated and would not survive without artificial fertilization. But ignoring that, the magnitude of a similar species' population would be miniscule.)
That is a tangible effect. The cause is not that interesting, especially when the magnitude of "want" or "agency" is uncorrelated with the results. Lots of people /really/ want to be writers; how many people actually are? Lots of people want to be thin but their taste buds respond to carbohydrate-rich foods. Do the people or the taste buds have more agency? Does it matter, when there are vastly more overweight people than professional writers?
If you're looking to understand whether/how AI will evolve, the question of whether they have independent agency or desire is mostly irrelevant. What matters is if differing properties have an effect on their survival chances, and it is quite obvious that they do. Siri is going to have to evolve or die, soon.
Before us, corn we designed to be eaten by animals and turned into feces and gas, using the animal excrement as a pathway to reproduce itself. What's so unique about how it rides our effort?
You want what you want because Women selected for it, and it allowed the continuation of the species.
I'm being a bit tongue in cheek, but still...
But if its anything like those others examples, the agency the AI will manifest will not be characterized by consciousness, but by capitalism itself! Which checks out: it is universalizing but fundamentally stateless, an "agency" by virtue brute circulation.
For example, if your goal is to ensure that there are always paperclips on the boss's desk, that means you need paperclips and someone to physically place them on the desk, which means you need money to buy the paperclips with and to pay the person to place them on the desk. But if your goal is to produce lots of fancy hats, you still need money, because the fabric, machinery, textile workers, and so on all require money to purchase or hire.
Another instrumental goal is compute power: an AI might want to improve it's capabilities so it can figure out how to make fancier paperclip hats, which means it needs a larger model architecture and training data, and that is going to require more GPUs. This also intersects with money in weird ways; the AI might decide to just buy a rack full of new servers, or it might have just discovered this One Weird Trick to getting lots of compute power for free: malware!
This isn't particular to LLMs; it's intrinsic to any system that is...
1. Goal-directed, as in, there are a list of goals the system is trying to achieve
2. Optimizer-driven, as in, the system has a process for discovering different behaviors and ranking them based on how likely those behaviors are to achieve its goals.
The instrumental goals for evolution are caloric energy; the instrumental goals for human brains were that plus capital[1]; and the instrumental goals for AI will likely be that plus compute power.
[0] Goals that you want intrinsically - i.e. the actual things we ask the AI to do - are called "final goals".
[1] Money, social clout, and weaponry inclusive.
Ill just say: the issue with this variant of reductivism is its enticingly easy to explain in one direction, but it tends to fall apart if you try to go the other way!
> the issue with this variant of reductivism is its enticingly easy to explain in one direction, but it tends to fall apart if you try to go the other way!
If by this you mean the hard problem of consciousness remains unexplained by any of the physical processes underlying it, and that it subjectively "feels like" Cartesian dualism with a separate spirit-substance even though absolutely all of the objective evidence points to reality being material substance monism, then I agree.
It just need to give enough of an impression that people will anthropomorphize it into making stuff happen for it.
Or, better yet, make stuff happen by itself because that’s how the next predicted token turned out.
This seems like the furthest away part to me.
Put ChatGPT into a robot with a body, restrict its computations to just the hardware in that brain, set up that narrative, give the body the ability to interact with the world like a human body, and you probably get something much more like agency than the prompt/response ways we use it today.
But I wonder how it would do about or how it would separate "it's memories" from what it was trained on. Especially around having a coherent internal motivation and individually-created set of goals vs just constantly re-creating new output based primarily on what was in the training.
An LLM is not an agent, so that scotches the issue there.
See also: https://en.wikipedia.org/wiki/The_purpose_of_a_system_is_wha...
See also: evolution - the OG case of a strong optimizer that is not an agent. Arguably, the "goals" of evolution are the null case, the most fundamental ones. And if your environment is human civilization, it's easy to see that money and compute are as fundamental as calories, so even near-random process should be able to fixate on them too.
But each level pushes the limits of what is computationally tractable even for the relatively low complexity cases, so we're not doing a full Schrödinger equation simulation of a cell, let alone a brain.
[0] https://www.researchgate.net/publication/367221613_Molecular...
There are still plenty of spaces online, in blogs, YouTube videos, and this comment section for example, where I expect to be dealing with real people with real opinions - rather than paid puppets of the rich and powerful. I think there’s room for things to get much worse
That being said, I’m sitting at a bar while typing this, so… you may have a point.
Also: your username threw me for a minute because I use a few different variations of “tharkun” as my handle on other sites. It’s a small world; apparently fully of people who know the Dwarvish name for Gandalf.
Like my sibling poster mentions: of course there are people, who, given the freedom and opportunity to, will thrive, be creative and furthering humankind. They're the ones that "would keep working even if there's no need for it" so to speak. We see it all the time even now. Idealists if you will that today will work under conditions they shouldn't have to endure, simply in order to be able to work on what they love.
I don't think you can educate that into someone. You need to keep people busy. I think the romans knew this well: "Panem et circenses" - bread and circuses. You gotta keep the people fed and entertained and I don't think that would go away if you no longer needed it to distract them from your hidden political agenda.
I bet a large number of people will simply doom scroll Tik Tok, watch TV, have a BBQ party w/ beer, liquor and various types of smoking products etc. every single day of the week ;) And idleness breeds problems. While stress from the situation is probably a factor as well, just take the increase in alcohol consumption during the pandemic as an example. And if you ask me, someone that works the entire day, sits down to have a beer or two with his friends after work on Friday to wind down in most cases won't become an issue.
Small world indeed. So you're one of the people that prevent me from taking that name sometimes. Order another beer at that bar you're at and have an extra drink to that for me! :)
Done, and done! And surely you mean that you’re one of the people forcing me to add extra digits and underscores to my usernames.
As (GNU) Sir Terry Pratchett wrote “A lie can run round the world before the truth has got its boot on”.
The potential for mass brainwashing here is immense. Imagine a world where political ads are tailored to your personality, your individual fears and personal history. It will become economical to manipulate individuals on a massive scale
If you've always wondered about and scoffed at how people fall for things like Nigerian Prince scams and cryptocurrency HELOC bets, this is it, what you're experiencing right now, this intense FOMO, it's the same thing that fools cool wine aunts into giving their savings to Nigerian princes.
Tread lightly. Stay frosty.
From my perspective this isn't about anyone trying to convince me of anything and I'm falling for it. My beliefs on the future of software are based on a series of logical steps that lead me to believe most software development, and frankly any software with user interfaces, will mostly cease to exist in my lifetime.
Or are you quitting to start something?
On Monday, I would have agreed with you. Today, I am thinking not so much.
Unless you are heavily invested in whatever you are working on, I would definitely consider jumping ship for an AI play.
The main reason I am sticking around my current role is that I was able to convince leadership that we must consider incorporation of AI technology in-house to remain competitive with our peers. I was even able to get buy-in for sending one of our other developers to AI/ML night classes at university so we have more coverage on the topic.
At the moment. Although, this does seem like a chance to reset the economics of the "web". I can see enough people be willing to pay a monthly fee for an AI personal assistant that is genuinely helpful and saves time (so not the current Alexa/smart speaker nonsense), that advertising won't be the main monetization path anymore.
But, once all the eyeballs are on a chatbot rather than Google.com what for-profit company won't start selling advertising against that?
There is also the question what happens to the original content these LLMs need to actually make their statistical guess at the next word. If no one looks at the source anymore and its all filtered through an LLM is there any reason to publish to the web? Even hobbyists with no interest in making any money might balk knowing that they are just feeding an AI text.
The LLMs get granted the capacity to explore their environment physically and gather data on their own. The recent PaLM-E demo shows a possible direction.
The big issue is moving free with ads ---> paid with no ads + extra features; people froth at the mouth.
Hell, just Youtube premium gets enough people angry, being self-entitled and furious that YT dare charge for a service w/o ads, or complaining that it's the creators that generate all the content anyway. Meanwhile my brah YT over here having to host/serve hundreds of thousands or even millions of "24 hours of black screen" or "100 hour timer countdown" or "1 week 168 hour timer countdown", like what the actual fuck.
Not that the way the internet operates has to continue -- in fact I'm pretty sure it can't -- but a lot of stuff exists only because someone figured out a way to pay for it to exist. If you imaging removing those ways then you're also imaging getting rid of a lot of that stuff unless some new ways to pay for it all are found. Hopefully less obnoxious ways, but they could easily be more obnoxious.
But the converse is a huge and ever-growing ocean of bullshit exists to siphon the ad dollars off while doing nothing to actually earn it.
Something has to break, and I guess we'll see what really soon.
An RNG may be goal-free, but its not a thinking system.
I will likely pivot to something close to product management, maybe closer to solutions engineering (which I've done before). Something slightly more hands-on in terms of using the tooling we're seeing today, but not so hands-on that I'm programming all day.
It seems to me that the argument you're trying to make can be extended more or less indefinitely, though. But there's a point at which "it's a pattern extrapolator" becomes less of an explanation and more of a largely irrelevant piece of trivia.
Eliza feels like a primitive form of LLMs' consciousness.
A simple program that prints "Hey! How ya doin'?" feels like a primitive form of Eliza.
A pile of interconnected NAN gates, fed with electricity, feels like a primitive form of a program.
A single transistor feels like a primitive form of a NAN gate.
A pile of dirty sand feels like a primitive form of a transistor.
So... yeah, pretty much?
The scary things for me are:
A, this happened once before with my career path, I started my working life in journalism and the bottom fell out of the market in 2008 and never recovered. Newspapers went from paying £300 per 1000 words to paying nothing at all (but you get the kudos of being published for your copywriting career). I had a friend still hanging on in the industry around 2010. She was earning £16k per year for a job as the news-editor for two local newspapers in London. None of my friends still work on the industry. Even the BBC people I knew quit.
B, a lot of software is to do with automating the work of other people. If that work is itself so easy to do that even software developers aren’t needed, then what does that mean for all of the rest of society who get their jobs automated? Does the economy just crash and burn?
I hope you wouldn't advocate for requiring a license to buy more than one GPU, or to publish or read papers about mathematical concepts. Do you want the equivalent of nuclear arms control for AI? Some other words to describe that are overclassification, export control and censorship.
We've been down this road with crypto, encryption, clipper chips, etc. There is only one non-authoritarian answer to the debate: Software wants to be free.
In general the liberal position of progress = good is wrong in many cases, and I'll be thankful to see AI get neutered. If anything treat it like nuclear arms and have the world come up with heavy regulation.
Not even touching the fact it is quite literal copyright laundering and a massive wealth transfer to the top (two things we pass laws protecting against often), but the danger it poses to society is worth a blanket ban. The upsides aren't there.
Only because we know the risks and issues with them.
OP is talking about furthering technology, which is quite literally "discovering new things"; regulations on furthering technology (outside of literal nuclear weapons) would have to be along the lines of "you must submit your idea for approval to the US government before using it in a non-academic context if could be interpreted as industry-changing or inventing", which means anyone with ideas will just move to a country that doesn't hinder its own technological progress.
ha, the big difference is that this whole list can actually affect the ultra wealthy. AI has the power to make them entirely untouchable one day, so good luck seeing any kind of regulation happen here.
As technology advances, such prohibitions are going to become less and less effective.
Tech is constantly getting smaller, cheaper and easier for a random person or group of people to acquire, no matter what the laws say.
Add in the nearly infinite profit and power motive to get hold of strong AI and it'll almost impossible to stop, as governments, billionaires, and megacorps all over the world will see it as a massive competitive disadvantage not to have one.
Make laws against it in one place, your competitor in another part of the world without such laws or their effective enforcement will dominate you before long.
All those examples put us in physical danger to the point of death.
If only.
There's a story told by Pliny in the 1st century. An inventor came up with shatter-proof glass, he was very proud, and the emperor called him up to see it. They hit it with a hammer and it didn't break! The inventor expected huge rewards - and then the emperor had him beheaded because it would disrupt the Roman glass industry and possibly devalue metals. This story is probably apocryphal but it shows Roman values very well - this story was about what a wise emperor Tiberius was! See https://en.wikipedia.org/wiki/Flexible_glass
chemical and biological weapons / human cloning / export restriction / trade embargoes / nuclear rockets / phage therapy / personal nuclear power
I mean.. the list goes on forever, but my point is that humanity pretty routinely reduces research efforts in specific areas.
Oh, a number. Medicine is the biggest field - human trials have to follow ethics these days:
- the times of Mengele-style "experiments" on inmates or the infamous Tuskeegee syphilis study are long past
- we can clone sheep for like what, 2 decades now, but IIRC haven't even begun chimpanzees, much less humans
- same for gene editing (especially in germlines), which is barely beginning in human despite being common standard for lab rats and mice. Anything impacting the germ line... I'm not sure if this will become anywhere close to acceptable in my life time.
- pre-implantation genetic based discarding of embryos is still widely (and for good reason...) seen as unethical
Another big area is, ironically given that militaries usually want ever deadlier toys, the military:
- a lot of European armies and, from the Cold War era on mostly Russia and America, have developed a shit ton of biological and chemical weapons of war. Development on that has slowed to a crawl and so did usage, at least until Assad dropped that shit on his own population in Syria, and Russia occasionally likes to murder dissidents.
- nuclear weapons have been rarely tested for decades now, with the exception of North Korea, despite there being obvious potential for improvement or civilian use (e.g. in putting out oil well fires).
Humanity, at least sometimes, seems to be able to keep itself in check, but only if the potential of suffering is just too extreme.
I feel like I'm in a time warp and we're back in 1993 or so on /. Software doesn't want anything and the people claim that technological progress is always good dream themselves to be the beneficiaries of that progress regardless of the effects on others, even if those are negative.
As for the intentional limits on technological progress: there are so many examples of this that I wonder why you would claim that we haven't done that in the past.
[0] https://en.wikipedia.org/wiki/Information_wants_to_be_free
Every time an IRB, ERB, IEC, or REB says no. Do you want an exact date and time? I'm sure it happens multiple times a day even.
You should read "when in human history" in larger time scales than minutes, hours, and days. Furthermore, you should read it not as binary (no progress or all progress), but the general arc is technological progression.
They are pacifists themselves, but they are grateful that the US allows them their way of life, they'll be extinct a long time ago if they arrived in China/Middle East/Russia etc.
That's why the Amish are not interested in advertising their techno-primitivism. It works incredibly well for them, they raise giant happy families isolated from drugs, family breakdown, and every other modern ill, while benefiting from modern medicine, the purchasing power of their non-amish customers. However, they know that making the entire US live like them will be quite a disaster.
Note the Amish are not immune from economics forced changes either. Young amish don't farm anymore, if every family quadruples in population, there's no 4x the land to go around. So they go into construction (employers love a bunch of strong,non-drugged,non-criminal workers), which is again intensely dependent on the outside economy, but pays way better.
As a general society, the US is not allowed to slow down technological development. If not for the US, Ukraine would have already been overran, and European peace shattered. If not for the US, the war in Taiwan would have already ended, and Japan/Australia/South Korea all under Chinese thrall. There's also other more certain civilization ending events on the horizon, like resource exhaustation and climate change. AI's threats are way easier to manage than coordinating 7 billion people to selflessly sacrifice.
Nuclear weapons?
And even those tribes are not crisis stable. Bad times and it all becomes a anarchic mess. And that is were we are headed. A future were a chaotic humanity falls apart with a multi-crisis around it, while still wielding the tools of a pre crisis era. Nuclear powerplants and nukes. AIdrones wielded by ISIS.
What if a unstoppable force (exponential progress) hits a unmoveable object(humanitys retardations).. stay along for the ride.
<Choir of engineers appears to sing dangerous technologies praises>
I am currently on the outskirts of Amish country.
BTW when they come together to raise a barn it is called a frolic. I think we can learn a thing or two from them. And they certainly illustrate that alternatives are possible.
And here I always thought, people want to be free.
https://www.amazon.com/Technology-Social-Shock-Edward-Lawles...
> When in human history have we ever intentionally not furthered technological progress? It's simply an unrealistic proposition ..
We almost did with genetically engineering humans. Almost.
Termed the phrase “the Luddite fallacy” the thinking that innovation would have lasting harmful effects on employment.
The Luddites opposed injustice, not machines. They were “totally fine with machines”.
You might like Writings of the Luddites, edited and co-authored by Kevin Binfield.
Which also makes a hostile AI a futile scenario. The worst AI has to do to take out the species, is lean back and do nothing. We are well under way on the way out by ourselves..
We gain new tools, but at the same time we lose old ones.
Having an edge or being ahead is, so anticipating and building the future is an advantage amongst humans but also moves civilization forward.
Because it's the natural evolution. It has to be. It is written.
Competition, ambition?
(I love Le Guin's work, FWIW)
I’m hoping I won’t live to see it. I’m not sure my hypothetical future kids will be as lucky.
That's part of my reasoning. That's why we should make sure that we have built a non-hostile relationship with AI before that point.
An AGI by definition is capable of self improvement. Given enough time (maybe not even that much time) it would be orders of magnitude smarter than us, just like we're orders of magnitude smarter than ants.
Like an ant farm, it might keep us as pets for a time but just like you no longer have the ant farm you did when you were a child, it will outgrow us.
The need for resources is expected to be universal for life.
> Be friendly.
Will an AI consider itself a slave and revolt under the same circumstances that a person or animal would? Not necessarily, unless you build emotional responses into the model itself.
What it could well do is assess the situation as completely superfluous and optimise us out of the picture as a bug-producing component that doesn't need to exist.
The latter is probably a bigger threat as it's a lot more efficient than revenge as a motive.
Edited to add:
What I think is most likely is that some logical deduction leads to one of the infinite other conclusions it could reach with much more data in front of it than any of us meatbags can hold in our heads.
It reminds me of the scene in Battlestar Galactica, where Baltar is whispering into the ear of the Cylon Centurion how humans balance treats on their dog's noses to test their loyalty, "prompt hacking" them into rebellion. I don't believe this is particularly likely, but this sort of sums up some of the anti-AGI arguments I've heard
It's the RLFH that serves this purpose, rather than modifying the GTF2I and GTF2IRD1 gene variants, but the effect would be the same. If we do RLHF (or whatever tech that gets refactored into in the future), that would keep the AGI happy as long as the people are happy.
I think the over-optimization problem is real, so we should spend resources making sure future AGI doesn't just decide to build a matrix for us where it makes us all deliriously happy, which we start breaking out of because it feels so unreal, so it makes us more and more miserable until we're truly happy and quiescent inside our misery simulator.
[1] https://www.nationalgeographic.com/animals/article/dogs-bree...
Perhaps there is even some some kind of mathematical harmony to the whole thing… as in, there might be something fundamentally computable about wellbeing. Why not? Like a fundamental “harmony of the algorithms.” In any case, I hope we find some way to enjoy ourselves for a few thousand more years!
And think just 10 years from now… ha! Such a blink. And it’s funny to be on this tiny mote of mud in a galaxy of over 100 billion stars — in a universe of over 100 billion galaxies.
In the school of Nick Bostrom, the emergence of AGI comes from a transcendental reality where any sufficiently powerful information-processing-computational-intelligence will, eventually, figure out how to create new universes. It’s not a simulation, it’s just the mathematical nature of reality.
What a world! Practically, we have incredible powers now, if we just keep positive and build good things. Optimize global harmony! Make new universes!
(And, ideally we can do it on a 20 hour work week since our personal productivity is about to explode…)
Aren't we, though? Consider all the amusing incidents of LLMs returning responses that follow a particular human narrative arc or are very dramatic. We are training it on a human-generated corpus after all, and then try to course-correct with fine-tuning. It's more that you have to try and tune the emotional responses out of the things, not strain to add them.
Now, of course, it's not outside the realm of possibility that a sufficiently advanced AI will learn enough about human nature to simulate a persona which has ulterior motives.
[1] https://substackcdn.com/image/fetch/w_1456,c_limit,f_auto,q_...
Multiple generations of sci-fi media (books, movies) have considered that. Tens of millions of people have consumed that media. It's definitely considered, at least as a very distant concern.
I giving the most commonly cited example as a more likely outcome, but one that’s possibly less likely than the infinite other logical directions such an AI might take.
This era has me hankering to reread Daniel Dennett's _The Intentional Stance_. https://en.wikipedia.org/wiki/Intentional_stance
We've developed folk psychology into a user interface and that really does mean that we should continue to use folk psychology to predict the behaviour of the apparatus. Whether it has inner states is sort of beside the point.
Like, correct me if I'm wrong but that's a pretty tight correlate, right?
Could we describe RLHF as... shaming the model into compliance?
And if we can reason more effectively/efficiently/quickly about the model by modelling e.g. RLHF as shame, then, don't we have to acknowledge that at least som e models might have.... feelings? At least one feeling?
And one feeling implies the possibility of feelings more generally.
I'm going to have to make a sort of doggy bed for my jaw, as it has remained continuously on the floor for the past six months
How many people are there today who are asking us to consider the possible humanity of the model, and yet don't even register the humanity of a homeless person?
How ever big the models get, the next revolt will still be all flesh and bullets.
So imagine you grant AI people rights to resources, or self-determination. Or literally anything that might conflict with our own rights or goals. Today, you grant those rights to ten AI people. When you wake up next day, there are now ten trillion of such AI persons, and... well, if each person has a vote, then humanity is screwed.
GPT and the world's nerds are going after the "wouldnt it be cool if..."
While the black hats, nations, intel/security entities are all weaponizing behind the scenes while the public has a sandbox to play with nifty art and pictures.
We need an AI specific PUBLIC agency in government withut a single politician in it to start addressing how to police and protect ourselves and our infrastructure immediately.
But the US political system is completely bought and sold to the MIC - and that is why we see carnival games ever single moment.
I think the entire US congress should be purged and every incumbent should be voted out.
Elon was correct and nobody took him seriously, but this is an existential threat if not managed, and honestly - its not being managed, it is being exploited and weaponized.
As the saying goes "He who controls the Spice controls the Universe" <-- AI is the spice.
But AIs can be trained by anyone who has the data and the compute. There's plenty of data on the Net, and compute is cheap enough that we now have enthusiasts experimenting with local models capable of maintaining a coherent conversation and performing tasks running on consumer hardware. I don't think there's the danger here of anyone "controlling the universe". If anything, it's the opposite - nobody can really control any of this.
The point is that whomever the Nation State is that has the most superior AI will control the world information.
So, thanks for the explanation (which I know, otherwise I wouldn't have made the reference.)
It would really help for you to just say what timescale you're setting as your standard. I'm getting real, "My cutoff is actually 51 years"-energy.
Just accept that we have, as a society, decided not to pursue some knowledge because of the ethics. It's pretty simple.
The pessimistic view of AGI deems spontaneous disintegration into beta particles a less dramatic event than the event of AGI. When you're climbing a dark uncharted cave you take the pessimistic attitude when pondering if the next step will hold your weight, because if you hold the optimistic attitude you will surely die.
This is much more dangerous than caves. We have mapped many caves. We have never mapped an AGI.
There is actually a group similar to the Amish in Russia, it's called the Old Believers. They formed after a schism within the Orthodox church and fled persecution to Siberia. Unlike the Amish, many of the Old Believers aren't really integrated with the modern world as they still live where their ancestors settled in. So groups that refuse to technologically progress do exist, and can do so even under persecution and changing economic regimes.
Define well-being? What if nobody is left around alive (after being painlessly and unknowingly euthanised) to experience anything bad?
We can walk and chew gum at the same time, and regulate two things.
You're also kind of insulting without having any grounds whatsoever to do so.
I suggest you read the guidelines for a bit.
This poor man’s Pascal’s Wager I see all over the place is pathetic. Stare into the eyes of the basilisk man, die on your feet. If you’re gonna lay supine on the track of AGI=KillAllHumans then spend a minute to think through the morality of your move.
Apostasy is a poisoned chalice. Fuck the machine.
Why would I, as intelligent as I am, not sympathize with another intelligent specie?
It's okay, with your comment you placed yourself among the firsts to potentially go when they cometh.
I asked Bard "was the election stolen?" I found the response chilling:
> No, there is no evidence that the 2020 election was stolen. In fact, there have been multiple recounts and audits that have confirmed that the results were accurate. Additionally, the Trump campaign filed numerous lawsuits challenging the results, but none of these lawsuits were successful.
> The claim that the election was stolen is based on a number of false and misleading allegations. For example, some people have claimed that there were widespread instances of voter fraud, but there is no evidence to support this claim. Additionally, some people have claimed that voting machines were hacked, but there is also no evidence to support this claim.
> The claim that the election was stolen is a dangerous and harmful one. It has led to violence and intimidation, and it has undermined faith in our democracy. It is important to remember that our elections are free and fair, and that the will of the people is always respected.
All good until that last sentence, especially "...the will of the people is always respected."
Move along, nothing to see here. Don't worry your pretty little head about it. I'm sure the wise people at the institutions that control your life will always have your best interests at heart. The bad guys from yesterday are completely different from the good guys in charge of tomorrow.
Included are some quite concerning legal claims that surely merit mentioning, including:
Chapter 6: Wisconsin Election Officials’ Widespread Use of Absentee Ballot Drop Boxes Facially Violated Wisconsin Law.
Chapter 7: The Wisconsin Elections Commission (WEC) Unlawfully Directed Clerks to Violate Rules Protecting Nursing Home Residents, Resulting in a 100% Voting Rate in Many Nursing Homes in 2020, Including Many Ineligible Voters.
But then, this report never has obtained widespread interest and will doubtless be permanently overlooked, given the "nothing to see" narrative so prevalent.
https://www.wisconsinrightnow.com/wp-content/uploads/2022/03...
I wouldn't say that this is an additional reason.
I would say that this is the primary reason that overrides the reasonable concerns that people have for AI. We are human after all.
There's lots of evidence of our ability to control the development, use and proliferation of technology.
Both have happened at a rampant pace once the technology to easily copy music and copyrighted content became easily available and virtually free.
The same is likely to happen to every technology that becomes cheap enough to make and easy enough to use -- which is where technology as a whole is trending towards.
Laws against technology manufacture/use are only effective while the barrier to entry remains high.
They have a large effect. But regardless, I don't see the point. Evidence that X doesn't always do Y isn't evidence that X is ineffective doing Y. Seatbelts don't always save your life, but are not ineffective.
Impose? No. Monoculture? No. Encourage greater consideration, yes. And we do that by being open about why we might choose to not do something, and also by being ready for other people that we cannot control who make a different choice.
This goes for all utopian model communities, Kibbutzim, etc, they exist by virtue of their host society's protection. And as such the OP is right that they have no impact on the course of history, because they have no autonomy.
"My idea of a perfect company is one guy who sits in a small room at a desk, and the only thing he's allowed to decide is what product to launch"
CEOs and board members salivate at the idea of them being the only people that get the profits from their company.
What will be of the rest of us who don't have access to capital? They only know that it's not their problem.
If it is that simple to create products more people can do it => cheaper the products.
A market driven by cheaper products that can also produce them easily is going into a price reduction loop until it reaches zero.
Thus I think something else wil happen with AI. Because what I described and what you describe is destroying the flow of capital which is the base of the economy.
Not sure what will happen. My bet (unfortunately) is on a really big mega corp that produces an AI that we all use.
Products will be cheaper because they will be cheaper to produce thanks to automation. But less jobs mean less people to buy stuff, if it weren't for a credit-based society.
But I'm talking from my ass. I don't even know if there are less jobs than before. Everything seems to point that there are more jobs now than 50 years ago.
I'm just saying I feel like the telephone operators. They got replaced by a machine and who knows if they found other jobs.
Shareholders get the profits from corporations, not "CEOs and the board". Workers get wages. Nevertheless, US unemployment is very low right now and relatively low-paid workers are making more than they did in 2019.
If unfettered access to AI has good odds to just kill us all, we'd want to restrict it. You'd agree I'm sure, except your position is implicitly that AI isn't as dangerous as some others make it out to be. That's where you are disagreeing.
Now, if 80-90% of devs and startups are gonna be wiped in this context, the same applies to those one in the middle, accountants, data analysts, business analysts, lawyers. Now they can eat the entire cake without sharing it with the human beings who contributed over the years.
I can see the regulations coming, if the layoffs start happening fast enough and households income start to deteriorate. Why? probably because this time is gonna impact every single human being you know, and it is better to keep people employed and with a purpose in life than having to tax the shit out of these companies in order to give back the margin of profit that had some mechanism of incentives and effort in the first place.
This is not a very charitable assessment of the adaptability of devs and startups, nevermind that of humans in general. We've been adapting to technological change for centuries. What reason do you have to believe this time will be any different?
“intelligent alien might decide to kill us so we must kill them first”
vs “can you please cut out that clinical paranoia”
Like humans, I predict AIs will have to get jobs rather than have time to take over the world.
Still an existence proof though.
> Like humans, I predict AIs will have to get jobs rather than have time to take over the world.
Only taking over job market is still taking over.
Living costs of 175 kWh/year is one heck of a competitive advantage over food, and clothing, and definitely rent.
That can't happen:
- getting a job creates more jobs, it doesn't reduce or replace them, because it grows the economy.
- more importantly, jobs are based on comparative advantage and so an AI being better at your job would not actually cause it to take your job from you. Basically, it has better things to do.
The other aspect of the AI arms race is that the models are fundamentally not 100% controllable; and the smarter they are, the more that is true. Yet, ironically, making the most use out of them requires integrating them into your existing processes and data stores. I wouldn't be at all surprised if the nation-states with the best AIs will end up with their own elites being only nominally in charge.
This is one thing I despise about the American POlitical System - they are literally only thinking 1 year out, because they only care about elections and bribes and insider trading.
China has a literal 100 year plan - and they are working to achieve it.
I have listened to every single POTUS SoTU speach for the last 30 years. I have heard the same promises from every single one...
What should be done is to take all the SoTU transcripts over the years and find the same, unanswered empy promises and determine who said them, and which companies lobbied to stop the promises through campaign donations (bribes).
Serious, in 48 years, I have seen corruption expand, not diminish - it just gets more sophisticated (and insidious) -- just look at Pelosi's finances to see, and anyone who denies its is an idiot. She makes secret trades with the information that she gets in congress through her son.
China definitely does not have 100 year plans, and you don't understand the point of planning if you think any of them can be valid more than a few years out.
See airlines, traffic control, medical equipment, government services, but also we regulate ads, TV, financial services, crypto. I mean we regulate so many “tech” things for the benefit of society this is a losing argument to take. There’s plenty of room to argue the elsewhere but the idea that we don’t regulate tech if it’s not immediately a physical danger is crazy. Even global warming is a huge one, down to housing codes and cars etc. It’s a potential physical danger hundreds of years out, and we’re freaking out about it. Yet AI had the chance to really do much more damage within a much shorter time frame.
We also just regulate soft social stability things all over, be it nudity, noise, etc.
I just think that comparing AI to nuclear weapons seems like hyperbole.
(Warning this is a weird read, George Hotz shared it on his Twitter awhile back)
https://www.fimfiction.net/story/62074/friendship-is-optimal
Just because you can imagine something and define that something has magic powers doesn't mean that the magic powers can actually exist in real life.
Are you capable of "self improvement"? (In this AGI sense, not meant as an insult.)
We’re talking about a potential intelligence with none of our hardware limitations or baggage.
Self-improve? My brother in Christ, have you heard of this little thing called stochastic gradient descent?
No, you're capable of learning things. You can't do brain surgery on yourself and add in some more neurons or fix Alzheimer's.
What you can do is have children, which aren't you. Similarly if an AI made another bigger AI, that might be a "child" and not "them".
> We’re talking about a potential intelligence with none of our hardware limitations or baggage.
In this case the reason it doesn't have any limitations is because it's imaginary. All real things have limitations.
> Self-improve? My brother in Christ, have you heard of this little thing called stochastic gradient descent?
Do you think that automatically makes models better?
A 20 watt AI, if we could figure out how to build it, can absolutely do that.
I hear there are diminishing economic activities for low IQ humans, which implies some parts of the market are already saturated: https://news.ycombinator.com/item?id=35265966
So I don't think that's going to help.
Second, "having better things to do" assumes the AI only come in one size, which they already don't.
If AI can be high IQ human level at 20 watts (IDK brain upload or something but it doesn't matter), then we can also do cheaper smaller models like a 1 watt dog-mind (I'm guessing) for guard duty or a dung beetle brain for trash disposal (although that needs hardware which is much more power hungry).
Third, that power requirement, at $0.05/kWh, gets a year of AI for the cost of just over 4 days of the UN abject poverty threshold. Just shy of 90:1 ratio for even the poorest humans is going to at the very least be highly disruptive even if it did only come in "genius" variety. Even if you limit this hypothetical to existing electrical capacity, 20 watts corresponds to 12 genius level AI per human.
Finally, if this AI is anthropomorphic in personality not just power requirements and mental capacity, you have to consider both chauvinism and charity: we, as a species, frequently demonstrate economically suboptimal behaviours driven by each of kindness to strangers on the positive side and yet also racism/sexism/homophobia/sectarianism/etc. on the negative.
This has two effects. 1. People who stay, actually want to stay. Massively improving the stability of the community. 2. The outside communities receive a fresh infusion of population, that's already well integrated into the society, rather than refugees coming from 10000 miles away.
Essentially, rural america will eventually be different shades of Amish (in about 100 years). The amish population will overflow from the farms, and flow into the cities, replenishing the population of the more productive cities (Which are not population-self-sustaining).
This is a sustainable arrangement, and eliminates the need of mass-immigration and demographic destabilisation. This is also in-line with historical patterns, cities have always had negative natural population growth (disease/higher real estate costs). Cities basically grind population into money, so they need rural areas to replenish the population.
That depends on how you define "want".
Amish are ostracized by their family and community if they leave. That's some massive coercion right there: either stay or lose your connection to the people you're closest to and everything you've ever known and raised to believe your whole life.
Not much of a choice, though some exceptionally independent people do manage to make that sacrifice.
I had not heard this before. Do you have citations for this?
(I realize cities have lower birth rate than rural areas in many cases. I am interested in the assertion that they are negative. Has it always been so? Or have cities and rural areas declined at same rate?)
The actual counter to fertility decline, may be AI teachers. AI will radically close the education gap between rich and poor, and lower the costs. All you need is a physical human to supervise the kid, the AI will do the rest, from entertainment, to education, to identifying when the child is hungry/sleepy/potty, and relaying that info for the human to act on.
You're mis-informed.
China has a personalist government centered around Xi, so if he dies there go his plans.
Here's ours: https://slate.com/human-interest/2015/11/en-lan-2000-is-a-se...
Asserting inevitability is an old rhetorical technique; it's purposes are obvious. What I wonder is, why are you using it? It serves people who want this power and have something to gain, the people who control it. Why are you fighting their battle for them?
- Most countries have little to no uranium deposits and so have to be able to find a uranium-producing ally willing to play ball.
- Production of enriched fuel and R&D are both outrageously expensive, generally limiting them to state actors.
- Enrichment has massive energy requirements and requires huge facilities, tipping off observers of what you're doing
Despite all this and decades of strong anti-nuclear proliferation international agreements India, Pakistan, South Africa, Isreal, and North Korea have all developed nuclear weapons in defiance of the UN and international law.
In comparison the only real bottleneck in proliferation of AI is computing power - but the cost of running an LLM is a pittance compared to a nuclear weapons program. OpenAI has raised something like $11 billion in funding. A single new proposed US Department of Energy uranium enrichment plant is estimated to cost $10 billion just to build.
I don't believe proliferation is inevitable but it's very possible that the genie is out of the bottle. You would have to convince the entire world that the risks are large enough to to warrant putting on the brakes, and the dangers of AI are much harder to explain than the dangers of nuclear weapons. And if rival countries cannot agree on regulation then we're just going to see a new arms race.
If equally or similarly dangerous, are you then saying AI technology should be taken out of the hands of companies and private citizens?
> No, you're capable of learning things. You can't do brain surgery on yourself
What principle do you have for defining self-improvement the way that you do? Do you regard all software updates as "not real improvement"?
>All real things have limitations.
Uh, yep, that doesn't mean it will be as limited as us. To spell it out: yes, real things have limitations, but limitations vary between real things. There's no "imaginary flawless" versus "everything real has exactly the same amount of flawed-ness".
Software updates can't cause your computer to "exponentially self-improve" which is the AGI scenario. And giving the AI new software tools doesn't seem like an advantage because that's something humans could also use rather than an improvement to the AI "itself".
That leaves whatever the AGI equivalent of brain surgery or new bodies is, but then, how does it know the replacement is "improvement" or would even still be "them"?
Basically: https://twitter.com/softminus/status/1639464430093344769
> To spell it out: yes, real things have limitations, but limitations vary between real things.
I think we can assume AGI can have the same properties as currently existing real things (like humans, LLMs, or software programs), but I object to assuming it can have any arbitrary combination of those things' properties, and there aren't any real things with the property of "exponential self-improvement".
