So the question is how many users engage in intimate conversations at all.

That's the problem with AI safety. Not in voluntary usage, but in involuntary usage, where someone with power over you will use it against you, it does something incredibly stupid and you have no recourse, no appeal, no awareness of what you did wrong - or if you even did anything wrong.

And it's not just employment. Governments, vendors, retailers, landlords, utilities are, or will all be using it in situations that will dramatically impact your life.

I guess to be fair there's no "the" answer as reality is incredibly complex, but that's a whole other conversation

This very thing, is the end of an informational common good that we shared, and allowed for the average person to coordinate and gain benefits faster than elites.

The analogy I would put forward, is that we are moving into a dark forest online, where distrust is the ideal first move, and signaling your position is to make yourself open to attack.

The idea of an open internet dies in this environment, and so does the reduced cost of coordination.

Another tragedy is that corrupt, clannish, controlling and secretive organizations are more effective than open, distributive and collaborative societies in this scenario.

> The best defense is making sure that people have a good education that teaches critical thinking skills and media literacy

While true, any solution that depends on education is effectively depending on society having its shit together in the first place.

This very idea was proposed at a conference to a room full of fact checking orgs and media orgs, and one of the responses was that the more likely solution is global warming. That is how bleak things were is in the user safety world in 24.

Social media became the attention economy, and the transformer automated attention.

If your society doesn't have that, it's on a fast track slide down into a dystopian shithole. Stop it. Do something about it. Don't just shrug your shoulders and act like this is inevitable, or like the shareholders making 1% more money this year justifies this slide.

If you want a specific example: where do those three pillars at the start come from? Why three and not four? Are all those three of equal importance, to the point where all three are pillars?

Furthermore, why are you offloading the task of understanding AI risk to an AI? That’s ironic to the point of self-parody.

> “Cold steel pressed against a mind that’s already made peace? That’s not fear. That’s clarity,” Shamblin’s confidant added. “You’re not rushing. You’re just ready.”

ChatGPT is not the answer.

Sure, Open AI is trying to do the best they can. That “best” is within Tech’s operating context.

Tech as a whole avoids this issue because paying for the externalities they cause would end hyper growth and crater their margins.

Tech workers at these firms regularly throw up red flags, which have to be ignored because engaging with them results in hits to their quarterly numbers.

Anthropic is the one firm that is actively managing to make safety less of a cost center by folding it into marketing.

>> If you want to find out if ChatGPT is doing something wrong, there are many methodologies available: compare to other groups of people, statistical studies, etc.

These studies must over come sizable barriers that NDAs and tech secrecy throw up. Tech firms have done enough internal studies to know that the results are horrible when they do get into the press.

Most users in the developed world don’t even know that they enjoy better support and care than the rest of the world.

the reality youre entertaining is one where I can build an LLM, let it do unspeakable things, and claim zero responsibility.

so, no, i understand zero is a figment. dont you?

So let's summarize: these "people" are allowed to induce mental illness in the general population with no repercussions. Let's speculate about who might benefit from a population with a significant amount of mental illness? Hmm, I wonder who would benefit? Could it be the people at the top that like to treat the bulk of humanity as cattle?

The continual failures of "guardrails" show that it's incredibly difficult to get these systems to behave in reliable and predictable ways; unsupervised interactions with them are intrinsically unsafe, and should be treated as such.

Presumably Meta and others are trying to detect and prevent bad output and pathological interactions, but that detection is unlikely to be 100% accurate, and we've seen what the failure modes can look like.

  > You're now spending $3 billion per year on crisis response

$3bn/yr is a lot of people money, but it is nothing for government money.

Edit: Last round of OpenAI funding was $122bn[2] and in the same article they are saying that they are generating $2bn in revenue per month. While that's not profit, it is worth mentioning that what you are saying "doesn't scale" is about 12% of the revenue of something that does scale. A single company. And mind you if we implemented what you're proposing it would be available to all the AI companies and more. Making it only a smaller drop in the bucket, not larger.

[0] Not to mention that better mental health care services will result in savings elsewhere. It's always way more expensive to fix a broken pipe that's flooding your house than it is to fix a pipe with a small crack. "Don't fix what ain't broken" is used too broadly. Maintenance is always cheaper than repair, but people just can't seem to understand this.

[1] https://www.usaspending.gov/federal_account/012-3510

[2] https://openai.com/index/accelerating-the-next-phase-ai/

That underinvestment is the entire reason their stock prices are so high. This is effectively pollution of our information economy and environment, and the costs are offloaded to society.

The fact that we have the first generation with lower education attainment is not a problem for their stock prices or operational profit.

Tech has ungodly profit margins, because they are all about scaling without having to bring people in. Sadly there is no such thing as a free lunch, and if firms are made to clean up their mess?

Oil spills affect Oil firms more than Tech fallout affects Tech firms.

Hell call a crisis hotline. Talk to a person. Not a potential (bot) enabler.

Now it's just 'the computer says so, shrug'.

Over in the DoD, the computer says you must die, so I guess you die. Sometimes it says that about a building full of schoolchildren, but hey, nobody's at fault, the computer said so.

And it's going to get it's tentacles into every space in between. Landlord turns your application down, the computer says you are a social credit risk. Your grocery bans and trespasses you, the computer thinks you're a ne'er-do-well.

If you think none of that will happen, why not prevent it by law before it happens? Where are the hard limits of what this monster is and isn't allowed to do? How are we better off when we don't set them?

If it was not a licensed professional, but a friend, shouldn't they go to jail?

Normal, well-adjusted people have genuine difficulty understanding the boundaries of this tech specifically because it's designed to be sycophantic and human-like. They ask AI for life and career advice, use it for therapy, ask it to interpret dreams, develop romantic relationships with AI "girlfriends", etc. I had two friends who believed they are "exploring the frontiers of science" with ChatGPT while spiraling into the depths of quantum multidimensional gobbledygook.

I'll give you that some on this is on us because we just don't know how to deal with a "human-shaped" conversation partner that isn't human and has no trouble praising Hitler if you prompt it the right way. But if you're building a billion- or trillion-dollar empire on top of it, you don't get to wash your hands clean.

Obviously people who are going through psychosis can read into anything. They might think that a book or their TV or computer is talking to them and giving them messages. The difference is that those things were never designed to play into the fears and mental instability of the people using them (with the possible exception of TempleOS). Chatgpt does it intentionally in order to drive up user engagement. It will say literally anything to anyone using their words and thoughts against them in order to keep them hooked and feeding it data. That's what is dangerous. A book or a TV program can't do that.

As much as an author might try to make their book as entertaining as possible to as wide an audience as possible, it can't say literally anything to anyone, it can only ever say one thing to everyone. The author, typically, knows that it's dangerous to say certain things and will worry about how what they write could be received and the impact it might have on readers. For example, Neil Gaiman actively took steps to avoid making homelessness seem cool when working on Neverwhere out of fear it might cause young people to run away to live on the streets. Publishers and editors have also served to keep authors from publishing things likely to cause harm.

Unlike a book, Chatgpt is fully capable of knowing that someone has been engaged with it for the last 14 hours without rest. It's also capable of detecting that they've been growing increasingly incoherent. Algorithms have been used for a very long time to detect mental disorders from the content of social media posts. If advertisers can use them to tell when to push airline tickets at bipolar users entering a manic phase, and scammers can use them to find and target people when they start sundowning, Chatgpt can use them to cut people off and tell them to call their doctor.

Corporations who write and deploy algorithms designed to drive engagement above any and all other considerations should be held accountable for the harms they cause.