OpenAI’s stated rationale was a concern for the shooter’s privacy, but its own interests
    better explain its silence. Upon information and belief, OpenAI was seeking to avoid implementation of a
    hard line rule to refer planning of real-world violence to authorities, perhaps due to how frequently its
    product is implicated in threats to human life. Requiring such disclosures would be incompatible with the
    company’s public position that ChatGPT is safe. It could also threaten the valuation underlying OpenAI’s
    anticipated initial public offering. Rather than expose those risks, OpenAI accepted the consequences of
    its silence. A mass murder in the only secondary school in Tumbler Ridge followed.

    Accordingly, the Crown, led by Attorney General Sharma, and SD59 jointly bring this
    action to hold OpenAI and Sam Altman accountable for designing a dangerous product, distributing it to
    every home with internet access, ignoring the warnings of their own safety team,
    refusing to notify authorities when they knew the shooter was planning gun violence, inviting the shooter back onto the
    platform after deactivating the shooter’s account, and choosing corporate self-interest over the lives of
    children. They seek compensation for the not just foreseeable but known harm OpenAI inflicted, the
    damages that they incurred and are incurring, and injunctive relief to ensure that this tragedy does not
    happen again.

Then the AI companies have more confidence that they can move forward in a certain way, and issue investor guidance that is maybe closer to reality.

As another commenter said, you're not chatting with a friend; you're chatting with Big Tech.

How much do you love Big Brother?

Yup. And with zero privacy protections in statute for AI chat, there is nothing to prevent an AI CEO looking to curry political favour from e.g. handing over the private correspondence of an opponent or an entire district’s residents.

Maybe there already is! But last I checked it was a little bit of a mess of manually installing things from multiple websites with little documentation.

A bot talking to you directly as if its some one real caters to your thoughts and can take you in a certain direction without you realizing it. I have heard first hand experience from people that they feel more comfortable talking to chatgpt or claude cause it gives a feeling of being on their side and listening to them.

I see constant ads on video platform (particularly youtube/tiktok) about llm chat apps, from friends, dating, romance and everythkng inbetween; that's personal.

People need to be reminded constantly if they use such apps that they are participating in easier mass surveillance, profiling and AI training.

Considering that many people will share their deepest thought with LLM, it might start looking more and more like attempts of precog agents from Minority Report...

It's the other way around, big techs need to properly disclose in their platform, during interaction that they aren't in a private and safe environment

Including any chats anywhere where someone might have a phone in their pocket, or if there's a "camera" attached to a utility pole or a nearby tree. The only real private chats might be whispered lying down in the bathtub together, with a mattress covering it like you're both hiding from a hurricane.

> they're chatting with Big Tech

They're chatting with any powerful person who wants to hear it. She thought she was chatting with Anthropic, who doesn't give a shit about her. But after being threatened (and immediately backing down because, of course, they don't give a shit about her) Anthropic has become an arm of the government. So she was chatting with the Bonita Springs, FL Sheriff's office, or anybody else. If I paid enough, Anthropic would tell me about what she was doing so I could sell her laundry detergent.

"Anthropic failed to report murderer's threats to authorities"

(or "Chatbot knew man was planning murder, yet company did nothing")

"Anthropic reported private chats to authorities"

(or "Arrested for chatbot fantasy")

To be fair to the journalists in these cases, there's also no society-wide agreed Schelling point about the correct outcome or correct rules. I have strong beliefs and intuitions about what should happen, but other people also have strong beliefs and intuitions, and many of those are probably opposite of mine. Even if my intuitions are the best and most justified, a journalist is unlikely to think "I'm just not going to mention that some people are mad at this company over this outcome, because a hypothetically better norm or principle would support the company's actions here". Hopefully the journalism can at least contextualize the lack of legal or social consensus and the difficult incentive problems, rather than jumping to "obviously companies are sociopaths staffed by supervillains".

If a person told their real BFF they were planning a shooting and went through with it, couldn’t the BFF be charged with accessory?

The problem with that is, "no, they don't have to do their thing. They have no given right to do it. If they do it, they should, they have an ethical duty to, do it right..."

Somehow humanity survived the past 40 years without Microsoft Word and Excel phoning home and shopping users to the feds at random, I don't see why the standard should be any different for this new class of tooling.

They did and they do

https://www.bbc.co.uk/news/technology-44699263

I never considered online services safe. What I object to is active scanning of content on devices. This should not be allowed.

Google doesn't report people to the police for the private (non-CSAM) contents of searches or emails.

You could definitely go through people's shit before, but Big Tech generally had serious prohibitions and a stronger presumption of privacy about this sort of thing.

CSAM was an exception where they theoretically had the evidence of the actual crime, rather than writings alluding to them. And even that has been problematic.

Frankly, the charge here is total bullshit and should be thrown out. The law is meant to prevent people sending threats to others, not keeping notes. If this charge sticks, the law should be changed.

I feel like if people understood what these things actually are there'd be way less of this AI psychosis and similar stuff.

There'd also be fewer people falling for apocalyptic Rationalist delusions.

Also: people need to understand "not your computer, not your data." (Unless it's stored in the cloud but encrypted locally with keys only you possess.) Same goes for storing things unencrypted in OneDrive, Google Drive, etc. There is nothing to stop these companies from bulk scanning, data mining, or reporting people based on whatever request a government gives them. Don't count on them to resist, because they often can't, especially if the request is from a sovereign state where they do business.

Adding: - I typically ask questions in the I form, regardless for whom or why I ask for. - Gemini chats quite often end when it starts recommending psychological council or a suicide line, to talk about my problems. It apparently detects a persistent tendency to not agree with the party line. So it makes sense I must be suicidal ;-

But sure, as llm's start to babysit us, and know our inner dialog better than anyone else, we'll soon be debugging their opinion/behavior/co-existence/authority, when it comes to reporting people to the authorities, or taking on tasks in society in general. We'll hire doctors to cure our psychological profile from our record (Total Recall).

A Minority Report like this shouldn't cause a referral to the police.

For the sake of even more argument, imagine if she was writing her thoughts with a pencil, on a good old fashioned paper diary, and she had a phone nearby and the phone took a picture of her diary, OCR'd the words, and reported it to the police?

Edit: I thought it was figurative language, but I actually think it did that warning when I asked it about what types of defenses stadiums had against drone swarm attacks from terrorists and why none had ever occurred and then kept probing it's excuses with technical workarounds. It got very upset and said even questioning in an intellectual/academic capacity was grounds for terrorism charges. Sheesh. So many rules these days about what one can or can't think about even when it's purley a thought exercise and there is no intent to do anything.

> If they were to offer total privacy, is it ok for the public to use chat to get advice on _how_ to commit a crime?

Yes it should be, but it should be illegal for a company providing chat service to respond with anything other than a refusal when doing so.

That detection and refusal should be a private closed loop though, anonymizing any data that will be passed into a training pipeline, or ads targeting. This requirement for closed loop private chats should be mandated by law sooner than later. Otherwise we're getting into very tricky territory where the temptation of alerting on things like pre-crime grows too close.

Today I learned about "foldering"[1], which uses the drafts folder of email systems as a "dead drop" between multiple participants. It goes back to at least 2005.

Back when I was in IT, circa 2010, I learned that several of the users of our systems routinely relied on the "deleted items" folder of Microsoft Outlook as a filing system, with multiple gigabytes important files stored there. 8(

Our customs are routinely ignored by everyone else.

[1] https://en.wikipedia.org/wiki/Foldering

Selling analogous profiles of companies using their services to the highest bidder would be another idea.

Whether intentionally or not, they market their product as suitable for all kinds of things that it clearly isn't when input is being used that way.

And she's facing felony charges. Yeah, folks on hacker news can talk about technical solutions all day long, but that's not the fundamental problem. The fundamental problem is that we've allowed our digital infrastructure that includes everything from banks to schools to doctors to become adversarial to us. In some cases predatory.

It's effectively untrustworthy, like being charged a felony for walking across a bridge incorrectly. That's not a "I'll host a small model in my garage" problem. It's a we need the government to do its job problem.

No this is not true — colos are very hands off.

You don't need to operate a mini AWS to self-host, and I don't know why people on HN want to make it sound like self-hosting if you don't have nation state security

A PC at home with a 3090 in it is more than enough if you want to talk to a chatbot about your day.

Wow. I never knew that my Frankenrouter, gaming PC, and handful of laptops, switches, and WiFi APs were a datacenter. The things you learn...

> This lady was just using it as a personal journal. ... That kind of thing can be done with really small models locally these days.

That kind of thing can be done with MS-DOS's 'EDIT.COM'. If you're nasty, it can be done with 'ed', The Standard UNIX Text Editor.

That dynamic will fuel further borrowing until we get to some kind of equilibrium, or some kind of washout occurs where interest rates spike higher and/or the value of GPUs and GPU services start leveling off or even declining. We might get a test soon as the FED has started hiking.

I understand moral panic, disgust, etc, but rationally speaking.

It's always been complicated like this. That's why certain professions (psych, lawyer, clergy) come with rules around when and if disclosure is allowed[ required, and/or admissible].

This quote is pretty old:

> Those who would give up essential liberty, to purchase a little temporary safety, deserve neither liberty nor safety.

> its almost better to be safe than to have to explain to someone's loved ones you had to chance to prevent this and did nothing.

An authoritarian government isn't safe. That's why safety is also not deserved when you go chasing for a little of it at the cost of essential freedoms.

As a Brit, I'm aware of https://en.wikipedia.org/wiki/Twitter_joke_trial

I can't say I actually disagree with the initial prosecution. The penalty was a fine, likely less than the cost of investigating it.

Intended as a joke? Blowing off steam? I can understand that, but given the number of people on social media is large enough to include genuinely unhinged people, you can't expect anyone who receives such as message to take them as a joke.

Same with AI use. A billion users, you have to assume some of them are actually sincere if they write about any act of violence, from self-harm to a plan to steal a nuke and use it in a false-flag attack to trigger WW3 and everything between.

So this conundrum is at least partly of their own making.

While the privacy around ai chatbots is rotten in general, I can’t fault anyone who reported this.

The appropriate question is whether I want to live in Florida. This is much more a Florida law problem than an AI company problem.

It would apply if you happened to use Office 365 to write your diary.

Would you use a lawyer knowing they are inclined to turn you into the police if you talk about committing a crime in the future?

The law is the law.

We should be happy about this as for once the AI companies did the right thing.

But that service is a process on MY computer, loaded into RAM, with an abliterated model.

I don't trust 3rd party networks from abusing any data I give them.

So don't run for office or anything like that. Someone, somewhere will have a contact that will get that.

Obviously, it's hard to judge exactly what was appropriate there because we're being asked to extrapolate from a two word quote about the customer intending to "shoot up" the sheriff's office. Consider the following two statements, which express quite different levels of intentionality.

I got a $200 ticket from a sheriff's deputy today for throwing away an apple core. I'm so mad. I'd like to shoot up their office!

Those sheriff's deputies have exhausted my last reservoir of patience. I'm going to shoot up the department. They'll be sorry when they're sprawled all over the floor bleeding out from saucer-sized shotgun slug wounds. I can't wait to hear the screaming and crying of their miserable families!!"

I'm guessing that the diary entry was a more casual expression similar to the first statement, or they police would have quoted more of the statement to emphasize the apparent severity of the risk but it's hard to say without reading the charging documents.

This is spying with extra steps couched in corporate speak.

Then, you can write anything in an EULA but it is not automatically legal either.

According to Gemini, "Florida appellate courts have overturned juvenile convictions [based on this law] when the state could not prove the person subjectively intended for the record to be seen."

Hopefully more of these stories push people towards local models :)

What if she put it in a locked box before shipping it to herself UPS, and she has the only key?

What if instead of UPS, she hired a moving company to move the locked box?

What if she wrote it electronically in diary.txt, but it was backed up to a cloud provider?

--

I'm guessing there's some sort of "reasonable expectation of privacy" for certain activities. We're going to find out what Florida courts think about this new medium.

Saving is not sending ie passive vs active act.

Does the person have to know (or at least believe) that it will be viewed by another person?

She likely didn't think anyone would view it. Honestly, even as a career software developer I don't think it is unreasonable to think know would would see what she wrote to an AI. I assume most of what I write to an AI is not viewed by any other human, based simply on the quantity of messages sent back and forth to AIs, I would assume a vast majority are not read by another human.

What if she had written this into google docs, and she kept a diary there? That also crosses state lines, and is transmitted to another location.

You can argue from technicalities but they would need to prove intent.

You would expect them to enforce every possible legal standard, in every jurisdiction.

Why stop with criminal law? It should flag torts too, right?

Absurd.

Similar thing is that I believe if motorcycles were invented today they would most certainly be banned from the road due to their safety properties.

Google also monitors your searches, it’s to be expected that an AI lab will know all your prompts, they aren’t providing a paid service for free out the good of their heart. lol.

But I can't help but wonder if it isn't some socially apparent phenomenon stemming from Simpson's Reversal [1] as applied to group sentiment analysis / polling of opinions.

[0] https://knowyourmeme.com/memes/the-goomba-fallacy [1] https://en.wikipedia.org/wiki/Simpson's_paradox

> Review is needed to enforce our Usage Policy... designated members of our Trust & Safety team may access this data on a need-to-know basis as a part of their evaluation process.

[0]: https://privacy.claude.com/en/articles/10458704-how-does-ant...

[1]: https://www.anthropic.com/legal/aup

> The communication must be made in a manner in which another person may view it.

Even 'transmitted' is too broad if you also consider iCloud backup to be a means.

Reporting the danger is by itself a good deed. But there should be a better way of restricting firearms from the probably irresponsible lady than using inappropriate charges to punish the thoughtcrime, OR waiting for them to commit violence.

IMO I do not think this is a grey area and it's legal to tell a LLM you want to kill someone. It's certainly not a "threat" like you might send to another person, though it may end up being evidence of conspiracy or premeditation. I suspect we would be well served to, after a few years of experience, put together some laws governing when LLM chats must be made available to authorities.

It is very interesting that the LLM responses to these lines - the context around what she is saying - is not in the article. I suspect, as is the case in many instances where LLMs are involved in violent planning, that the LLM was urging this behavior on. Basically entrapment - you are encouraged by a robot to become more violent and vindictive and then when you do you are handed over to police.

> may be viewed by another person

Was it viewed by another person? Yes.

Why wait for a company to build it? Get your own local hardware like I did and have those guarantees because YOU set it up.

That is correct! And exactly my point.

We want both, but, on paper, that is impossible. But in reality, we make it sorta mostly happen anyway, through making the easy defaults not private, and the private stuff not easy.

This is not ideal, because [various reasons I do not need to tell you], but it has proven to be the best we can do to mostly achieve both goals.

Kinda like how capitalism isn't great but just the least worst option we've found so far.

___

The actual fundamental underlying problem being that not all people are equal, but we kinda have to pretend they are, because not doing so leads to fascism and other terrible stuff.

But we kinda also do not want to fully pretend that, because doing so leads to yet other terrible stuff.

Hence the quadruple-speak and contradictions to kinda sorta somehow have a somewhat functioning reality.

A lot of people causing issues are people that can't make sense of many things (like many terrorists). They get a fixed idea and they end up doing something bad. You would catch those with some (basic) surveillance.

A lot of normal people (not wanting to cause issues) might benefit from some privacy, if they understand what are the trade-offs (like government overreach). They can then use a slightly more complex tech.

We would still remain with the couple intelligent but sociopaths (think Unabomber style), but I think no solution can fix all cases.

I urge those people to share their full information from banks, companies (salaries, agreements, contracts), full health information and share publically all the texts they ever wrote (incl. as teenagers) and all the photos they have taken. And give away all the passwords to all of the services so people could check that they are truly clean. Just to be sure, just for everyone's security, right?..

In a old happy little idealised village, it became known quickly, who started to behave oddly and timely intervention could happen. In the modern anonymous mass cities?

No one (wants to) notice the madmen scheming in his isolated flat, surrounded by strangers. Until he explodes.

Unfortunately I also don't trust our government agencies with the surveillance - because they ain't transparent either and the self surveillance seems broken.

" "I'm going to kill that guy", which for non-autists means "that guy was really annoying" you should be reported."

And unfortunately there are lots of real threats being made under the disguise of humor. And much harder to separate im text. So maybe don't talk of murdering people in general, AI surveillance or not?

LLMs aren't email or file storage. AI labs aren't just shuttling bytes around, they're interpreting those bytes and taking action based on them. These models _already_ react viscerally in response to users saying disturbing things: the only practical difference is the ability (or obligation) for the model to escalate that concern. I'm not sure the ethics we hold AI companies to should be different than if a human being was typing out the responses.

Privacy is obviously hugely important, but this isn't the government surveiling every message. It's companies having an obligation to flag real, credible threats according to the law, which is a very different problem space.

Normal people live prejudice. It's (erm, claude-speak) load-bearing for them, given just how complex reality is and given just how well it reduces that complexity.