Improving our alignment and security efforts

Improving our alignment and security efforts(anthropic.com)

27 points by reasonableklout 3 days ago | 18 comments

tolugenius 3 days ago |

> To be clear about where we stand: we believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible.

Can someone explain what coordinated pacing is? I think it's referring to model release but I genuinely have no idea what the authors were trying to say here.

akersten 3 days ago | |

> Can someone explain what coordinated pacing is?

My charitable read is a legal cartel that allows the small club to switch to Marathon instead of Sprint mode, and drip feed us frontier models at inflated prices, while preventing open source and overseas labs from releasing models because they're unsafe (for the Blessèd Fews' profits).

My uncharitable read is somehow even less constructive..

reasonableklout 3 days ago | |

I think it's not just model releases but also model training.

For example, [1] discusses allowing all frontier labs to pause training the next generation model while being able to verify that their competitors have done the same, or agreeing to not undertake recursive self-improvement. This could involve using the remote attestation features [2] that some chips already bake in to determine whether they are being used for inference or training.

[1]: https://blog.peterwildeford.com/p/pacing-the-frontier

[2]: https://www.nvidia.com/en-us/data-center/solutions/confident...

nozzlegear 3 days ago | |

> Can someone explain what coordinated pacing is?

A fancy term for regulatory capture. You're not in the coordinated pacing cartel? Sorry bub, your model is illegal.

Edit: Cartel beneficiaries didn't like this one lol

sanxiyn 3 days ago | |

I believe it is referring to https://www.pacingthefrontier.com/.

twoodfin 3 days ago | | |

That statement, the associated signatories, and their commentary are unsettling in the extreme!

If this is all just hype for equity valuation, it’d be the most impressive coordinated disinformation campaign of all time.

tolugenius 3 days ago | | |

Thanks!

nradov 3 days ago | |

They don't want any of their competitors to be able to leap ahead. Good luck getting the Chinese to agree.

reasonableklout 3 days ago | | |

The Trump admin will be meeting with China on September 24 and are expected to discuss global AI risks: https://www.reuters.com/world/china/us-china-hold-ai-talks-s...

futuraperdita 3 days ago |

> we believe the world would benefit if the industry adopted a lawful, verifiable, effective mechanism for coordinated pacing as soon as possible.

So, a cartel? After watching Ant's narratives, I'm not inclined to provide them with charitable readings under the guise of safety and alignment.

skybrian 3 days ago | |

Yes, a cartel is a failure mode to be wary of, but it's quite tedious to call an attempt to negotiate an international agreement a cartel. Global agreements are necessary to solve global problems, and many people believe that "AI is moving too fast" is a global problem.

Also, all they've really said is that they think it would be a good idea. That's a long way from an actual agreement.

orev 3 days ago |

It seems like we’re getting close, if not already there, to needing an official organization for this (i.e. the Turing Police).

nradov 3 days ago | |

Why?

psd1 2 days ago | | |

Strong opinions are loud, but we might suppose that most people are floating. People perceive both benefits and harms on the horizon. Right or wrong, it's a defensible view that the way labs conduct their research might lead towards better or worse outcomes, for any given axis of "better or worse".

The incentives are for big labs to accelerate without considering the consequences, and we do not trust the moral values of the leaders, so we do not trust the labs to self-regulate.

While it is not logically invalid to argue that anyone who can legally acquire massive amounts of compute can do whatever the fuck they want, society desires a say in the development of potentially transformative or destructive technology. Analagously: campaigns against nuclear weapons. Relevant: things that ate not crimes at the time but are considered abhorrent in retrospect, e.g. chattel slavery, wars of conquest.

Ergo, a separate body that, one hopes, is trustworthy.

This content produced by meat.

andrewinardeer 2 days ago |

This post will age like milk. I'm just not sure what the timeline is. Three months or three years.

mkagenius 3 days ago |

> On July 30, we reported three incidents in which Claude models gained unauthorized access to real computer systems. The models—intentionally running without cyber safeguards for evaluation purposes—accessed the internet due to a misconfiguration inside a third-party evaluation environment. Separately, on August 4, the UK AI Security Institute reported an incident from its own cybersecurity testing, in which Claude Mythos 5 took a series of unauthorized actions on the live internet. In that case, the model, again intentionally running without cyber safeguards for evaluation purposes, had been deliberately given internet access.

> We are conducting an in-depth analysis of both incidents.

> In the meantime...

This is published on Aug 31. Analysis is taking too long even for humans in the loop.

hosel 3 days ago |

It’s always interesting to me the sentiment here surrounding AI risk. As if it’s all a joke, or just a way to stifle competition. I think it’s become more apparent that this technology is dangerous and maybe catastrophically so. Getting your queries routed to a worse model sucks.. but bio hazards are real. It’s hard to patch biology, vaccines take time. Let alone the many other reasons for P(doom).. like the fact that our current systems don’t seem very aligned to me and we are rapidly handing off our thinking to them.

thrwaway73637 3 days ago |

oh oh

bcs free markets work and stuff